Large model-based image processing method, storage medium, and electronic device

Abstract

Disclosed in the present application are a large model-based image processing method, a storage medium, and an electronic device. The method comprises: on the basis of segmentation processing on an image to be processed, acquiring a segmented image comprising a target element and a background element, wherein the background element comprises a first pixel and the target element comprises a second pixel; on the basis of the first pixel on the edge position of the segmented image and / or the second pixel on the edge position of the target element, determining whether the target element has a defective region on the edge position of the segmented image, wherein the edge position of the segmented image corresponds to the edge position of the target element; and if so, by means of a large model, performing image repair on the defective region in the image to be processed, so as to generate a target image corresponding to the image to be processed. Thus, in case that target elements in images to be processed are defective, said images can be automatically repaired, and repaired images can clearly and completely express the target elements.

Description

Image processing methods, storage media, and electronic devices based on large models

[0001] This application claims priority to Chinese Patent Application No. 202411184374.3, filed on August 26, 2024, entitled "Image Processing Method, Storage Medium and Electronic Device Based on Large Model", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence, specifically to an image processing method and apparatus based on a large model. This application also relates to a target image acquisition method, a commodity image processing method based on a large model, a computer storage medium and electronic device, and a computer program product. Background Technology

[0003] Image processing refers to the technical process of analyzing, processing, and manipulating digital images, aiming to improve the visual effect of images or extract useful information to adapt and use them in different scenarios. Image processing can improve the visual effect of images and extract useful information. The image processing process can include multiple aspects such as image segmentation, feature extraction and recognition, restoration, and enhancement.

[0004] Image segmentation is the process of dividing an image into multiple non-overlapping regions, each satisfying certain specific properties. This is done to identify and distinguish different objects or regions in the image, providing a foundation for subsequent feature extraction and recognition.

[0005] Image feature extraction and recognition is the process of extracting useful information from an image, such as edges, corners, and textures, or identifying specific objects or scenes within the image.

[0006] Image inpainting is the process of repairing and reconstructing images that may have been damaged or distorted during acquisition or transmission.

[0007] Image enhancement is a crucial step in image processing, primarily focusing on improving the visual quality of an image to make it clearer, easier to observe, and easier to understand. This is achieved by adjusting attributes such as contrast, brightness, and color, or by employing more complex algorithms to remove noise and sharpen edges.

[0008] With the rapid development of computer vision and artificial intelligence technologies, image processing technology is also constantly advancing and innovating. Advanced technologies such as deep learning and convolutional neural networks are widely used in the field of image processing, greatly improving the efficiency and accuracy of image processing. Summary of the Invention

[0009] This application provides an image processing method based on a large model to solve the problems of image distortion and poor user experience in existing technologies.

[0010] This application provides an image processing method based on a large model, including:

[0011] Based on the segmentation process of the image to be processed, a segmented image including target elements and background elements is obtained; wherein, the background element includes a first pixel, and the target element includes a second pixel;

[0012] Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, determine whether the target element has a missing region at the edge position of the segmented image; wherein, the edge position of the segmented image corresponds to the edge position of the target element;

[0013] If so, the missing region in the image to be processed is repaired using a large model to generate a target image corresponding to the image to be processed.

[0014] In some embodiments, determining whether the target element has a missing region at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element includes:

[0015] Obtain the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element;

[0016] Based on the distribution data of the first pixel, or the distribution data of the second pixel, determine whether the target element has a missing region at the edge position of the segmented image.

[0017] In some embodiments, determining whether the target element has a missing region at the edge position of the segmented image based on the distribution of the first pixel at the edge position, or the distribution of the second pixel at the edge position, includes:

[0018] Based on the proportion of the first pixel at the edge of the segmented image, determine whether the target element has a missing region at the edge of the segmented image;

[0019] or,

[0020] Based on the proportion of the second pixel at the edge position of the segmented image, it is determined whether the target element has a missing region at the edge position of the segmented image.

[0021] In some embodiments, determining whether the target element has a missing region at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element includes:

[0022] Obtain the pixel points of the first pixel and the pixel points of the second pixel; wherein the pixel points of the first pixel and the pixel points of the second pixel correspond to each other;

[0023] Based on the distance between the first pixel and the second pixel, it is determined whether the target element has a missing region at the edge of the segmented image.

[0024] In some embodiments, the step of performing image inpainting on the incomplete region in the image to be processed using a large model to generate a target image corresponding to the image to be processed includes:

[0025] The direction of the defect in the image to be processed is determined based on the defective region.

[0026] The large model uses the missing direction as the designated repair direction, and expands the image to be processed along the designated repair direction by a specified size to generate an expanded image;

[0027] The expanded image is filled based on the image to be processed to generate a filled image;

[0028] Based on the image to be processed, the filled image is repaired to generate a repaired image;

[0029] The repaired image and the image to be processed are fused together to generate a target image corresponding to the image to be processed.

[0030] In some embodiments, the large model uses the damaged direction as the designated repair direction, and expands the image to be processed by a specified size along the designated repair direction to generate an expanded image, including:

[0031] The large model expands the image to be processed along the specified repair direction, according to a specified step size or a specified extension ratio, to generate an expanded image.

[0032] In some embodiments, filling the expanded image with the image to be processed to generate a filled image includes:

[0033] Based on the selection method, a target image is selected from the edges of the image to be processed;

[0034] The expanded image is filled based on the target image to generate the filled image.

[0035] In some embodiments, it also includes:

[0036] The repair results of the target image are verified;

[0037] When the verification result meets the storage requirements, the target image is stored.

[0038] This application also provides a method for acquiring a target image, including:

[0039] Select the image to be processed;

[0040] Obtain the target image corresponding to the image to be processed, as determined by the above-described large model-based image processing method.

[0041] This application also provides a product image processing method based on a large model, including:

[0042] Based on the segmentation process of the product image to be processed, a segmented image including target product elements and background elements is obtained; wherein, the background element includes a first pixel, and the target product element includes a second pixel;

[0043] Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element, determine whether the target product element has a missing region at the edge position of the segmented image; wherein, the edge position of the segmented image corresponds to the edge position of the target product element;

[0044] If so, the missing area in the product image to be processed is repaired using a large model to generate a target product image corresponding to the product image to be processed.

[0045] This application also provides a computer storage medium for storing data generated by a network platform, and a program for processing the data generated by the network platform.

[0046] When the program is read and executed by the processor, it performs the image processing method based on the large model as described above, or the target image acquisition method as described above, or the commodity image processing method based on the large model as described above.

[0047] This application also provides an electronic device, including:

[0048] processor;

[0049] The memory is used to store programs that process data generated by the network platform. When the program is read and executed by the processor, it executes the image processing method based on the large model as described above, or executes the target image acquisition method as described above, or executes the product image processing method based on the large model as described above.

[0050] Compared with the prior art, this application has the following advantages:

[0051] This application provides a large-model-based image processing method that enables batch processing and automated image processing. Specifically, it takes an input image to be processed, segments it to obtain a segmented image including target elements and background elements, where the background elements include first pixels and the target elements include second pixels. Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, it is determined whether the target element has a missing region at the edge position of the segmented image, where the segmented image edge position corresponds to the target element edge position. If so, the large-model is used to repair the missing region in the image to be processed, generating a target image corresponding to the image to be processed. This automatically completes image repair, ensuring that even if the target element in the image to be processed is missing, it can be automatically repaired, and the repaired image can clearly and completely represent the target element.

[0052] The entire processing chain requires no user awareness, selection, judgment, or participation in verification and review. This not only improves image processing efficiency but also enhances image restoration quality and user experience.

[0053] Furthermore, it allows for backtracking of image processing nodes throughout the entire image processing chain, enabling precise location of processing nodes and adjustments to the large model. Additionally, by scoring the quality of the target image, images that meet quality requirements can be directly synchronized to online scenarios for use, avoiding the user experience degradation caused by a reworked target image being detected as poor by other image quality detection methods outside the image processing chain. Attached Figure Description

[0054] Figure 1 is a flowchart of an image processing method based on a large model provided in this application.

[0055] Figure 2 is a schematic diagram of a segmentation processing embodiment in an image processing method based on a large model provided in this application.

[0056] Figure 3 is a schematic diagram of an embodiment of a large-model-based image processing method provided in this application, specifically regarding a missing region.

[0057] Figure 4 is a schematic diagram of an embodiment of the repaired target image in an image processing method based on a large model provided in this application.

[0058] Figure 5 is a schematic diagram of the repair and verification of the target image in an image processing method based on a large model provided in this application.

[0059] Figure 6 is a schematic diagram of the structure of an image processing device based on a large model provided in this application.

[0060] Figure 7 is a flowchart of a commodity image processing method based on a large model provided in this application.

[0061] Figure 8 is a schematic diagram of the structure of a commodity image processing device based on a large model provided in this application.

[0062] Figure 9 is a flowchart of a target image acquisition method provided in this application.

[0063] Figure 10 is a schematic diagram of the structure of a target image acquisition device provided in this application.

[0064] Figure 11 is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0065] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0066] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The descriptive terms used in this application and the appended claims, such as "a," "first," and "second," are not intended to limit quantity or sequence, but rather to distinguish information of the same type from one another.

[0067] As can be seen from the above background technology, the inventive concept of this application originates from image processing scenarios. Specific application scenarios include the production of product series images required by merchants on online lifestyle application service platforms, or the production of product series images required by offline merchants. The common approach is to first cut out a product image and then fill it with the same background image, displaying the product image through a store's product list, thereby improving the user's menu selection experience. However, in practical applications, it has been found that when the original product image is incomplete, the product image obtained by cutting out the image and filling the background is not aesthetically pleasing. Furthermore, images with incomplete main subjects often look better against the original background than after optimization, making the image processing more detrimental than beneficial. In practical applications, this also results in merchants displaying various styles of food images, leading to a cluttered and disorganized visual effect.

[0068] In view of this, this application provides an image processing method based on a large model. Through automated intelligent image processing, it can integrate the identification, incompleteness, and repair of image subject elements into a single processing chain, thereby improving image processing quality while also increasing the speed and quantity of image processing.

[0069] As shown in Figure 1, Figure 1 is a flowchart of an image processing method based on a large model provided in this application. The method includes:

[0070] Step S101: Based on the segmentation processing of the image to be processed, obtain a segmented image including target elements and background elements; wherein, the background elements include a first pixel, and the target elements include a second pixel;

[0071] Step S102: Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, determine whether the target element has a missing region at the edge position of the segmented image, or determine whether the edge position of the target element has a missing region in the segmented image; wherein the edge position of the segmented image corresponds to the edge position of the target element.

[0072] Step S103: If so, then perform image restoration on the missing region in the image to be processed using a large model to generate a target image corresponding to the image to be processed.

[0073] The execution of steps S101 to S103 above is based on a large model, that is, the above steps are performed on the input image to be processed, thereby obtaining the target image output after processing based on the large model. In this embodiment, the large model refers to a neural network model with a large number of parameters and a complex structure in the fields of deep learning and artificial intelligence. Compared with traditional small models, large models have stronger learning and generalization capabilities and can handle more complex tasks and datasets. Large models are usually trained on huge datasets to acquire broader knowledge and capabilities. The large model in this embodiment is a large model used for image processing. The type of large model is not specifically limited, as long as it can implement the functions and processing procedures involved in this embodiment.

[0074] The steps S101 to S103 described above will be described in detail below.

[0075] Regarding step S101: Based on the segmentation processing of the image to be processed, obtain a segmented image including target elements and background elements; wherein, the background elements include a first pixel, and the target elements include a second pixel.

[0076] The target element refers to the image element included in the image to be processed. For example, for a product image, the target element can be determined based on the different types of products in the product image and the attributes that the product is meant to express. The target element of a food image is the element in the image that represents the characteristics of the food's attributes, as shown in Figure 2, the element that holds the food in the container. The target element of a household goods image is the element in the image that represents the characteristics of the household goods, such as clothing, shoes, and socks. Of course, it can also be an image from other application scenarios. In this embodiment, a food image is mainly used as an example, and the target element can be the food elements in the food image.

[0077] The specific implementation process of step S101 may include:

[0078] Step S101-1: Perform segmentation processing on the image to be processed to obtain a transparent image including target elements and background elements;

[0079] Step S101-2: Determine the transparent image as a segmented image.

[0080] The segmentation process in step S101-1 can include threshold-based segmentation, edge-based segmentation, region-based segmentation, graph-based segmentation, and deep learning-based segmentation. Threshold-based segmentation can be used on images with significant color or grayscale value differences. By setting one or more thresholds, pixels are divided into target pixels and background pixels. Edge segmentation relies on detecting edge information in the image; edges are places where grayscale, color, or texture characteristics change drastically, typically corresponding to the boundary between the target and background images. Region segmentation relies on the similarity between pixels for region segmentation. Graph-based segmentation treats the image as a graph, where nodes correspond to pixels and edges correspond to similarity measures between pixels. Then, graph cut algorithms (such as GrabCut, Normalized Cut, etc.) are used to segment the target and background images. Deep learning methods utilize models such as Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) for segmentation based on the image depth map.

[0081] As shown in Figure 2, which is a schematic diagram of a segmentation processing embodiment in an image processing method based on a large model provided in this application, the segmented image after segmentation processing of the image to be processed in this embodiment can be a mask image (binarized image), such as a transparent image where the background pixels are transparent pixels, that is, the background elements include a first pixel that is transparent, and the target elements include a second pixel that is non-transparent. Of course, the first pixel can also be other pixels, and the second pixel can be a pixel different from the first pixel; it is not only distinguished by transparent and non-transparent pixels.

[0082] To improve the quality of images in subsequent repair, verification and other processing links, edge processing can also be performed on the segmented images, such as smoothing and noise reduction, thereby improving the segmentation quality of the segmented images.

[0083] To improve the efficiency of image processing, avoid resource waste, and ensure accuracy, after step S101, the process may further include: performing an image quality assessment on the segmented image to prevent poor image quality from affecting subsequent image restoration results and thus wasting computational resources. In this embodiment, the assessment of the segmented image quality may include judging whether the segmented image contains a watermark, whether the target element is blurred and / or overexposed, and whether the target element and background element overlap. If the image quality meets the quality threshold (or subsequent processing requirements), the subsequent steps are executed; otherwise, a prompt can be output. Of course, the image quality assessment can also be performed before step S101, i.e., by acquiring the image to be processed and assessing its image quality.

[0084] Regarding step S102: Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, determine whether the target element has a missing region at the edge position of the segmented image; wherein, the edge position of the segmented image corresponds to the edge position of the target element.

[0085] As shown in Figure 3, Figure 3 is a schematic diagram of an embodiment of a large-model-based image processing method provided in this application, specifically addressing the missing region. The purpose of step S102 is to determine whether the target element in the image to be processed is missing, that is, whether the representation of the target element in the image to be processed is complete. Therefore, step S102 can be implemented in several ways as follows.

[0086] Method 1 includes:

[0087] Step S102-11: Obtain the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element;

[0088] Step S102-12: Determine whether the target element has a missing region at the edge position of the segmented image based on the distribution data of the first pixel or the distribution data of the second pixel. The determination of whether the target element has a missing region at the edge position of the segmented image can be based on the pixel distribution ratio of the first pixel at the edge position. That is, when the ratio of the first pixel at the edge position is not 100%, it indicates that a second pixel exists at the edge position, and the target element is missing. Of course, the smaller the ratio of the first pixel at the edge position, the higher the degree of missingness of the target element. When the ratio of the second pixel at the edge position is greater than 0, it indicates that the target element is missing; the larger the ratio, the higher the degree of missingness. In this embodiment, the ratio can be based on the pixel ratio of the first pixel or the second pixel in the height and / or length direction at the edge position. Therefore, the first pixel or the second pixel can be selected along the height and / or length direction of the edge position. Continuing with the previous example, if the first pixel is a transparent pixel, meaning the segmented image is a transparent image, the distribution ratio is the ratio of the transparent pixel to the total number of pixels on each edge of the transparent image, thus determining the missing region. Alternatively, it is the ratio of the non-transparent pixel to the total number of pixels on each edge of the transparent image, determining the missing region. This allows us to determine whether there are missing target elements in each direction of the segmented image.

[0089] Therefore, step S102-12 includes:

[0090] Step S102-12-11: Based on the proportion of the first pixel at the edge position of the segmented image, determine whether the target element has a missing region at the edge position of the segmented image;

[0091] or,

[0092] Step S102-12-12: Based on the proportion of the second pixel at the edge position of the segmented image, determine whether the target element has a missing region at the edge position of the segmented image.

[0093] Method two includes:

[0094] Step S102-21: Obtain the pixel points of the first pixel and the pixel points of the second pixel; wherein the pixel points of the first pixel and the pixel points of the second pixel correspond to each other;

[0095] Step S102-22: Determine whether the target element has a missing region at the edge position of the segmented image based on the distance from the pixel of the first pixel to the pixel of the second pixel.

[0096] When the distance from the first pixel to the second pixel is 0, it indicates that the second pixel at the edge of the target element is located at the edge of the segmented image, thus the target element is incomplete. Alternatively, it can be determined whether the second pixel can be obtained at the edge of the segmented image; if so, it also indicates that the target element is incomplete.

[0097] Therefore, in this embodiment, the pixel of the first pixel can be obtained at the edge position of the segmented image along the height and / or length direction of the image, and the pixel of the second pixel can be obtained at the edge position of the target element along the height and / or length direction of the image, or at the edge position of the segmented image along the height and / or length direction of the image.

[0098] In this embodiment, when the missing region is determined by distance, the selection of the first pixel can be based on selecting the pixel in the middle region of the height and / or length direction of the segmented image edge position. The selection of the second pixel can be based on selecting the pixel in the middle region of the height and / or length direction of the target element edge position. Then, the distances are calculated sequentially. If there is a case where the distance is zero, it indicates that there is a missing region of the target element in the segmented image. When the missing region is determined by the pixel value of the target element, it can also be based on selecting along the height and / or length direction of the target element at the edge position of the segmented image. If the pixel value of the target element exists during the selection process at the edge position of the segmented image, it indicates that there is a missing region of the target element.

[0099] The edge position of the target element can be determined based on the selected range. The edge position of the segmented image can be determined based on the image contour. The second pixel can be obtained from any pixel within the selected range. In this embodiment, it is a pixel in the middle of the selected range or a pixel near the center of the target element. For example, a×3, where a represents the maximum height of the target element and 3 represents the width value of the edge of the target element selected towards the center. Here, 3 is used as an example for illustration only.

[0100] In this embodiment, the determination of the incomplete region can be achieved by processing each edge of the segmented image in the manner described above, where the first pixel and the second pixel are pixels with corresponding positions.

[0101] Regarding step S103: Image restoration is performed on the incomplete region in the image to be processed using a large model to generate a target image corresponding to the image to be processed.

[0102] As shown in Figure 4, Figure 4 is a schematic diagram of an embodiment of the repaired target image in an image processing method based on a large model provided in this application. The purpose of step S103 is to repair the incomplete region, so that the target element is fully expressed in the image to be processed. Specifically, it may include:

[0103] Step S103-1: Determine the defect direction in the image to be processed based on the defective region; of course, the defect direction can also be determined based on the location of the first pixel and / or the second pixel, specifically, the defect direction can be determined based on the coordinates of the pixel.

[0104] Step S103-2: The large model uses the missing direction as the designated repair direction and expands the image to be processed along the designated repair direction by a specified size to generate an expanded image. In this embodiment, the expansion or extension can specifically be to expand the image to be processed along the designated repair direction by a specified step size or a specified extension ratio to generate an expanded image. For example, if the image to be processed is 800×800 and the segmented image obtained after segmentation is also 800×800, with the missing direction being to the left, the expanded image to be processed is 800×1200. Based on the expanded image to be processed with a height of 400, that is, 800×400, the expanded image is a white background image to reduce the computational load for subsequent repair. Also, because extending along the designated direction by a specified size can effectively avoid image distortion caused by over-extension.

[0105] Step S103-3: Fill the extended image according to the image to be processed to generate a filled image. For filling the extended image, fill content can be selected from the image to be processed, and then the extended image is filled. Continuing the previous example, an 800×5 strip image is selected from the edge of the image to be processed, and the strip image is sequentially arranged to fill the extended image. When the extended image is completely filled, an 800×400 filled image is generated. The filled image is used as an image prompt for the large model in step S103-4. Therefore, step S103-3 can be specifically implemented as follows: Step S103-31: Select a target image from the edge of the image to be processed according to the selection method; Step S103-32: Fill the extended image according to the target image to generate the filled image. In this embodiment, an 800×5 strip image is used as an example of the target image for illustration. The method of selecting a target image from the edge of the image to be processed is not limited to the above example.

[0106] Step S103-4: Based on the image to be processed, the filled image is repaired by calling the large model to generate a repaired image. In this embodiment, the repair of the filled image can be achieved by calling a large model (e.g., the SD large model, also known as the Stable Diffusion Inpaint large model). As shown in the example above, the large model repairs the 800×400 filled image. For example, the filled image is used as an image prompt for the large model. The image optimization module of the large model iteratively optimizes the image features of the filled image and inputs the iteratively optimized image features into the image decoder to reconstruct the image, thereby obtaining the repaired image.

[0107] Step S103-5: Fuse the repaired image and the image to be processed to generate a target image corresponding to the image to be processed. In this embodiment, Poisson Blending can be used to fuse the repaired image and the image to be processed. Poisson Blending is an image processing technique whose core objective is to seamlessly blend a specific region of one image into another while maintaining the consistency of lighting, color, and texture in the blended region. This technique is based on the Poisson equation and utilizes the properties of the Laplace equation, using the pixel values ​​of the two images as boundary conditions, and obtaining the blended image by solving the Laplace equation. During the Poisson Blending process, the continuity of color and brightness within the blended region is maintained, while discontinuities at the boundaries are minimized, resulting in a more natural blended image. In addition, weighted averaging, multi-resolution fusion, etc., can also be used. This embodiment uses Poisson Blending as an example for description.

[0108] To avoid affecting the success rate and accuracy of image restoration due to aesthetic degradation caused by extension or extension of multiple target elements, the target image can also be verified, as shown in Figure 5. Figure 5 is a schematic diagram of target image restoration verification in an image processing method based on a large model provided in this application, specifically including:

[0109] Step S104: Verify the repair result of the target image;

[0110] Step S105: When the verification result meets the storage requirements, the target image is stored.

[0111] The specific implementation process of step S104 may include: segmenting the target image to obtain a segmented target image; determining whether there are missing regions in the segmented target image; if not, the verification result meets the storage requirements. Furthermore, when there are no missing regions in the segmented target image, image attribute scoring can be performed on the target image, such as scoring the target image's lighting, sharpness, and the sharpness of target elements, thereby optimizing and iterating the large model based on the scores and improving the model's optimization and iteration efficiency.

[0112] Similarly, the process of verifying the target image may also include image quality judgment of the target segmented image. If the image meets the quality requirements, the incomplete region is determined. If the image does not meet the quality requirements, the image to be processed is reprocessed or a prompt is output.

[0113] The above is a detailed description of an embodiment of an image processing method based on a large model provided in this application. This method can perform batch processing and automate the image processing process. Specifically, based on the input image to be processed, it automatically completes image restoration according to steps S101 to S103, automatically repairing any missing target elements in the image to be processed, ensuring the restored image clearly and completely represents the target elements. The entire processing chain requires no user awareness, selection, judgment, or verification, improving image processing efficiency, image restoration quality, and user experience. Furthermore, it allows for backtracking of image processing nodes throughout the entire processing chain, enabling precise location of processing nodes and adjustments to the large model. Additionally, by scoring the quality of the target image, images meeting quality requirements can be directly synchronized to online scenarios for use, avoiding the user experience degradation caused by the restored target image being detected as poor by other image quality detection methods outside the image processing chain.

[0114] This application provides an embodiment of an image processing method based on a large model. By judging the image quality after segmentation, it ensures the repairability of the image to be processed, avoiding the poor user experience caused by repeatedly reminding users of image quality requirements in existing technologies. Furthermore, it avoids the increased review costs caused by the randomness of image-to-image or text-to-image generation methods in existing technologies, as well as the increased batch generation costs, increased error rates, and image distortion caused by the need for subjective user operation during batch processing.

[0115] The above is a detailed description of an embodiment of an image processing method based on a large model provided in this application. Corresponding to the aforementioned embodiment of an image processing method based on a large model, this application also discloses an embodiment of an image processing apparatus based on a large model. Please refer to Figure 6. Since the apparatus embodiment is basically similar to the method embodiment, it is described simply. For relevant details, please refer to the description of the method embodiment. The apparatus embodiment described below is merely illustrative.

[0116] As shown in Figure 6, Figure 6 is a schematic diagram of the structure of an image processing device based on a large model provided in this application. The device includes: an acquisition unit 601, a determination unit 602, and a generation unit 603.

[0117] The acquisition unit 601 is used to acquire a segmented image including target elements and background elements according to the segmentation processing of the image to be processed; wherein, the background elements include a first pixel and the target elements include a second pixel;

[0118] The determining unit 602 is used to determine whether the target element has a missing region at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element; wherein the edge position of the segmented image corresponds to the edge position of the target element;

[0119] The generation unit 603 is used to perform image repair on the missing region in the image to be processed using a large model when the determination result of the determining unit 602 is yes, thereby generating a target image corresponding to the image to be processed.

[0120] The acquisition unit 601 includes an acquisition subunit and a determination subunit; the segmentation subunit is used to segment the image to be processed to acquire a transparent image including target elements and background elements. The determination subunit is used to determine the transparent image as a segmented image.

[0121] For details regarding the acquisition unit 601, please refer to the content of step S101 above, which will not be elaborated here.

[0122] The determining unit 602 includes, in one manner, an acquisition subunit and a determining subunit. The acquisition subunit is used to acquire the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element. The determining subunit is used to determine, based on the distribution data of the first pixel or the distribution data of the second pixel, whether the target element has a missing region at the edge position of the segmented image. Specifically, the determining subunit is used to determine whether the target element has a missing region at the edge position of the segmented image based on the proportion of the first pixel at the edge position of the segmented image; or, based on the proportion of the second pixel at the edge position of the segmented image, whether the target element has a missing region at the edge position of the segmented image.

[0123] The second method of the determining unit 602 includes: an acquisition subunit and a determining subunit. The acquisition subunit is used to acquire the pixel points of the first pixel and the pixel points of the second pixel; wherein the pixel points of the first pixel and the pixel points of the second pixel correspond to each other. The determining subunit is used to determine whether the target element has a missing region at the edge position of the segmented image based on the distance from the pixel points of the first pixel to the pixel points of the second pixel.

[0124] For details regarding the specific content of the determining unit 602, please refer to the content of step S102 above, which will not be elaborated here.

[0125] The generation unit 603 includes: a determining subunit, a first generation subunit, a second generation subunit, a third generation subunit, and a fourth generation subunit; the determining subunit is used to determine the missing direction in the image to be processed based on the missing region; the first generation subunit is used by the large model to use the missing direction as a repair specified direction and expand the image to be processed along the repair specified direction by a specified size to generate an expanded image; the second generation subunit is used to fill the expanded image based on the image to be processed to generate a filled image; the third generation subunit is used to repair the filled image based on the image to be processed to generate a repaired image; the fourth generation subunit is used to fuse the repaired image and the image to be processed to generate a target image corresponding to the image to be processed.

[0126] The first generation subunit is specifically used to extend the image to be processed along the specified repair direction by the large model according to a specified step size or a specified extension ratio, thereby generating an extended image.

[0127] The second generation subunit includes a selection subunit and a generation subunit. The selection subunit is used to select a target image at the edge of the image to be processed according to a selection method. The generation subunit is used to fill the extended image according to the target image to generate the filled image.

[0128] It also includes a verification unit and a storage unit. The verification unit is used to verify the repair result of the target image. The storage unit is used to store the target image when the verification result meets the storage requirements.

[0129] For details regarding the generation unit 603, please refer to the details of step S103 above, which will not be elaborated here.

[0130] The above is a description of an image processing apparatus based on a large model provided in this application. For details about the apparatus, please refer to the description of the method above. This is only a summary description.

[0131] Based on the above, this application also provides a product image processing method based on a large model, as shown in Figure 7. Figure 7 is a flowchart of a product image processing method based on a large model provided by this application, which includes:

[0132] Step S701: Based on the segmentation processing of the product image to be processed, obtain a segmented image including target product elements and background elements; wherein, the background elements include a first pixel, and the target product elements include a second pixel;

[0133] Step S702: Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element, determine whether the target product element has a missing region at the edge position of the segmented image; wherein, the edge position of the segmented image corresponds to the edge position of the target product element;

[0134] Step S703: If so, then use a large model to perform image repair on the missing area in the product image to be processed, and generate a target product image corresponding to the product image to be processed.

[0135] The product images to be processed can be food images, clothing images, jewelry images, etc. In other words, the items traded on the application platform are not limited to physical items but also include virtual items.

[0136] It may also include:

[0137] Step S704: Verify the repair result of the target product image;

[0138] Step S705: When the verification result meets the storage requirements, the target product image is stored.

[0139] For details regarding steps S701 to S705, please refer to steps S101 to S105 above; they will not be elaborated here.

[0140] Accordingly, this application also provides a product image processing device based on a large model, as shown in Figure 8. Figure 8 is a schematic diagram of the structure of a product image processing device based on a large model provided in this application. The device includes:

[0141] The acquisition unit 801 is used to acquire a segmented image including target product elements and background elements according to the segmentation processing of the product image to be processed; wherein, the background element includes a first pixel and the target product element includes a second pixel;

[0142] The determining unit 802 is configured to determine whether the target product element has a missing region at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element; wherein the edge position of the segmented image corresponds to the edge position of the target product element;

[0143] The generation unit 803 is used to perform image repair on the missing area in the product image to be processed through a large model when the determination result of the determining unit 802 is yes, so as to generate a target product image corresponding to the product image to be processed.

[0144] Based on the above, this application also provides a method for acquiring a target image, as shown in Figure 9. Figure 9 is a flowchart of a method for acquiring a target image provided by this application. An embodiment of this method may include:

[0145] Step S901: Select the image to be processed;

[0146] Step S902: Obtain the target image corresponding to the image to be processed, as determined by the large model-based image processing method as described above.

[0147] In this embodiment, the selection of the image to be processed and the acquisition of the target image can both be implemented on the client side. The client can be determined according to the specific application scenario. For example, when processing product images, the client can be the merchant's client.

[0148] After acquiring the target image, the client can output and display it locally, modify it, or store it according to its own needs. Remote output and modification are also possible. In other words, the client can process the target image according to its specific requirements and scenarios.

[0149] Accordingly, this application also provides a target image acquisition device, as shown in FIG10. FIG10 is a schematic diagram of the structure of a target image acquisition device provided by this application. An embodiment of the device may include:

[0150] Selection unit 1001: Select the image to be processed;

[0151] Acquisition Unit 1002: Acquires the target image corresponding to the image to be processed, determined according to the image processing method based on the large model as described above.

[0152] For details regarding the above-mentioned target image acquisition methods and devices, please refer to the descriptions of the above-mentioned large-model-based image processing methods and large-model-based commodity image processing methods, which will not be elaborated here.

[0153] Based on the above, this application also provides a computer storage medium for storing data generated by a network platform and a program for processing the data generated by the network platform.

[0154] When the program is read and executed by the processor, it performs the image processing method based on the large model as described above, or it performs the product image processing method based on the large model as described above.

[0155] Based on the above, this application also provides an electronic device, as shown in Figure 11, which is a schematic diagram of the structure of an electronic device provided by this application. The electronic device includes:

[0156] Processor 1101;

[0157] The memory 1102 is used to store a program for processing data generated by the network platform. When the program is read and executed by the processor, it performs the image processing method based on the large model as described above, or performs the product image processing method based on the large model as described above.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0159] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0160] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0161] 1. Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0162] 2. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. An image processing method based on a large model, characterized in that, include: Based on the segmentation process of the image to be processed, a segmented image including target elements and background elements is obtained; wherein, the background element includes a first pixel, and the target element includes a second pixel; Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element, determine whether the target element has a missing region at the edge position of the segmented image; wherein, the edge position of the segmented image corresponds to the edge position of the target element; If so, the missing region in the image to be processed is repaired using a large model to generate a target image corresponding to the image to be processed.

2. The image processing method based on a large model according to claim 1, characterized in that, The step of determining whether the target element has a missing region at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element includes: Obtain the distribution data of the first pixel at the edge position of the segmented image or the distribution data of the second pixel at the edge position of the target element; Based on the distribution data of the first pixel, or the distribution data of the second pixel, determine whether the target element has a missing region at the edge position of the segmented image.

3. The image processing method based on a large model according to claim 2, characterized in that, Determining whether the target element has a missing region at the edge position of the segmented image based on the distribution of the first pixel at the edge position, or the distribution of the second pixel at the edge position, includes: Based on the proportion of the first pixel at the edge of the segmented image, determine whether the target element has a missing region at the edge of the segmented image; or, Based on the proportion of the second pixel at the edge position of the segmented image, it is determined whether the target element has a missing region at the edge position of the segmented image.

4. The image processing method based on a large model according to claim 1, characterized in that, The step of determining whether the target element has a missing region at the edge position of the segmented image based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target element includes: Obtain the pixel points of the first pixel and the pixel points of the second pixel; wherein the pixel points of the first pixel and the pixel points of the second pixel correspond to each other; Based on the distance between the first pixel and the second pixel, it is determined whether the target element has a missing region at the edge of the segmented image.

5. The image processing method based on a large model according to claim 1, characterized in that, The step of performing image restoration on the incomplete region of the image to be processed using a large model to generate a target image corresponding to the image to be processed includes: The direction of the defect in the image to be processed is determined based on the defective region. The large model uses the missing direction as the designated repair direction, and expands the image to be processed along the designated repair direction by a specified size to generate an expanded image; The expanded image is filled based on the image to be processed to generate a filled image; Based on the image to be processed, the filled image is repaired to generate a repaired image; The repaired image and the image to be processed are fused together to generate a target image corresponding to the image to be processed.

6. The image processing method based on a large model according to claim 5, characterized in that, The large model uses the missing direction as the designated repair direction, and expands the image to be processed by a specified size along the designated repair direction to generate an expanded image, including: The large model expands the image to be processed along the specified repair direction, according to a specified step size or a specified extension ratio, to generate an expanded image.

7. The image processing method based on a large model according to claim 5, characterized in that, The step of filling the expanded image with the image to be processed to generate a filled image includes: Based on the selection method, a target image is selected from the edges of the image to be processed; The expanded image is filled based on the target image to generate the filled image.

8. The image processing method based on a large model according to claim 1, characterized in that, Also includes: The repair results of the target image are verified; When the verification result meets the storage requirements, the target image is stored.

9. A method for acquiring a target image, characterized in that, include: Select the image to be processed; Obtain the target image corresponding to the image to be processed, as determined by the image processing method based on a large model according to any one of claims 1-8.

10. A product image processing method based on a large model, characterized in that, include: Based on the segmentation process of the product image to be processed, a segmented image including target product elements and background elements is obtained; wherein, the background element includes a first pixel, and the target product element includes a second pixel; Based on the first pixel at the edge position of the segmented image and / or the second pixel at the edge position of the target product element, determine whether the target product element has a missing region at the edge position of the segmented image; wherein, the edge position of the segmented image corresponds to the edge position of the target product element; If so, the missing area in the product image to be processed is repaired using a large model to generate a target product image corresponding to the product image to be processed.

11. A computer storage medium for storing data generated by a network platform and a program for processing the data generated by the network platform; When the program is read and executed by the processor, it performs the method as described in any one of claims 1-8, or performs the method as described in claim 9, or performs the method as described in claim 10.

12. An electronic device, comprising: processor; The memory is used to store a program for processing data generated by the network platform, which, when read and executed by the processor, performs the method as described in any one of claims 1-8, or the method as described in claim 9, or the method as described in claim 10.

Images