An AI-generated image self-checking method

Abstract

The present invention relates to the field of image data processing technology, and specifically to an AI-generated image self-inspection method, the method comprising: determining the problem area of ​​the generated image and the source area of ​​the target screen element as a whole corresponding to the problem area; determining the content weight of the problem area based on the geometric relationship between the problem area and the source area; determining the connection elements in the adjacent elements of the problem area whose connection probability with the target screen element is greater than a preset connection threshold; determining the correlation strength between the connection elements and the problem area, and determining the target repair area corresponding to the problem area using the content weight and the correlation strength; and repairing the target repair area using the target mask corresponding to the target repair area until the problem area no longer exists. The present invention analyzes the influence of the problem area itself and combines it with its correlation with the adjacent elements, obtains the diffusion status of the influence of the problem area in each element, and determines a more accurate repair area, thereby improving the self-inspection and repair quality of the generated image.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an AI-generated image self-checking method. Background Art

[0002] The main process for AI-generated images is as follows: the user enters a text description, and the AI ​​model, using models such as the Variational Autoencoder (VAE), converts the input text into vectors to understand the user's input. The AI-generated image then generates an image based on the input text. The output of AI-generated images may contain logical errors or semantic deviations. Therefore, algorithms are often required to verify that the generated image conforms to physical rules, semantic logic, and common sense. The output is then tested, modified, and feedback is provided to ensure the quality of the generated image.

[0003] During the self-check process of AI-generated images, if a problem area is found, it needs to be reconstructed and repaired, and then re-checked to see if there are any problems. Existing techniques for reconstructing problem areas often simply set the problem area and a certain range adjacent to it as the repair area. This fails to consider that the specific relationships between different elements will affect the corresponding areas to varying degrees, resulting in low quality self-check and repair of the generated image. Summary of the Invention

[0004] In order to solve the technical problem that the existing restoration technology has low self-check restoration quality for AI-generated images, the purpose of the present invention is to provide an AI-generated image self-check method. The technical solution adopted is as follows:

[0005] The present invention provides an AI-generated image self-checking method, the method comprising:

[0006] Determine the problem area of ​​the generated image and the source area of ​​the target screen element as a whole corresponding to the problem area;

[0007] Determine the content weight of the problem area based on the geometric relationship between the problem area and the source area;

[0008] Determine the connected elements among the adjacent elements of the problem area whose connection probability with the target screen element is greater than a preset connection threshold;

[0009] Determine the association strength between the connecting elements and the problem area, and use the content weight and association strength to determine the target repair area corresponding to the problem area;

[0010] Use the target mask corresponding to the target repair area to repair the target repair area until there is no problem area;

[0011] The content weight represents the degree of impact of changes in the problem area on other content.

[0012] Furthermore, determining the problem area of ​​the generated image and the source area of ​​the entire target screen element corresponding to the problem area includes:

[0013] Generate images based on text content and use generative adversarial models to identify problematic areas in the generated images.

[0014] The entire pixel area of ​​the target screen element corresponding to the problem area is taken as the source area.

[0015] Furthermore, determining the content weight of the problem area based on the geometric relationship between the problem area and the source area includes:

[0016] Extract the problem area skeleton of the problem area and the source area skeleton of the source area respectively;

[0017] Determine the closest corresponding points between the problem area skeleton and the source area skeleton, and use the difference between the corresponding points to obtain the skeleton matching degree between the problem area skeleton and the source area skeleton;

[0018] Determine the subject evaluation of the target screen elements corresponding to the problem area, and use the subject evaluation and skeleton matching to determine the content weight of the problem area;

[0019] The skeleton matching degree represents the similarity between the problem region and the source region in terms of shape and trend.

[0020] Furthermore, determining the subject evaluation of the target screen element corresponding to the problem area includes:

[0021] Determine the cosine similarity between the target semantic vector of the target screen element corresponding to the problem area and other semantic vectors of other screen elements;

[0022] The total number of elements in the generated image is obtained, and the subject evaluation of the target image element is calculated using cosine similarity, a preset subject threshold, and the total number of elements.

[0023] Furthermore, the method of determining the content weight of the problem area by using the subject evaluation and the skeleton matching degree includes:

[0024] Obtain the problem pixel area of ​​the problem area and the source pixel area of ​​the source area as well as the pixel area difference between the two;

[0025] The content weight of the problem area is calculated using subject evaluation, skeleton matching and pixel area difference.

[0026] Furthermore, determining the connected elements among the adjacent elements of the problem area whose connection probability with the target screen element is greater than a preset connection threshold includes:

[0027] determining the subject evaluation difference between the subject evaluation of the target screen element and the subject evaluation of adjacent elements in the problem area;

[0028] Determine the cosine similarity between the target screen element and the adjacent elements in the problem area;

[0029] Using the subject evaluation difference and cosine similarity, the connection probability between the adjacent elements of the problem area and the target screen elements is calculated;

[0030] Adjacent elements with a connection probability greater than a preset connection threshold are regarded as connection elements.

[0031] Furthermore, the strength of the connection between the connected elements and the problem area is determined, including:

[0032] The area formed by the pixel points in the connection element that are within a preset Euclidean distance range from the problem area is determined as the transition area corresponding to the connection element;

[0033] Determine the color difference between the pixels in the target group in the transition area and the pixels in the problem area that are closest in color to the pixels in the target group, as well as the total number of pixels in the target group.

[0034] Using the color difference and the total number of pixels, the bias evaluation of the target group in the transition area to the problem area is calculated;

[0035] Using target grouping to evaluate the bias of the problem area and the connection element area, the strength of the association between the connection element and the problem area is determined;

[0036] The pixels in the target group correspond to the same Euclidean distance sub-range, and the preset Euclidean distance range includes a plurality of different Euclidean distance sub-ranges.

[0037] Furthermore, the bias evaluation of the problem area and the connection element area by using the target grouping to determine the strength of association between the connection element and the problem area includes:

[0038] The larger bias evaluation of the target group on the problem area and the connection element area is determined as the bias dominance of the target group;

[0039] When the bias dominance of adjacent target groups changes, the changed target groups are determined as disguised groups, and the number of disguised groups is determined;

[0040] determining the difference in bias evaluation between the target group's bias evaluation of the problem area and the target group's bias evaluation of the connected element area;

[0041] The association strength between the connected elements and the problem area is calculated by using the number and bias evaluation differences of the disguised groups.

[0042] Furthermore, the method of determining the target repair area corresponding to the problem area by using the content weight and the association strength includes:

[0043] The basic diffusion coefficient of the connected elements is calculated using content weight and association strength;

[0044] The basic diffusion coefficient is updated by using the biased evaluation of the problem area by the target grouping to obtain the updated diffusion coefficient;

[0045] The problem area is used as the initial repair area, and the pixels in the target group corresponding to the updated diffusion coefficient greater than the preset repair threshold are added to the initial repair area to obtain the target repair area corresponding to the problem area.

[0046] Furthermore, the method of repairing the target repair area by using the target mask corresponding to the target repair area until no problem area exists includes:

[0047] Determine the target mask corresponding to all pixels in the target repair area and determine the mask ratio of the target mask in the generated image;

[0048] If the mask ratio is greater than the preset global threshold, the entire generated image is repaired;

[0049] If the mask ratio is less than or equal to the preset global threshold, the target repair area is locally repaired until there is no problem area in the generated image.

[0050] The present invention has the following beneficial effects:

[0051] The present invention analyzes the influence of the problem area itself and combines it with the correlation with adjacent elements to obtain the diffusion status of the problem area's influence in each element to determine a more accurate repair area, thereby improving the self-inspection and repair quality of the generated image.

[0052] Specifically, the present invention obtains AI-generated images and extracts problem areas, analyzes the subjectivity of the elements corresponding to the problem areas in the picture, and obtains the skeleton matching degree by comparing the overall skeleton differences between the problem areas and their corresponding elements, and obtains the content weight of the problem areas in combination with the area ratio of the problem areas in their corresponding elements. Analyze the elements adjacent to the problem areas and record them as connecting elements. Obtain the association strength based on the transition between the problem areas and the connecting elements. Based on the content weight and association strength, obtain the diffusion of the influence of the problem areas in each element, obtain a more accurate repair area for corresponding repair, and realize AI-generated image self-inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 A flowchart of the steps of an AI-generated image self-checking method provided by one embodiment of the present invention;

[0055] Figure 2 This is a detailed flowchart of step S2 in an AI-generated image self-checking method provided by one embodiment of the present invention;

[0056] Figure 3 This is a detailed flowchart of step S3 in an AI-generated image self-checking method provided by one embodiment of the present invention;

[0057] Figure 4 This is a detailed flowchart of step S4 in an AI-generated image self-checking method provided by one embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram of the hardware operating environment of the AI-generated image self-checking device involved in an embodiment of the present invention;

[0059] Figure 6 This is a schematic diagram of the framework structure of the AI-generated image self-checking system involved in the embodiment of the present invention;

[0060] Figure 7 Schematic diagram of the problem region skeleton and source region skeleton involved in an AI-generated image self-checking method provided by one embodiment of the present invention;

[0061] Figure 8 A schematic diagram of a problem area deviation curve and a connection element deviation curve involved in an AI-generated image self-checking method provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in combination with the accompanying drawings and preferred embodiments, describes in detail an AI-generated image self-checking method proposed in accordance with the present invention, its specific implementation, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics of one or more embodiments may be combined in any suitable form.

[0063] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0064] The following describes in detail a specific solution of an AI-generated image self-checking method provided by the present invention with reference to the accompanying drawings.

[0065] Example 1:

[0066] For the AI-generated image self-checking method provided by the present invention, please refer to Figure 1 , which shows a flowchart of the steps of the AI-generated image self-inspection method provided by one embodiment of the present invention.

[0067] The method comprises:

[0068] Step S1, determining the problem area of ​​the generated image and the source area of ​​the entire target screen element corresponding to the problem area;

[0069] Specifically, the step S1 includes:

[0070] Generate images based on text content and use generative adversarial models to identify problematic areas in the generated images.

[0071] The entire pixel area of ​​the target screen element corresponding to the problem area is taken as the source area.

[0072] In this embodiment, the generated image and the problem area therein are first obtained. The specific steps are as follows:

[0073] The user inputs text into the AI ​​image generation model. Using a model such as the BERT (Bidirectional Encoder Representations from Transformers) model, the natural language text is converted into a text encoding vector, and the semantic vector used to generate the image is extracted to parse the input text. The image is then generated based on the semantic vector. Using a model such as the GAN (Generative Adversarial Network) model, the generated image is back-propagated into the latent space. Reconstruction errors are used to identify local distortions, revealing the problem area in the image and the corresponding target image element (corresponding to the text content). This results in the problem area in the generated image. All pixels in the entire region corresponding to the target image element in the problem area are then collected and recorded as the source area.

[0074] Step S2: determining the content weight of the problem area based on the geometric relationship between the problem area and the source area; wherein the content weight represents the degree of influence of changes in the problem area on other content;

[0075] In the prompt text entered by the user, the screen elements corresponding to different words have different influences. Therefore, when repairing the problem area, in order to accurately obtain the mask of the repair area, it is necessary to analyze the importance of the elements corresponding to the problem area itself (the target screen elements) in the image, that is, the degree of influence.

[0076] When repairing content in a problem area, because the content in the generated image is interrelated and correlated, when repairing content in a problem area, related and adjacent content also needs to be adjusted to match the repair content to avoid image fragmentation. Therefore, when repairing the problem area, the actual repair area that needs to be adjusted will be larger than the original problem area. Therefore, it is necessary to analyze the correlation between other content and the problem area to understand the impact of the problem area and then determine the repair area.

[0077] Since the problem area may not be the entire content of a certain screen element, but often a local area of ​​an element, for the problem area itself, if there is a problem with the local details of the corresponding element, the impact on other parts of the element and other elements may be relatively small, but if it is a key part, it is more likely to affect other parts or even other elements.

[0078] The content weight of the problem area is obtained based on the overall shape and area relationship between the problem area and its corresponding element. The more consistent the overall trend of the problem area and its corresponding element and the larger its area, the more likely it is to have a greater impact; otherwise, the more likely it is to have a smaller impact.

[0079] For details, please refer to Figure 2 In one embodiment, step S2 includes:

[0080] Step S21, extracting the problem area skeleton of the problem area and the source area skeleton of the source area respectively;

[0081] Step S22: determining the closest corresponding points between the problem region skeleton and the source region skeleton, and using the difference between the corresponding points to obtain a skeleton matching degree between the problem region skeleton and the source region skeleton; wherein the skeleton matching degree represents the similarity in shape and trend between the problem region and the source region;

[0082] Step S23 : determining the subject evaluation of the target screen element corresponding to the problem area, and determining the content weight of the problem area using the subject evaluation and the skeleton matching degree.

[0083] In this embodiment, the skeletons of the problem region and the entire region of the corresponding target screen element (source region) are extracted, and the geometric relationship between the problem region and the entire region is expressed by the skeleton matching degree.

[0084] Specifically, using an existing skeleton extraction algorithm such as the Zhang-Suen thinning algorithm, the problem region skeleton of the problem region and the source region skeleton of the source region are extracted respectively;

[0085] The point closest to each pixel in the source region skeleton and its problem region skeleton is recorded as the corresponding point, and the center point of the minimum circumscribed rectangle of the source region skeleton is used as the reference point. The skeleton matching degree is obtained based on the difference between the corresponding points. , specific skeleton matching The calculation method is:

[0086]

[0087] Where: Represents the total number of all pixel points in the source region skeleton, represents the Euclidean distance (the difference between corresponding points) from any m-th pixel in the source region skeleton to the corresponding point in the problem region skeleton. Represents the angle between the m-th pixel point and its corresponding point and the line connecting the reference point. The model is used here for inverse normalization. For the problem region skeleton and source region skeleton mentioned here, please refer to Figure 7 , Figure 7 Schematic diagram of the problem region skeleton and source region skeleton involved in an AI-generated image self-checking method provided by one embodiment of the present invention.

[0088] Regarding step S23, the content weight of the problem area is determined by using the subject evaluation and the skeleton matching degree, specifically including:

[0089] Obtain the problem pixel area of ​​the problem area and the source pixel area of ​​the source area as well as the pixel area difference between the two;

[0090] The content weight of the problem area is calculated using subject evaluation, skeleton matching and pixel area difference.

[0091] The skeleton matching degree shows the similarity between the problem area and its source area in terms of shape and trend. The greater the similarity, the more likely the problem area is to be an important content in the source area. The content weight of the problem area is further obtained by combining the area ratio of the problem area to the source area. The calculation method is:

[0092]

[0093] Where: Indicates the subject evaluation of the target screen element corresponding to the problem area. represents the skeleton matching degree of the problem area, Indicates the area of ​​the problem area (number of pixels, problem pixel area), Indicates the area of ​​the source region (source pixel area).

[0094] At this point, the content weight of the problem area is obtained. The above process obtains the content weight of the problem area by comparing the shape and area of ​​the problem area with its corresponding screen element and the main evaluation of its corresponding element. The content weight is used to express the influence of the problem area on other content in the screen when it changes.

[0095] In step S23, determining the subject evaluation of the target screen element corresponding to the problem area specifically includes:

[0096] Determine the cosine similarity between the target semantic vector of the target screen element corresponding to the problem area and other semantic vectors of other screen elements;

[0097] The total number of elements in the generated image is obtained, and the subject evaluation of the target image element is calculated using cosine similarity, a preset subject threshold, and the total number of elements.

[0098] The subjectivity of the target screen element is analyzed based on the relationship between the target screen element and other elements. As a decorative secondary element, it is often attached to the main body of the screen. Therefore, the main body of the screen will be related to multiple elements. The stronger the relationship between a certain screen element and other elements, the more likely it is to be the main body of the screen. The subject evaluation of each element is obtained by the relationship between the text semantic vector corresponding to the screen element and other vectors. The subject evaluation of the kth element (as the target screen element) is: The calculation method is:

[0099]

[0100] Where: represents the cosine similarity between the semantic vector corresponding to the kth screen element (target screen element) and the ith semantic vector corresponding to other screen elements (other semantic vectors), Indicates the total number of elements in the picture. Indicates the preset subject threshold, which can be set according to actual conditions. Represents the ReLU function, the expression of the function is .

[0101] Similarly, we can also obtain the subject evaluation of each element in the picture, which reflects the possibility of each element as the subject of the picture, and serves as the basis for judging the influence of each element in the picture.

[0102] Step S3, determining the connected elements among the adjacent elements of the problem area whose connection probability with the target screen element is greater than a preset connection threshold;

[0103] For details, please refer to Figure 3 In one embodiment, step S3 includes:

[0104] Step S31, determining the subject evaluation difference between the subject evaluation of the target screen element and the subject evaluation of the adjacent elements in the problem area;

[0105] Step S32, determining the cosine similarity between the target image element and the adjacent elements in the problem area;

[0106] Step S33, using the subject evaluation difference and cosine similarity, calculate the connection probability between the adjacent elements of the problem area and the target image element;

[0107] Step S34: adjacent elements whose connection probability is greater than a preset connection threshold are used as connection elements.

[0108] The content weight reflects the influence of the problem area itself. However, when the connection relationship is different, the influence on the element areas adjacent to the problem area will be different. Therefore, the connection relationship between the elements adjacent to the problem area and the problem area also needs to be considered.

[0109] Adjacent screen elements may have strong logical connections with the problem area. When adjusting the problem area, these elements will be significantly affected, and these adjacent elements are more likely to require matching and repair. Therefore, this step analyzes the relationships between screen elements adjacent to the problem area.

[0110] Although some elements are adjacent to the problem area, they may not be connected to the problem area. For example, if one adjacent area is the foreground subject and the other is the background area, in this case, although adjacent, they are not connected, so the impact of the change is relatively small. Based on the difference in the subject evaluation of each adjacent element and its semantic relevance to the corresponding element in the problem area, the adjacent elements are analyzed to identify those that may have a connection with the problem area.

[0111] Connection evaluation of any m-th adjacent element The calculation method is:

[0112]

[0113] Where: It represents the cosine similarity between the mth adjacent element and the target screen element corresponding to the problem area, represents the principal evaluation of the kth adjacent element, Indicates the subject evaluation of the target screen element corresponding to the problem area. represents the linear normalization function.

[0114] The connection evaluation represents the connection probability between each adjacent element and the target screen element corresponding to the problem area. The stronger the connection relationship, the greater the impact of the change on the element.

[0115] Exclude irrelevant elements through connection relationships. A connection threshold of 0.3 can be preset. All adjacent elements with a connection evaluation greater than the preset connection threshold are recorded as connected elements, and the rest are recorded as irrelevant elements. In the subsequent analysis of the specific correlation strength, irrelevant elements are excluded to avoid involving too many irrelevant elements when obtaining the repair area.

[0116] Step S4, determining the association strength between the connection element and the problem area, and determining the target repair area corresponding to the problem area using the content weight and the association strength;

[0117] In AI-generated images, problematic areas often experience adhesion between adjacent areas. This means that the connected elements are logically chaotic and intertwined. The more chaotic the intertwining between the elements and the problematic area, the greater the impact of the problematic area on the element. This step uses the adhesion and intertwining between connected elements and the problematic area to determine the strength of the association between each connected element and the problematic area.

[0118] For details, please refer to Figure 4 In one embodiment, in step S4, determining the strength of association between the connection element and the problem area includes:

[0119] Step S41, determining the area formed by the pixels in the connection element that are within a preset Euclidean distance range from the problem area as the transition area corresponding to the connection element;

[0120] Step S42, determining the color difference between the pixels in the target group in the transition area and the pixels in the problem area with the closest corresponding colors, as well as the total number of pixels in the target group; wherein the pixels in the target group correspond to the same Euclidean distance sub-range, and the preset Euclidean distance range includes multiple different Euclidean distance sub-ranges;

[0121] Step S43, using the color difference and the total number of pixels, calculate the bias evaluation of the target group in the transition area to the problem area;

[0122] Taking any h-th connection element as an example, analyze the adhesion and interlacing of each connection element with the problem area through the following steps:

[0123] The Euclidean distance between the pixel corresponding to the hth connected element and the problem area is rounded up to a distance unit. The distance unit is divided into a group of 5 units (for example, all pixels with a distance of 1-5 units are the first group, all pixels with a distance of 6-10 units are the second group, and so on). A total of 40 groups are taken as the transition area.

[0124] Analyze each pixel group's bias toward the problem area and the connected element area. Use color differences to reflect the bias of each pixel group.

[0125] Taking the problem area as an example, the bias evaluation of the tth group (target group) of pixels in the transition area of ​​the hth connected element towards the problem area is The calculation method is:

[0126]

[0127]

[0128] Where: Indicates the color difference of the j-th pixel in the t-th group in the h-th transition area, which represents the color difference value between this pixel and the point with the closest color in the problem area. Indicates the total number of channels used for color comparison. Here, RGB channels are used, so L=3. Represents the value of the j-th pixel in the t-th group in the h-th transition area in the l-th channel, is the value of any pixel in the problem area in the lth channel, Represents the total number of pixels in the tth group in the hth transition region (the total number of pixels).

[0129] Step S44: using the target grouping to evaluate the bias of the problem area and the connection element area, the strength of the association between the connection element and the problem area is determined.

[0130] Similarly, obtain the bias evaluation of each group (each target group) in the transition area of ​​the hth connected element to the problem area, and replace the problem area pixels with the non-transition area pixels of the connected element to obtain the bias evaluation of the transition area to the connected element area itself, and draw a bias curve based on the distance from each group to the problem area from near to far. Figure 8 As shown, Figure 8 A schematic diagram of a problem area deviation curve and a connection element deviation curve involved in an AI-generated image self-checking method provided by one embodiment of the present invention.

[0131] The step S44 specifically includes:

[0132] The larger bias evaluation of the target group on the problem area and the connection element area is determined as the bias dominance of the target group;

[0133] When the bias dominance of adjacent target groups changes, the changed target groups are determined as disguised groups, and the number of disguised groups is determined;

[0134] determining the difference in bias evaluation between the target group's bias evaluation of the problem area and the target group's bias evaluation of the connected element area;

[0135] The association strength between the connected elements and the problem area is calculated by using the number and bias evaluation differences of the disguised groups.

[0136] The group with the greater bias evaluation between the problem area and the connecting element area itself is recorded as the bias dominant group. If the bias dominant group is different from the previous group, the group is recorded as the changing direction group. The bias of each group in the transition area shows the situation of regional adhesion and interlacing. When the bias of the transition area changes frequently and the bias evaluation of the two is close, it means that the content of the transition section is more likely to be chaotic, and the group is more affected by the connecting element, and its association strength is greater. The association strength of the hth connecting element The calculation method is:

[0137]

[0138] Where: represents the number of direction-changing groups in the h-th connection element, Indicates the total number of groups where T=40, Indicates the bias evaluation of the t-th group of pixels in the transition area of ​​the h-th connected element on the problem area, It represents the bias evaluation of the tth group of pixels in the transition area of ​​the hth connected element to the connected element area itself. It is the denominator anti-zero parameter, which can be set as needed, for example, it can be 0.0000001.

[0139] Similarly, the association strength of all connected elements can be obtained.

[0140] At this point, the association strength of all connected elements is obtained. The magnitude of the association strength represents the degree of influence on the connected elements with stronger connections among the elements adjacent to the problem area.

[0141] Regarding step S4, determining the target repair area corresponding to the problem area using the content weight and the association strength includes:

[0142] The basic diffusion coefficient of the connected elements is calculated using content weight and association strength;

[0143] The basic diffusion coefficient is updated by using the biased evaluation of the problem area by the target grouping to obtain the updated diffusion coefficient;

[0144] The problem area is used as the initial repair area, and the pixels in the target group corresponding to the updated diffusion coefficient greater than the preset repair threshold are added to the initial repair area to obtain the target repair area corresponding to the problem area.

[0145] The content weight of the problem area itself determines the basis of its influence, and the different association strengths of the areas connected to it lead to different degrees of influence. Therefore, the diffusion coefficient of each connected element is obtained based on the content weight of the problem area itself and the association strength of each element.

[0146] The basic diffusion coefficient of the h-th connected element The calculation method is:

[0147]

[0148] Where: Indicates the content weight of the problem area, Represents the association strength of the h-th connected element.

[0149] The problem area is recorded as the initial repair area, and each group of pixels in the transition area of ​​the h-th connected element is added to the repair area in sequence. The diffusion coefficient is updated every time a group is added. The attenuation after the t-th group of the transition area of ​​the h-th connected element is added is The calculation method is:

[0150]

[0151] Where: Indicates that the preset hyperparameter controls the size of the attenuation here, which is 0.3. Indicates the target group's biased evaluation of the problem area.

[0152] Update the diffusion coefficient , the preset repair threshold is 0.6, when When , the pixels in the t+1th group are added to the initial repair area, otherwise the judgment of the hth connected element is terminated. Similarly, the pixels that meet the conditions in all connected elements are added to the initial repair area, and the target repair area in the generated image is obtained.

[0153] Step S5: Repair the target repair area using the target mask corresponding to the target repair area until no problem area exists.

[0154] Specifically, step S5 includes:

[0155] Determine the target mask corresponding to all pixels in the target repair area and determine the mask ratio of the target mask in the generated image;

[0156] If the mask ratio is greater than the preset global threshold, the entire generated image is repaired;

[0157] If the mask ratio is less than or equal to the preset global threshold, the target repair area is locally repaired until there is no problem area in the generated image.

[0158] Generate a mask based on all pixels in the repair area and obtain the percentage of the mask in the entire image. Determine the repair strategy based on the mask and provide corresponding repair feedback. The specific steps are as follows:

[0159] The preset global threshold is 15%. When the target mask percentage exceeds the global threshold, the entire image is repaired. If it is less than or equal to the global threshold, only the masked area (corresponding to the target repair area) is repaired. The content to be repaired is regenerated, and the problem areas and corresponding masks in the repaired image are again determined. The above steps are repeated until there are no problem areas in the image. This completes the self-checking of AI-generated images.

[0160] The embodiments of the present invention analyze the influence of the problem area itself and combine it with its correlation with adjacent elements to obtain the diffusion status of the influence of the problem area in each element to determine a more accurate repair area, thereby improving the self-inspection and repair quality of the generated image.

[0161] Specifically, the present invention obtains AI-generated images and extracts problem areas, analyzes the subjectivity of the elements corresponding to the problem areas in the picture, and obtains the skeleton matching degree by comparing the overall skeleton differences between the problem areas and their corresponding elements, and obtains the content weight of the problem areas in combination with the area ratio of the problem areas in their corresponding elements. Analyze the elements adjacent to the problem areas and record them as connecting elements. Obtain the association strength based on the transition between the problem areas and the connecting elements. Based on the content weight and association strength, obtain the diffusion of the influence of the problem areas in each element, obtain a more accurate repair area for corresponding repair, and realize AI-generated image self-inspection.

[0162] Example 2:

[0163] The embodiment of the present invention further provides an AI-generated image self-checking device. The AI-generated image self-checking device can be a data computing and processing device such as a computer, a server, or a combination of multiple devices.

[0164] like Figure 5 As shown, Figure 5 It is a structural diagram of the hardware operating environment of the AI-generated image self-inspection device involved in the embodiment of the present invention.

[0165] like Figure 5As shown, the AI-generated image self-test device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Communication bus 1002 is used to enable communication between these components. User interface 1003 may include a display and an input unit, such as a control panel. Optionally, user interface 1003 may also include a standard wired interface or a wireless interface. Network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WiFi interface). Memory 1005 may be high-speed RAM or non-volatile memory, such as disk storage. Memory 1005 may also be a storage system independent of processor 1001. Memory 1005, a computer storage medium, may include an AI-generated image self-test program.

[0166] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation of the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0167] Continue to refer to Figure 5 , Figure 5 The memory 1005 as a computer-readable storage medium may include an operating system, a user interface module, a network communication module, and an AI-generated image self-test program.

[0168] exist Figure 5 In the embodiment, the network communication module is mainly used to connect to the server and can communicate data with the server; and the processor 1001 can call the AI ​​generated image self-test program stored in the memory 1005 and execute the steps in the above embodiments.

[0169] The hardware structure of the above-mentioned AI-generated image self-checking device is used to implement various embodiments of the AI-generated image self-checking method of the present invention.

[0170] In addition, the present invention also provides an AI generated image self-checking system, please refer to Figure 6 , the AI ​​generated image self-inspection system includes:

[0171] Impact analysis module A10 is used to determine the problem area of ​​the generated image and the source area of ​​the target image element; based on the geometric relationship between the problem area and the source area, determine the content weight of the problem area;

[0172] The association analysis module A20 is configured to determine, among the adjacent elements of the problem area, connected elements whose connection probability with the target image element is greater than a preset connection threshold; determine the strength of the association between the connected elements and the problem area, and determine the target repair area corresponding to the problem area using the content weight and the strength of the association;

[0173] The reconstruction and repair module A30 is used to repair the target repair area using the target mask corresponding to the target repair area until no problem area exists.

[0174] Furthermore, the impact analysis module A10 is further configured to:

[0175] Generate images based on text content and use generative adversarial models to identify problematic areas in the generated images.

[0176] The entire pixel area of ​​the target screen element corresponding to the problem area is taken as the source area.

[0177] Furthermore, the impact analysis module A10 is further configured to:

[0178] Extract the problem area skeleton of the problem area and the source area skeleton of the source area respectively;

[0179] Determine the closest corresponding points between the problem area skeleton and the source area skeleton, and use the difference between the corresponding points to obtain the skeleton matching degree between the problem area skeleton and the source area skeleton;

[0180] Determine the subject evaluation of the target screen elements corresponding to the problem area, and use the subject evaluation and skeleton matching to determine the content weight of the problem area;

[0181] The skeleton matching degree represents the similarity between the problem region and the source region in terms of shape and trend.

[0182] Furthermore, the impact analysis module A10 is further configured to:

[0183] Determine the cosine similarity between the target semantic vector of the target screen element corresponding to the problem area and other semantic vectors of other screen elements;

[0184] The total number of elements in the generated image is obtained, and the subject evaluation of the target image element is calculated using cosine similarity, a preset subject threshold, and the total number of elements.

[0185] Furthermore, the impact analysis module A10 is further configured to:

[0186] Obtain the problem pixel area of ​​the problem area and the source pixel area of ​​the source area as well as the pixel area difference between the two;

[0187] The content weight of the problem area is calculated using subject evaluation, skeleton matching and pixel area difference.

[0188] Furthermore, the association analysis module A20 is further configured to:

[0189] determining the subject evaluation difference between the subject evaluation of the target screen element and the subject evaluation of adjacent elements in the problem area;

[0190] Determine the cosine similarity between the target screen element and the adjacent elements in the problem area;

[0191] Using the subject evaluation difference and cosine similarity, the connection probability between the adjacent elements of the problem area and the target screen elements is calculated;

[0192] Adjacent elements with a connection probability greater than a preset connection threshold are regarded as connection elements.

[0193] Furthermore, the association analysis module A20 is further configured to:

[0194] The area formed by the pixel points in the connection element that are within a preset Euclidean distance range from the problem area is determined as the transition area corresponding to the connection element;

[0195] Determine the color difference between the pixels in the target group in the transition area and the pixels in the problem area that are closest in color to the pixels in the target group, as well as the total number of pixels in the target group.

[0196] Using the color difference and the total number of pixels, the bias evaluation of the target group in the transition area to the problem area is calculated;

[0197] Using target grouping to evaluate the bias of the problem area and the connection element area, the strength of the association between the connection element and the problem area is determined;

[0198] The pixels in the target group correspond to the same Euclidean distance sub-range, and the preset Euclidean distance range includes a plurality of different Euclidean distance sub-ranges.

[0199] Furthermore, the association analysis module A20 is further configured to:

[0200] The larger bias evaluation of the target group on the problem area and the connection element area is determined as the bias dominance of the target group;

[0201] When the bias dominance of adjacent target groups changes, the changed target groups are determined as disguised groups, and the number of disguised groups is determined;

[0202] determining the difference in bias evaluation between the target group's bias evaluation of the problem area and the target group's bias evaluation of the connected element area;

[0203] The association strength between the connected elements and the problem area is calculated by using the number and bias evaluation differences of the disguised groups.

[0204] Furthermore, the association analysis module A20 is further configured to:

[0205] The basic diffusion coefficient of the connected elements is calculated using content weight and association strength;

[0206] The basic diffusion coefficient is updated by using the biased evaluation of the problem area by the target grouping to obtain the updated diffusion coefficient;

[0207] The problem area is used as the initial repair area, and the pixels in the target group corresponding to the updated diffusion coefficient greater than the preset repair threshold are added to the initial repair area to obtain the target repair area corresponding to the problem area.

[0208] Furthermore, the reconstruction and repair module A30 is further configured to:

[0209] Determine the target mask corresponding to all pixels in the target repair area and determine the mask ratio of the target mask in the generated image;

[0210] If the mask ratio is greater than the preset global threshold, the entire generated image will be fully repaired;

[0211] If the mask ratio is less than or equal to the preset global threshold, the target repair area is locally repaired until there is no problem area in the generated image.

[0212] The specific implementation of the AI-generated image self-checking system of the present invention is basically the same as the embodiments of the above-mentioned AI-generated image self-checking method, and will not be repeated here.

[0213] In addition, the present invention further provides a computer-readable storage medium having an AI-generated image self-test program stored thereon, wherein when the AI-generated image self-test program is executed by a processor, the steps of the above-mentioned AI-generated image self-test method are implemented.

[0214] Among them, the method implemented when the AI-generated image self-test program is executed can refer to the various embodiments of the AI-generated image self-test method of the present invention, and will not be repeated here.

[0215] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0216] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0217] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0218] The above description is only a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. All equivalent structural / method transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the scope of protection of the present invention.

Claims

1. An AI-generated image self-checking method, characterized in that: The method comprises: Determine the problem area of ​​the generated image and the source area of ​​the target screen element as a whole corresponding to the problem area; Determine the content weight of the problem area based on the geometric relationship between the problem area and the source area; Determine the connected elements among the adjacent elements of the problem area whose connection probability with the target screen element is greater than a preset connection threshold; Determine the association strength between the connecting elements and the problem area, and use the content weight and association strength to determine the target repair area corresponding to the problem area; Use the target mask corresponding to the target repair area to repair the target repair area until there is no problem area; The content weight represents the degree of impact of changes in the problem area on other content.

2. The AI-generated image self-checking method according to claim 1, characterized in that: Determine the problem area of ​​the generated image and the source area of ​​the target screen element corresponding to the problem area, including: Generate images based on text content and use generative adversarial models to identify problematic areas in the generated images. The entire pixel area of ​​the target screen element corresponding to the problem area is taken as the source area.

3. The AI-generated image self-checking method according to claim 1, characterized in that: The determining of the content weight of the problem area based on the geometric relationship between the problem area and the source area includes: Extract the problem area skeleton of the problem area and the source area skeleton of the source area respectively; Determine the closest corresponding points between the problem area skeleton and the source area skeleton, and use the difference between the corresponding points to obtain the skeleton matching degree between the problem area skeleton and the source area skeleton; Determine the subject evaluation of the target screen elements corresponding to the problem area, and use the subject evaluation and skeleton matching to determine the content weight of the problem area; The skeleton matching degree represents the similarity between the problem region and the source region in terms of shape and trend.

4. The AI-generated image self-checking method according to claim 3, characterized in that: Determining the subject evaluation of the target screen element corresponding to the problem area includes: Determine the cosine similarity between the target semantic vector of the target screen element corresponding to the problem area and other semantic vectors of other screen elements; The total number of elements in the generated image is obtained, and the subject evaluation of the target image element is calculated using cosine similarity, a preset subject threshold, and the total number of elements.

5. The AI-generated image self-checking method according to claim 3, characterized in that: The method of determining the content weight of the problem area by using the subject evaluation and the skeleton matching degree includes: Obtain the problem pixel area of ​​the problem area and the source pixel area of ​​the source area as well as the pixel area difference between the two; The content weight of the problem area is calculated using subject evaluation, skeleton matching and pixel area difference.

6. The AI-generated image self-checking method according to claim 1, characterized in that: The step of determining the connected elements in the adjacent elements of the problem area whose connection probability with the target screen element is greater than a preset connection threshold includes: determining the subject evaluation difference between the subject evaluation of the target screen element and the subject evaluation of adjacent elements in the problem area; Determine the cosine similarity between the target screen element and the adjacent elements in the problem area; Using the subject evaluation difference and cosine similarity, the connection probability between the adjacent elements of the problem area and the target screen elements is calculated; Adjacent elements with a connection probability greater than a preset connection threshold are regarded as connection elements.

7. The AI-generated image self-checking method according to claim 1, characterized in that: Determine the strength of the connection between the connecting element and the problem area, including: The area formed by the pixel points in the connection element that are within a preset Euclidean distance range from the problem area is determined as the transition area corresponding to the connection element; Determine the color difference between the pixels in the target group in the transition area and the pixels in the problem area that are closest in color to the pixels in the target group, as well as the total number of pixels in the target group. Using the color difference and the total number of pixels, the bias evaluation of the target group in the transition area to the problem area is calculated; Using target grouping to evaluate the bias of the problem area and the connection element area, the strength of the association between the connection element and the problem area is determined; The pixels in the target group correspond to the same Euclidean distance sub-range, and the preset Euclidean distance range includes a plurality of different Euclidean distance sub-ranges.

8. The AI-generated image self-checking method according to claim 7, characterized in that: The method of using target grouping to evaluate the bias of the problem area and the connection element area to determine the strength of association between the connection element and the problem area includes: The larger bias evaluation of the target group on the problem area and the connection element area is determined as the bias dominance of the target group; When the bias dominance of adjacent target groups changes, the changed target groups are determined as disguised groups, and the number of disguised groups is determined; determining the difference in bias evaluation between the target group's bias evaluation of the problem area and the target group's bias evaluation of the connected element area; The association strength between the connected elements and the problem area is calculated by using the number and bias evaluation differences of the disguised groups.

9. The AI-generated image self-checking method according to claim 7, characterized in that: The method of determining the target repair area corresponding to the problem area by using the content weight and the association strength includes: The basic diffusion coefficient of the connected elements is calculated using content weight and association strength; The basic diffusion coefficient is updated by using the biased evaluation of the problem area by the target grouping to obtain the updated diffusion coefficient; The problem area is used as the initial repair area, and the pixels in the target group corresponding to the updated diffusion coefficient greater than the preset repair threshold are added to the initial repair area to obtain the target repair area corresponding to the problem area.

10. The AI-generated image self-checking method according to claim 1, characterized in that: The method of repairing the target repair area by using the target mask corresponding to the target repair area until no problem area exists includes: Determine the target mask corresponding to all pixels in the target repair area and determine the mask ratio of the target mask in the generated image; If the mask ratio is greater than the preset global threshold, the entire generated image will be fully repaired; If the mask ratio is less than or equal to the preset global threshold, the target repair area is locally repaired until there is no problem area in the generated image.

Images