A data preprocessing method and system based on image feature refinement

Abstract

The present invention relates to the field of image data processing technology, and in particular to a data preprocessing method and system based on image feature refinement. The method comprises the following steps: obtaining sample image data, dividing the sample image data into three levels, thereby obtaining a three-level divided data set; performing segment aggregation on the three-level divided data set, thereby obtaining aggregated segment features; constructing a multi-model set for the aggregated segment features based on the three-level divided data set and using a generative adversarial network algorithm, thereby obtaining a detail extension model set; performing multi-transfer learning on the detail extension model set, thereby obtaining an image feature refinement model; obtaining original image data, and preprocessing the original image data using an image feature refinement model, thereby obtaining a preprocessed image data set. The present invention performs data preprocessing on images through a data preprocessing platform or system based on image feature refinement to achieve more effective image enhancement and detail extension.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a data preprocessing method and system based on image feature refinement. Background Art

[0002] With the continuous development of image technology, people are using image feature refinement technology more and more widely in various industries. For industrial images, many industrial images have lost their analytical value because the defect information of a large number of industrial images is not clear enough. As a result, there are not enough clear sample images in the industry to support neural networks to use huge data training sets to further analyze and process image data. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide a data preprocessing method based on image feature refinement to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a data preprocessing method based on image feature refinement includes the following steps:

[0005] Step S1: obtaining sample image data, dividing the sample image data into three levels, thereby obtaining a three-level divided data set;

[0006] Step S2: performing segment aggregation on the three-level divided data set to obtain aggregate segment features;

[0007] Step S3: Based on the three-level divided data set and using the adversarial generative network algorithm, a multi-model set is constructed for the aggregated fragment features to obtain a detailed extended model set;

[0008] Step S4: performing multi-transfer learning on the detail expansion model set to obtain an image feature refinement model;

[0009] Step S5: Acquire original image data, and preprocess the original image data using an image feature refinement model, thereby obtaining a preprocessed image data set.

[0010] The present invention first divides the sample image data into three levels to obtain a three-level divided data set, which contains information at different levels of the sample image, can effectively extract the structure, texture and detail features of the image, and provides a rich data source for subsequent image processing. By performing segment aggregation on the three-level divided data set, the aggregated segment feature is obtained, which can meaningfully combine the local area of ​​the image, enhance the semantic expression ability of the image, and reduce the redundancy and noise of the data, providing high-quality input for subsequent image generation. Based on the three-level divided data set and using the adversarial generative network algorithm, a multi-model set is constructed for the aggregated segment feature, thereby obtaining a detail extension model set, which can utilize the powerful learning ability and diversity of the adversarial generative network algorithm to generate multiple image models of different styles and details, and provide multiple choices and references for subsequent image feature refinement. Multi-transfer learning is performed on the detail extension model set to obtain an image feature refinement model, which can utilize the technology of multi-transfer learning to fuse and optimize each model in the detail extension model set, and obtain an image feature refinement model with higher accuracy and better effect, providing a powerful tool for subsequent image preprocessing. The original image data is obtained and preprocessed using the image feature refinement model to obtain a preprocessed image data set. This preprocessing can improve and enhance the original image data in terms of clarity, brightness, contrast, color, etc., making the original image data more suitable for human viewing or machine recognition, providing a better foundation for subsequent image applications.

[0011] Preferably, the present invention further provides a data preprocessing system based on image feature refinement, which is used to execute the data preprocessing method based on image feature refinement as described above. The data preprocessing system based on image feature refinement includes:

[0012] A three-level division module is used to obtain sample image data and divide the sample image data into three levels, thereby obtaining a three-level division data set;

[0013] The fragment aggregation module is used to aggregate the fragments of the three-level partitioned data set to obtain the aggregated fragment features;

[0014] A model set construction module is used to construct a multi-model set based on the three-level partitioned data set and aggregated fragment features using a generative adversarial network algorithm, thereby obtaining a detailed extended model set;

[0015] A multi-transfer learning module is used to perform multi-transfer learning on the detail expansion model set to obtain an image feature refinement model;

[0016] The data preprocessing module is used to obtain the original image data and preprocess the original image data using the image feature refinement model to obtain a preprocessed image data set.

[0017] In summary, the present invention provides a data preprocessing system based on image feature refinement, which is composed of a three-level division module, a fragment aggregation module, a model set construction module, a multi-transfer learning module and a data preprocessing module. It can realize any data preprocessing method based on image feature refinement described in the present invention. The internal structure of the system cooperates with each other to extract features of different levels of the image, and then enhance and express the local area of ​​the combined image, and then use the adversarial generative network algorithm to generate multiple different image refinement models, and then fuse and optimize each image model to obtain a high-precision and high-effect image feature refinement model to improve and enhance the original image data, thereby improving the processing efficiency of the data preprocessing system based on image feature refinement. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0019] Figure 1 A schematic diagram of the steps of a data preprocessing method based on image feature refinement according to the present invention;

[0020] Figure 2 for Figure 1 Detailed step flow diagram of step S1;

[0021] Figure 3 for Figure 2 Detailed step flow chart of step S18 in FIG. DETAILED DESCRIPTION

[0022] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0024] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0025] To achieve this, please refer to Figures 1 to 3 The present invention provides a data preprocessing method based on image feature refinement, the method comprising the following steps:

[0026] Step S1: obtaining sample image data, dividing the sample image data into three levels, thereby obtaining a three-level divided data set;

[0027] Step S2: performing segment aggregation on the three-level divided data set to obtain aggregate segment features;

[0028] Step S3: Based on the three-level divided data set and using the adversarial generative network algorithm, a multi-model set is constructed for the aggregated fragment features to obtain a detailed extended model set;

[0029] Step S4: performing multi-transfer learning on the detail expansion model set to obtain an image feature refinement model;

[0030] Step S5: Acquire original image data, and preprocess the original image data using an image feature refinement model, thereby obtaining a preprocessed image data set.

[0031] In the embodiment of the present invention, please refer to Figure 1 FIG. 1 is a schematic diagram of the steps of the data preprocessing method based on image feature refinement of the present invention. In this example, the steps of the data preprocessing method based on image feature refinement include:

[0032] Step S1: obtaining sample image data, dividing the sample image data into three levels, thereby obtaining a three-level divided data set;

[0033] The embodiment of the present invention randomly or purposefully selects a certain number of images from an industrial image database as sample image data, which include normal images and defective images, and the defective images include blurred images and clearer images. The sample image data is formatted and detected to obtain defect data of each region in the image, and clustering is performed based on these data to divide the image data into multiple image data of different defect types, and then the image data is further divided according to the defect characteristics of each region to obtain a data set with higher defect resolution, and finally multiple different targets are used as defect indicators for division, and the index of mutual influence of different indicators is calculated to obtain target feature data, and the image data is divided for the last time based on the target feature data and using the regional growing method, that is, three-level division, to obtain a three-level division data set.

[0034] Step S2: performing segment aggregation on the three-level divided data set to obtain aggregate segment features;

[0035] The embodiment of the present invention first compresses and segments the three-level divided data set to obtain a fragment data set, and then extracts the features of the image topology and shape of the fragment data set. Here, a convolutional neural network algorithm is used to extract the features of the fragment data set to obtain a feature vector of each sub-image, and then the feature vectors of the sub-images are spliced ​​according to the categories to which they belong, so as to obtain aggregated fragment features.

[0036] Step S3: Based on the three-level divided data set and using the adversarial generative network algorithm, a multi-model set is constructed for the aggregated fragment features to obtain a detailed extended model set;

[0037] The embodiment of the present invention first performs multi-range information capture and local and global feature comparison and extraction on the aggregated fragment features, thereby obtaining a data set for complete defect feature discrimination in a region, and uses a generative adversarial network algorithm to construct a generator and a discriminator for the data set, thereby constructing multiple model sets, and then uses a conditional generative adversarial network algorithm to perform dual discrimination of the true and false features of the aggregated fragment features, and calculates and generates a loss function based on the constructed model set, thereby improving the construction of each model set.

[0038] Step S4: performing multi-transfer learning on the detail expansion model set to obtain an image feature refinement model;

[0039] The embodiment of the present invention performs adversarial iterative data analysis on the detail extension model set to obtain the adversarial process data of the generator and the discriminator in its iteration, and uses the detail extension model set to perform feature detail extension on the sample image data to obtain a detail extension image set. Based on this enhanced image set and the obtained adversarial process data, the model can be sub-gradient optimized to improve the gradient strength relationship between each generator and discriminator. Subsequently, the generator and discriminator in the detail extension model set are migrated to new industrial image data using the transfer learning method, that is, the generator and discriminator in the detail extension model set are fine-tuned with new aggregated fragment features and sub-images respectively to obtain new generators and discriminators. Finally, the new generator and discriminator are combined into an image feature refinement model.

[0040] Step S5: Acquire original image data, and preprocess the original image data using an image feature refinement model, thereby obtaining a preprocessed image data set.

[0041] The embodiment of the present invention selects a certain number of unprocessed images as original image data from an industrial image database, including normal images and defective images, divides the original image data into sub-images and extracts features required by the model, assigns corresponding defect type labels to the sub-images according to the results of the feature extraction, and passes the sub-images and defect type labels as input to a generator in a generative adversarial network model, uses the generator to generate new sub-images, and compares and analyzes the new sub-images with the original sub-images. If the new sub-image has more refined and abundant defect features, the original sub-image is replaced; otherwise, the original sub-image is retained; the replaced or retained sub-images are recombined into a new image and saved as a feature-refined data set, thereby obtaining a preprocessed image data set.

[0042] The present invention first divides the sample image data into three levels to obtain a three-level divided data set, which contains information at different levels of the sample image, can effectively extract the structure, texture and detail features of the image, and provides a rich data source for subsequent image processing. By performing segment aggregation on the three-level divided data set, the aggregated segment feature is obtained, which can meaningfully combine the local area of ​​the image, enhance the semantic expression ability of the image, and reduce the redundancy and noise of the data, providing high-quality input for subsequent image generation. Based on the three-level divided data set and using the adversarial generative network algorithm, a multi-model set is constructed for the aggregated segment feature, thereby obtaining a detail extension model set, which can utilize the powerful learning ability and diversity of the adversarial generative network algorithm to generate multiple image models of different styles and details, and provide multiple choices and references for subsequent image feature refinement. Multi-transfer learning is performed on the detail extension model set to obtain an image feature refinement model, which can utilize the technology of multi-transfer learning to fuse and optimize each model in the detail extension model set, and obtain an image feature refinement model with higher accuracy and better effect, providing a powerful tool for subsequent image preprocessing. The original image data is obtained and preprocessed using the image feature refinement model to obtain a preprocessed image data set. This preprocessing can improve and enhance the original image data in terms of clarity, brightness, contrast, color, etc., making the original image data more suitable for human viewing or machine recognition, providing a better foundation for subsequent image applications.

[0043] Preferably, step S1 comprises the following steps:

[0044] Step S11: acquiring sample image data;

[0045] Step S12: performing binary conversion on the sample image data to obtain binary image data;

[0046] Step S13: performing edge detection on the binary image data to obtain regional defect data;

[0047] Step S14: using a clustering method to perform a primary division on the regional defect data, thereby obtaining a primary division data set;

[0048] Step S15: performing key feature mining on the first-level partitioned data set to obtain a key feature set, which includes an original feature set and a defect feature set;

[0049] Step S16: performing a secondary division on the primary division data set based on the key feature set and using the defect feature division calculation formula, thereby obtaining a secondary division data set;

[0050] Step S17: labeling the defect dimension of the secondary divided data set, thereby obtaining defect labeling data;

[0051] Step S18: Based on the defect annotation data, the two-level divided data set is divided into three levels by using the region growing method, so as to obtain a three-level divided data set.

[0052] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in FIG. 1 , in this embodiment, step S1 includes the following steps:

[0053] Step S11: acquiring sample image data;

[0054] The embodiment of the present invention randomly selects a certain number of images from the preprocessed image data set as sample image data, which includes normal images and defective images.

[0055] Step S12: performing binary conversion on the sample image data to obtain binary image data;

[0056] The embodiment of the present invention grayscales each sample image by using a threshold method, and then converts it into a black and white binary image according to the size of the grayscale value, thereby forming binary image data.

[0057] Step S13: performing edge detection on the binary image data to obtain regional defect data;

[0058] The embodiment of the present invention uses the Canny algorithm to perform edge detection on each binary image to obtain the boundary contour of each defect area, and then marks it as different regional defects according to the shape and position of the contour, thereby forming regional defect data. Among them, regional defects can be divided into different types such as point defects, line defects, and surface defects.

[0059] Step S14: using a clustering method to perform a primary division on the regional defect data, thereby obtaining a primary division data set;

[0060] The embodiment of the present invention clusters each regional defect in the regional defect data by using the K-means algorithm, and divides it into several categories according to the similarity of its feature vector, thereby forming a primary division data set. The feature vector includes information such as the type, size, location, and direction of the regional defect.

[0061] Step S15: performing key feature mining on the first-level partitioned data set to obtain a key feature set, which includes an original feature set and a defect feature set;

[0062] The embodiment of the present invention uses principal component analysis to extract features from each category in the first-level divided data set, obtains the principal component feature vector of each category, and then selects the principal component feature vector as the key feature according to its importance and difference, thereby forming a key feature set. The key feature set includes an original feature set and a defect feature set, the original feature set represents the key features that match all images, and the defect feature set represents the key defect features that match the defective images.

[0063] Step S16: performing a secondary division on the primary division data set based on the key feature set and using the defect feature division calculation formula, thereby obtaining a secondary division data set;

[0064] The embodiment of the present invention evaluates and calculates each regional defect in the first-level divided data set by using the defect feature division calculation formula to obtain the feature distance between the defect data between regions, so as to analyze and judge the degree of correlation between the defect features between regions, so as to achieve the effect of dividing the data set according to the defect feature distance, and finally obtain the second-level divided data set, wherein the defect feature division calculation formula can be designed according to different industrial scenarios and tasks to achieve the best recognition effect.

[0065] Step S17: labeling the defect dimension of the secondary divided data set, thereby obtaining defect labeling data;

[0066] The embodiment of the present invention labels each defect type area defect in the secondary divided data set, and provides its dimensional information such as position, size, shape, color, etc. in the image, thereby forming defect labeling data.

[0067] Step S18: Based on the defect annotation data, the two-level divided data set is divided into three levels by using the region growing method, so as to obtain a three-level divided data set.

[0068] The embodiment of the present invention segments each defect type regional defect in the secondary divided data set by using the region growing method, and further refines it into several sub-regions according to its similarity with surrounding pixels, thereby forming a three-level divided data set.

[0069] The present invention firstly obtains sample image data to provide an original data source for subsequent image processing, and the data source can reflect the real situation and characteristics of the image. By performing binary conversion on the sample image data, binary image data is obtained, and the data can convert the grayscale information of the image into black and white information, simplifying the complexity of the image and providing convenience for subsequent edge detection. By performing edge detection on the binary image data, regional defect data is obtained, and the data can extract the edge information of the image, highlight the outline and shape of the image, and provide a basis for subsequent regional division. The regional defect data is divided into a first level using a clustering method, so as to obtain a first-level division data set, and the data set can group the regional defect data according to similarity, reduce the confusion and duplication of the data, and provide structured data for subsequent feature mining. The key feature mining is performed on the first-level division data set, so as to obtain a key feature set, which includes the original feature set and the defect feature set. The feature set can extract the important information and abnormal information in the regional defect data respectively, enhance the expression ability and recognition ability of the data, and provide a basis for subsequent defect division. Based on the key feature set and using the defect feature partitioning calculation formula, the first-level partitioned data set is divided into two levels to obtain the second-level partitioned data set. This data set can classify the regional defect data according to the degree of defect, distinguish defects of different levels and types, and provide a reference for subsequent defect labeling. The defect dimension of the second-level partitioned data set is labeled to obtain defect labeling data. This data can describe and record the location, size, shape, color, etc. of each defect in the regional defect data, providing detailed information for subsequent defect repair. Based on the defect labeling data and using the regional growing method, the second-level partitioned data set is divided into three levels to obtain a three-level partitioned data set. This data set can further refine and optimize each defect in the regional defect data, eliminate possible noise and errors, and provide more accurate and complete data for subsequent defect repair.

[0070] Preferably, the defect feature classification calculation formula in step S16 is specifically:

[0071] ;

[0072] Where D ij is the feature distance between the defect data of the i-th region and the defect data of the j-th region, N is the total number of original features, M is the total number of defect features, and w nm is the feature weight value between the nth original feature and the mth defect feature, f nm is the correlation coefficient between the nth original feature and the mth defect feature, F in is the degree of approximation between the average value of all original features in the ith region and the defect of the nth original feature, F jnis the degree of approximation between the average value of all original features in the jth region and the defect of the nth original feature, f nM is the correlation coefficient between the nth original feature and the average value of all defect features, α is the correlation coefficient between the nth original feature and the defect area, β is the correlation coefficient between the nth original feature and the defect shape feature, ε is the correlation coefficient between the nth original feature and the defect degree, z is the correlation coefficient between the nth original feature and the defect feature position, K is the total number of dimensions of the defect index, w k is the indicator weight value of the kth indicator, F k is the index value of the kth index.

[0073] The present invention constructs a defect feature division calculation formula for calculating the defect feature distance between image regions to provide a data basis for subsequent defect feature division. The formula takes into account the correlation and weight values ​​between features, including the feature weight value between the nth original feature and the mth defect feature and the correlation coefficient between the nth original feature and the mth defect feature, so as to reflect the correlation between all original features in the data and all defect features, and can adjust the corresponding weight value according to actual conditions to obtain a more reasonable feature distance. The formula also calculates the difference between the average value of all original features between two regions and the defect approximation degree of the nth original feature, so as to use the defect approximation degree difference between the two regions as the actual feature distance reference index, which can better reflect the overall defect feature distance between regions and reduce the amount of calculation to a certain extent, without the need to compare each feature of each region with that of another region. The defect approximation degree difference is calculated for each feature, and feature overfitting is avoided to a certain extent. At the same time, the results can also be adjusted accordingly through relevant weight values ​​and correlation coefficients to avoid distortion of feature distance accuracy. In addition, the formula also considers the power relationship of the sum of the correlation coefficient between the nth original feature and the average value of all defect features and the correlation coefficient between the nth original feature and the defect area size, the correlation coefficient between the nth original feature and the defect shape feature, the correlation coefficient between the nth original feature and the defect degree, and the correlation coefficient between the nth original feature and the defect feature position, to reflect the importance of the nth original feature in different defect feature dimensions, thereby increasing the discrimination and sensitivity of the feature distance, and the power can be adjusted according to different industrial scenarios and tasks, so as to realize dynamic adjustment and optimization of the feature distance, thereby realizing comprehensive calculation and evaluation of the feature distance. Finally, the formula subtracts , to reflect the overall characteristics of each defect, thereby reducing the noise and deviation of the characteristic distance. According to the dimensions of different defect indicators, the error of the characteristic distance between regions caused by the different indicator dimensions can be reduced, and the indicator weight value w can be adjusted. kThe indicators are actually adjusted to make the formula more reasonable and effective. Therefore, the formula can realize reasonable and efficient calculation of defect feature distance between image data regions, thereby providing a good data division basis for subsequent defect feature division.

[0074] Preferably, step S18 comprises the following steps:

[0075] Step S181: Defect dimension stratification is performed on the defect annotation data, thereby obtaining dimension stratification data, where the dimension stratification data includes defect type dimension data, defect location dimension data, and defect degree dimension data;

[0076] Step S182: performing type target analysis on the defect type dimension data, thereby obtaining type target data;

[0077] Step S183: performing position target analysis on the defect position dimension data to obtain position target data;

[0078] Step S184: performing degree target analysis on the defect degree dimension data, thereby obtaining degree target data;

[0079] Step S185: performing multidimensional fuzzy set calculation on the type target data, the position target data and the degree target data, thereby obtaining multidimensional similarity data;

[0080] Step S186: Based on the multi-dimensional similarity data and using a feature fusion method, target feature fusion is performed on the type target data, the position target data, and the degree target data, thereby obtaining target feature data;

[0081] Step S187: Based on the target feature data and using the region growing method, the two-level divided data set is divided into three levels, thereby obtaining a three-level divided data set.

[0082] As an embodiment of the present invention, refer to Figure 3 As shown, Figure 2 FIG. 1 is a schematic diagram of a detailed step flow chart of step S18 in FIG. 1 . In this embodiment, step S18 includes the following steps:

[0083] Step S181: Defect dimension stratification is performed on the defect annotation data, thereby obtaining dimension stratification data, where the dimension stratification data includes defect type dimension data, defect location dimension data, and defect degree dimension data;

[0084] The embodiment of the present invention uses a multi-layer perceptron to classify each defect area in the defect annotation data, and divides it into several subcategories according to its type, position and degree in the image, thereby forming dimensional hierarchical data. The dimensional hierarchical data includes defect type dimensional data, defect position dimensional data and defect degree dimensional data. The defect type dimensional data indicates the type of each defect area, such as point, line, surface, etc. The defect position dimensional data indicates the position of each defect area in the image, such as upper left, lower right, center, etc. The defect degree dimensional data indicates the degree of each defect area, such as slight, medium, severe, etc.

[0085] Step S182: performing type target analysis on the defect type dimension data, thereby obtaining type target data;

[0086] The embodiment of the present invention evaluates each subcategory in the defect type dimension data and divides it into two categories, namely, target class and non-target class, according to the degree of difference between it and the normal image, thereby forming type target data. The target class represents a subcategory that is significantly different from the normal image, and the non-target class represents a subcategory that is not significantly different from the normal image.

[0087] Step S183: performing position target analysis on the defect position dimension data to obtain position target data;

[0088] The embodiment of the present invention evaluates each subcategory in the defect location dimension data and divides it into two categories, namely, target category and non-target category, according to its importance to the normal image, thereby forming location target data. The target category represents the subcategory that has an important impact on the normal image, and the non-target category represents the subcategory that has no important impact on the normal image.

[0089] Step S184: performing degree target analysis on the defect degree dimension data, thereby obtaining degree target data;

[0090] The embodiment of the present invention evaluates each subcategory in the defect degree dimension data and divides it into two categories, namely, target class and non-target class, according to the degree of similarity with the normal image, thereby forming degree target data. The target class represents a subcategory with a lower similarity with the normal image, and the non-target class represents a subcategory with a higher similarity with the normal image.

[0091] Step S185: performing multidimensional fuzzy set calculation on the type target data, the position target data and the degree target data, thereby obtaining multidimensional similarity data;

[0092] The embodiment of the present invention calculates each subcategory in the type target data, position target data and degree target data by using multidimensional fuzzy set theory to obtain its membership value in three dimensions, and then converts it into a similarity value according to the size of the membership value, thereby forming multidimensional similarity data. The multidimensional similarity data represents the similarity between each subcategory and the normal image in the three dimensions of type, position and degree.

[0093] Step S186: Based on the multi-dimensional similarity data and using a feature fusion method, target feature fusion is performed on the type target data, the position target data, and the degree target data, thereby obtaining target feature data;

[0094] The embodiment of the present invention fuses each subcategory in the multidimensional similarity data by using the feature fusion method, and merges them into a single feature value according to the weighted average of their similarity values ​​in three dimensions, thereby forming target feature data. Among them, the feature fusion method can adjust the weight coefficient according to different industrial scenarios and tasks to achieve the best fusion effect.

[0095] Step S187: Based on the target feature data and using the region growing method, the two-level divided data set is divided into three levels, thereby obtaining a three-level divided data set.

[0096] The embodiment of the present invention segments each defect type regional defect in the secondary divided data set by using the region growing method, and further refines it into several sub-regions according to its similarity with surrounding pixels, thereby forming a three-level divided data set.

[0097] The present invention first performs defect dimension stratification on the defect annotation data to obtain dimension stratification data. The dimension stratification data includes defect type dimension data, defect location dimension data and defect degree dimension data. The data can divide the defect annotation data according to different dimensions, providing a multi-angle perspective for subsequent target analysis. A type target analysis is performed on the defect type dimension data to obtain type target data. The data can identify and classify the type of each defect in the defect type dimension data, providing type information for subsequent target feature fusion. A position target analysis is performed on the defect location dimension data to obtain position target data. The data can locate and measure the position of each defect in the defect location dimension data, providing position information for subsequent target feature fusion. A degree target analysis is performed on the defect degree dimension data to obtain degree target data. The data can evaluate and quantify the degree of each defect in the defect degree dimension data, providing degree information for subsequent target feature fusion. Multidimensional fuzzy set calculation is performed on the type target data, position target data and degree target data to obtain multidimensional similarity data. This data can use the method of multidimensional fuzzy set calculation to calculate the similarity between different defects in type, position and degree, and provide similarity information for subsequent target feature fusion. Based on multidimensional similarity data and using feature fusion method, target feature fusion is performed on type target data, position target data and degree target data to obtain target feature data. This data can use the technology of feature fusion method to integrate and optimize target data of different dimensions to obtain target feature data with higher accuracy and better effect, providing a powerful tool for subsequent regional division. Based on target feature data and using regional growing method, the secondary division data set is divided into three levels to obtain a three-level division data set. This data set can use the technology of regional growing method to further refine and optimize each region in the secondary division data set, eliminate possible noise and errors, and provide more accurate and complete data for subsequent regional repair.

[0098] Preferably, step S2 comprises the following steps:

[0099] Step S21: performing vector compression on the three-level divided data set, thereby obtaining a vector compression set;

[0100] The embodiment of the present invention extracts features from each data set in the three-level divided data set to obtain a feature vector for each data block, and based on the previously obtained principal component feature vector, the feature vector of each data block is reduced to a low-dimensional feature value according to the importance and difference between the principal component feature vector and the feature vector of each newly obtained data block, thereby forming a vector compression set. The vector compression set represents the feature representation of each sub-region in a low-dimensional space;.

[0101] Step S22: segmenting the vector compression set into segments, thereby obtaining a segment data set;

[0102] The embodiment of the present invention clusters each eigenvalue in the vector compression set and divides it into several categories according to its distance and similarity in the low-dimensional space, thereby forming a fragment data set, wherein the fragment data set represents a set of sub-regions contained in each category, that is, a fragment.

[0103] Step S23: extracting topological structure features from the fragment data set, thereby obtaining a topological structure feature set;

[0104] The embodiment of the present invention analyzes each fragment in the fragment data set to obtain the topological structure information of each fragment, such as the number of nodes, the number of edges, the connectivity, the number of rings, etc., and then combines this information into a feature vector to form a topological structure feature set. The topological structure feature set represents the topological structure features of each fragment.

[0105] Step S24: extracting shape features from the segment data set to obtain a shape feature set;

[0106] The embodiment of the present invention analyzes each segment in the segment data set to obtain shape information of each segment, such as area, perimeter, roundness, direction, etc., and then combines this information into a feature vector to form a shape feature set. The shape feature set represents the shape features of each segment.

[0107] Step S25: performing segment aggregation on the topological structure feature set and the shape feature set to obtain aggregated segment features.

[0108] The embodiment of the present invention forms an aggregated fragment feature by splicing and fusing each fragment in the topological structure feature set and the shape feature set, and merging them into a single feature value according to the weighted average of their feature values ​​in two dimensions. The aggregated fragment feature represents the comprehensive features of each fragment in the two dimensions of topological structure and shape.

[0109] The present invention performs vector compression on the three-level partitioned data set to obtain a vector compression set, which can represent each region in the three-level partitioned data set with a vector, reducing the dimension and size of the data and providing simplified data for subsequent segmentation. The vector compression set is segmented to obtain a segment data set, which can cut each region in the vector compression set according to certain rules to obtain multiple segments with certain lengths and shapes, providing basic units for subsequent feature extraction. The segment data set is subjected to topological structure feature extraction to obtain a topological structure feature set, which can perform topological structure analysis on each segment in the segment data set to obtain information on its connectivity, connectivity, adjacency, etc., and provide structural information for subsequent segment aggregation. The segment data set is subjected to shape feature extraction to obtain a shape feature set, which can perform shape feature analysis on each segment in the segment data set to obtain information on its length, width, curvature, angle, etc., and provide shape information for subsequent segment aggregation. The topological structure feature set and the shape feature set are fragmented to obtain the aggregated fragment feature. This feature can use the information of topological structure features and shape features to meaningfully combine similar or adjacent fragments to obtain an aggregated fragment feature with higher level and richer content, providing high-quality input for subsequent image generation.

[0110] Preferably, step S3 comprises the following steps:

[0111] Step S31: performing multi-range information capture on the aggregated segment features, thereby obtaining multi-range information data;

[0112] The embodiment of the present invention extracts information from each eigenvalue in the aggregated fragment features to obtain information representations of each eigenvalue at different scales and angles, and then combines these information representations into a multi-dimensional information vector to form multi-range information data. The multi-range information data represents the information expression of each eigenvalue in multiple ranges.

[0113] Step S32: Calculate the defect distribution probability of the multi-range information data to obtain defect distribution probability data;

[0114] The embodiment of the present invention performs probability calculation on each information vector in the multi-range information data to obtain the defect distribution probability value of each information vector in the image, and then combines these probability values ​​into a one-dimensional probability vector to form defect distribution probability data. The defect distribution probability data represents the defect distribution possibility of each feature value in the image.

[0115] Step S33: extracting local features from the three-level divided data set, thereby obtaining local feature data;

[0116] The embodiment of the present invention extracts features from each data block in the three-level divided data set to obtain local texture features of each data block, and then combines these features into a one-dimensional feature vector to form local feature data, wherein the local feature data represents the local texture features of each data block.

[0117] Step S34: extracting global features from the three-level divided data set, thereby obtaining global feature data;

[0118] The embodiment of the present invention extracts features from the original complete image formed by combining each data block in the three-level divided data set to obtain the global texture features of each image, and then combines these features into a one-dimensional feature vector to form global feature data. The global feature data represents the global texture features of each image.

[0119] Step S35: globally integrating the local feature data to obtain globally integrated data;

[0120] The embodiment of the present invention encodes and decodes each feature vector in the local feature data to obtain a reconstructed representation of each feature vector, and then splices these reconstructed representations with the original representation to form global integrated data. The global integrated data represents the original and reconstructed expressions of each sub-region.

[0121] Step S36: performing feature discrimination on the global integrated data and the global feature data, thereby obtaining feature discrimination data;

[0122] The embodiment of the present invention performs feature discrimination on each feature vector in the global integrated data and the global feature data, and divides them into two categories, namely, target class and non-target class, according to their distinction from normal images. Feature discrimination is performed on the global integrated data and the global feature data based on the target class data to obtain identical, similar, and different data features, thereby forming feature discrimination data.

[0123] Step S37: Based on the defect distribution probability data and feature discrimination data and using the generative adversarial network algorithm, a multi-model set is constructed for the aggregated fragment features to obtain a detailed extended model set.

[0124] The embodiment of the present invention uses a conditional adversarial generative network to train each feature value in the aggregated fragment feature, and obtains a model set composed of multiple generators and discriminators. Each generator can generate a corresponding high-resolution sub-region based on the input feature value and defect distribution probability value, and each discriminator can determine whether the sub-region is real based on the input sub-region and feature discrimination value. By continuously optimizing the parameters of the generator and the discriminator, the generator can generate more realistic sub-regions, thereby obtaining a detailed expansion model set.

[0125] The present invention obtains multi-range information data by performing multi-range information capture on the aggregated fragment features. The data can use the technology of multi-range information capture to obtain information related to the aggregated fragment features from different data sources and angles, providing rich data for the subsequent defect distribution probability calculation. The defect distribution probability is calculated on the multi-range information data to obtain defect distribution probability data. The data can use the method of defect distribution probability calculation to calculate the probability of occurrence of defects of different types and degrees in different regions and locations, providing probability information for the subsequent construction of multiple model sets. The three-level divided data set is subjected to local feature extraction to obtain local feature data. The data can use the local feature extraction technology to extract the local features of each region in the three-level divided data set, such as color, texture, brightness, etc., and provide local information for subsequent feature discrimination. The three-level divided data set is subjected to global feature extraction to obtain global feature data. The data can use the global feature extraction technology to extract the global features of the entire image in the three-level divided data set, such as contour, shape, symmetry, etc., and provide global information for subsequent feature discrimination. The local feature data is globally integrated to obtain global integrated data. This data can use the global integration technology to compare and match the local features of each region in the local feature data with the global features of the entire image, and obtain a global integrated data with higher consistency and better effect, which provides integrated information for subsequent feature discrimination. Feature discrimination is performed on the global integrated data and the global feature data to obtain feature discrimination data. This data can use the feature discrimination technology to analyze and evaluate the features of each region in the global integrated data and the global feature data to obtain information such as whether there are defects and the type and degree of defects, which provides discrimination information for the subsequent construction of multiple model sets. Based on the defect distribution probability data and feature discrimination data and using the adversarial generative network algorithm to construct a multi-model set for the aggregated fragment features, a detailed extension model set is obtained. This model set can use the powerful learning ability and diversity of the adversarial generative network algorithm to generate multiple image models with different styles and details, which provides a variety of options and references for subsequent image restoration.

[0126] Preferably, step S37 includes the following steps:

[0127] Step S371: Based on the defect distribution probability data, the attention mechanism algorithm is used to perform information attention fusion on the aggregated fragment features, thereby obtaining information fusion data;

[0128] The embodiment of the present invention uses the self-attention mechanism to perform information fusion on each feature value in the aggregated fragment features, obtains the information weight of each feature value in different ranges, and then weighted averages them according to the size of the information weight, thereby forming information fusion data. The information fusion data represents the information fusion representation of each feature value in multiple ranges.

[0129] Step S372: generating a generator strategy for the information fusion data, thereby obtaining a generator construction strategy;

[0130] The embodiment of the present invention generates a strategy for each information fusion representation in the information fusion data to obtain a generator construction strategy corresponding to each information fusion representation, thereby forming a generator construction strategy set. The generator construction strategy represents the parameter selection and optimization method of each information fusion representation when generating a high-resolution sub-region.

[0131] Step S373: performing dual discrimination on the aggregated segment features based on the feature discrimination data and using a conditional generative adversarial network algorithm, thereby obtaining dual discrimination data;

[0132] In the embodiment of the present invention, a conditional generative adversarial network is used to perform dual discrimination on each feature value in the aggregated fragment feature, and the authenticity judgment result of each feature value under two different discriminators is obtained, thereby forming dual discrimination data. The dual discrimination data represents the authenticity probability value of each feature value under two different discriminators.

[0133] Step S374: Predicting the probability of authenticity of the dual-discrimination data, thereby obtaining the prediction data of the probability of authenticity;

[0134] The embodiment of the present invention predicts each authenticity probability value in the dual discriminant data by using a logistic regression method to obtain the authenticity evaluation result of each authenticity probability value in the image, thereby forming authenticity probability prediction data. The authenticity probability prediction data represents the authenticity score of each authenticity probability value in the image.

[0135] Step S375: generating a discriminator strategy for the true and false probability prediction data, thereby obtaining a discriminator construction strategy;

[0136] The embodiment of the present invention generates a strategy for each authenticity score in the authenticity probability prediction data to obtain a discriminator construction strategy corresponding to each authenticity score, thereby forming a discriminator construction strategy set. The discriminator construction strategy represents the parameter selection and optimization method for each authenticity score when discriminating a sub-region.

[0137] Step S376: collaboratively generate loss functions for the generator construction strategy and the discriminator construction strategy, thereby obtaining a collaborative loss function;

[0138] The embodiment of the present invention generates a loss function for each strategy in the generator construction strategy set and the discriminator construction strategy set by using collaborative filtering method to obtain a collaborative loss function corresponding to each strategy, thereby forming a collaborative loss function set. The collaborative loss function represents the objective function of each strategy during collaborative optimization.

[0139] Step S377: Perform adversarial loss function generation on the generator construction strategy and the discriminator construction strategy to obtain an adversarial loss function;

[0140] The embodiment of the present invention generates a loss function for each strategy in the generator construction strategy set and the discriminator construction strategy set by using the adversarial learning method to obtain an adversarial loss function corresponding to each strategy, thereby forming an adversarial loss function set. The adversarial loss function represents the objective function of each strategy during adversarial optimization.

[0141] Step S378: using the loss function weight calculation formula to perform weight distribution calculation on the collaborative loss function and the adversarial loss function, thereby obtaining the overall loss function;

[0142] The embodiment of the present invention uses the loss function weight calculation formula to perform weight distribution calculation on each loss function in the collaborative loss function set and the adversarial loss function set to obtain the weight value of each loss function, and then weighted averages them according to the size of the weight value to form an overall loss function. The overall loss function represents the objective function of each feature value during overall optimization.

[0143] Among them, the loss function weight calculation formula is as follows:

[0144] ;

[0145] In the formula, L is the loss function weight value of the collaborative loss function, λ is the overall loss adjustment coefficient, and L g is the loss weight value of the generator, b is the loss adjustment coefficient of the discriminator, γ is the global loss adjustment coefficient, δ is the regional loss adjustment coefficient, L d is the loss weight value of the discriminator, is the global loss weight value of the generator, is the regional loss weight value of the generator, e is a natural constant, a is the sensitivity parameter of the weight allocation, l g is the collaborative loss function, R g is the weight reduction coefficient of the concordant loss function to the adversarial loss function;

[0146] The present invention constructs a loss function weight calculation formula. The L obtained by this formula is used to measure the synergy between the generator and the discriminator, as well as the synergy between the global and regional. λ is used to adjust the size of the overall loss function to adapt to different data sets and tasks. The larger λ is, the larger the overall loss function is, the more sensitive the model training is, and L g It is used to measure the generation ability of the generator and the difference between the generated results and the real data. g The smaller it is, the more realistic the generator can generate, and the more natural the detail extension is. b is used to adjust the loss weight value of the discriminator to balance the adversarial relationship between the generator and the discriminator. The larger b is, the larger the loss weight value of the discriminator is, the more the discriminator can distinguish between true and false data, and the more distinct the detail extension is. γ is used to adjust the global loss weight value of the generator to balance the synergistic relationship between the global and regional. The larger γ is, the larger the global loss weight value of the generator is, the higher the global consistency between the generated results and the real data is, and the more complete the detail extension is. δ is used to adjust the regional loss weight value of the generator to balance the synergistic relationship between the global and regional. The larger δ is, the larger the regional loss weight value of the generator is, the more similar the regional details between the generated results and the real data are, and the more accurate the detail extension is. L d It is used to measure the discriminative ability of the discriminator and the difference between the discriminative result and the true label. d The smaller it is, the more accurately the discriminator can distinguish true and false data, and the more effective the detail expansion is. Used to measure the global consistency between the generated results and the real data. The smaller it is, the smaller the global difference between the generated results and the real data is, and the more complete the detail expansion is. Used to measure the similarity of regional details between the generated results and the real data. The smaller the a is, the smaller the regional difference between the generated result and the real data is, and the more accurate the detail expansion is. a is used to control the shape of the weight distribution curve of the loss function. The larger a is, the steeper the curve is and the more sensitive the weight distribution is. g It is used to measure the synergy between the generator and the discriminator, as well as the synergy between the global and regional. g The smaller it is, the better the synergy effect is and the finer the detail expansion is. g Used to adjust the impact of the concordant loss function on the adversarial loss function. g The larger it is, the smaller the influence of the concordant loss function on the adversarial loss function, the stronger the adversarial relationship, and the more distinct the detail expansion. This part is composed of the loss weight values ​​of the generator and the discriminator, as well as the global and regional loss weight values ​​of the generator, which are weighted by an exponential function. The size of the collaborative loss function can be dynamically adjusted according to different parameter values ​​to balance the collaborative effect between the generator and the discriminator, and the collaborative effect between the global and regional. This part is composed of a constant 1 and an exponential function, which can prevent the loss function weight value of the collaborative loss function from being zero or negative, thereby ensuring the stability of model training. Specifically, when the loss difference between the generator and the discriminator is large, or the loss difference between the global and the regional is large, the value of this part will be close to 1, so that the loss function weight value of the collaborative loss function is close to the value of the numerator. When the loss difference between the generator and the discriminator is small, or the loss difference between the global and the regional is small, the value of this part will be much greater than 1, so that the loss function weight value of the collaborative loss function is much smaller than the value of the numerator. Therefore, this formula can obtain a comprehensive and reasonable loss function weight distribution ratio, and then obtain the loss function weight value of the collaborative loss function, so as to infer the loss function weight value of the adversarial loss function, and provide a functional basis that is more in line with the actual needs of data processing for the subsequent model construction of the generative adversarial network.

[0147] Step S379: Use the generative adversarial network algorithm to model the overall loss function, generator construction strategy, and discriminator construction strategy to obtain a detailed expansion model.

[0148] The embodiment of the present invention uses a generative adversarial network to construct a model of the overall loss function, the generator construction strategy, and the discriminator construction strategy, thereby obtaining a model set consisting of multiple generators and discriminators. Each generator can generate a corresponding high-resolution sub-region based on the input feature value and defect distribution probability value, and each discriminator can determine whether the sub-region is real based on the input sub-region and the true and false probability prediction value. By continuously optimizing the parameters of the generator and the discriminator, the generator can generate more realistic sub-regions, thereby obtaining a detailed expansion model set.

[0149] The present invention obtains information fusion data by performing multi-range information capture on aggregated fragment features. This data can use the technology of the attention mechanism algorithm to weight and fuse each information in the multi-range information data to obtain information fusion data with higher attention and better effect, providing fusion information for subsequent generator strategy generation. Generator strategy generation is performed on the information fusion data to obtain a generator construction strategy. This strategy can use the generator strategy generation method to generate a generator construction strategy suitable for aggregated fragment features according to different information and probabilities in the information fusion data, providing strategy information for subsequent multi-model set construction. Double discrimination is performed on the aggregated fragment features to obtain double discrimination data. This data can use the technology of the conditional generative adversarial network algorithm to input the aggregated fragment features into two different discriminators respectively, and obtain two discrimination results with different angles and standards, providing dual information for subsequent true and false probability prediction. The authenticity probability of the dual-discrimination data is predicted to obtain authenticity probability prediction data. This data can use the authenticity probability prediction method to calculate the authenticity probability of the aggregated fragment features in the two discriminators according to the two discrimination results in the dual-discrimination data, providing probability information for the subsequent discriminator strategy generation. The discriminator strategy is generated for the authenticity probability prediction data to obtain the discriminator construction strategy. This strategy can use the discriminator strategy generation method to generate a discriminator construction strategy suitable for the aggregated fragment features according to the different probabilities and standards in the authenticity probability prediction data, providing strategy information for the subsequent loss function collaborative generation. The loss function is collaboratively generated for the generator construction strategy and the discriminator construction strategy to obtain the collaborative loss function. This function can use the loss function collaborative generation method to generate a collaborative loss function that can coordinate the relationship between the two and balance the effects between the two according to the different parameters and goals in the generator construction strategy and the discriminator construction strategy, providing collaborative information for the subsequent loss function weight calculation. The generator construction strategy and the discriminator construction strategy are subjected to adversarial loss function generation to obtain the adversarial loss function. This function can use the adversarial loss function generation method to generate an adversarial loss function that can enhance the competition between the two and improve the effect between the two according to the different parameters and goals in the generator construction strategy and the discriminator construction strategy, and provide adversarial information for the subsequent loss function weight calculation. The weight distribution calculation of the collaborative loss function and the adversarial loss function is performed using the loss function weight calculation formula to obtain the overall loss function. This function can use the loss function weight calculation formula technology to calculate an overall loss function that can combine the advantages and disadvantages of the collaborative loss function and the adversarial loss function according to the different information and effects in the collaborative loss function and the adversarial loss function, and provide optimization information for the subsequent model construction.The generative adversarial network algorithm is used to model the overall loss function, generator construction strategy and discriminator construction strategy to obtain a detail extension model. This model can use the technology of the generative adversarial network algorithm to build a detail extension model that can generate high quality and high effect according to different parameters and goals in the overall loss function, generator construction strategy and discriminator construction strategy, providing a powerful tool for subsequent image restoration.

[0150] Preferably, step S4 comprises the following steps:

[0151] Step S41: performing adversarial iterative analysis on the detail expansion model set, thereby obtaining iterative analysis data;

[0152] The embodiment of the present invention performs iterative analysis on each generator and discriminator in the detail expansion model set to obtain indicators such as loss value, accuracy, stability, etc. of each generator and discriminator at different iteration times, and then combines these indicators into a multi-dimensional analysis vector to form iterative analysis data. The iterative analysis data represents the performance of each generator and discriminator at different iteration times.

[0153] Step S42: using the detail extension model set to perform feature detail extension on the sample image data, thereby obtaining a detail extension image set;

[0154] The embodiment of the present invention uses a detail expansion model set to expand the feature details of each image in the sample image data, obtains high-resolution sub-regions of each image under different generators, and then splices these sub-regions into a high-resolution image, thereby forming a detail expansion image set. The detail expansion image set represents a high-resolution image of each image under different generators.

[0155] Step S43: performing adversarial gradient estimation on the detail extended image set, thereby obtaining a gradient estimation data set;

[0156] The embodiment of the present invention performs gradient estimation on each high-resolution image in the detail expansion image set to obtain the gradient value of each high-resolution image under different discriminators, and then combines these gradient values ​​into a one-dimensional gradient vector to form a gradient estimation data set. The gradient estimation data set represents the gradient value of each high-resolution image under different discriminators.

[0157] Step S44: performing sub-gradient optimization on the detail extension model set based on the iterative analysis data and the gradient estimation data set, thereby obtaining a detail optimization model set;

[0158] The embodiment of the present invention optimizes each analysis vector and gradient vector in the iterative analysis data and the gradient estimation data set to obtain the parameter update value of each generator and discriminator under different sub-gradients, and then updates the parameters of each generator and discriminator according to the parameter update value, thereby forming a detailed optimization model set. The detailed optimization model set represents the model after the parameter update of each generator and discriminator under different sub-gradients.

[0159] Step S45: performing multi-domain labeling on the detail extended image set, thereby obtaining multi-domain label data;

[0160] The embodiment of the present invention labels each high-resolution image in the detail extension image set and gives its label in different industrial fields and tasks, thereby forming multi-field label data. The multi-field labeling tool can provide different labeling modes and functions according to different industrial fields and tasks to improve the efficiency and accuracy of labeling.

[0161] Step S46: Based on the multi-domain label data and using the transfer learning method, multi-transfer learning is performed on the detail optimization model set to obtain the image feature refinement model.

[0162] The embodiment of the present invention uses the transfer learning method to learn each label in the multi-domain label data to obtain the transfer model corresponding to each label, and then fuses these transfer models with the generator and discriminator in the detail optimization model set to form an image feature refinement model. The image feature refinement model represents the adaptability model of each generator and discriminator in different industrial fields and tasks.

[0163] The present invention performs adversarial iterative analysis on the detail extension model set, thereby obtaining iterative analysis data, which can use the adversarial iterative analysis technology to analyze the generation effect and discrimination effect of each model in the detail extension model set, obtain information on its number of iterations, iteration error, iteration convergence, etc., and provide analysis information for subsequent sub-gradient optimization. The detail extension model set is used to perform feature detail expansion on the sample image data, thereby obtaining a detail extension image set, which can use the generation ability of each model in the detail extension model set to perform feature detail expansion on each region in the sample image data, obtain multiple images with higher clarity and richer details, and provide extended images for subsequent adversarial gradient estimation. The detail extension image set is subjected to adversarial gradient estimation, thereby obtaining a gradient estimation data set, which can use the adversarial gradient estimation method to calculate the gradient value and direction of each region according to the difference between each image in the detail extension image set and the original image, and provide gradient information for subsequent sub-gradient optimization. Based on the iterative analysis data and the gradient estimation data set, the detail extension model set is sub-gradient optimized to obtain the detail optimization model set. This model set can use the sub-gradient optimization technology to adjust and optimize the parameters of each model in the detail extension model set according to the different information and indicators in the iterative analysis data and the gradient estimation data set, and obtain a detail optimization model set with lower error and faster convergence, which provides an optimization model for subsequent multi-domain labeling. Multi-domain labeling is performed on the detail extension image set to obtain multi-domain label data. This data can use the multi-domain labeling technology to label each image according to the different domains and categories to which each image in the detail extension image set belongs, and provide label information for subsequent multi-transfer learning. Based on the multi-domain label data and using the transfer learning method, multi-transfer learning is performed on the detail optimization model set to obtain the image feature refinement model. This model can use the transfer learning method to fuse and optimize the various models in the detail optimization model set according to the different labels and categories in the multi-domain label data, and obtain an image feature refinement model with higher accuracy and better effect, which provides a powerful tool for subsequent image preprocessing.

[0164] Preferably, step S46 includes the following steps:

[0165] Step S461: performing domain difference analysis on multi-domain label data to obtain multi-domain difference data;

[0166] The embodiment of the present invention analyzes each label in the multi-field label data to obtain the degree of difference of each label in different industrial fields and tasks, and then combines these degree of difference into a one-dimensional difference vector to form multi-field difference data. The multi-field difference data represents the degree of difference of each label in different industrial fields and tasks.

[0167] Step S462: extracting the hierarchical structure of the domain difference data, thereby obtaining hierarchical structure data;

[0168] The embodiment of the present invention clusters each difference vector in the domain difference data by using a hierarchical clustering method, and divides them into several levels according to their distance and similarity in one-dimensional space, thereby forming hierarchical structure data. The hierarchical structure data represents the hierarchical division of each difference vector in one-dimensional space.

[0169] Step S463: performing multi-layer data splitting on the multi-domain difference data based on the hierarchical structure data and using a multi-task learning method, thereby obtaining a multi-layer split data set;

[0170] The embodiment of the present invention uses a multi-task learning method to learn each level in the hierarchical data to obtain a multi-task model corresponding to each level, and then splits these multi-task models with each difference vector in the multi-domain difference data to form a multi-layer split data set. The multi-layer split data set represents the sub-task division of each difference vector at different levels.

[0171] Step S464: quantizing the feature matrix of the multi-layer split data set to obtain a feature matrix set;

[0172] The embodiment of the present invention quantizes each subtask in the multi-layer split data set by using a feature matrix quantization method to obtain a feature matrix of each subtask, and then combines these feature matrices into a two-dimensional feature matrix set, wherein the feature matrix set represents a two-dimensional feature representation of each subtask.

[0173] Step S465: performing migration trend analysis on the feature matrix set to obtain migration trend data;

[0174] The embodiment of the present invention analyzes each feature matrix in the feature matrix set to obtain the migration trend of each feature matrix in different industrial fields and tasks, and then combines these migration trends into a one-dimensional migration vector to form migration trend data. The migration trend data represents the migration possibility of each feature matrix in different industrial fields and tasks.

[0175] Step S466: Based on the migration trend data and using the migration learning method, multi-transfer learning is performed on the detail optimization model set to obtain an image feature refinement model.

[0176] The embodiment of the present invention uses the transfer learning method to learn each transfer vector in the transfer trend data to obtain the transfer model corresponding to each transfer vector, and then fuses these transfer models with the generator and discriminator in the detail optimization model set to form an image feature refinement model. The image feature refinement model represents the adaptability model of each generator and discriminator in different industrial fields and tasks.

[0177] The present invention obtains multi-domain difference data by performing domain difference analysis on multi-domain label data. The data can use the technology of domain difference analysis to analyze the difference and similarity between each domain and category in the multi-domain label data, and provide difference information for subsequent hierarchical structure extraction. Perform hierarchical structure extraction on the domain difference data to obtain hierarchical structure data. The data can use the technology of hierarchical structure extraction to extract the hierarchy and relationship in the multi-domain label data according to the different differences and similarities in the domain difference data, and provide structural information for subsequent multi-layer data splitting. Based on the hierarchical structure data and using the multi-task learning method, multi-domain difference data is split and distributed to obtain a multi-layer split data set. The data set can use the technology of the multi-task learning method to split and distribute the multi-domain difference data according to the different levels and relationships in the hierarchical structure data, and obtain a plurality of sub-data sets with different tasks and goals, and provide a sub-data set for subsequent feature matrix quantization. Perform feature matrix quantization on the multi-layer split data set to obtain a feature matrix set. The feature matrix set can use the method of feature matrix quantization to convert it into a feature matrix according to the characteristics and attributes of each sub-data set in the multi-layer split data set, and provide matrix information for subsequent migration trend analysis. Perform migration trend analysis on the feature matrix set to obtain migration trend data. This data can use the migration trend analysis technology to analyze the migration trends and directions between different sub-data sets based on the correlation and similarity between each feature matrix in the feature matrix set, providing trend information for subsequent multi-transfer learning. Based on the migration trend data and using the transfer learning method, multi-transfer learning is performed on the detail optimization model set to obtain an image feature refinement model. This model can use the transfer learning method to fuse and optimize the various models in the detail optimization model set according to the different trends and directions in the migration trend data, and obtain an image feature refinement model with higher accuracy and better results, providing a powerful tool for subsequent image preprocessing.

[0178] Preferably, the present invention further provides a data preprocessing system based on image feature refinement, which is used to execute the data preprocessing method based on image feature refinement as described above. The data preprocessing system based on image feature refinement includes:

[0179] A three-level division module is used to obtain sample image data and divide the sample image data into three levels, thereby obtaining a three-level division data set;

[0180] The fragment aggregation module is used to aggregate the fragments of the three-level partitioned data set to obtain the aggregated fragment features;

[0181] A model set construction module is used to construct a multi-model set based on the three-level partitioned data set and aggregated fragment features using a generative adversarial network algorithm, thereby obtaining a detailed extended model set;

[0182] A multi-transfer learning module is used to perform multi-transfer learning on the detail expansion model set to obtain an image feature refinement model;

[0183] The data preprocessing module is used to obtain the original image data and preprocess the original image data using the image feature refinement model to obtain a preprocessed image data set.

[0184] In summary, the present invention provides a data preprocessing system based on image feature refinement, which is composed of a three-level division module, a fragment aggregation module, a model set construction module, a multi-transfer learning module and a data preprocessing module. It can realize any data preprocessing method based on image feature refinement described in the present invention. The internal structure of the system cooperates with each other to extract features of different levels of the image, and then enhance and express the local area of ​​the combined image, and then use the adversarial generative network algorithm to generate multiple different image refinement models, and then fuse and optimize each image model to obtain a high-precision and high-effect image feature refinement model to improve and enhance the original image data, thereby improving the processing efficiency of the data preprocessing system based on image feature refinement.

[0185] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0186] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A data preprocessing method based on image feature refinement, characterized in that: The following steps are involved: Step S1: obtaining sample image data, dividing the sample image data into three levels, thereby obtaining a three-level divided data set; Step S2: performing segment aggregation on the three-level divided data set to obtain aggregate segment features; wherein step S2 includes the following steps: Step S21: performing vector compression on the three-level divided data set, thereby obtaining a vector compression set; Step S22: segmenting the vector compression set into segments, thereby obtaining a segment data set; Step S23: extracting topological structure features from the fragment data set, thereby obtaining a topological structure feature set; Step S24: extracting shape features from the segment data set to obtain a shape feature set; Step S25: performing segment aggregation on the topological structure feature set and the shape feature set, thereby obtaining aggregate segment features; Step S3: Based on the three-level divided data set and using the adversarial generative network algorithm, a multi-model set is constructed for the aggregated fragment features to obtain a detailed extended model set; wherein step S3 includes the following steps: Step S31: performing multi-range information capture on the aggregated segment features, thereby obtaining multi-range information data; Step S32: Calculate the defect distribution probability of the multi-range information data to obtain defect distribution probability data; Step S33: extracting local features from the three-level divided data set, thereby obtaining local feature data; Step S34: extracting global features from the three-level divided data set, thereby obtaining global feature data; Step S35: globally integrating the local feature data to obtain globally integrated data; Step S36: performing feature discrimination on the global integrated data and the global feature data, thereby obtaining feature discrimination data; Step S37: constructing a multi-model set for the aggregated fragment features based on the defect distribution probability data and the feature discrimination data and using the adversarial generative network algorithm to obtain a detailed extended model set; wherein step S37 includes the following steps: Step S371: Based on the defect distribution probability data, the attention mechanism algorithm is used to perform information attention fusion on the aggregated fragment features, thereby obtaining information fusion data; Step S372: generating a generator strategy for the information fusion data, thereby obtaining a generator construction strategy; Step S373: performing dual discrimination on the aggregated segment features based on the feature discrimination data and using a conditional generative adversarial network algorithm, thereby obtaining dual discrimination data; Step S374: Predicting the probability of authenticity of the dual-discrimination data, thereby obtaining the prediction data of the probability of authenticity; Step S375: generating a discriminator strategy for the true and false probability prediction data, thereby obtaining a discriminator construction strategy; Step S376: collaboratively generate loss functions for the generator construction strategy and the discriminator construction strategy, thereby obtaining a collaborative loss function; Step S377: Perform adversarial loss function generation on the generator construction strategy and the discriminator construction strategy to obtain an adversarial loss function; Step S378: using the loss function weight calculation formula to perform weight distribution calculation on the collaborative loss function and the adversarial loss function, thereby obtaining the overall loss function; Among them, the loss function weight calculation formula is as follows: ； In the formula, L is the loss function weight value of the collaborative loss function, λ is the overall loss adjustment coefficient, and L g is the loss weight value of the generator, b is the loss adjustment coefficient of the discriminator, γ is the global loss adjustment coefficient, δ is the regional loss adjustment coefficient, L d is the loss weight value of the discriminator, is the global loss weight value of the generator, is the regional loss weight value of the generator, e is a natural constant, a is the sensitivity parameter of the weight allocation, l g is the collaborative loss function, R g is the weight reduction coefficient of the collaborative loss function to the adversarial loss function; Step S379: constructing a model of the overall loss function, the generator construction strategy, and the discriminator construction strategy using a generative adversarial network algorithm, thereby obtaining a detailed expansion model; Step S4: performing multi-transfer learning on the detail expansion model set to obtain an image feature refinement model; wherein step S4 includes the following steps: Step S41: performing adversarial iterative analysis on the detail expansion model set, thereby obtaining iterative analysis data; Step S42: using the detail extension model set to perform feature detail extension on the sample image data, thereby obtaining a detail extension image set; Step S43: performing adversarial gradient estimation on the detail extended image set, thereby obtaining a gradient estimation data set; Step S44: performing sub-gradient optimization on the detail extension model set based on the iterative analysis data and the gradient estimation data set, thereby obtaining a detail optimization model set; Step S45: performing multi-domain labeling on the detail extended image set, thereby obtaining multi-domain label data; Step S46: Based on the multi-domain label data and using the transfer learning method, multi-transfer learning is performed on the detail optimization model set to obtain the image feature refinement model; wherein step S46 includes the following steps: Step S461: performing domain difference analysis on multi-domain label data to obtain multi-domain difference data; Step S462: extracting the hierarchical structure of the multi-field difference data, thereby obtaining hierarchical structure data; Step S463: performing multi-layer data splitting on the multi-domain difference data based on the hierarchical structure data and using a multi-task learning method, thereby obtaining a multi-layer split data set; Step S464: quantizing the feature matrix of the multi-layer split data set to obtain a feature matrix set; Step S465: performing migration trend analysis on the feature matrix set to obtain migration trend data; Step S466: performing multi-transfer learning on the detail optimization model set based on the transfer trend data and using the transfer learning method, thereby obtaining an image feature refinement model; Step S5: Acquire original image data, and preprocess the original image data using an image feature refinement model, thereby obtaining a preprocessed image data set.

2. The data preprocessing method based on image feature refinement according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: acquiring sample image data; Step S12: performing binary conversion on the sample image data to obtain binary image data; Step S13: performing edge detection on the binary image data to obtain regional defect data; Step S14: using a clustering method to perform primary division on the regional defect data, thereby obtaining a primary division data set; Step S15: performing key feature mining on the first-level partitioned data set to obtain a key feature set, which includes an original feature set and a defect feature set; Step S16: performing a secondary division on the primary division data set based on the key feature set and using the defect feature division calculation formula, thereby obtaining a secondary division data set; Step S17: labeling the defect dimension of the secondary divided data set, thereby obtaining defect labeling data; Step S18: Based on the defect annotation data, the two-level divided data set is divided into three levels by using the region growing method, so as to obtain a three-level divided data set.

3. The data preprocessing method based on image feature refinement according to claim 2, characterized in that: The defect feature classification calculation formula in step S16 is specifically: ； Where D ij is the feature distance between the defect data of the i-th region and the defect data of the j-th region, N is the total number of original features, M is the total number of defect features, and w nm is the feature weight value between the nth original feature and the mth defect feature, f nm is the correlation coefficient between the nth original feature and the mth defect feature, F in is the degree of approximation between the average value of all original features in the ith region and the defect of the nth original feature, F jn is the degree of approximation between the average value of all original features in the jth region and the defect of the nth original feature, f nM is the correlation coefficient between the nth original feature and the average value of all defect features, α is the correlation coefficient between the nth original feature and the defect area, β is the correlation coefficient between the nth original feature and the defect shape feature, ε is the correlation coefficient between the nth original feature and the defect degree, z is the correlation coefficient between the nth original feature and the defect feature position, K is the total number of dimensions of the defect index, w k is the indicator weight value of the kth indicator, F k is the index value of the kth index.

4. The data preprocessing method based on image feature refinement according to claim 2, characterized in that: The specific steps of step S18 are: Step S181: Defect dimension stratification is performed on the defect annotation data, thereby obtaining dimension stratification data, where the dimension stratification data includes defect type dimension data, defect location dimension data, and defect degree dimension data; Step S182: performing type target analysis on the defect type dimension data, thereby obtaining type target data; Step S183: performing position target analysis on the defect position dimension data to obtain position target data; Step S184: performing degree target analysis on the defect degree dimension data, thereby obtaining degree target data; Step S185: performing multidimensional fuzzy set calculation on the type target data, the position target data and the degree target data, thereby obtaining multidimensional similarity data; Step S186: Based on the multi-dimensional similarity data and using a feature fusion method, target feature fusion is performed on the type target data, the position target data, and the degree target data, thereby obtaining target feature data; Step S187: Based on the target feature data and using the region growing method, the two-level divided data set is divided into three levels, thereby obtaining a three-level divided data set.

5. A data preprocessing system based on image feature refinement, characterized in that: Used to execute the data preprocessing method based on image feature refinement as claimed in claim 1, the data preprocessing system based on image feature refinement comprises: A three-level division module is used to obtain sample image data and divide the sample image data into three levels, thereby obtaining a three-level division data set; The fragment aggregation module is used to perform vector compression on the three-level divided data set to obtain a vector compression set; perform fragment segmentation on the vector compression set to obtain a fragment data set; perform topological structure feature extraction on the fragment data set to obtain a topological structure feature set; perform shape feature extraction on the fragment data set to obtain a shape feature set; perform fragment aggregation on the topological structure feature set and the shape feature set to obtain aggregated fragment features; The model set construction module is used to capture multi-range information of aggregated fragment features, thereby obtaining multi-range information data; calculate the defect distribution probability of multi-range information data, thereby obtaining defect distribution probability data; extract local features of the three-level divided data set, thereby obtaining local feature data; extract global features of the three-level divided data set, thereby obtaining global feature data; globally integrate the local feature data, thereby obtaining global integration data; perform feature discrimination on the global integration data and the global feature data, thereby obtaining feature discrimination data; based on the defect distribution probability data and using the attention mechanism algorithm, perform information attention fusion on the aggregated fragment features, thereby obtaining information fusion data; perform generator strategy generation on the information fusion data, thereby obtaining Generator construction strategy; based on feature discrimination data and using the conditional generative adversarial network algorithm to perform dual discrimination on the aggregated fragment features, thereby obtaining dual discrimination data; predict the authenticity probability of the dual discrimination data, thereby obtaining authenticity probability prediction data; generate the discriminator strategy for the authenticity probability prediction data, thereby obtaining the discriminator construction strategy; perform loss function collaborative generation on the generator construction strategy and the discriminator construction strategy, thereby obtaining a collaborative loss function; perform loss function adversarial generation on the generator construction strategy and the discriminator construction strategy, thereby obtaining an adversarial loss function; use the loss function weight calculation formula to perform weight distribution calculation on the collaborative loss function and the adversarial loss function, thereby obtaining the overall loss function; wherein, the loss function weight calculation formula is as follows: ； In the formula, L is the loss function weight value of the collaborative loss function, λ is the overall loss adjustment coefficient, and L g is the loss weight value of the generator, b is the loss adjustment coefficient of the discriminator, γ is the global loss adjustment coefficient, δ is the regional loss adjustment coefficient, L d is the loss weight value of the discriminator, is the global loss weight value of the generator, is the regional loss weight value of the generator, e is a natural constant, a is the sensitivity parameter of the weight allocation, l g is the collaborative loss function, R g The weight reduction coefficient of the collaborative loss function to the adversarial loss function is obtained; the adversarial generative network algorithm is used to construct a model for the overall loss function, the generator construction strategy, and the discriminator construction strategy, so as to obtain a detailed expansion model; A multi-transfer learning module is used to perform adversarial iterative analysis on the detail extension model set, thereby obtaining iterative analysis data; perform feature detail extension on the sample image data using the detail extension model set, thereby obtaining a detail extension image set; perform adversarial gradient estimation on the detail extension image set, thereby obtaining a gradient estimation data set; perform sub-gradient optimization on the detail extension model set based on the iterative analysis data and the gradient estimation data set, thereby obtaining a detail optimization model set; perform multi-domain labeling on the detail extension image set, thereby obtaining multi-domain label data; perform domain difference analysis on the multi-domain label data, thereby obtaining multi-domain difference data; perform hierarchical structure extraction on the multi-domain difference data, thereby obtaining hierarchical structure data; perform multi-layer data splitting on the multi-domain difference data based on the hierarchical structure data and using a multi-task learning method, thereby obtaining a multi-layer split data set; perform feature matrix quantization on the multi-layer split data set, thereby obtaining a feature matrix set; perform migration trend analysis on the feature matrix set, thereby obtaining migration trend data; perform multi-transfer learning on the detail optimization model set based on the migration trend data and using a transfer learning method, thereby obtaining an image feature refinement model; The data preprocessing module is used to obtain the original image data and preprocess the original image data using the image feature refinement model to obtain a preprocessed image data set.

Images