Recognition Method for Surface Morphology Structure Diagram of Wool and Cashmere Fibers Based on Image Enhancement

Abstract

A method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement, which respectively cuts fiber segments from wool and cashmere fibers and obtains image data by photographing; preprocesses and digitizes the obtained image data, then annotates and stores it in a matrix; sets a set of image enhancement methods; constructs a sampling model based on reinforcement learning training or a sampling model based on a masking algorithm to obtain the image enhancement methods and corresponding enhanced data during the training of the recognition model; constructs an image recognition model, takes the enhanced data as the input of the Softmax layer, trains the enhanced data in each round of iterative training to obtain a recognition result, and uses the error of the recognition result as feedback to further optimize the model until the optimal enhancement strategy is obtained to accurately identify the wool and cashmere fiber images. The present invention realizes the accurate identification of wool and cashmere fiber images for the problem of sparse effective features in a small amount of image data.

Description

Technical Field

[0001] The present invention belongs to the technical field of textile detection and identification, and particularly relates to a method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement. Background Art

[0002] Cashmere fibers have excellent properties such as fineness, strong elasticity, softness and warmth retention, and are raw materials for high-grade textile products. However, their resources are rare and the price is relatively high. In order to seek benefits, there are illegal acts of using relatively cheap wool fibers to replace cashmere fraudulently and mixing them into cashmere during the production and processing process. Since the basic component of animal fibers is protein, chemical methods cannot be used to qualitatively and quantitatively identify and analyze cashmere and wool fibers. At the same time, the appearance morphologies of the two types of fibers are very similar, and there are large subjective factors and errors in the identification through observing the microscopic structure of the fiber surface scales under an optical microscope. The identification of the types of cashmere and wool fibers has become a difficult point in the field of fiber inspection. Summary of the Invention

[0003] Aiming at the strict requirements of the existing digital image recognition-based methods for data volume and data accuracy, the purpose of the present invention is to provide a method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement, aiming to solve the problems in the identification of wool and cashmere images such as high cost of obtaining images, scarce image data, few image features, single features, and low recognition accuracy, reduce the high threshold of data requirements for training the recognition model using deep learning, and achieve accurate identification of wool and cashmere fiber images for the problem of sparse effective features in a small amount of image data.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] A method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement, comprising the following steps:

[0006] S1: Cut fiber segments from wool and cashmere fibers respectively, and photograph to obtain image data;

[0007] S2: Perform preprocessing operations on the obtained image data to obtain preprocessed fiber images, including: background removal, grayscale processing, denoising, and binarization; digitize the preprocessed fiber images to obtain digital images, label the digital images and store them in matrices;

[0008] S3: Set a set of image enhancement methods;

[0009] S4: Consider the set of image enhancement methods as a function mapping, each type of enhancement method as a function, and the corresponding function execution probability and operation amplitude as weights to form a function mapping relationship, and construct a sampling model based on reinforcement learning training to fit the function mapping relationship; or, use a masking algorithm to perform masking on the set of enhancement methods in the feedback information generated by the continuous iterative training of the image recognition model until a masking scheme suitable for the current wool and cashmere fiber image recognition is found; finally, according to the sampling model or the masking scheme, obtain the image enhancement methods and corresponding enhanced data during the training process of the recognition model.

[0010] S5: Construct an image recognition model, use the enhanced data obtained in S4 as the input of the Softmax layer, train the enhanced data in each round of iterative training to obtain a recognition result, and use the error of the recognition result as feedback to further optimize the model until the optimal enhancement strategy is obtained to accurately recognize the wool and cashmere fiber image.

[0011] In one embodiment, in S1, for the cut fiber segments, perform electron microscope fiber sampling and microscope fiber sampling respectively to obtain an electron microscope fiber image and a microscope fiber image.

[0012] In one embodiment, the method of electron microscope fiber sampling is:

[0013] Use a Haas microtome to cut fiber segments with a length of 0.4 - 0.6 mm from wool and cashmere fibers respectively, put them into a glass test tube, drop ethyl acetate, stir evenly, wait for the solvent to evaporate, paste them on the sample stage, coat them with a gold film using a vacuum evaporator, and then put them into the sample chamber of a scanning electron microscope to take longitudinal fiber morphology images.

[0014] The method of microscope fiber sampling is:

[0015] Use a Haas microtome to cut fiber segments with a length of 0.4 - 0.6 mm from wool and cashmere fibers respectively, place them on a watch glass, drop liquid paraffin or glycerol, stir thoroughly until the fibers are evenly dispersed, then transfer them to a glass slide, and use an optical microscope to take longitudinal fiber morphology images.

[0016] In one embodiment, S3 includes:

[0017] (1) Set N types of image enhancement methods;

[0018] (2) Set the execution probability and operation amplitude of each enhancement method. In the initial state, the execution probability is randomly selected, and there is a default value for the operation amplitude;

[0019] (3) Discretize the execution probability and operation amplitude to obtain a discrete matrix representation that can be used for gradient solution in the neural network. Among them: for the execution probability, discretize it to be not less than 23 = 8 evenly discrete values; for the operation amplitude, it is discretized into 10 evenly discrete values; the set of image enhancement methods can derive (N × 8 × 10) 2 subsets for randomly enhancing fiber images during the training process.

[0020] In one embodiment, in S4, when obtaining the image enhancement method and corresponding enhancement data through the sampling model:

[0021] (1) Construct a sampling model based on reinforcement learning that can receive reinforcement training reward signals;

[0022] (2) By dynamically integrating two training methods, cross-entropy training and reinforcement training, complete the parameter update of the sampling model itself and the acquisition of the feedback signal of the image recognition model. The model training uses deep deterministic policy gradient as the optimization algorithm;

[0023] (3) Embed the predicted output of Softmax in multiple rounds of training and use it as the input of the image recognition model.

[0024] In one embodiment, in S4, when obtaining the image enhancement method and corresponding enhancement data through the masking algorithm: Mask the set of image enhancement methods to generate random enhancement methods, and the steps are as follows:

[0025] (1) In the initial state, randomly select several enhancement methods from the set of image enhancement methods to generate enhancement data, participate in the training of the image recognition model, and dynamically perform masking sampling on the set of image enhancement methods during the training. The execution algorithm is as follows:

[0026] (1.1) Represent the set of image enhancement methods in vector form, and give four variables: the set of image enhancement methods: AS = {AS 1 , …, AS i}, i ∈ [1, N]; increment: Δ; batch training size: batchsize and the subset of enhancement methods: Sub AS ; AS i represents the i-th enhancement method;

[0027] (1.2) In each round of training iteration, start with the initially randomly generated Sub AS , and use the enhancement methods in Sub AS to generate enhancement data in units of a batchsize;

[0028] (1.3) In each subsequent batchsize training, use Δ as the increment to gradually mask the remaining elements in Sub AS , that is, the enhancement methods, which is expressed as: Sub AS {ASi+Δ}, where \(i + \Delta\in[i, N]\), and record Sub using the loss of model training as the measurement criterion AS the elements in it that can reduce the loss value;

[0029] (2) After each round of training is completed, use the accuracy of the set recognition model validation set as an indicator to modify the masking method of Sub in the next round of training AS until the accuracy of the recognition model reaches the threshold.

[0030] In one embodiment, the S5:

[0031] (1) The image recognition model is trained using any supervised or semi-supervised learning method according to the data scale. During iterative training, use the embedded representation output in S4 as the sampling basis for the Softmax layer of the image recognition model, normalize the generated enhanced data, and participate in the training of the image recognition model;

[0032] (2) Two types of error losses are obtained during the training stage of the recognition model: ① Participate in the gradient update of the image recognition model in the form of recognition error, ② Feed back to S4 for reinforcement training in the form of a reward signal to provide a basis for parameter update;

[0033] (3) During the co-training of the image recognition model and the sampling model, the convergence condition of the two models is: the accuracy of the validation set is not lower than the set threshold

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] (1) The present invention can complete the recognition of the surface morphology structure diagram of wool and cashmere fibers under extremely low image resources, and the recognition accuracy can meet the auxiliary requirements of enterprises for the recognition task. The auxiliary requirements are generally set as the recognition accuracy not lower than 80%.

[0036] (2) Under low-resource conditions, the present invention can completely replace manual discrimination and exceed the manual recognition accuracy, generally with an accuracy not lower than 98%.

[0037] (3) In the case of scarce image data, the data augmentation method claimed in the present invention can help the recognition model to mine more features and semantic associations in the image from limited training data.

[0038] (4) The set of multiple data augmentation methods of the present invention can help the recognition model to establish a more information-rich embedded representation between features in the sparse storage of input data.

[0039] (5) Compared with traditional single data augmentation methods, the present invention performs function mapping on multiple augmentation methods, and on this basis, fits the augmentation methods, execution probabilities, and operation amplitudes using a neural network, so as to more accurately find an augmentation scheme suitable for the current training data.

[0040] (6) The present invention combines a reinforcement training strategy with an identification model, which can significantly reduce the recording of redundant information by the model during training, enabling the model to converge efficiently and quickly within fewer iteration cycles. Among them, redundant information includes: parameters such as the repeated recording of image edge feature parameters in gradient backpropagation and neurons that cannot guarantee randomness and participate in feedforward calculations during model training, which are not helpful for updating the model in the next round.

[0041] (7) Compared with the Dropout-like methods that increase the randomness of parameter states in traditional identification models, the method based on reinforcement learning in the present invention replaces randomly dropped neurons and feeds reward signals back to neurons with great contributions, improving the training efficiency.

[0042] (8) Compared with traditional identification models, the training effect of the present invention based on data augmentation methods will be significantly improved as the amount of data expands. The significant improvement is not only reflected in the help of big data for deep learning, but also in that when adding the same amount of training data, the method can generate exponentially more image features compared with traditional data augmentation methods by fitting the execution probabilities and operation amplitudes of multiple data augmentation methods. Description of the Drawings

[0043] Figure 1 It is a process diagram of reinforcement training.

[0044] Figure 2 It is an effect diagram of removing the background of the fiber image.

[0045] Figure 3 It is an effect diagram of grayscale processing of the fiber image.

[0046] Figure 4 It is an effect diagram of denoising the fiber image.

[0047] Figure 5 It is a process diagram of the embedded representation of the fiber image.

[0048] Figure 6 It is a collection diagram of fiber image augmentation methods.

[0049] Figure 7 It is a process diagram of image augmentation.

[0050] Figure 8 It is a flowchart of fiber image augmentation based on the masking method.

[0051] Figure 9 It is a process diagram of training the identification model.

[0052] Figure 10 It is an architecture diagram for wool and cashmere fiber image recognition based on data augmentation.

[0053] Figure 11 It is a flowchart for the execution of the training process. Specific implementation manners

[0054] The following will describe the implementation manners of the present invention in detail with reference to the accompanying drawings and embodiments.

[0055] The recognition method based on deep learning can obtain high training efficiency and accurate results, but the training of its model highly depends on the scale and quality of the training data, which has certain human and material thresholds for the textile industry. A large number of wool and cashmere fiber images that can obtain effective features are characterized by high acquisition cost and great acquisition difficulty, and the fiber image features obtained from the same type of fiber samples are very similar, and sufficient effective features cannot be obtained in large-scale neural unit calculations.

[0056] Therefore, the present invention provides a method for recognizing the surface morphology structure diagram of wool and cashmere fibers based on image enhancement, which mainly includes the following steps:

[0057] S1: Cut fiber segments from wool and cashmere fibers respectively, and shoot to obtain image data.

[0058] For the cut fiber segments of the present invention, electron microscope fiber sampling and microscope fiber sampling are respectively performed to obtain electron microscope fiber images and microscope fiber images. In subsequent embodiments, electron microscope fiber images or microscope fiber images can be used alone, but a better way is obviously to extract features by combining the two images.

[0059] Exemplarily, the method of electron microscope fiber sampling is:

[0060] Use a Haas microtome to cut fiber segments with a length of 0.4 - 0.6 mm from wool and cashmere fibers respectively, put them into a glass test tube, drop ethyl acetate, stir evenly, wait for the solvent to evaporate, paste them on the sample stage, coat them with a gold film using a vacuum evaporator, and then put them into the sample chamber of a scanning electron microscope to shoot the longitudinal morphology images of the fibers.

[0061] Correspondingly, the method of microscope fiber sampling is:

[0062] Use a Haas microtome to cut fiber segments with a length of 0.4 - 0.6 mm from wool and cashmere fibers respectively, place them on a watch glass, drop liquid paraffin or glycerin, stir well until the fibers are evenly dispersed, transfer them to a glass slide, cover the glass slide, place it on the stage of an optical microscope, and use the optical microscope to shoot the longitudinal morphology images of the fibers.

[0063] Only a small amount of fiber image data needs to be obtained in this step, and the data volume can be expanded through subsequent enhancement methods.

[0064] S2: Perform preprocessing operations on the obtained image data to obtain preprocessed fiber images, including: background removal, grayscale processing, denoising, binarization, etc. Digitalize the preprocessed fiber images to obtain digital images, annotate the digital images, and store them in matrices.

[0065] The preprocessing operations are to be able to obtain more effective features in subsequent recognition models and sampling models. Effective features refer to non-redundant image features in fiber samples that can participate in the training of recognition models.

[0066] S3: Set a set of image enhancement methods, which contains several enhancement methods suitable for fiber images. The set can select different enhancement methods according to the characteristics of the image data, or a unified operation set PIL (Python image library) can be set.

[0067] This step specifically includes:

[0068] S3.1: Set N types of image enhancement methods. For wool and cashmere fiber images, the present invention lists but is not limited to 14 types, including: image flipping, image rotation, automatic contrast, image equalization, scaling, contrast change, brightness change, sharpness change, fiber cross-section, sample pairing, shearing, pixel translation, image exposure, image layering.

[0069] S3.2: Set the execution probability and operation amplitude of each enhancement method. In the initial state, the execution probability is randomly selected, and there is a default value for the operation amplitude.

[0070] S3.3: Discretize the execution probability and operation amplitude to obtain a discrete matrix representation that can perform gradient solution in the neural network. The specific scheme is as follows: For the execution probability, it is discretized into no less than 2 3 = 8 uniformly discrete values. For the operation amplitude, it is discretized into 10 uniformly discrete values. Therefore, the set of image enhancement methods set in (1) can derive (N × 8 × 10) 2 subsets for randomly enhancing fiber images during the training process. In actual recognition model training, this enhancement method can provide at most (N × 8 × 10) 14×2 seed enhancement methods in each round of iterative training. N is the maximum number of participants. In actual training, according to the computing power and the number of model parameters, the number of enhancement methods participating in the training in each scheme can be limited by hyperparameters.

[0071] In this step, first, it is necessary to perform a labeling operation on the set of image enhancement methods, mainly to uniquely label each enhancement method in the set. The purpose is to be able to quickly index the corresponding enhancement method through the label during the training of the sampling model. Since the sampling model performs probability operations on the elements in the set, there is no order requirement, and the binary labeling method, i.e., [True / False] labeling, can be adopted for constructing the labeling of the set. Secondly, it is necessary to perform set storage. The set storage uses a three-dimensional array to record the execution probability, operation amplitude, and label information of the set elements respectively. The initial value of the execution probability is generated by a random seed and then determined by updating the parameters of the sampling model according to the feedback information obtained from the reinforcement training. The operation amplitude can be set by empirical values according to the image features of the wool and cashmere images or updated following the model gradient as one of the parameters of the sampling model.

[0072] S4: Consider the set of image enhancement methods as a function mapping. Each type of enhancement method is regarded as a function, and the corresponding function execution probability and operation amplitude are regarded as weights to form a function mapping relationship. Construct a sampling model based on reinforcement learning training to fit the function mapping relationship. Or use the masking algorithm to perform masking on the set of enhancement methods in the feedback information generated by continuously iteratively training the enhancement methods in the image recognition model until a masking scheme suitable for the current wool and cashmere fiber image recognition is found. Obtain the image enhancement methods and corresponding enhanced data during the training process of the recognition model.

[0073] This step provides two schemes for image enhancement of a small number of fiber image samples participating in the training of the recognition model. When using the sampling model, it specifically includes:

[0074] S4.1.1: For the original fiber image, construct a sampling model based on reinforcement learning that can receive reinforcement training reward signals, which can be a recurrent neural network, a multi-layer perceptron, etc. In this invention, Transformer is taken as an example.

[0075] S4.1.2: By dynamically integrating two training methods, cross-entropy training and reinforcement training, complete the parameter update of the sampling model itself and the acquisition of the feedback signal of the image recognition model. The model training uses the Deep Deterministic Policy Gradient (DDPG) as the reinforcement training algorithm and performs iterative training for no less than 80 epochs (in this invention, the training scale is taken as an example of 200 samples).

[0076] S4.1.3: Embed the prediction output of Softmax in multiple rounds of training (taking Embedding as an example), and use it as the input of the image recognition model. The effective features that can be obtained in each round of training of the image recognition model are (the number of elements in S3 * batchsize) times that of the original recognition image, where batchsize is the number of image samples input to the model in each round of training.

[0077] For the scheme of generating enhanced data in S4, the set in S3 can also be randomly masked according to the masking algorithm to generate a random enhancement method. This scheme does not limit the number of methods participating in the enhancement, and continues until the optimal combination of enhancement methods is found under the constraint of recognition accuracy. Specifically, it includes:

[0078] S4.2.1: In the initial state, randomly select several enhancement methods from the set of image enhancement methods to generate enhanced data, which are used in the training of the image recognition model. During the training, perform masked sampling on the set of image enhancement methods dynamically, and use the masking algorithm to mask the enhancement methods in the set during the continuous iterative training of the recognition model until a masking scheme suitable for the current wool and cashmere image recognition is found. The algorithm is as follows:

[0079] S4.2.1.1: Represent the set of image enhancement methods in vector form, and assign four variables: the set of image enhancement methods: AS = {AS 1 , …, AS i}, i ∈ [1, N]. Increment: Δ. Batch training size: batchsize, and the subset of enhancement methods: Sub AS . AS i represents the i-th enhancement method.

[0080] S4.2.1.2: Start with the initially randomly generated Sub AS in each round of training iteration. Take batchsize as a unit, and use the enhancement methods in Sub AS to generate enhanced data.

[0081] S4.2.1.3: In each subsequent batchsize training, use Δ as the increment (Δ is set as a hyperparameter and can be set according to the number of parameters in the constructed model), and gradually mask the remaining elements in Sub AS , that is, the enhancement methods, which is expressed as: Sub AS {AS i+Δ}, i + Δ ∈ [i, N], and record the elements in Sub AS that can reduce the loss value with the loss of model training as the measurement standard.

[0082] S4.2.2: After each round of training, use the accuracy of the set validation set of the set recognition model as an index to modify Sub in the next round of training.AS The masking method is used until the accuracy rate of the recognition model reaches the threshold value (in the present invention, the threshold value of the recognition success rate of wool and cashmere fiber images is set to 99%). Among them, the effective features that can be obtained in each round of training of the image recognition model are (the number of S3 elements) times that of the original recognition image.

[0083] S5: Construct an image recognition model, use the enhanced data obtained in S4 as the input of the Softmax layer, train the enhanced data in each round of iterative training to obtain a recognition result, and use the error of the recognition result as feedback to further optimize the model until the optimal enhancement strategy is obtained to accurately recognize wool and cashmere fiber images.

[0084] This step specifically includes:

[0085] S5.1: The image recognition model is trained using any supervised or semi-supervised learning method according to the data scale. In the iterative training, the embedded representation output in S4 is used as the sampling basis for the Softmax layer of the image recognition model, and the generated enhanced data is normalized and participates in the training of the image recognition model.

[0086] S5.2: Two types of error losses are obtained during the training stage of the recognition model:

[0087] ① Participate in the gradient update of the image recognition model in the form of recognition error.

[0088] ② Feed back to the S4 reinforcement training in the form of a reward signal to provide a basis for parameter update.

[0089] S5.3: During the co-training of the image recognition model and the sampling model, the convergence condition of the two models is: the accuracy rate of the validation set is not lower than the set threshold value.

[0090] Taking the wool and cashmere microscope image as an example below, a recognition network is constructed based on ResNet, and feedback information is generated according to the classification result to update the sampling model and mask algorithm parameters based on reinforcement training, so as to determine an effective enhancement method.

[0091] Among them, in order to make the method claimed in the present invention easier to understand, some model parameters and image processing details are concretely described, but do not restrict the principle of the method claimed in the present invention.

[0092] Reference Figure 1 As shown, the wool and cashmere fiber image recognition method based on image enhancement in this embodiment includes the following steps:

[0093] S1, obtain a small amount of fiber image data of wool and cashmere.

[0094] The specific method is:

[0095] S1.1. Cut and make fiber samples from wool and cashmere fibers with various forms and processes. Different forms refer to wool and cashmere fibers with different colors, different spots, presence or absence of medulla, etc. intercepted from various materials. Different processes refer to textile processes including but not limited to knitting, yarn, combing, weaving, etc.

[0096] S1.2. Make glass slide samples from the obtained materials to get fiber samples that can be observed under a microscope.

[0097] S1.3. Image a small amount of fiber materials through a microscope to obtain corresponding fiber images.

[0098] S2. Preprocess the obtained wool and cashmere fiber images.

[0099] S2.1. Background removal: Use tools such as Matlab or Python that can be used for rough image processing to perform operations such as background removal and background gray level unification on the fiber images. The purpose is to facilitate feature extraction and normalization processing of image input in deep neural network training. The processing effect is as Figure 2 shown.

[0100] S2.2. Gray level processing: When the channels of the image are set to be the same, the color of the fiber image is represented as a gray color, and the gray level range is [0 - 255]. The methods of gray level processing include but not limited to brightness thresholding, histogram equalization, logarithmic gray level change, pseudo-color change, etc. The processing effect is as Figure 3 shown.

[0101] S2.3. Denoising: For common noises such as shot noise, readout noise, and pixel response non-uniformity in the fiber images, use methods including but not limited to filter-based methods, model-based methods, learning-based methods, etc. to perform denoising. The processing effect is as Figure 4 shown.

[0102] S2.4. Embedding representation: Perform a digital representation with associated meanings on the fiber images. Use methods including but not limited to Embedding and Fingerprint methods to convert the images into an n-dimensional vector space, and through normalization methods, enable the embedding representation of the images to participate in the training of subsequent sampling models and recognition models. The digital representation form of wool and cashmere fiber images is as Figure 5 shown.

[0103] S3. Set a set of image enhancement methods, as Figure 6 shown.

[0104] The set can set a reasonable set of enhancement methods according to the characteristics of wool and cashmere fiber images, including but not limited to: image flipping, automatic contrast, image equalization, scaling, brightness change, sharpness change, fiber cross-section, shearing, pixel translation, image exposure, image layering, etc.

[0105] Image flipping: Obtain the length, width, and height of the fiber image, and use a function similar to flip() in OpenCV with image flipping function to arbitrarily flip the fiber image to obtain 90°, 180°, and 270° flipped images corresponding to the original image.

[0106] Automatic contrast processing, brightness change, image exposure: After obtaining the image size, use a function similar to CreatTrackbar() in OpenCV with brightness and color gamut change functions to obtain fiber images with obvious features such as contrast, brightness, and exposure.

[0107] Scaling: After obtaining the image size, use a function similar to resize() with image scaling function to perform non-uniform scaling on the original fiber image. The non-uniform setting is because the effective features of the wool and cashmere fiber images are concentrated in a fixed area of the image.

[0108] Fiber cross-section: Use a function similar to 2DGamma() in OpenCV with gamma change function to perform 2D gamma transformation on the fiber image to obtain a fiber cross-section image with obvious features.

[0109] Image equalization: First, obtain the histogram of the fiber image through a function similar to CalcHise() with histogram conversion function, then obtain the corresponding index through a function similar to minMaxloc() with annotation index function, and then obtain the equalized fiber image through a function similar to equalizeHist() with image equalization function.

[0110] Sharpness change: Use a function similar to resize() with image sharpness improvement function to perform interpolation processing on the fiber image to further improve the sharpness of the image.

[0111] Pixel translation: Use a function similar to warpAffine() in OpenCV with image affine transformation function to perform affine transformation on the original fiber image to obtain a linear transformation image or translation image corresponding to the original image, and then use a function similar to translate() with image pixel translation function to perform pixel translation transformation on different coordinate axes.

[0112] Image layering: To obtain multiple features of the stacked image, image layering is performed on multiple pixels of different images at the same position using methods including but not limited to mean, median, maximum value, etc.

[0113] PIL (Python image library) can also be used as a set of enhancement methods based on existing operations. This set provides basic image processing functions, such as: changing the image size, rotating the image, image format conversion, color space conversion, image enhancement, histogram processing, interpolation, and filtering, etc.

[0114] S4. Use the set of enhancement methods AS obtained in S3 to perform image enhancement on a small number of fiber image samples and generate an enhanced subset Sub AS , such as Figure 7 as shown. The specific steps are as follows:

[0115] S4.1. The present invention provides two sampling methods for AS: S4.1.1 and S4.1.2.

[0116] S4.1.1: Regard the set of enhancement methods AS as a function mapping. Each type of enhancement method is regarded as an element, and the corresponding function execution probability and operation amplitude are regarded as weights to form a function mapping relationship. Construct a neural network to fit the function relationship to obtain the image enhancement method and corresponding enhancement data during the training process of the recognition model.

[0117] S4.1.2: Use the masking algorithm to mask the enhancement methods in AS during the continuous iterative training of the recognition model until a masking scheme suitable for the wool and cashmere image recognition of the current batch of training samples (batchsize) is found, and further obtain the image enhancement method and corresponding enhancement data during the training process of the recognition model. The specific process is as Figure 8 shown.

[0118] For S4.1.1, the present invention provides a method for describing the problem of finding the best image enhancement strategy as a discrete sampling problem. The specific steps are as follows:

[0119] S4.1.1.1, respectively construct a search algorithm and a search space.

[0120] ① Sample the enhancement methods in the set AS. AS not only contains different enhancement methods AS in the set i , but also contains the probability of using this sampling in each batch of training samples and the amplitude information of this enhancement method.

[0121] ② The sampling of the set AS is used to train a neural network with a fixed structure, and its verification accuracy will be fed back in the form of a feedback signal to update the neural network.

[0122] ③ Since the model parameters are trained using reinforcement learning, this feedback signal is non-differentiable in the specific mathematical solution. Therefore, this sampling model will be updated through the policy gradient method.

[0123] In S4.1.1.2, the sampling model selects a network structure that can receive reinforcement learning feedback signals, such as recurrent neural networks, multi-layer perceptrons, Transformers, etc., and the training algorithm uses but is not limited to policy optimization algorithms.

[0124] ① Construct a model based on reinforcement training as the sampling model for the AS set.

[0125] ② Set model parameters such as the number of network layers, the number of attention heads, and the regularization method according to the image pixels and data volume.

[0126] ③ In each round of training, the prediction of the sampling model generates a sampling scheme by Softmax, and then its output result is transformed into an embedded representation and fed into the next step.

[0127] ④ The sampling model combines multiple Softmax prediction results (specifically depending on parameters such as network depth and number of parameters) to obtain the final subset Sub of enhancement methods. AS .

[0128] For S4.1.2, the specific method is as follows:

[0129] Use the masking algorithm to mask the enhancement methods in the set AS during the continuous iterative training of the recognition model until a masking scheme suitable for the recognition of wool and cashmere fiber images in the current batch of samples is found, and further obtain the image enhancement methods and corresponding enhanced data during the training process of the recognition model.

[0130] For the scheme of generating enhanced data, the number of methods participating in the enhancement is not limited in this scheme. Until the optimal combination of enhancement methods is found under the constraint of recognition accuracy, the specific steps are as described in the aforementioned S4.2.1.1 - S4.2.1.3.

[0131] S4.2, construct the sampling model loss.

[0132] The parameter update of the sampling model itself and the acquisition of the recognition model feedback signal are completed through the dynamic integration of two training methods: cross-entropy training and reinforcement training.

[0133] S4.2.1, map the sampling model with reinforcement training.

[0134] Map the reinforcement learning algorithm to the model sampling process. The learning body, environment, parameter optimization method, exploration action, learning body parameter state, and reward in reinforcement learning respectively correspond to the search module, input at different time steps, model training method, search result decoding, network parameters, and recognition accuracy in the sampling algorithm.

[0135]

[0136] The inputs at different time steps, i.e., different states, are the subset Sub AS and the enhancement method AS i in AS . The search results, i.e., the sequence, are decoded into a one-element unit decoding.

[0137] S4.2.2. Based on mapping, the sampling model obtains rewards and punishments in the exploration action by continuously optimizing the network parameters from random states, and finds the optimal network parameters under the reward and punishment mechanism, and then continuously updates to make the optimal choice.

[0138] In one embodiment, the sampling process is regarded as a process of exploring rewards for a single variable by the Markov decision method, i.e.:

[0139] The learning entity continuously changes its parameter state through iterative exploration actions to obtain more feedback rewards, and continuously interacts with the environment on the basis of this iterative training. The process is expressed as M = <S, A, P{s, a}, R>, where s ∈ S, s represents the parameter state of the learning entity, S is the finite set of the parameter states of the learning entity, used to represent the network parameters of the sampling model in different states during training; a ∈ A, a represents the exploration action, A is the finite set of exploration actions, used to record the exploration actions of the learning entity in different states; P{s, a} represents that the learning entity predicts the next parameter state s′ according to the parameter state s and the exploration action a at the current time step; R represents the immediate reward obtained after the learning entity takes the exploration action a.

[0140] S4.2.3, Cross-entropy training.

[0141] In each state, the sampling model takes the element vector AS i ∈ AS and the output h t of the neural network hidden layer t+1 as inputs, calculates the output h 1 of the hidden layer at the next time step 2 , until the sampling model finally outputs the complete sampling vector AS′, where AS is expressed as AS <AS i , AS N , …, AS i , …, AS 1 , AS 2 , …, AS i , … AS N >, AS i is the element vector encoded at the i-th time step in AS, N is the vector length of AS; AS′ is expressed as AS′ <AS′ t , AS′

[0142] S4.2.4, Reinforcement Training. The sampling model makes corresponding decoding actions according to the deep deterministic policy gradient algorithm with the accuracy of the recognition model as the optimization objective, and observes the feedback reward after each output sampling subset by comparing the enhancement methods in the candidate set composed of multiple sampling results output by the current model according to probability with the best enhancement method.

[0143] The specific steps of the reinforcement training algorithm are as follows:

[0144] ① Create two copy networks: an online network O and a target network T, which are used to update the policy parameters and the gradients of the parameters of the reinforcement learning algorithm (in this invention, the Q-Learning reinforcement algorithm is taken as an example) respectively. The purpose is to reduce the repeated gradient calculation of the policy network parameters by the Q-learning parameters during the gradient update in the reinforcement training process.

[0145] ② Use the policy network to obtain the functional relationship π() between the action space a and the state space s.

[0146] ③ Q-Learning is used to determine the feedback information r∈R corresponding to a in the s state. The gradient update process of the reinforcement training in this invention can be expressed as:

[0147]

[0148]

[0149] Different from the traditional reinforcement learning algorithm, since the method claimed in this invention is under the condition of less wool and cashmere fiber image data volume and single features, in order to avoid the feature repetition problem, the training process of the policy network no longer uses random sampling in the optimal solution, but uses a deterministic function to fit the action of behavior a in the state space s, and this functional relationship can be expressed as:

[0150] a~π(s|θ π ) (3)

[0151] where π() and π′() are probability distribution functions, including but not limited to joint probability distribution, etc., and θ π and θ π′ are model parameters.

[0152] ④ Q-Learning network training, θ Q and θ Q′ respectively represent the parameters of two Q-Learning networks Q π and Q′, and the sampling (s i ,a i )~(a i |s i) The training process can be expressed as the joint solution of the feedback information for the next sampling:

[0153] Q π (s i ,a i ) = E[r(s i ,a i ) + αQ π (s i+1 , π(s i+1 ))] (4)

[0154] α is a hyperparameter. For the evaluation function D() that determines the function π(), different from the traditional reinforcement training method, since the sampling object is only a single label index, the probability distribution factor is removed, and the obtained feedback information is mainly used.

[0155] ⑤ Design the evaluation function. The specific evaluation function can be regarded as the expectation of the feedback information of the recognition model after the feedback information is continuously represented, and the representation form is as follows:

[0156]

[0157] ⑥ Design the soft update method.

[0158] The update method of the policy network parameters is expressed as follows:

[0159] θ π′ → βθ π + (1 - β)θ π′ (6) Similarly, the update method of Q-Learning parameters is as follows:

[0160] θ Q′ → βθ Q + (1 - β)θ Q′ (7)

[0161] β is set as a hyperparameter.

[0162] ⑦ In the training of each batchsize, update the parameters of the online network through stochastic gradient descent, and then update the parameters of the target network through the soft update method.

[0163] The overall training process of the sampling model is shown in the following algorithm:

[0164]

[0165]

[0166] S4.3, Sampling model training.

[0167] S4.3.1. Through no less than 80 rounds of iterative training (this iteration period is based on a training scale of 200 samples as an example).

[0168] S4.3.2. Embed the predicted outputs of Softmax in multiple rounds of training (including but not limited to the Embedding method), and use them as the input to the recognition model. The embedding process is as Figure 5 shown, and the specific steps are as follows:

[0169] ① Take each row vector of the weight matrix in the Softmax layer as the Embedding of the training image.

[0170] ② Store the Embedding in the search space matrix.

[0171] ③ Obtain the final calculation result of the fiber image Embedding through the inner product operation of vectors, and use it as the candidate sampling result of the sampling model.

[0172] ④ Sort the candidate results, and take multiple elements with the highest probabilities as the elements of the enhanced set subset Sub AS . The number of subsets can be set as a hyperparameter or can be set as a weight and updated through model training.

[0173] ⑤ Use the enhanced image results and operation amplitudes corresponding to the enhancement methods in Sub AS as the input to the recognition model.

[0174] S5. Build an image recognition model. The recognition model can adopt any recognition network structure based on deep learning. On this basis, statistically analyze the loss function of the model and convert it into a feedback signal form for storage and feedback to the sampling model as the basis for updating its parameters, as Figure 9 shown.

[0175] S5.1. Use the enhanced data obtained in S4 as the input to the Softmax layer.

[0176] S5.2. Train the enhanced data in each round of iterative training to obtain recognition results.

[0177] S5.3. Use the error of the recognition results as feedback to further optimize the model built in S4 until the optimal enhancement strategy is obtained to accurately recognize wool and cashmere fiber images.

[0178] Furthermore, during the training process of S5.2, the recognition model can adopt any recognition network structure based on deep learning.

[0179] At this time, the number of features C that the traditional recognition model can obtain through training on the original image (input size: L*W) data can be expressed as:

[0180]

[0181] Among them, s is the feature extraction step size, the kernel width is f×f, and the padding pixels are pad.

[0182] On this basis, through the sampling model to enhance the image of the original data, the effective features that can be obtained by the recognition model in each round of training are (Sub AS the number of elements m*batchsize) times that of the original recognition image.

[0183] For the mask enhancement method, through the mask sampling method to enhance the image of the original data, the effective features that can be obtained by the recognition model in each round of training are (Sub AS the number of elements m) times that of the original recognition image.

[0184] The difference between the present invention and the recognition model usually based on a deep neural network is that, based on the constructed recognition model, the loss function of the model is statistically calculated and converted into a feedback signal form for storage and feedback to the sampling model as the basis for updating its parameters. Given a training set x of fiber diagrams to be recognized containing n samples, for the predicted classification label y(x), the conversion method of the specific loss function with the correct sample P(x) is:

[0185] Loss=-∑ X P(X)logy(X) R(X) (9)

[0186] R(X)=γR Top-1 +(1-γ)R Top-5 (10)

[0187]

[0188]

[0189] Among them, according to the complexity of the task, γ can be set as a hyperparameter. When the predicted label y(x) is equal to the label P(x) of the correct sample, the discriminant function d(y(X),P(x))=0.

[0190] In summary, the recognition model architecture of the wool and cashmere fiber diagram constructed by the present invention is as Figure 10 shown, and its training execution process is as Figure 11 shown.

Claims

1. A method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement, characterized in that, it includes the following steps: S1: Cut fiber segments from wool and cashmere fibers respectively, and capture image data; S2: Perform preprocessing operations on the obtained image data to obtain preprocessed fiber images, including: background removal, grayscale processing, denoising, and binarization; digitize the preprocessed fiber images to obtain digital images, annotate the digital images and store them in a matrix; S3: Set a set of image enhancement methods, including: (1) Set N types of image enhancement methods; (2) Set the execution probability and operation amplitude of each enhancement method. In the initial state, the execution probability is randomly selected, and there is a default value for the operation amplitude; (3) Discretize the execution probability and the operation amplitude to obtain a discrete matrix representation that can be used for gradient solving in a neural network, where: for the execution probability, discretize it into no less than 2 3 = 8 uniformly discrete values; for the operation amplitude, discretize it into 10 uniformly discrete values; the set of image enhancement methods can derive (N × 8 × 10) 2 subsets for randomly enhancing fiber images during the training process; S4: Regard the set of image enhancement methods as a function mapping, regard each type of enhancement method as a function, and regard the corresponding function execution probability and operation amplitude as weights to form a function mapping relationship. Construct a sampling model based on reinforcement learning training to fit the function mapping relationship; or, use the masking algorithm to perform masking on the set of image enhancement methods in the feedback information generated by continuous iterative training of the image recognition model until a masking scheme suitable for the current wool and cashmere fiber image recognition is found; finally, according to the sampling model or the masking scheme, obtain the image enhancement methods and corresponding enhancement data during the training of the recognition model; where: When obtaining image enhancement methods and corresponding enhancement data through the sampling model: (1) Construct a sampling model based on reinforcement learning that can receive reinforcement training reward signals; (2) The model training uses deep deterministic policy gradient as the optimization algorithm; (3) Embed the predicted outputs of Softmax in multiple rounds of training and use them as the input of the image recognition model; When obtaining image enhancement methods and corresponding enhancement data through the masking algorithm: Perform masking on the set of image enhancement methods to generate random enhancement methods. The steps are as follows: (1) In the initial state, randomly select several enhancement methods from the set of image enhancement methods to generate enhancement data, participate in the training of the image recognition model, and dynamically perform masking sampling on the set of image enhancement methods during the training. The execution algorithm is as follows: (1.1) Vectorize the set of image enhancement methods and given four variables: the set of image enhancement methods: AS = {AS 1 , …, AS i}, i ∈ [1, N]; increment: Δ; batch training size: batchsize and enhancement method subset: Sub AS ; AS i represents the i-th enhancement method; (1.2) At the beginning of each training iteration with an initially randomly generated Sub AS as the starting point, taking a batchsize as a unit, use the augmentation method in Sub AS to generate augmented data; (1.3) In each subsequent batchsize training, gradually mask Sub with an increment of Δ AS The remaining elements in are the enhancement methods, denoted as: Sub AS {AS i+Δ}, i + Δ ∈ [i, N], and record Sub using the loss of model training as the measurement criterion AS The elements in that can reduce the loss value; (2) After each round of training, the masking method for Sub AS is modified in the next round of training using the accuracy of the validation set of the recognition model set as an indicator until the accuracy of the recognition model reaches the threshold; AS ​ S5: Construct an image recognition model, use the enhancement data obtained in S4 as the input of the Softmax layer, train the enhancement data in each round of iterative training to obtain the recognition result, and use the error of the recognition result as feedback to further optimize the model until the optimal enhancement strategy is obtained to accurately recognize the wool and cashmere fiber images.

2. The method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement according to claim 1, characterized in that, in S1, for the cut fiber segments, perform electron microscope fiber sampling and microscope fiber sampling respectively to obtain electron microscope fiber images and microscope fiber images.

3. The method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement according to claim 1, characterized in that, the method of electron microscope fiber sampling is: Use a Haas microtome to cut fiber segments with a length of 0.4 - 0.6 mm from wool and cashmere fibers respectively. Put them into a glass test tube, drop in ethyl acetate, stir evenly. After the solvent evaporates, paste them on the sample stage, coat them with a gold film using a vacuum evaporator, and then put them into the sample chamber of a scanning electron microscope to take longitudinal fiber morphology images; The method for fiber sampling under a microscope is as follows: Use a Haas microtome to cut fiber segments with a length of 0.4 - 0.6 mm from wool and cashmere fibers respectively. Place them on a watch glass, drop in liquid paraffin or glycerol, stir thoroughly until the fibers are evenly dispersed, then transfer them to a glass slide and use an optical microscope to take longitudinal fiber morphology images.

4. The method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement according to claim 1, characterized in that, by dynamically integrating two training methods, cross-entropy training and reinforcement training, the parameter update of the sampling model itself and the acquisition of the feedback signal of the image recognition model are completed.

5. The method for identifying the surface morphology structure diagram of wool and cashmere fibers based on image enhancement according to claim 1, characterized in that, S5: (1) The image recognition model is trained using any supervised or semi-supervised learning method according to the data scale. In iterative training, the embedded representation output in S4 is used as the sampling basis for the Softmax layer of the image recognition model, and the generated enhanced data is normalized and participates in the training of the image recognition model; (2) Two types of error losses are obtained during the training stage of the recognition model: ① Participate in the gradient update of the image recognition model in the form of recognition error, ② Feed back to the S4 reinforcement training in the form of a reward signal to provide a basis for parameter update; (3) In the co-training of the image recognition model and the sampling model, the convergence condition for the two models is that the accuracy of the validation set is not lower than the set threshold.

Images