Grayscale measurement expert knowledge generalization model construction method

Abstract

The invention discloses a gray measurement expert knowledge generalization model construction method, which comprises the following steps of: constructing a dual-task deep learning model sharing an encoder by synchronously acquiring quantitative scores and visual focus data of experts, so that the dual-task deep learning model learns and predicts expert scores and regions of interest at the same time; the method comprises the following steps of: establishing a deep model, migrating implicit knowledge of the deep model to an interpretable shallow model by utilizing knowledge distillation, extracting dominant and readable discrimination rules, and finally establishing a man-machine collaborative closed loop: submitting highly uncertain samples of the rules and the model to an expert for auditing and correcting, and utilizing feedback iteration to finely adjust the model to realize continuous optimization of performance and interpretability. According to the method, recessive and intuitive knowledge and dominant knowledge of experts are effectively fused, the generalization performance, robustness and interpretability of the model are remarkably improved, and a solid technical support is provided for high-precision and high-reliability automatic gray scale measurement and calculation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, and in particular relates to a method for constructing a grayscale measurement expert knowledge generalization model. Background Technology

[0002] In many scientific research and industrial production fields, such as surface quality inspection of industrial products, lesion diagnosis in medical images such as CT or MRI, and microstructure analysis of new materials, grayscale measurement is a fundamental and crucial task. It typically refers to providing a quantitative assessment, classification, or judgment by analyzing the grayscale values, contrast, texture, and other features of specific regions in an image. For a long time, this work has heavily relied on well-trained domain experts. These experts, with their profound professional knowledge and long-accumulated practical experience, are able to make accurate judgments on complex and ambiguous image information.

[0003] However, the traditional model that relies on human experts has many bottlenecks: First, expert resources are scarce and costly, and training a qualified expert requires a long period of time.

[0004] Secondly, manual testing has limited efficiency, making it difficult to meet the needs of large-scale, high-throughput production and analysis. Furthermore, prolonged work can lead to fatigue, thus affecting the stability and consistency of judgments.

[0005] Furthermore, experts' knowledge is often "tacit," internalized in their intuition and experience, and difficult to describe completely and unambiguously using explicit language or mathematical formulas. This makes the transmission and replication of expert knowledge extremely difficult.

[0006] To address these issues, researchers have attempted to introduce machine learning techniques to build automated grayscale measurement models. A common approach is supervised learning, which involves collecting a batch of image samples labeled with quantitative evaluation values ​​by experts, such as "qualified / unqualified" or "defect level 1 / 2 / 3," and then using these samples to train a classification or regression model, such as a Support Vector Machine (SVM) or the increasingly popular Convolutional Neural Network (CNN). For example, Chinese patent CN110349126A discloses a "method for detecting surface defects of steel plates based on labeled convolutional neural networks," which achieves automatic defect classification by training a CNN on a large number of labeled defect images.

[0007] However, existing technologies have obvious limitations: First, traditional supervised learning models only learn the mapping relationship between "input image" and "output label". This mapping may only capture some superficial correlations in the training data, without touching the deeper logic and causal relationships behind expert judgments. Once encountering new types of samples that have not appeared in the training set or samples collected under different lighting and angles, the model performance will drop sharply.

[0008] Second, models like deep neural networks are often considered "black boxes" because their internal decision-making processes are opaque; people cannot understand the underlying reasons and basis for their specific judgments. In fields with extremely high requirements for reliability and safety, such as medical diagnosis and aerospace component inspection, a model that cannot explain its decision-making process is difficult to trust and apply.

[0009] Third, building a high-performance deep learning model usually requires tens of thousands of accurately labeled samples, and obtaining such a large amount of expert-labeled data is extremely costly and time-consuming, making it impractical in many professional fields.

[0010] In summary, how to effectively extract, solidify, and generalize the tacit knowledge of experts, and construct an automated grayscale measurement model that is both accurate and interpretable, while not overly reliant on massive amounts of labeled data, is a core challenge that urgently needs to be addressed in the current technological field. Summary of the Invention

[0011] To overcome the problems of weak generalization ability, poor interpretability, and high dependence on large-scale labeled data in existing automated grayscale measurement models, this invention provides a method for constructing a grayscale measurement expert knowledge generalization model.

[0012] To achieve the above objectives, the present invention provides the following technical solution: A method for constructing a gray-scale measurement expert knowledge generalization model includes the following steps: Step S1: Simultaneously acquire a first data stream containing task data and a second data stream containing attention map data through a dual-stream data acquisition module. This step innovatively captures both the expert's evaluation results and the visual focus during the evaluation process, addressing the questions of "what the model needs to do" and "where the model needs to look," thus laying a data foundation for the model to learn the complete cognitive process of the expert.

[0013] Step S2: Construct and train a dual-task deep learning model. This model, through a shared feature extraction module and two parallel task output modules, is designed to simultaneously predict quantified evaluation values ​​and reproduce expert attention maps. Through a joint optimization strategy, the model is guided to learn essential features that can both explain the evaluation results and locate key regions, rather than superficial correlations.

[0014] Step S3: Employing a knowledge distillation and extraction mechanism, the complex, implicit knowledge from the deep model is transferred to a simple, interpretable, shallow student model. The deep model, acting as the teacher model, generates a large amount of pseudo-labeled data for training student models such as decision trees, enabling the student model to learn the essence of the teacher model at low cost. More importantly, human-understandable "IF-THEN" discriminative rules can be extracted from the trained student model, completely opening the model's "black box."

[0015] Step S4: Establish a human-machine collaborative iterative optimization loop, submitting the rules extracted by the model and the problematic samples that the model itself finds "confusing" to experts for review. Experts can directly revise the rules or provide gold standard annotations for the problematic samples. This high-quality feedback is used to iteratively fine-tune the model, forming a virtuous cycle of "model learning - expert guidance - model relearning," continuously improving the model's performance and generalization ability.

[0016] Preferably, the attention map data in step S1 is obtained through at least one of the following methods: a) Using eye-tracking devices, the gaze trajectory and dwell time of experts' eyes are recorded in real time when they observe grayscale measurement samples, and converted into an attention map in the form of a heat map; b) In the human-computer interaction interface, experts are required to use a mouse or touch operation to highlight or circle the key areas on which they base their judgments on the grayscale measurement sample image before making a quantitative assessment, and then convert the highlighted or circled areas into a binarized or grayscale attention map.

[0017] Preferably, the specific structure of the dual-task deep learning model in step S2 is as follows: The shared feature extraction module uses a convolutional neural network (CNN) or visual transformer (ViT) structure to extract deep semantic features from the input grayscale measurement samples; The evaluation value prediction module is a regression or classification network consisting of one or more fully connected layers, used to output quantitative evaluation values; The attention map prediction module is a decoder structure with upsampling capabilities, such as the decoder part of U-Net, used to restore the deep semantic feature map to the original input resolution and output the predicted attention map.

[0018] Preferably, the joint loss function used in the joint optimization of the dual-task deep learning model is... The calculation formula is: in, The loss function for the evaluation value prediction module, The actual quantitative assessment value given by experts. The evaluation value predicted by the model; Let be the loss function of the attention map prediction module. A true attention map of experts. The attention map predicted by the model; is a hyperparameter used to balance the weights of the two tasks, and 0 < <1.

[0019] Preferably, the Choose the mean squared error loss (MSE), mean absolute error loss (MAE), or cross-entropy loss based on the task type; Choose either pixel-level binary cross-entropy loss or Dice loss.

[0020] Preferably, the interpretable shallow student model in step S3 is a gradient boosting decision tree (GBDT), a random forest, or a fuzzy inference system.

[0021] Preferably, when the student model is a gradient boosting decision tree (GBDT), the method for extracting its discrimination rule set includes: traversing all decision trees in the trained GBDT model, extracting each path from the root node to the leaf node, and combining all splitting conditions on the path, i.e. features and their thresholds, into a rule of the form "IF-THEN", the conclusion of which is the output value of the corresponding leaf node.

[0022] Preferably, in step S4, the method for identifying difficult samples includes at least one of the following: a) Based on uncertainty sampling, calculate the entropy or variance of the output probability distribution of the expert knowledge depth model when predicting the evaluation value, and select the sample with the largest entropy or variance as the difficult sample. b) Based on the Query-by-Committee method, multiple deep models with different structures or initialization parameters are trained to form a committee, and the sample with the largest difference in prediction results among committee members is selected as the difficult sample.

[0023] In step S4, the method of fine-tuning the model using rules confirmed by experts is as follows: the rules are transformed into logical constraints or regularization terms and added to the joint loss function described in step S2 to penalize predictions that violate prior human knowledge.

[0024] The grayscale measurement samples are surface defect images of industrial products, lesion area images in medical imaging, or microstructure images in materials science.

[0025] Compared with the prior art, the present invention has the following advantages: 1. This invention, through dual-stream data acquisition and dual-task learning, for the first time synchronously models the explicit knowledge of expert quantitative judgment with the implicit knowledge of visual attention, so that the model not only knows the result, but also understands the basis of the judgment, fundamentally improving the depth of mining expert knowledge.

[0026] 2. Because the model of this invention learns the underlying logic behind expert judgments, that is, which feature regions need to be focused on, rather than the statistical regularities on the surface of the data, it exhibits stronger adaptability and robustness when facing unseen samples or environmental changes, and its generalization ability is significantly better than that of traditional supervised learning models.

[0027] 3. This invention, through a knowledge distillation-extraction mechanism, successfully transforms the decision-making logic of uninterpretable deep learning models into an explicit set of rules that human experts can understand, review, and correct. This greatly enhances the model's credibility and removes obstacles to its application in high-risk fields such as medicine and aerospace.

[0028] 4. This invention utilizes an iterative optimization loop through human-machine collaboration, allowing experts to focus their valuable time on the most important tasks, such as reviewing rules and labeling challenging samples, rather than repetitive simple labeling. This proactive learning paradigm can achieve or even surpass the model accuracy of traditional methods with far less data, significantly reducing data collection costs and project timelines.

[0029] Therefore, through the synergistic effect of the above four key technologies, this invention realizes a brand-new expert knowledge modeling paradigm, solves the core problems of unclear adjustment direction, large error and weak anti-interference ability in the prior art, and provides solid technical support for high-performance and high-reliability automated intelligent measurement systems. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0031] Figure 1 This is a schematic diagram of the system flow of a gray-scale measurement expert knowledge generalization model construction method proposed in this invention.

[0032] Figure 2 This is a schematic diagram of the structure of the dual-task deep learning model in an embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1 refer to Figure 1 This embodiment presents a complete process for constructing a grayscale measurement expert knowledge generalization model. Taking the automatic detection of surface microcracks on a turbine blade of a certain type of aero-engine as an example, the method includes the following steps: Step S1: First, prepare a batch of high-resolution grayscale images of turbine blades as measurement samples. Invite three senior nondestructive testing experts to form an expert group.

[0035] The first data stream, i.e., task data acquisition, involves the expert group independently evaluating each blade image and providing a quantitative evaluation value based on predefined standards. For example, 0 represents no crack, 1 represents a minor crack, 2 represents a significant crack, and 3 represents a severe crack. Inconsistent evaluation results are discussed and agreed upon by the expert group. The final dataset is then generated. ,That It is the first Zhang Image It is the corresponding crack level.

[0036] The second data stream, attention mapping data acquisition, involves using a high-frequency eye tracker such as the TobiiPro Spectrum to record each expert's gaze point and saccade path while observing the image during expert evaluation. The eye-tracking data from multiple experts are then overlaid, smoothed, and used to generate a visual attention heatmap representing the "expert consensus." In the image, areas with higher brightness represent areas of greater expert attention. The final dataset... .

[0037] Step S2: Reference Figure 2 A dual-task deep learning model is constructed. A ResNet-50 network pre-trained on the ImageNet dataset is used as a shared feature extraction module to obtain general image feature extraction capabilities.

[0038] The evaluation prediction module 102 is connected after the global average pooling layer of ResNet-50 and consists of two fully connected layers and a Softmax layer with an output size of 4, used to predict the four levels of cracks. The attention map prediction module 103 adopts a decoder structure similar to U-Net.

[0039] It fuses feature maps from different stages of ResNet-50, namely conv2, conv3, conv4, and conv5, through upsampling and skip connections to gradually restore the resolution of the feature maps, ultimately outputting an image identical to the input image. Predictive attention maps of the same size with pixel values ​​between [0,1] The joint loss function is in, It is the cross-entropy loss used to evaluate value prediction. This is a pixel-level binary cross-entropy loss used for attention map prediction. In this embodiment, the weight α is set to 0.5, indicating equal emphasis on both tasks. The Adam optimizer is used on the dataset... and The entire model is trained end-to-end until the loss function converges, resulting in an expert knowledge deep model. .

[0040] Step S3: Select an efficient gradient boosting decision tree algorithm, LightGBM, as the shallow student model. Prepare a large-scale dataset of unlabeled turbine blade images, which can be 10-100 times larger than the labeled dataset. Input these unlabeled images into a pre-trained teacher model. In this process, the predicted crack level probability distribution and the predicted attention map are obtained for each image.

[0041] right The training data is used to construct the feature. For each unlabeled image, its feature consists of two parts: one part is the global statistical features of the image, such as mean, variance, and various texture descriptors; the other part is based on... The predicted attention map A^ is a local statistical feature calculated within the key regions. Its label is then directly used. Predicted crack grade. Training was conducted using the data constructed above. .

[0042] After training, the feature importance analysis function built into LightGBM can be used to determine which features are most important for judging cracks. Furthermore, all decision trees in the model are traversed, and all paths are transformed into an explicit rule set such as "IF[feature a>threshold x]AND[feature b<threshold y]...THEN[crack level=2]".

[0043] Step S4: Extract the top 20 most important rules from step S3, and During prediction, the 5% of samples with the highest entropy values ​​output by Softmax, which are the most uncertain and problematic samples in the model, are presented to the expert group through a web interface.

[0044] The expert panel may find that the rule "IF [average gray level of a certain area < 35] THEN [crack grade = 2]" is too absolute and will revise it to "IF [average gray level of a certain area < 35] AND [edge gradient of the area > 50] THEN [crack grade = 2]".

[0045] Meanwhile, the expert panel provided gold-standard annotations for the submitted challenging samples. The rules revised by the experts were transformed into regularization terms; for example, if the model's predictions violated the rule, a penalty term was added to the loss function, and the newly added challenging sample annotations were expanded into the original training set.

[0046] Using this new information, return to steps S2 and S3, for and Perform a round of fine-tuning. Repeat this iterative process 2-3 times until the model no longer shows significant improvement in accuracy on independent test sets.

[0047] Example 2 This embodiment differs from Embodiment 1 in that it modifies the method of acquiring attention map data in step S1. Instead of using an expensive eye tracker, a software interface is designed.

[0048] When evaluating images, experts are asked to use a mouse, like a brush, to paint over areas they believe contain cracks or require special attention. The system records the painting trajectory and converts it into a binary attention map, where painted areas are represented by 1 and other areas by 0. This method is less expensive and easier to deploy, and while its accuracy may be slightly lower than eye trackers, it is effective enough in many scenarios.

[0049] Example 3 This embodiment differs from Embodiment 1 in that it modifies the student model and rule extraction method in step S3. A fuzzy inference system (FIS) is used as the student model. Features of key regions, such as average brightness, contrast, and texture complexity, are extracted from the attention map and used as input variables for the fuzzy inference system.

[0050] Membership functions of input variables are determined using methods such as fuzzy clustering, and algorithms such as Adaptive Neural Fuzzy Inference Systems (ANFIS) are used to... The output serves as a supervisory signal, automatically generating a fuzzy rule base. The extracted rules take the form of "IF brightness is 'dark' AND texture is 'complex' THEN crack risk is 'high'", a rule format that is closer to the thinking of human experts.

[0051] Example 4 This embodiment supplements the difficult sample identification method in step S4 based on embodiment 1. It employs a query-by-committee method. In step S2, three slightly different structures are trained simultaneously. The models, for example, use ResNet-50, ResNet-101 and EfficientNet as backbone networks respectively, thus forming a "committee".

[0052] When screening for challenging samples, samples where committee members' predictions are inconsistent are selected and submitted to experts; for example, samples that one model classifies as level 1 while two others classify as level 2. This method can more effectively identify the boundary regions of the model's generalization ability.

[0053] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0054] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0055] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0056] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0057] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a gray-scale measurement expert knowledge generalization model, characterized in that, Includes the following steps: Step S1: Using the dual-stream data acquisition module, simultaneously acquire the first data stream and the second data stream for the same batch of grayscale measurement samples; the first data stream contains the grayscale measurement samples and the task data containing the quantitative evaluation values ​​given by domain experts to the samples; the second data stream contains the grayscale measurement samples and the attention map data of the spatial distribution of the visual attention of the experts on the samples during the evaluation. Step S2: Construct and train a dual-task deep learning model; the dual-task deep learning model includes a shared feature extraction module and two parallel task output modules, namely an evaluation value prediction module and an attention graph prediction module; input the first data stream obtained in step S1 into the evaluation value prediction module for training, and simultaneously input the second data stream into the attention graph prediction module for training. By jointly optimizing the loss functions of the two modules, a pre-trained expert knowledge deep model is obtained. Step S3: Using a knowledge distillation and extraction mechanism, construct and train an interpretable shallow student model; use the expert knowledge deep model trained in Step S2 as the teacher model to predict massive amounts of unlabeled grayscale measurement samples, generating quantitative evaluation values ​​and attention maps containing pseudo-labels; use these pseudo-label data to train the shallow student model, and after training, extract a structured, human-readable set of grayscale measurement discrimination rules from the student model; Step S4: Establish an iterative optimization closed loop for human-machine collaboration; submit the discrimination rule set extracted in Step S3, along with the difficult samples with low confidence in the prediction process of the expert knowledge depth model, to the expert review interface; have domain experts verify, correct, or supplement the discrimination rule set, and accurately label the difficult samples; use the rules confirmed by experts and the newly added difficult sample labeling data to iteratively fine-tune the expert knowledge depth model in Step S2 and the shallow student model in Step S3 until the model performance converges to the preset target accuracy.

2. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 1, characterized in that, The attention map data in step S1 is obtained through at least one of the following methods: a) Using eye-tracking devices, the gaze trajectory and dwell time of experts' eyes are recorded in real time when they observe grayscale measurement samples, and converted into an attention map in the form of a heatmap; b) In the human-computer interaction interface, experts are required to use a mouse or touch operation to highlight or circle the key areas on which they base their judgments on the grayscale measurement sample image before making a quantitative assessment, and then convert the highlighted or circled areas into a binarized or grayscale attention map.

3. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 2, characterized in that, The specific structure of the dual-task deep learning model in step S2 is as follows: The shared feature extraction module uses a convolutional neural network or a visual Transformer structure to extract deep semantic features from the input grayscale measurement samples. The evaluation value prediction module is a regression or classification network consisting of one or more fully connected layers, used to output quantitative evaluation values; The attention map prediction module is a decoder structure with upsampling capabilities, such as the decoder part of U-Net, used to restore the deep semantic feature map to the original input resolution and output the predicted attention map.

4. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 3, characterized in that, The joint loss function used in the joint optimization of the dual-task deep learning model The calculation formula is: in, The loss function for the evaluation value prediction module, The actual quantitative assessment value given by experts. The evaluation value predicted by the model; Let be the loss function of the attention map prediction module. A true attention map of experts. The attention map predicted by the model; is a hyperparameter used to balance the weights of the two tasks, and 0 < <1.

5. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 4, characterized in that, The The loss method is selected as mean squared error loss, mean absolute error loss, or cross-entropy loss based on the task type; Choose either pixel-level binary cross-entropy loss or Dice loss.

6. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 5, characterized in that, The interpretable shallow student model in step S3 is a gradient boosting decision tree, random forest, or fuzzy inference system.

7. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 6, characterized in that, When the student model is a gradient boosting decision tree, the method for extracting its discrimination rule set includes: traversing all decision trees in the trained GBDT model, extracting each path from the root node to the leaf node, and combining all splitting conditions on the path, i.e. features and their thresholds, into a rule of the form "IF-THEN". The conclusion of the rule is the output value of the corresponding leaf node.

8. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 1, characterized in that, In step S4, the method for identifying difficult samples includes at least one of the following: a) Uncertainty-based sampling: Calculate the entropy or variance of the output probability distribution of the expert knowledge depth model when predicting the evaluation value, and select the sample with the largest entropy or variance as the difficult sample; b) Based on the committee query method: train multiple deep models with different structures or initialization parameters to form a committee, and select the sample with the largest difference in prediction results among committee members as the difficult sample.

9. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 1, characterized in that, In step S4, the method of fine-tuning the model using rules confirmed by experts is as follows: the rules are transformed into logical constraints or regularization terms and added to the joint loss function described in step S2 to penalize predictions that violate prior human knowledge.

10. The method for constructing a grayscale measurement expert knowledge generalization model according to claim 1, characterized in that, The grayscale measurement samples are surface defect images of industrial products, lesion area images in medical imaging, or microstructure images in materials science.

Images