Boiler water wall defect detection method and system based on multi-modal agent

By employing a detection method based on multimodal intelligent agents, combined with a visual language model and a defect knowledge base, hierarchical reasoning and incremental learning are used to solve the problems of low efficiency and poor accuracy in boiler water-cooled wall defect detection, achieving efficient, interpretable detection and continuous learning.

CN123071307APending Publication Date: 2026-11-10HUANENG (FUJIAN) ENERGY DEVELOPMENT LIMITED COMPANY FUZHOU BRANCH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611335704.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-31
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies for detecting defects in boiler water-cooled walls suffer from problems such as low detection efficiency, high subjectivity, harsh environments, difficulty in detecting early defects, insufficient robustness, lack of knowledge accumulation, insufficient utilization of reference samples, lack of human-computer interaction mechanisms, and weak incremental learning capabilities, resulting in poor detection accuracy and interpretability.

Method used

A detection method based on multimodal intelligent agents is adopted. By acquiring images and preprocessing them, global semantic parsing, cross-image visual feature comparison analysis and fusion reasoning are performed by combining a defect knowledge base and a visual language model to generate a defect detection report. Incremental learning is achieved through user feedback to build a knowledge management system.

Benefits of technology

It achieves efficient and accurate defect detection, has good interpretability and continuous learning capabilities, reduces data preparation costs, and improves detection accuracy and system adaptability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123071307A_ABST
    Figure CN123071307A_ABST
Patent Text Reader

Abstract

The present application provides a boiler water wall defect detection method and system based on a multi-modal intelligent agent, belonging to the technical field of industrial visual detection, which can at least partially solve the problems of lack of systematic knowledge management, insufficient utilization of reference samples, absence of human-computer interaction mechanism, poor interpretability and weak incremental learning ability in existing technologies. The present application includes: acquiring an image of the area to be measured and performing preprocessing, constructing a first prompt word driven VLM model to perform global semantic analysis on the image of the area to be measured; selecting a plurality of candidate defect types with high confidence from the full defect types; constructing a second prompt word to drive the VLM model to perform cross-image visual feature comparison analysis and determine the target defect subtype; and constructing a comprehensive prompt word to drive the VLM model to perform fusion reasoning and generate a defect detection report. The present application utilizes the image understanding and reasoning capabilities of the multi-modal large model to realize defect detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of industrial vision inspection technology, specifically relating to a method and system for detecting defects in boiler water-cooled walls based on multimodal intelligent agents. Background Technology

[0002] As a core component of energy equipment such as thermal power and nuclear power generation, the safe operation of boiler water-cooled walls directly affects the stability and reliability of the entire power generation system. The heating surfaces of water-cooled walls are subjected to complex conditions such as high temperature, high pressure, corrosion, and wear over long periods, making them prone to various defects such as thinning of the tube wall, mechanical damage, abnormal wear-resistant tiles, bending deformation, and cracking. Statistics show that boiler heating surface tube failures account for a significant proportion of unplanned shutdowns of power plant boilers, seriously threatening the safety of power production. Therefore, timely and accurate detection of water-cooled wall defects is of great importance to ensuring the safe operation of power plants.

[0003] Traditional defect detection methods mainly rely on manual visual inspection or conventional image processing techniques. Manual visual inspection has the following problems: low inspection efficiency, as an experienced inspection engineer can only inspect a limited number of pipe banks per day; strong subjectivity, as different inspectors have different judgment standards, which easily leads to missed or false detections; harsh working environment, with high temperature, high humidity, and high dust inside the boiler, posing a threat to the health of inspectors; and difficulty in detecting early defects, as the human eye has limited ability to identify subtle defects, often only detecting them when they have developed to a more serious stage.

[0004] While conventional image processing methods (such as traditional computer vision methods based on edge detection, texture analysis, and template matching) have improved detection efficiency to some extent, they also have significant limitations: feature engineering relies on manual design, requiring specialized feature extraction algorithms for different defect types, resulting in poor generalization ability; they are sensitive to changes in lighting, angle, and background, and the robustness of traditional methods is insufficient in the complex and variable environment of actual industrial sites; they struggle to handle complex defect types, and traditional segmentation and classification methods are ineffective for defects with diverse shapes and blurred boundaries; and they lack a knowledge accumulation mechanism, as each detection is independent and cannot leverage historical detection experience to improve performance.

[0005] In recent years, the rapid development of Large Language Models (LLMs) and Visual Language Models (VLMs) has brought new opportunities to industrial visual inspection. Multimodal large models possess powerful image understanding and natural language reasoning capabilities, enabling them to understand complex visual scenes and output structured analysis results. However, applying multimodal large models to industrial defect detection still faces the following technical challenges: First, there is a lack of systematic knowledge management. Traditional defect detection methods only output judgment results, cannot accumulate and reuse detection experience, and need to analyze again each time, making it difficult to form an evolving intelligent system. A large amount of valuable information generated during the detection process, such as defect feature description, judgment basis, and similar cases, cannot be effectively saved and utilized.

[0006] Second, reference sample utilization is insufficient. Existing methods usually use an end-to-end training mode, which requires a large amount of labeled data to train deep learning models. For industrial defect detection scenarios, defect samples are often scarce and costly to obtain, making it difficult to meet training needs. At the same time, existing methods cannot flexibly utilize a small number of reference samples for small sample learning, and have poor adaptability to new types of defects.

[0007] Third, there is a lack of human-computer interaction mechanism. Existing systems are mostly "black box" mode, ending after detection is completed, and cannot effectively collect and utilize user feedback. When the detection result is wrong, the system cannot learn and improve from it; when a new type of defect is encountered, the system cannot quickly adapt through user guidance. This one-way detection mode limits the continuous improvement of system performance.

[0008] Fourth, the explainability is poor. The "black box" nature of deep learning models makes it difficult to explain the detection results, and industrial site personnel cannot understand why the model makes certain judgments, resulting in low trust in the detection results and affecting actual application promotion. Industrial scenarios have high requirements for the explainability of detection results, and need to know the judgment basis and reasoning process.

[0009] Fifth, the incremental learning ability is weak. Traditional models need to be retrained to learn new knowledge, which is time-consuming and labor-intensive, and may cause catastrophic forgetting (forgetting learned knowledge). Online learning and knowledge updating cannot be achieved, making it difficult to adapt to the changing detection needs of industrial sites.

[0010] Therefore, there is an urgent need for a technical solution that can fully utilize the image understanding and reasoning capabilities of multi-modal large models, combine the autonomous decision-making and learning capabilities of intelligent agents, and achieve defect detection, knowledge accumulation, and system evolution integration.

[0011] To this end, we propose a boiler water wall defect detection method and system based on a multi-modal intelligent agent. SUMMARY

[0012] The present application aims to at least solve one of the technical problems existing in the prior art, and provides a boiler water wall defect detection method and system based on a multi-modal intelligent agent.

[0013] In one aspect of the present application, a boiler water wall defect detection method based on a multi-modal intelligent agent is provided, comprising the following steps: S1: acquiring an image of a to-be-detected region, performing preprocessing on the image of the to-be-detected region to obtain a preprocessed image, and constructing a first prompt word containing the preprocessed image to drive a VLM model to perform global semantic analysis on the preprocessed image and extract analysis text; S2: based on the analysis text, combining a pre-set defect knowledge base, and screening a plurality of candidate defect types with high confidence from full-quantity defect types; S3: retrieving a reference image matching the candidate defect type from a pre-set data set, constructing a second prompt word containing the preprocessed image of the to-be-detected region and the reference image to drive the VLM model to perform cross-image visual feature comparison analysis on the preprocessed image of the to-be-detected region and the reference image, and determining a target defect sub-type; and S4: constructing a comprehensive prompt word containing the image of the to-be-detected region, the reference image, the analysis text, the candidate defect type, and the target defect sub-type to drive the VLM model to perform fusion reasoning on the comprehensive prompt word and generate a defect detection report.

[0014] Further, in the step S1, the step of preprocessing the image of the to-be-detected region specifically comprises: performing format checking, size adjustment, and file compression on the image of the to-be-detected region to meet the interface input requirements of the VLM model.

[0015] Further, the specific steps of the step S2 comprise: acquiring feature description information of each defect type in the defect knowledge base; calculating the text similarity between the analysis text and each feature description information; performing confidence ranking on each defect type based on the text similarity, and selecting a plurality of defect types with high ranking as the candidate defect types.

[0016] Further, the pre-set data set organizes a folder structure according to the defect types, each defect type corresponds to an independent folder, and only all reference images of the defect type are stored in the independent folder.

[0017] Further, in the step S3, the step of retrieving a reference image matching the candidate defect type from a pre-set data set specifically comprises: constructing a picture storage path according to the name of the candidate defect type and locating a corresponding target folder; reading the complete paths of all picture files in the target folder that meet the pre-set format to generate a path list; and inputting the path list as input data of the cross-image visual feature comparison analysis into the VLM model.

[0018] Further, the content of the defect detection report includes: a determination result of whether there is a defect, a defect type classification, a confidence value, a judgment reason and a recommended measure, and also contains complete reasoning chain information, which is used to support result display and feedback collection.

[0019] Further, it also includes: collecting user evaluation on the defect detection report, and realizing incremental learning of the defect knowledge base based on the evaluation type.

[0020] Further, the step of collecting user evaluation on the defect detection report and realizing incremental learning of the defect knowledge base based on the evaluation type specifically includes: when the user evaluation on the defect detection report is correct, automatically creating or positioning the corresponding defect type directory, copying the to-be-tested area image to the preset data set and performing a duplicate file processing, and synchronously updating the visual description of the new sample and the label information containing the user confirmation and the automatically added identification to the defect knowledge base to realize incremental learning; when the user evaluation on the defect detection report is wrong or uncertain, only recording feedback information without updating the defect knowledge base, and leaving the related sample to be manually audited.

[0021] Another aspect of the present application provides a boiler water wall defect detection system based on a multi-modal intelligent agent, which is used to realize the boiler water wall defect detection method based on a multi-modal intelligent agent described in any of the above aspects, and the system includes an input module for acquiring a to-be-tested area image and a defect detection reasoning module, and the defect detection reasoning module includes: a preliminary analysis module, which is used to perform a preprocessing operation on the to-be-tested area image, and construct a first prompt word containing the preprocessed to-be-tested area image, so as to drive a VLM model to perform global semantic analysis on the preprocessed to-be-tested area image and extract an analysis text; a rough screening candidate module, which is used to screen out a candidate defect type from a plurality of defect types based on the analysis text and in combination with a preset defect knowledge base; a fine screening matching module, which is used to retrieve a reference image corresponding to the candidate defect type, construct a second prompt word containing the to-be-tested area image and the reference image, call the VLM model to perform a comparative analysis, and determine a target defect sub-type; and a final judgment module, which is used to construct a comprehensive prompt word containing the analysis text, the candidate defect type, the target defect sub-type and the reference image, so as to call the VLM model to generate a final defect detection report.

[0022] Further, it also includes a knowledge management and incremental learning module, which is configured to build and maintain a defect knowledge base and a data set, provide knowledge support for the defect detection reasoning module, and perform incremental learning based on user feedback on the defect detection report, so as to realize continuous improvement of the detection capability of the system.

[0023] The present application has the following beneficial effects: A defect detection mechanism based on four-stage reasoning is realized, through a hierarchical reasoning process of preliminary analysis, coarse screening, fine screening and final judgment, the visual understanding ability of the multi-modal large model is fully utilized, small sample learning is carried out combined with reference samples, the defect detection accuracy is significantly improved, and good explainability is also possessed, and detailed judgment reasons can be output. Compared with the traditional deep learning method which needs a large amount of training data, the present application can realize effective detection only by using a small amount of reference samples, and greatly reduces the data preparation cost.

[0024] A defect knowledge base management system is constructed, the detection experience is stored and reused in a structured form, dynamic updating and extension of knowledge are supported, the problem of lacking knowledge accumulation in traditional methods is solved, and the system has long-term memory ability. The knowledge base is stored in JSON format, which is convenient for querying, updating and version management, and forms a heritageable detection knowledge asset.

[0025] A human-computer cooperative feedback learning mechanism is designed, the knowledge base is automatically updated by collecting user feedback, continuous learning and evolution of the system are realized, correct detection confirmed by the user is automatically included in the data set, and the detection ability and coverage of the system are continuously improved. This incremental learning method avoids the catastrophic forgetting problem of traditional models and realizes online accumulation of knowledge.

[0026] Good explainability is possessed, the intermediate results and reasoning basis of each stage of the four-stage reasoning process are output, the final judgment contains detailed reason explanation, so that the industrial site personnel can understand and trust the detection result, and it is beneficial to practical application and popularization. Compared with the 'black box' characteristics of the deep learning model, the transparent reasoning process of the present application is more in line with the needs of industrial applications. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The overall flowchart of the boiler water wall defect detection method based on the multi-modal intelligent agent of the embodiment of the present application is shown in the figure. Figure 2 The detailed work flowchart of the defect detection and knowledge base incremental learning of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0028] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0029] Please refer to Figure 1The embodiment of the present application provides a boiler water wall defect detection method based on a multi-modal intelligent agent, and the method comprises the following steps: S1: acquiring an image of a to-be-detected area, performing pretreatment on the image of the to-be-detected area to obtain a pretreated image, and constructing a first prompt word containing the pretreated image to drive a VLM model to perform global semantic analysis on the pretreated image and extract analysis text; S2: based on the analysis text, combining a preset defect knowledge base, and screening a plurality of candidate defect types with high confidence from full-quantity defect types; S3: searching a reference image matching the candidate defect type from a preset data set, constructing a second prompt word containing the pretreated image of the to-be-detected area and the reference image, driving the VLM model to perform cross-image visual feature comparison analysis on the pretreated image of the to-be-detected area and the reference image, and determining a target defect sub-type; and S4: constructing a comprehensive prompt word containing the image of the to-be-detected area, the reference image, the analysis text, the candidate defect type and the target defect sub-type, driving the VLM model to perform fusion reasoning on the comprehensive prompt word, and generating a defect detection report.

[0030] Specifically, please refer to the following Figure 2 In the embodiment, the system adopts a four-stage detection process combining a visual language model (VLM) and a dynamic knowledge base, and realizes continuous learning and adaptive evolution of the system through a user feedback mechanism. The specific working process is as follows: when the image of the to-be-detected heating surface area is input, the system sequentially performs the following four reasoning stages: first, the VLM model performs global scanning on the image of the to-be-detected area to generate analysis text, and the analysis text includes preliminary visual description and abnormal area positioning. Then, the preset defect knowledge base is called, and the analysis text is combined to screen the three candidate defect types with the highest confidence from the predefined defect classification system to narrow down the scope of investigation. Then, on the basis of the candidate defect types, the VLM model again combines the detailed feature description in the defect knowledge base to perform fine identification and matching on the specific defect type, determines the target defect sub-type, and finally compares the features of the image of the to-be-detected area with the standard samples in the reference atlas in a cross-modal manner, and outputs the final defect detection report. After the system outputs the defect detection report, the artificial feedback link is introduced. The user judges the defect detection report according to the actual experience, and the feedback result is divided into three cases: correct result, incorrect result and uncertain result. The system executes differentiated subsequent processing logic according to the feedback type: If the user confirms that the defect detection report is correct, the system triggers an automatic update mechanism, including: Knowledge base update: automatically adding the feature description and label of the current to-be-detected area image to the defect knowledge base to enrich the sample diversity of known defects.

[0031] Dataset archiving: The image of the region to be tested is automatically stored as a positive sample in the corresponding defect directory of the dataset's data / anomaly / directory for subsequent model training.

[0032] If the user determines that the defect detection report is incorrect, the system initiates an interactive correction process, including: Interactive correction: Users can specify the correct defect type or create a new defect type on the interface.

[0033] Knowledge base correction: The system automatically updates the defect knowledge base based on the user's correction actions, correcting the associated descriptions of errors or adding new type definitions.

[0034] Sample archiving: Store the corrected image data in the correct directory of the dataset's data / anomaly / directory to ensure the accuracy of the training data.

[0035] If the user deems the defect detection report uncertain or difficult to judge, the system executes a long-tail sample processing procedure, including: Automatic archiving: The image of the area to be tested is automatically moved to the output / uncertain_samples / directory for temporary storage.

[0036] Manual review triggered: The system monitors the number of samples in this directory. When the preset review threshold (controlled by Config) is reached, it automatically reminds experts to conduct centralized manual review.

[0037] Review and feedback: Samples that have been manually reviewed and confirmed will be reclassified into the correct dataset directory to complete the knowledge loop.

[0038] Through the above process, all data verified by users (whether it's confirmation of correctness, correction of errors, or confirmation after review) is ultimately transformed into the system's knowledge assets. The system utilizes these newly added high-quality samples and updated defect knowledge to periodically fine-tune or incrementally learn the VLM model, thereby achieving continuous improvement in detection accuracy and adaptability to new defect types.

[0039] In this embodiment, the defect knowledge base specifically adopts the form of a Wiki knowledge base to provide flexible text editing and hyperlink association functions. However, those skilled in the art should understand that the specific storage medium of the defect knowledge base is not limited to this. In other embodiments, the defect knowledge base may also be at least one of a relational database, graph database, vector database, document database, or knowledge graph. Regardless of the underlying storage architecture used, the defect knowledge base is configured to store defect type definitions, feature descriptions, reference image indexes, and association relationships to support the knowledge retrieval and reasoning needs in the four-stage detection process.

[0040] Furthermore, in step S1, the preprocessing step of the image of the region to be tested specifically includes: performing format checks, size adjustments, and file compression on the image of the region to be tested to meet the interface input requirements of the VLM model.

[0041] Specifically, in step S1, firstly, standardized preprocessing is performed on the image of the region to be tested, including format verification, size adjustment, and file compression, to ensure that the image data conforms to the input specifications of the VLM model interface. Secondly, a first prompt word containing specific task instructions is constructed and input into the VLM model along with the preprocessed image of the region to be tested, guiding the model to perform global semantic analysis of the visual features of the heated surface in the image. Finally, the structured analysis text generated by the VLM model is extracted as context input for subsequent coarse and fine screening stages. This step adopts an analysis strategy of "global semantic understanding first, followed by local detail focus," effectively overcoming the limitation of traditional detection methods that lack contextual support when directly classifying defects, and significantly improving the accuracy and robustness of defect detection in complex scenarios.

[0042] Further, the specific steps of step S2 include: obtaining feature description information of each defect type in the defect knowledge base; calculating the text similarity between the analysis text and each feature description information; ranking each defect type by confidence based on the text similarity, and selecting the top-ranked defect types as candidate defect types.

[0043] Specifically, in step S2, the system first acquires the preliminary analysis text generated in the previous stage and uses it as a query index to interact with the defect knowledge base. Specifically, the system uses a text semantic similarity matching algorithm to compare the analysis text with various pre-stored defect standard descriptions in the defect knowledge base. Based on the comparison results, the system selects the top N (e.g., Top 3) candidate defect types with the highest confidence from the full defect type library. This step, by introducing prior knowledge constraints, converges the detection scope from the global category space to a local candidate subset, effectively eliminating interference from irrelevant categories, significantly reducing the computational complexity of subsequent fine-tuning stages, and improving the overall response efficiency of the detection process.

[0044] In other embodiments, the zero-shot reasoning capability of the VLM model can also be used to compare the analyzed text with various pre-stored defect standard descriptions in the defect knowledge base.

[0045] Furthermore, the preset dataset is organized into folder structures according to the defect types, with each defect type corresponding to an independent folder, and the independent folder contains only all reference images of that defect type.

[0046] Furthermore, in step S3, the step of retrieving reference images matching the candidate defect type from the preset dataset specifically includes: constructing an image storage path based on the name of the candidate defect type and locating the corresponding target folder; reading the complete paths of all image files conforming to a preset format within the target folder and generating a path list; and inputting the path list as input data for the cross-image visual feature comparison analysis into the VLM model.

[0047] Specifically, in step S3, in this embodiment, the fine-grained matching step aims to achieve precise mapping from broad defect types to specific subtypes. The specific execution logic is as follows: First, based on the candidate defect types locked in the previous stage, multiple corresponding reference images are retrieved from a pre-set dataset. Then, a second cue word containing the image of the region to be tested and the reference images is constructed and input into the VLM model. Through visual feature alignment and semantic reasoning, the similarity between the image of the region to be tested and each reference image in terms of texture, shape, and distribution is analyzed to determine the most matching specific target defect subtype. This step utilizes the contextual learning capability of the VLM model, providing a small number of high-quality reference images as "visual anchors" and employing the idea of ​​few-shot learning to effectively guide the model to achieve high-precision fine-grained classification even with a lack of large amounts of training data.

[0048] Furthermore, the defect detection report includes: the determination result of whether a defect exists, the defect type classification, the confidence level value, the reason for the judgment, and the recommended measures. It also includes complete reasoning chain information to support result display and feedback collection.

[0049] Specifically, in step S4, a comprehensive prompt word is constructed, comprising the image of the region to be tested, the reference image, the analysis text, the candidate defect type, and the target defect subtype. This prompt word is then input into the VLM model to perform the final defect determination. After comprehensive reasoning, the model outputs a standardized JSON format detection result, whose core fields include at least: defect existence, defect type classification, determination confidence, determination reason, and subsequent recommended measures. In particular, the JSON output contains embedded complete reasoning chain information. This information not only provides interpretability support for the final detection conclusion, facilitating result visualization in downstream systems, but also provides precise data traceability for subsequent user feedback collection and incremental learning of the system.

[0050] Furthermore, it also includes: collecting user evaluations of the defect detection report, and implementing incremental learning of the defect knowledge base based on the evaluation type.

[0051] Furthermore, the steps of collecting user evaluations of the defect detection report and incrementally learning the defect knowledge base based on the evaluation type specifically include: when the user's evaluation of the defect detection report is correct, automatically creating or locating the corresponding defect type directory, copying the image of the area to be tested to the preset dataset and performing duplicate file name processing, and simultaneously updating the visual description of the new sample and the tag information containing user confirmation and automatically added identification to the defect knowledge base to achieve incremental learning; when the user's evaluation of the defect detection report is incorrect or uncertain, only the feedback information is recorded without updating the defect knowledge base, and the relevant samples are left for manual review.

[0052] Another aspect of the present invention provides a boiler water-cooled wall defect detection system based on a multimodal intelligent agent, for implementing the boiler water-cooled wall defect detection method based on a multimodal intelligent agent as described in any of the preceding claims. The system includes an input module for acquiring an image of the area to be tested and a defect detection inference module, the defect detection inference module comprising: The preliminary analysis module is used to perform preprocessing operations on the image of the region to be tested and construct a first prompt word containing the preprocessed image of the region to be tested, so as to drive the VLM model to perform global semantic parsing on the preprocessed image of the region to be tested and extract the analysis text. The candidate screening module is used to select candidate defect types from multiple defect types based on the analyzed text and in conjunction with a preset defect knowledge base. The fine screening matching module is used to retrieve reference images corresponding to the candidate defect type, construct a second cue word containing the image of the region to be tested and the reference image, call the VLM model to perform comparative analysis, and determine the target defect subtype. The final judgment module is used to construct a comprehensive prompt word that includes the analysis text, the candidate defect type, the target defect subtype, and the reference image, so as to call the VLM model to generate the final defect detection report.

[0053] Furthermore, it also includes a knowledge management and incremental learning module, which is configured to build and maintain a defect knowledge base and dataset, provide knowledge support for the defect detection reasoning module, and perform incremental learning based on user feedback on the defect detection report, so as to continuously improve the system's detection capabilities.

[0054] In this embodiment, the knowledge management and incremental learning module includes defect knowledge base management functionality, which provides a complete knowledge base management interface, specifically covering three dimensions: data persistence, dynamic management of defect types, and knowledge retrieval services. JSON format files are used as the persistent storage medium for Wiki data. During the loading phase, if the target file is detected as non-existent, the system will automatically initialize and generate an empty knowledge base structure; during the saving phase, the system automatically updates the last modification timestamp each time a data write operation is performed to ensure strict traceability of knowledge asset changes.

[0055] The system supports dynamic expansion and maintenance of defect types, allowing the addition of new defect types or the updating of descriptions for existing types. For each operation, the system automatically records its creation and update times. When adding a new reference image to a specified defect type, the system requires structured information including the image name, visual description, and labels. It also has a built-in duplicate checking mechanism that automatically intercepts and avoids redundant additions of identical samples, ensuring high-quality datasets.

[0056] The system provides multi-dimensional knowledge retrieval interfaces to support the four-stage detection process. In the coarse screening stage, it supports obtaining all defect types and their descriptions, or retrieving complete information for a specified type. In the fine screening stage, it supports accurately extracting and matching corresponding defect subtypes by analyzing the visual descriptions of images. Furthermore, the system has dynamic prompt word generation capabilities, automatically integrating defect type descriptions with typical manifestations to generate Wiki prompt words specifically for VLM model inference, thus providing strong prior knowledge support for the model's visual semantic analysis.

[0057] The following describes the specific implementation of the present invention in detail, taking the defect detection of the heating surface of the boiler water-cooled wall as an example. Those skilled in the art can implement it in engineering based on this description and specific configurations.

[0058] The system's reference dataset contains five defect type folders: cracking, pipe wall thinning, pipe bending deformation, pipe mechanical damage, and wear-resistant tile anomalies. The cracking category includes sub-types such as transverse cracks, longitudinal cracks, weld cracks, and fatigue cracks, with typical morphologies including fin burn-out cracks in the intermediate mixing header, longitudinal weld cracks in the front wall mixing header, transverse cracks in the fins near the observation holes in the rear wall, fatigue cracks in water-cooled wall tubes, and pipe weld cracks. The pipe wall thinning category includes sub-types such as wear thinning, corrosion thinning, and blow-damage thinning, with typical manifestations including wear thinning at the lower elbow of the cold ash hopper, corrosion thinning on the back side of the cold ash hopper tubes, corrosion pitting thinning in the vertical section of the water-cooled wall tubes, corrosion groove thinning in the water-cooled wall, and wear morphologies. Pipe bending deformation displays typical morphologies of pipe plastic deformation. Pipe mechanical damage includes typical damage such as mechanical damage to cold ash hopper tubes, mechanical damage to water-cooled wall tubes, and pipe abrasion. Abnormalities in wear-resistant tiles include deformation, holes, burns, blow-through, shifting, gaps, and other abnormal conditions.

[0059] The Wiki knowledge base uses JSON format for storage and contains structured descriptions of the five defect types mentioned above. Each defect type includes a description field, an image list, a created_at creation time, and an updated_at update time. Image information includes an image_name image name, a visual_description visual description, a tag list, and an added_at addition time. For example, the description of cracking is that it mainly manifests as various forms of cracks, including transverse cracks, longitudinal cracks, and network cracks. Cracks may appear in welds, heat-affected zones, or on the surface of the base material, and are usually caused by factors such as stress concentration, fatigue, and corrosion.

[0060] In the defect detection inference process, the system executes a four-stage inference process on the image to be detected. In the preliminary analysis stage, the VLM model analyzes the input image of the region to be tested and outputs a description of the image content, including equipment type, component structure, surface condition, and abnormal features. In the coarse candidate screening stage, based on the preliminary analysis results, the system selects the most likely candidate defect type from five defect types. In the fine matching stage, for each candidate defect type, the system retrieves reference images from the corresponding folder and determines the most matching target defect subtype through comparative analysis. In the final judgment stage, integrating all information, the VLM model outputs a final judgment, including whether a defect exists, the defect type, the confidence level, and the reasoning behind the judgment.

[0061] After the inspection is completed, the system displays a defect detection report and collects user feedback. Users view the inspection results through an interactive interface, including the defect type, confidence level, reasoning, and recommended measures, and can select "correct," "incorrect," or "uncertain" for evaluation. When the user confirms "correct," the system automatically performs incremental learning, copies the image of the region to be tested to the dataset folder corresponding to the defect type, automatically appends a number if a file with the same name exists, and updates the Wiki knowledge base, adding new sample descriptions and label information. When the user selects "incorrect" or "uncertain," the system records the feedback information but does not automatically update the knowledge base, leaving it for manual review and processing.

[0062] Through the above implementation process, the system has achieved a complete closed loop from image input to intelligent judgment, knowledge accumulation and system evolution, and has good detection capabilities and continuous learning capabilities.

[0063] In summary, the embodiments disclosed herein have at least the following technical effects: This invention implements a defect detection mechanism based on four-stage reasoning. Through a hierarchical reasoning process of preliminary analysis, coarse screening, fine screening, and final judgment, it fully leverages the visual understanding capabilities of a multimodal large-scale model and combines it with small-sample learning using reference samples. This significantly improves defect detection accuracy while also exhibiting good interpretability, capable of outputting detailed judgment reasons. Compared to traditional deep learning methods that require a large amount of training data, this invention achieves effective detection with only a small number of reference samples, greatly reducing data preparation costs.

[0064] A defect knowledge base management system was constructed to store and reuse detection experience in a structured format, supporting dynamic updates and expansion of knowledge. This solves the problem of insufficient knowledge accumulation in traditional methods and enables the system to have long-term memory capabilities. The knowledge base is stored in JSON format, facilitating querying, updating, and version management, forming a transferable detection knowledge asset.

[0065] A human-machine collaborative feedback learning mechanism was designed. By collecting user feedback and automatically updating the knowledge base, the system achieves continuous learning and evolution. Correct detections confirmed by users are automatically added to the dataset, continuously improving the system's detection capabilities and coverage. This incremental learning approach avoids the catastrophic forgetting problem of traditional models, enabling online accumulation of knowledge.

[0066] It possesses excellent interpretability; each stage of the four-stage reasoning process outputs intermediate results and reasoning basis, and the final judgment includes detailed explanations, enabling industrial personnel to understand and trust the test results, which is beneficial for practical application and promotion. Compared to the "black box" nature of deep learning models, the transparent reasoning process of this invention is more in line with the needs of industrial applications.

[0067] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting defects in boiler water-cooled walls based on multimodal intelligent agents, characterized in that, Includes the following steps: S1: Obtain the image of the region to be tested, perform preprocessing on the image of the region to be tested to obtain a preprocessed image; construct a first cue word containing the preprocessed image to drive the VLM model to perform global semantic parsing on the preprocessed image and extract and analyze the text; S2: Based on the analyzed text and combined with the preset defect knowledge base, select multiple candidate defect types with high confidence from the full range of defect types; S3: Retrieve a reference image that matches the candidate defect type from the preset dataset, construct a second cue word containing the preprocessed image of the region to be tested and the reference image, so as to drive the VLM model to perform cross-image visual feature comparison analysis on the preprocessed image of the region to be tested and the reference image to determine the target defect subtype; S4: Construct a comprehensive prompt word that includes the image of the region to be tested, the reference image, the analysis text, the candidate defect type, and the target defect subtype, so as to drive the VLM model to perform fusion reasoning on the comprehensive prompt word and generate a defect detection report.

2. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to claim 1, characterized in that, In step S1, the preprocessing step of the image of the region to be tested specifically includes: The image of the region to be tested is subjected to format checks, size adjustments, and file compression to meet the interface input requirements of the VLM model.

3. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to claim 1, characterized in that, The specific steps of step S2 include: Obtain feature description information for each defect type from the defect knowledge base; Calculate the text similarity between the analyzed text and each of the feature descriptions; Based on the text similarity, each defect type is ranked by confidence, and the top-ranked defect types are selected as candidate defect types.

4. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to claim 1, characterized in that, The preset dataset is organized into folders according to the defect type, with each defect type corresponding to an independent folder, and the independent folder contains only all reference images of that defect type.

5. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to claim 4, characterized in that, In step S3, the step of retrieving a reference image from the preset dataset that matches the candidate defect type specifically includes: Construct the image storage path and locate the corresponding target folder based on the name of the candidate defect type; Read the complete paths of all image files in the target folder that conform to the preset format, and generate a path list; The path list is input into the VLM model as input data for the cross-image visual feature comparison analysis.

6. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to claim 1, characterized in that, In step S4, The defect detection report includes: the determination of whether a defect exists, the defect type classification, the confidence level value, the reason for the judgment, and the recommended measures. It also includes complete reasoning chain information to support result display and feedback collection.

7. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to any one of claims 1 to 6, characterized in that, Also includes: Collect user evaluations of the defect detection reports and perform incremental learning of the defect knowledge base based on the evaluation type.

8. The boiler water-cooled wall defect detection method based on multimodal intelligent agents according to claim 7, characterized in that, The steps of collecting user evaluations of the defect detection reports and incrementally learning the defect knowledge base based on the evaluation types specifically include: When the user evaluates the defect detection report as correct, the corresponding defect type directory is automatically created or located, the image of the area to be tested is copied to the preset dataset and file name duplication processing is performed, and the visual description of the new sample and the tag information containing user confirmation and automatically added identification are synchronously updated to the defect knowledge base to achieve incremental learning. When a user evaluates the defect detection report as incorrect or uncertain, only the feedback information is recorded without updating the defect knowledge base, and the relevant samples are left for manual review.

9. A boiler water-cooled wall defect detection system based on multimodal intelligent agents, characterized in that, For implementing the boiler water-cooled wall defect detection method based on a multimodal intelligent agent according to any one of claims 1 to 8, the system includes an input module for acquiring an image of the area to be tested and a defect detection inference module, wherein the defect detection inference module includes: The preliminary analysis module is used to perform preprocessing operations on the image of the region to be tested and construct a first prompt word containing the preprocessed image of the region to be tested, so as to drive the VLM model to perform global semantic parsing on the preprocessed image of the region to be tested and extract the analysis text. The candidate screening module is used to select candidate defect types from multiple defect types based on the analyzed text and in conjunction with a preset defect knowledge base. The fine screening matching module is used to retrieve reference images corresponding to the candidate defect type, construct a second cue word containing the image of the region to be tested and the reference image, call the VLM model to perform comparative analysis, and determine the target defect subtype. The final judgment module is used to construct a comprehensive prompt word that includes the analysis text, the candidate defect type, the target defect subtype, and the reference image, so as to call the VLM model to generate the final defect detection report.

10. The boiler water-cooled wall defect detection system based on multimodal intelligent agents according to claim 9, characterized in that, It also includes a knowledge management and incremental learning module, which is configured to build and maintain a defect knowledge base and dataset, provide knowledge support for the defect detection reasoning module, and perform incremental learning based on user feedback on the defect detection report, so as to continuously improve the system's detection capabilities.