A segmentation and classification collaborative disease annotation method for mural multi-disease scene

Abstract

The application discloses a segmentation and classification collaborative disease annotation method for mural multi-disease scenes, and relates to the technical fields of cultural heritage digital protection and image processing. The method comprises the following steps: constructing a segmentation-classification collaborative model comprising a disease area segmentation module and a disease type identification module; generating a corresponding disease segmentation mask through the segmentation-classification collaborative model based on disease area prompt information, and extracting a disease boundary area to generate a disease candidate area image; identifying the disease type based on the input range of the disease candidate area constraint disease type identification module, outputting the corresponding disease type and probability distribution; structurally storing the disease segmentation result, and generating disease area and proportion statistical information based on the disease segmentation result. The application reduces the interference of mural background texture and non-disease areas on the disease type identification process by using the disease segmentation result to constrain the disease classification range, and realizes the collaborative processing between disease area positioning and disease semantic identification.

Description

Technical Field

[0001] This application relates to the field of digital preservation and image processing technology of cultural heritage, and in particular to a segmentation, classification and collaborative disease labeling method for murals with multiple diseases. Background Technology

[0002] As an important carrier of cultural heritage, murals are prone to various damages such as paint peeling, ground layer peeling, water stains, and cracks due to long-term natural weathering, environmental changes, and human factors. Accurately locating, identifying, and quantitatively analyzing mural damage is a crucial foundation for the protection, restoration, and digital management of cultural heritage.

[0003] Existing deep learning-based mural disease detection methods treat disease detection as an image classification task, identifying the type of disease on the entire mural. When extracting features from the entire mural, these methods extract texture, color, and morphological features from the image to obtain the probability distribution of various diseases (such as paint peeling, cracks, water stains, etc.) contained in the mural, and take the category with the highest probability as the disease type identification result.

[0004] However, mural surfaces typically contain intricate pigment patterns, lines, and traces of historical restoration. The texture features of these background areas are highly similar to those of the diseased areas in terms of low-level visual features such as color and gradient. When the classification network extracts features from the entire mural, the texture features of the background areas inevitably participate in the encoding process of the disease features, which can easily cause overlap between the disease features and the background features, leading to ambiguity in feature expression and ultimately affecting the accuracy of disease type identification. Summary of the Invention

[0005] Therefore, it is necessary to provide a segmentation, classification, and collaborative disease labeling method for murals with multiple diseases, addressing the aforementioned technical issues.

[0006] The following technical solution is adopted in this specification: Obtain information indicating damage to the murals; Based on the disease indication information, determine the local disease analysis image block to be processed; The local disease analysis image block is binarized and segmented by the disease region segmentation module to generate a disease segmentation mask and determine the disease candidate region image block; the texture features, color features and morphological features of the disease candidate region image block are extracted by the disease type recognition module to obtain the disease category contained in the disease candidate region image block and the corresponding probability distribution, and the disease category with the highest probability is taken as the disease type of the disease candidate region image block; Based on the disease type identification results, corresponding correction strategies are applied to correct different disease types to complete the labeling of disease areas.

[0007] Furthermore, corresponding correction strategies are applied to different types of diseases, including: When the disease type is linear disease, the skeleton of the disease segmentation mask is extracted, the broken area is connected according to the distance between the endpoints of the skeleton, and the abnormal width area is constrained according to the preset line width range to complete the labeling of the linear disease. When the disease type is a planar disease, the holes inside the disease segmentation mask are filled, the boundary of the disease segmentation mask is smoothed, and small connected regions with an area lower than a preset threshold are filtered to complete the labeling of the planar disease. When the confidence level of the disease type identification result is lower than the preset threshold, the preset structural constraint rules are invoked according to the corresponding disease type to perform adaptive correction processing on the disease segmentation result.

[0008] Further, based on the disease indication information, the local disease analysis image block to be processed is determined, including: Acquire the original image and the displayed image of the mural, and establish the coordinate mapping relationship between the original image and the displayed image; Based on the disease indication information, the image region to be processed in the displayed image is determined, the image region to be processed is cropped, and the cropped image region to be processed is mapped to the original image resolution space through the coordinate mapping relationship to obtain a local disease analysis image block.

[0009] Furthermore, the determination of the candidate disease region image patch includes: The minimum bounding boundary region corresponding to the diseased area is obtained based on the disease segmentation mask, and a preset range is expanded outside the boundary region to generate a diseased candidate region image.

[0010] Furthermore, the disease area segmentation module is constructed based on a pre-trained visual segmentation network, and the disease type identification module is constructed based on a deep neural network.

[0011] Furthermore, the marking of the diseased areas includes: Auxiliary annotation masks for disease areas are generated by using regional auxiliary annotation methods for area-like diseases and linear auxiliary annotation methods for linear diseases. The auxiliary annotation mask is fused with the disease segmentation mask to complete the integrity annotation of complex disease areas.

[0012] Furthermore, the large-size mural image is divided into multiple local area windows according to the preset window size and window step size by the image local area reasoning method, and then the disease prompt information of each local area window is obtained.

[0013] Furthermore, the disease warning information includes positive warning information for diseased areas and negative warning information for non-diseased areas.

[0014] Furthermore, the types of diseases include linear diseases and surface diseases; the surface diseases include pigment layer peeling, ground layer peeling, and water stains; the linear diseases include fissure diseases.

[0015] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: By using a disease segmentation mask to establish a spatial constraint relationship between disease regions and disease type identification, the disease type identification process is confined to the disease candidate region, reducing the participation of non-disease region texture features in the classification process, reducing feature aliasing, and improving the accuracy of disease type identification. In addition, the disease categories and confidence scores output by the classification module guide the segmentation module to automatically call different methods for morphological correction based on the inherent morphology of the disease. This makes the cracks connected by a linear skeleton and the detached parts filled smoothly by a planar filling. This creates a two-way collaborative closed loop between the segmentation module and the classification module, where the segmentation constrains the classification range and the classification guides the segmentation correction. This significantly improves the consistency of disease area labeling in multi-disease scenarios. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This document presents a flowchart illustrating a segmentation, classification, and collaborative disease labeling method for murals with multiple diseases, as provided in this specification. Figure 2 This is a schematic diagram of the auxiliary labeling process for areas with planar defects provided in this manual; Figure 3 This is a schematic diagram of the auxiliary annotation process for linear diseases provided in this manual. Figure 4 This is a schematic diagram of the collaborative processing flow for disease area segmentation and disease type identification provided in this manual. Figure 5 This is a schematic diagram illustrating the differentiated texture encoding for different disease types provided in this manual; Figure 6 This is a schematic diagram illustrating the reasoning process for a local area of ​​the ultra-large mural image provided in this manual. Figure 7 A schematic diagram of the compact storage structure for the disease segmentation results provided in this manual; Figure 8 This is a schematic diagram of the structure of the statistical analysis results of the diseases provided in this manual. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0019] Furthermore, existing disease analysis methods lack auxiliary annotation mechanisms for different disease morphologies, and there is a lack of differentiated annotation methods for linear diseases such as cracks and planar diseases such as flaking. At the same time, for ultra-large mural images, existing methods struggle to balance overall reasoning ability with the ability to identify local disease details. Moreover, existing disease analysis methods typically lack the ability to perform structured disease statistics and quantitative analysis, making it difficult to automatically generate information on the area and proportion of various diseases, which is detrimental to long-term disease monitoring and digital management in cultural heritage protection.

[0020] Therefore, this invention provides a segmentation-classification collaborative disease labeling method for murals with multiple diseases. By constructing a collaborative processing mechanism between disease area segmentation and disease type identification, it achieves integrated processing of mural disease area location, disease type identification, disease segmentation result feedback correction, disease statistical analysis, and structured management of disease results. The method includes the following steps:

[0021] S101: Obtain the original image of the mural, preprocess the original image of the mural, and establish the coordinate mapping relationship between the displayed image and the original image.

[0022] S102: Construct a collaborative model for multi-disease segmentation and classification, including a disease region segmentation module and a disease type identification module. The disease region segmentation module is used to generate a disease segmentation mask, and the disease type identification module is used to generate disease candidate regions based on the disease segmentation mask and output the corresponding disease type information.

[0023] S103: Obtain the disease area prompt information input by the user, and map the prompt information to the original image resolution space based on the coordinate mapping relationship, and input it into the disease area segmentation module to generate a disease segmentation mask.

[0024] S104: Extract disease candidate region image blocks based on the disease segmentation mask, perform disease type recognition processing based on the disease candidate region image blocks, and output the corresponding disease type information and confidence information.

[0025] S105: Perform feedback correction processing on the disease segmentation results based on the disease type identification results; wherein, when the disease type corresponds to linear disease, perform skeleton continuity constraint, break connection processing or line width constraint processing on the disease segmentation results; when the disease type corresponds to planar disease, perform hole filling, boundary smoothing processing or small connected region filtering processing on the disease segmentation results.

[0026] S106: Perform auxiliary annotation processing; including linear trajectory auxiliary annotation for linear diseases and region closure auxiliary annotation for area diseases, and integrate the auxiliary annotation results with the disease segmentation results.

[0027] S107: Perform local area segmentation reasoning processing on ultra-large mural images, and map the results of local disease analysis back to the original image space for fusion.

[0028] S108: The disease segmentation results corresponding to different disease types are visualized using a differentiated texture encoding method, and the disease segmentation results are stored in a structured manner to generate disease statistical reports corresponding to various disease area information and percentage information.

[0029] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart illustrating a segmentation, classification, and collaborative disease labeling method for murals with multiple diseases, as described in this specification. The method includes the following steps: Step 1: Image Loading and Preprocessing The original image of the mural is acquired and preprocessed, including image scaling, color normalization, and noise suppression.

[0031] For ultra-large mural images, the system generates display images for interactive processing and establishes a coordinate mapping relationship between the display images and the original images to achieve a correspondence between low-resolution display interactive processing and high-resolution disease analysis.

[0032] Let the original image size be... The preset display side length threshold is ,when The scaling ratio is calculated using the following formula: The displayed image sizes are as follows: Display coordinates of the points in the image Coordinates of the original image The following mapping relationship exists between them: By establishing the coordinate mapping relationship, the spatial correspondence between the processing results of local areas of ultra-large mural images and the diseased areas at the original resolution is realized. This reduces the consumption of interactive processing computing resources while maintaining the consistency of diseased area positioning between the displayed image space and the original image space.

[0033] Step 2: Construction of a Collaborative Model for Multi-Disease Segmentation and Classification A diseased area segmentation module was constructed based on a pre-trained visual segmentation network and trained using a mural disease dataset.

[0034] The mural disease dataset includes mural images and corresponding disease area masks. After preprocessing the mural images, they are input into a pre-trained visual segmentation network, enabling the disease area segmentation module to learn the texture and morphological features of the mural diseases and achieve disease area localization.

[0035] Furthermore, the parameters of the pre-trained visual segmentation network are optimized to improve the adaptability of the disease area segmentation module to complex mural disease boundaries and multi-disease scenarios.

[0036] To achieve semantic type recognition of diseased areas, a disease type recognition module is constructed based on a deep neural network. The module is trained using image patches of candidate diseased areas as input and disease type labels as supervisory information to output the probability distribution of the corresponding disease category. The disease types include at least two of the following: pigment layer peeling, ground layer peeling, water stains, and cracks.

[0037] Furthermore, the disease type identification module extracts the texture, color, and morphological features of the diseased area to distinguish between different disease types.

[0038] Construct a segmentation-classification collaborative model that includes a disease area segmentation module and a disease type identification module. The working mechanism of the segmentation-classification collaborative model is as follows: Figure 4 As shown, the disease segmentation mask output by the disease region segmentation module is used to generate disease candidate region image blocks and constrain the input range of the disease type recognition module.

[0039] By utilizing disease segmentation results to limit the scope of disease type identification and reducing the involvement of texture features from non-disease areas in the disease type identification process, this invention reduces feature aliasing caused by complex mural backgrounds. This allows the disease type identification process to focus on the diseased area itself, achieving collaborative processing between disease area localization and disease semantic recognition. Unlike disease analysis methods where segmentation and classification are independent, this invention achieves joint optimization of disease spatial localization information and disease semantic information through bidirectional collaborative constraints between disease area segmentation results and disease type identification results.

[0040] Step 3: Localized disease segmentation driven by disease area indicators The system obtains the disease area prompt information input by the user, which includes positive prompt information for diseased areas and negative prompt information for non-diseased areas.

[0041] The system performs local area defect segmentation processing based on the original image area corresponding to the current display area, in order to reduce the computational resource consumption caused by the overall inference of ultra-large mural images.

[0042] Specifically, it includes the following steps: (1) Based on the coordinate mapping relationship established in step one, the coordinates of the prompt information in the displayed image are mapped to the original image resolution space; (2) Determine the image area to be processed according to the location corresponding to the prompt information, and crop the image area to be processed to generate a local disease analysis image block; (3) Input the local disease analysis image block and the corresponding prompt information into the disease area segmentation module to generate the corresponding disease segmentation mask; (4) Post-processing of the disease segmentation mask, including disease area connection processing and connected region filtering processing, to improve the continuity of disease area boundaries and reduce the impact of local noise areas on disease segmentation results.

[0043] Step 4: Disease Type Identification and Treatment Based on the disease segmentation mask generated in step three, the system automatically performs disease type identification processing.

[0044] Specifically, it includes the following steps: (1) Determine the boundary region corresponding to the disease area according to the disease segmentation mask, and generate a disease candidate region image block from the original image based on the boundary region; (2) Perform boundary expansion processing on the disease candidate area image to preserve the local context information around the disease area; (3) Preprocess the disease candidate area image and input it into the disease type identification module. The disease type identification module outputs the probability distribution corresponding to each disease category. (4) Based on the probability distribution of each disease category, determine the disease category with the highest probability value as the disease type identification result, and generate the corresponding disease type information and confidence information.

[0045] Among them, the disease segmentation mask output by the disease area segmentation module is used to constrain the input range of the disease type recognition module, so that the disease type recognition process focuses on the disease area itself, thereby reducing the interference of mural background texture, non-disease areas and complex mural patterns on the disease type recognition process, and realizing the collaborative processing between disease area localization and disease semantic recognition.

[0046] Step 5: Feedback and Correction of Disease Segmentation Results Based on the disease type identification results generated in step four, the system performs structural constraint correction processing on the disease segmentation results to improve the continuity of the boundary of complex disease areas, structural integrity, and consistency of disease segmentation results.

[0047] Specifically, the steps include the following: (1) Obtain the disease type results and corresponding confidence information output by the disease type identification module; (2) Determine the corresponding disease morphology type based on the disease type results; among them, when the disease type is fissure disease, determine that the current disease area belongs to linear disease; when the disease type is pigment layer peeling, ground layer peeling or water stain disease, determine that the current disease area belongs to surface disease. (3) When the disease type identification result corresponds to a linear disease, perform linear structure constraint correction processing on the disease segmentation mask; The linear structure constraint correction process includes: Extract the skeleton of the disease segmentation mask; connect the fracture areas according to the distance between the endpoints of the skeleton; perform line width constraint processing on abnormal width areas according to the preset line width range specified by national standards; the line width constraint includes: maintaining the preset line width in areas with relatively wide cracks to ensure that the line width does not change due to changes in the width of the crack.

[0048] (4) When the disease type identification result corresponds to a planar disease, planar structure constraint correction processing is performed on the disease segmentation mask; The planar structure constraint correction process includes: The holes inside the defect-segmenting mask should be filled. Perform boundary smoothing on the boundaries of the diseased area; Perform filtering on small connected regions whose area is below a preset threshold; (5) When the automatic prediction type is incorrect, it can be manually modified. The result will be saved after modification, and the model will continue to learn. The system's disease type prediction function will become more and more accurate with use. (6) Output the corrected disease segmentation results.

[0049] By performing structural constraint correction processing on the disease segmentation results based on the disease type identification results, synergistic optimization between disease spatial region constraints and disease semantic feature constraints is achieved, thereby improving the boundary continuity and structural consistency of disease areas in complex murals. Since different disease types have different spatial morphological characteristics, using disease type identification results to perform corresponding structural constraint correction on the disease segmentation results can improve the continuity of disease boundaries and the structural consistency of disease areas.

[0050] Step Six: Auxiliary Annotation Processing To address the problem that complex disease boundary areas and linear disease areas are difficult to accurately segment using prompt information, this invention provides an auxiliary annotation processing mechanism, including a region auxiliary annotation method for area-like diseases and a line auxiliary annotation method for linear diseases.

[0051] For area-like defects, a closed region is generated based on the region boundary input by the user, and a corresponding region mask is generated based on the closed region. During the generation process, the coordinates corresponding to the region boundary are mapped to the original image resolution space, and a defect region mask is generated based on the region boundary.

[0052] For linear diseases, the corresponding linear disease region is generated based on the linear trajectory input by the user, and a linear disease mask is generated in combination with the preset region width. During the generation process, the coordinates corresponding to the linear trajectory are mapped to the original image resolution space, and a disease region mask is generated based on the linear trajectory.

[0053] Furthermore, the generated auxiliary annotation mask is fused with the disease segmentation mask generated by the disease region segmentation module to improve the annotation integrity and boundary continuity of complex disease regions.

[0054] Step 7: Local Area Reasoning Processing of Extra-Large Mural Images Because mural images typically have ultra-high resolution, directly performing disease inference on the entire mural image can easily lead to excessive consumption of computing resources and affect the ability to express details in local diseased areas.

[0055] Therefore, this invention employs a localized, block-based reasoning approach to analyze defects in ultra-large mural images, such as... Figure 6 As shown, by dividing the mural image into multiple local area windows and performing disease segmentation processing within each local area window, disease detection of ultra-large mural images can be achieved.

[0056] Assume the original mural image size is The system generates multiple local area windows according to the preset window size and window step size, and each local area window covers the corresponding local area of ​​the mural.

[0057] For each local region window, the system performs the following processing: (1) Based on the position of the local area window, cut out the corresponding local disease analysis image block from the original mural image; (2) Generate prompt information for disease area detection in the local disease analysis image block, and input it into the disease area segmentation module to generate the corresponding local disease segmentation result; (3) Post-process the results of local disease segmentation and generate corresponding local disease area masks; (4) Map the local disease area mask corresponding to each local area window back to the original mural image space, and perform fusion processing to generate the disease distribution result corresponding to the complete mural image.

[0058] By fusing local area block reasoning with global disease results, the computational resources consumed for overall reasoning of ultra-large mural images are reduced while maintaining the ability to identify details in local disease areas, thus enabling disease analysis and distribution detection of ultra-large mural images.

[0059] Step 8: Visualization, structured storage, and statistical analysis of disease results. To achieve differentiated display between different disease types, this invention employs the following method: Figure 5 The differentiated texture encoding method shown provides a visual representation of the disease segmentation results.

[0060] Among them, such as Figure 3 As shown, linear diseases are encoded using a linear texture structure, generating corresponding linear textures by extracting the skeletal structure of the diseased area; for example... Figure 2 The surface-shaped diseases shown are encoded using a region-filled texture structure. Different disease types are distinguished by the boundary of the disease area and the internal texture filling.

[0061] Different disease types correspond to different texture structure codes, and the corresponding texture structures are superimposed on the disease segmentation area to achieve a visual representation of disease area distribution information in multi-disease scenarios.

[0062] Furthermore, to reduce the storage resource consumption of disease segmentation results, this invention employs the following... Figure 7 The compact storage method shown manages the disease segmentation results.

[0063] For disease segmentation masks, the boundary region corresponding to the disease area is first calculated; then, only the local disease mask data within the boundary region, as well as the corresponding disease type information, prompt information, and spatial location information, are stored.

[0064] When loading the disease segmentation results, the local disease mask data is mapped back to the original image space based on the spatial location information corresponding to the boundary region in order to reconstruct the complete disease segmentation results.

[0065] Furthermore, the system automatically generates disease statistics based on all disease segmentation results, such as... Figure 8 As shown.

[0066] Let the total area of ​​the mural images be: Here, H and W represent the height and width of the mural image, respectively.

[0067] For disease type t, the corresponding disease area is denoted as . The percentage of diseased area is: The total area of ​​all diseased areas is recorded as: The overall percentage of diseases is: The system generates a structured disease statistical report based on the above statistical results. The disease statistical report includes area information, percentage information and overall disease statistics for various types of diseases, which are used for quantitative analysis, long-term monitoring and digital management of mural diseases.

[0068] based on Figure 1 The segmentation, classification, and collaborative disease labeling method for murals with multiple diseases, as shown, has the following advantages compared to existing technologies: First, considering the diverse types of damage, complex background textures, rich color variations, and irregular boundaries of mural cultural heritage in preservation scenarios, this invention constructs a segmentation-classification collaborative damage labeling method for multi-damage scenarios. Compared to general image segmentation or general defect detection methods, this invention can better adapt to the unique characteristics of mural damage in terms of texture structure, color distribution, and spatial morphology, reducing the interference of complex backgrounds on the damage identification process, thereby improving the accuracy and stability of mural damage labeling and enhancing the applicability of the method in cultural relic preservation scenarios.

[0069] Second, this invention utilizes a disease segmentation mask to establish a spatial constraint relationship between disease regions and disease type identification, thereby limiting the disease type identification process to the disease candidate region, reducing the participation of non-disease region texture features in the classification process, thus reducing feature aliasing and improving the reliability of disease type identification in multi-disease scenarios.

[0070] Third, this invention constructs a collaborative closed-loop processing mechanism of "disease segmentation—candidate region generation—disease type identification—segmentation result feedback correction." By introducing the disease type identification result to impose a reverse constraint on the segmentation result, the semantic information of the disease can participate in the optimization of the disease spatial structure. Compared with the independent processing method of segmentation and classification, this invention achieves the collaborative optimization of spatial information and semantic information, improving the boundary consistency and identification accuracy of complex disease areas.

[0071] Fourth, this invention provides an auxiliary annotation method for linear and area diseases, and integrates the auxiliary annotation results with the automatic segmentation results, enabling the system to adapt to the annotation needs of slender crack diseases, irregular peeling diseases, and complex boundary areas, thereby improving the consistency between manual interactive annotation and automatic recognition results.

[0072] Fifth, this invention establishes a coordinate mapping relationship between the displayed image and the original image for ultra-large mural images, and combines a local area block reasoning mechanism to reduce the overall computing resource consumption while maintaining the positioning accuracy and detail expression ability of the diseased area in the original resolution space, which is suitable for practical application scenarios of high-resolution mural images.

[0073] Sixth, the present invention adopts a compact storage method for disease segmentation results, storing only the local mask data within the disease boundary area and its corresponding disease type information and spatial location information, and reconstructing the complete disease segmentation results when needed, thereby reducing data redundancy and improving the storage efficiency and recovery efficiency of ultra-large mural disease data.

[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A segmentation, classification, and collaborative disease labeling method for murals with multiple diseases, characterized in that, include: Obtain information indicating damage to the murals; Based on the disease indication information, determine the local disease analysis image block to be processed; The local disease analysis image block is binarized and segmented by the disease region segmentation module to generate a disease segmentation mask and determine the disease candidate region image block; the texture features, color features and morphological features of the disease candidate region image block are extracted by the disease type recognition module to obtain the disease category contained in the disease candidate region image block and the corresponding probability distribution, and the disease category with the highest probability is taken as the disease type of the disease candidate region image block; Based on the disease type identification results, corresponding correction strategies are applied to the disease segmentation mask areas of different disease types to complete the labeling of disease areas.

2. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, For different types of diseases, corresponding correction strategies are applied to the areas within the disease segmentation mask for correction, including: When the disease type is linear disease, the skeleton of the disease segmentation mask is extracted, the broken area is connected according to the distance between the endpoints of the skeleton, and the abnormal width area is constrained according to the preset line width range to complete the labeling of the linear disease. When the disease type is a planar disease, the holes inside the disease segmentation mask are filled, the boundary of the disease segmentation mask is smoothed, and small connected regions with an area lower than a preset threshold are filtered to complete the labeling of the planar disease. When the confidence level of the disease type identification result is lower than the preset threshold, the preset structural constraint rules are invoked according to the corresponding disease type to perform adaptive correction processing on the disease segmentation result.

3. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, Based on the disease indication information, the local disease analysis image blocks to be processed are determined, including: Acquire the original image and the displayed image of the mural, and establish the coordinate mapping relationship between the original image and the displayed image; Based on the disease indication information, the image region to be processed in the displayed image is determined, the image region to be processed is cropped, and the cropped image region to be processed is mapped to the original image resolution space through the coordinate mapping relationship to obtain a local disease analysis image block.

4. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, The determination of the candidate disease region image patch includes: The minimum bounding boundary region corresponding to the diseased area is obtained based on the disease segmentation mask, and a preset range is expanded outside the boundary region to generate a diseased candidate region image.

5. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, The disease area segmentation module is built based on a pre-trained visual segmentation network, and the disease type identification module is built based on a deep neural network.

6. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, The marking of the diseased areas includes: Auxiliary annotation masks for disease areas are generated using a region-assisted annotation method for area-like diseases and a line-assisted annotation method for line-like diseases. The auxiliary annotation mask is fused with the disease segmentation mask to complete the integrity annotation of complex disease areas.

7. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, The large-size mural image is divided into multiple local area windows according to the preset window size and window step size by the image local area reasoning method, and then the disease prompt information of each local area window is obtained.

8. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, The disease warning information includes positive warning information for diseased areas and negative warning information for non-diseased areas.

9. The method for segmentation, classification, and collaborative disease labeling in mural multi-disease scenarios as described in claim 1, characterized in that, The disease types include linear diseases and surface diseases; the surface diseases include pigment layer peeling, ground layer peeling, and water stains; the linear diseases include fissure diseases.

Images