Dam hidden danger classification data set construction method based on semi-automatic active learning

Abstract

The invention relates to the technical field of data processing and artificial intelligence, in particular to a dam hidden danger classification data set construction method based on semi-automatic active learning, and the method comprises the steps: calling a dam safety analysis component to generate a multi-dimensional analysis abstract for unlabeled multi-modal data; screening out records with content contradictions and information sparseness, and labeling the records by experts; dynamically updating the instruction set according to the expert label and carrying out generative cross-modal expansion; organizing all the marked hidden danger records and the expanded hidden danger records into a structured data unit containing positive, negative and difficult samples, and constructing a domain knowledge index through sample comparison; and continuously accumulating the structured data units in the closed loop step, and finally constructing a dam hidden danger classification data set. According to the method, the domain knowledge index constructed in the circulation serves as a new round of information retrieval tool, continuous accumulation of knowledge is achieved, and the efficiency and quality of data set construction are improved.

Description

Technical Field

[0001] This invention relates to the fields of data processing and artificial intelligence technology, specifically to a method for constructing a dam hazard classification dataset based on semi-automatic active learning. Background Technology

[0002] During the use of critical infrastructure such as dams, a large amount of data, including inspection records and monitoring reports, is generated. Effectively organizing, classifying, and managing the hidden information on potential hazards such as deformation, seepage, and cracks contained within this data is the core data foundation for ensuring facility safety and making risk predictions. Traditional data management methods mainly rely on domain experts for manual reading, screening, and archiving, which is inefficient and results in poor consistency and reusability of data assets.

[0003] To address this issue, existing technologies primarily employ artificial intelligence for assisted processing. For example, large language models are used to perform preliminary data summarization, followed by expert review. While this human-machine collaborative model improves efficiency to some extent, its core bottleneck lies in how to efficiently construct a high-quality "labeled dataset" for a specific classification task for the AI ​​model.

[0004] A model's domain expertise requires training on a dataset containing tens of thousands of precisely labeled samples. Traditional dataset construction relies entirely on manual annotation by domain experts, which is extremely costly and time-consuming. Existing AI-assisted methods also typically lack intelligent sample selection mechanisms, failing to proactively identify and filter out the most valuable, ambiguous, or contradictory "difficult samples" for expert guidance, thus hindering the efficient use of expert resources.

[0005] In summary, a common problem with existing technologies in constructing datasets for specific domain classification tasks is the lack of a proactive and efficient data selection and labeling strategy, making it difficult to quickly construct high-quality datasets that can significantly improve the performance of downstream classification models while keeping costs under control.

[0006] To address this, a method for constructing a dam hazard classification dataset based on semi-automatic active learning is proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a method for constructing a dam hazard classification dataset based on semi-automatic active learning, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for constructing a dam hazard classification dataset based on semi-automatic active learning includes:

[0010] Establish an instruction set that includes query and record-derived instructions; invoke the dam safety analysis component, process the unlabeled multimodal data according to the instruction set, and generate a multidimensional analytical summary; filter the multidimensional analytical summary, extract records with contradictory content and sparse information, and hand them over to experts for annotation to obtain annotated hazard records;

[0011] Based on the marked hazard records, the instruction set is dynamically updated, and the optimized instruction set is used to perform generative cross-modal expansion on the marked hazard records to obtain expanded hazard records; the marked hazard records and expanded hazard records are organized into structured data units containing positive, negative and difficult samples;

[0012] By comparing samples and encoding instructions, a domain knowledge index is constructed using the structured data units.

[0013] The domain knowledge index is used as a new information retrieval tool. The closed-loop steps of multidimensional parsing summary generation, record filtering, generative cross-modal expansion and index optimization are repeatedly executed until the domain knowledge index performance reaches the preset index. The structured data units accumulated in the loop are used as the dam hazard classification dataset.

[0014] Preferably, the process of generating the multidimensional parsing summary specifically includes: for unlabeled multimodal data, the dam safety analysis component generates a key entity list and a semantic summary for the text portion of the record according to the instruction set, and generates a target entity list containing the identified objects and spatial locations for the image portion of the record; the dam safety analysis component performs preset statistical calculations on the identification confidence scores of the entities contained in the key entity list and the target entity list to obtain an identification confidence score, and counts the total number of entities to quantify the information richness score; finally, the key entity list, semantic summary, target entity list, identification confidence score, and information richness score are combined together to form a structured record as the multidimensional parsing summary, and a one-to-one mapping relationship is established with the original data record.

[0015] Preferably, the process of filtering the multidimensional parsing summary specifically includes: calling a pre-trained image description generation model, inputting the target entity list of the image part into the image description generation model, and generating structured image content description text; using the Sentence-BERT model to calculate the corresponding feature vectors for the semantic summary of the text part and the image content description text, and determining the content contradiction degree based on these two feature vectors. The rule for determining the content contradiction degree is: the lower the semantic similarity between the two feature vectors, the higher the content contradiction degree; setting an upper threshold for content contradiction degree, a lower threshold for recognition confidence, and a lower threshold for information richness; and determining the original data records corresponding to the multidimensional parsing summary that fail to simultaneously meet all of the following conditions as records to be filtered and extracted for output by experts for subsequent annotation processing: the content contradiction degree is not higher than the upper threshold for content contradiction degree; the recognition confidence score is not lower than the lower threshold for recognition confidence; and the information richness score is not lower than the lower threshold for information richness.

[0016] Preferably, the process of dynamically updating the instruction set specifically includes: analyzing the classification tags and descriptive text given by experts to the marked hazard records, and extracting high-frequency domain terms and unique expression paradigms; pre-setting instruction templates containing keyword placeholders, wherein the instruction templates fixedly include sentence transformation templates and scene detail addition templates; filling the extracted high-frequency domain terms as keywords into the keyword placeholders of the instruction templates, and combining them to generate new record-derived instructions; incorporating the new record-derived instructions into the instruction set, and increasing the calling priority of the new record-derived instructions in subsequent generative cross-modal expansion tasks.

[0017] Preferably, the generative cross-modal augmentation process specifically includes: for the marked hazard records that are determined to be positive samples, calling an image content restoration model, removing the hazard visual features from the image portion based on the hazard entity location information in the record, generating a counterfactual negative sample, and using the counterfactual negative sample as the augmented hazard record; for the marked hazard records that are determined to be negative samples, calling a text-guided image editing model, using expert descriptive text in the positive sample as semantic guidance, synthesizing the corresponding hazard visual features in the image portion of the negative sample, generating an enhanced positive sample, and using the enhanced positive sample as the augmented hazard record; difficult samples are not augmented.

[0018] Preferably, the process of organizing the structured data unit specifically includes: creating a data structure containing multiple fields for the marked hazard records and expanded hazard records; the fields include a unique identifier for the data record, expert-marked correct classification labels as positive samples, a set of explicitly excluded incorrect classification labels as negative samples, and expert-marked confidence scores as difficult samples; wherein, the set of negative samples is provided by experts during the marking process, and the confidence score is the expert's assessment of the authenticity of the marking results.

[0019] Preferably, constructing the domain knowledge index specifically includes: generating high-dimensional feature vectors for structured data units; adjusting vector positions in the feature vector space by comparing data records, with the adjustment rule being: minimizing the vector distance between positive samples and corresponding expanded hidden danger records, while maximizing the vector distance between positive samples and negative samples; the domain knowledge index is constructed as a vector database supporting efficient vector retrieval, the content of which is the high-dimensional feature vectors of all structured data units; the instruction encoding refers to converting externally input text query instructions into query vectors through a text encoder identical to the data record encoder; subsequent information retrieval is accomplished by calculating the similarity between the query vector and the vectors of all data units in the vector database.

[0020] Preferably, the process of using the domain knowledge index as a new round of information retrieval tool specifically includes: in the next cycle, before generating a multidimensional analytical summary for new unlabeled multimodal data, using the domain knowledge index to retrieve the neighboring data records most similar to the unlabeled multimodal data in the labeled hazard records; extracting the classification labels and metadata features of the neighboring data records as context information, and dynamically generating temporary query instructions; using the temporary query instructions to call the dam safety analysis component to generate a new multidimensional analytical summary.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] 1. To address the problem of passive expert intervention and the inability to intelligently assess data value in existing technologies, this invention proposes a proactive data selection strategy. By identifying "content contradictions" and "information sparsity" among multimodal data, this method can proactively filter out the key samples with the most ambiguous content and the highest information entropy from massive amounts of unlabeled data. This ensures that scarce expert resources are precisely allocated to solving the most challenging problems, thereby maximizing the improvement of subsequent classification performance with minimal annotation costs and solving the problem of how to efficiently expand high-quality labeled data.

[0023] 2. To address the bottleneck issues of existing methods relying on large-scale pre-labeled datasets and high startup costs, this invention constructs a closed-loop self-optimizing method. This method internalizes the knowledge from each round of expert annotation into a searchable domain knowledge index and applies it as a tool to the next round of data processing. This progressive knowledge accumulation and self-evolution mechanism enables starting with very little initial labeled data and becoming increasingly intelligent as more data is processed, thereby gradually reducing reliance on expert intervention and solving the efficiency and cost challenges of building professional domain datasets from scratch.

[0024] 3. Compared to general knowledge base construction methods that only focus on relationship discovery, this invention specifically optimizes the dataset construction process for "classification tasks." By organizing data into refined structured data units containing positive, negative, and difficult samples, and by adjusting the vector space using contrastive learning, not only is the intrinsic quality of the final dataset improved, but the downstream classification model trained using this dataset also ensures stronger robustness and generalization ability, thus solving the problem of mismatch between existing technical processes and classification application goals. Attached Figure Description

[0025] Figure 1 This is an overall flowchart of the method for constructing a dam hazard classification dataset based on semi-automatic active learning, provided in an embodiment of the present invention.

[0026] Figure 2 A schematic diagram of the multidimensional parsing summary filtering process for the dam hazard classification dataset construction method based on semi-automatic active learning provided in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram illustrating the conceptual architecture of a method for constructing a dam hazard classification dataset based on semi-automatic active learning, as provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1-3 The specific implementation steps of the method proposed in this invention include:

[0030] Establish an instruction set that includes query and record-derived instructions; invoke the dam safety analysis component, process the unlabeled multimodal data according to the instruction set, and generate a multidimensional analytical summary; filter the multidimensional analytical summary, extract records with contradictory content and sparse information, and hand them over to experts for annotation to obtain annotated hazard records;

[0031] Based on the marked hazard records, the instruction set is dynamically updated, and the optimized instruction set is used to perform generative cross-modal expansion on the marked hazard records to obtain expanded hazard records; the marked hazard records and expanded hazard records are organized into structured data units containing positive, negative and difficult samples;

[0032] By comparing samples and encoding instructions, a domain knowledge index is constructed using the structured data units.

[0033] The domain knowledge index is used as a new information retrieval tool. The closed-loop steps of multidimensional parsing summary generation, record filtering, generative cross-modal expansion and index optimization are repeatedly executed until the domain knowledge index performance reaches the preset index. The structured data units accumulated in the loop are used as the dam hazard classification dataset.

[0034] Example 1

[0035] This embodiment provides a specific application of the method for constructing a dam hazard classification dataset based on semi-automatic active learning.

[0036] A typical application scenario of this embodiment is as follows: To process its accumulated large amount of unlabeled multimodal data containing inspection photos and text descriptions, Reservoir Management Bureau A introduced the dam hazard classification dataset construction method based on semi-automatic active learning described in this invention. Please refer to... Figure 1 The specific implementation steps of this invention are as follows:

[0037] The overall conceptual framework of this invention can be found in [reference needed]. Figure 3 .

[0038] Furthermore, an instruction set including query and record-derived instructions is established; the dam safety analysis component is invoked to generate multidimensional analytical summaries of unlabeled multimodal data based on the instruction set; the multidimensional analytical summaries are filtered, and records with contradictory content and sparse information are extracted and submitted to experts for annotation to obtain annotated hazard records. The specific process is as follows:

[0039] In the initial stage, this method processes 10,000 unlabeled multimodal data points. Simultaneously, an initial instruction set is established, containing basic text analysis instructions such as "extract inspection locations, times, and equipment status from the report" and visual information extraction instructions such as "identify cracks, seepage points, and corrosion in the image." A multimodal dam safety analysis component based on the Transformer architecture is invoked to process these 10,000 records according to the instruction set. For one record, its text portion is processed to generate a list of key entities ["spillway," "sidewall"] and a semantic summary: "The report describes a vertical crack approximately 0.3 mm wide existing in the spillway sidewall." Its image portion is processed to generate a list of target entities [(crack, [x1, y1, x2, y2]), (concrete, [x3, y3, x4, y4])], where [x1, y1, x2, y2] are the coordinates of the crack in the image, and [x3, y3, x4, y4] are the coordinates of the concrete in the image. The dam safety analysis component assigned an average identification confidence score of 0.92 to the identified entities and quantified the information richness score to 4.0 based on the total number of the four identified entities. This information collectively constitutes the multidimensional parsed summary of this record.

[0040] By generating a structured, multidimensional parsing summary for each piece of unlabeled multimodal data, including key entities, semantic summaries, target entities, and quantified recognition confidence and information richness scores, this invention achieves deep, multidimensional parsing and quantification of raw unstructured data. This refined metadata not only provides a basis for subsequent automated screening, avoiding the uncertainty caused by subjective judgment in traditional methods, but also ensures the traceability of the entire data processing process by establishing a one-to-one mapping with the original data.

[0041] Next, all generated multidimensional parsed summaries are filtered. See the diagram for a detailed flowchart of this filtering process. Figure 2 A pre-trained image description generation model was invoked, and the aforementioned list of target entities was input to generate the image content description text: "The image shows cracks on the concrete surface." Subsequently, the Sentence-BERT model was used to calculate the feature vectors of this image description text and the aforementioned semantic summary, and the cosine similarity between them was found to be 0.95, so the content contradiction was calculated to be 0.05. This invention pre-set screening thresholds: an upper limit for content contradiction of 0.2, a lower limit for recognition confidence of 0.85, and a lower limit for information richness of 3.0. Since all indicators of this record met the conditions, it was judged as high-quality data. After screening all 10,000 records, the first batch of 2,000 records were finally extracted and judged as pending screening because their content contradiction was higher than 0.2, confidence was lower than 0.85, or information richness was lower than 3.0. These records were submitted to water conservancy experts for manual annotation.

[0042] The screening strategy proposed in this invention constructs a multi-dimensional network for identifying difficult samples by setting thresholds for three dimensions: content contradiction, identification confidence, and information richness. This method ensures that the value of expert annotation work is maximized and effectively avoids wasting manpower on a large number of simple and repetitive samples.

[0043] Furthermore, based on the marked hazard records, the instruction set is dynamically updated, and the optimized instruction set is used to perform generative cross-modal expansion on the marked hazard records to obtain expanded hazard records; the marked hazard records and expanded hazard records are organized into structured data units containing positive, negative, and difficult samples, specifically as follows:

[0044] Water conservancy experts precisely labeled these 2,000 records on a dedicated annotation interface. For one record, the experts confirmed that its hazard type belonged to "structural safety hazard," thus making it a positive sample label. Simultaneously, based on experience, the experts explicitly stated that it did not belong to "seepage safety hazard," forming a negative sample set. Due to the relatively blurry image in this record, the experts gave their judgment an 80% subjective confidence score, marking this record as a difficult sample.

[0045] After experts complete the annotation of the selected records, an automatic threshold calibration step is performed. The proportion of records with error-free multidimensional parsing summaries in the batch is calculated and defined as the invalid screening rate. Simultaneously, the proportion of records annotated with a score higher than the preset high confidence score is calculated and defined as the low-value screening rate. When the invalid screening rate exceeds the preset upper limit for invalid screening, the main trigger thresholds leading to invalid screening are tracked, and the main trigger thresholds are relaxed according to a preset adjustment coefficient based on the percentage exceeding the upper limit. When the low-value screening rate exceeds the preset upper limit for low-value screening, the lower limit thresholds for identification confidence and information richness are tightened according to another preset adjustment coefficient based on the percentage exceeding the upper limit. Finally, based on the adjusted lower limit threshold for identification confidence, the upper limit threshold for content contradiction is reset using a preset inverse proportional function.

[0046] The automatic threshold calibration step distinguishes between two different sources of efficiency loss: invalid screening and low-value screening, and performs targeted, unambiguous threshold adjustments accordingly. It improves the problem of potentially valuable data being filtered out due to overly stringent screening conditions by selectively relaxing the main trigger threshold; and it reduces the number of simple, unchallenging samples submitted to experts by tightening thresholds related to data information content. This method maintains its ability to discover challenging samples while effectively reducing the invalid review load on experts, making the human resource investment in the entire dataset construction closed-loop process more targeted and improving the overall operational efficiency of the method.

[0047] Subsequently, the descriptive text provided by experts in 2000 marked hazard records was analyzed, automatically extracting high-frequency domain terms such as "pockmarked surface," "exposed reinforcement," and "stripping." The extracted term "pockmarked surface" was then inserted into a pre-defined instruction template that included sentence variations and scene detail additions, creating new record-derived instructions such as "check for pockmarked surface." These new instructions were then incorporated into the instruction set, and their priority was increased in subsequent generative cross-modal extension tasks.

[0048] By analyzing expert annotations, this invention automatically extracts high-frequency domain terms and unique expression paradigms, integrating them into preset instruction templates. This establishes a dynamic instruction optimization mechanism capable of learning from expert knowledge and evolving independently. This makes subsequent generative cross-modal expansions no longer random or generic transformations, but rather targeted content generation highly tailored to the characteristics of the dam hazard domain, enhancing the domain relevance and diversity of the expanded data.

[0049] Using this updated instruction set, generative cross-modal augmentation was performed on 2000 marked hazard records. For example, for a record marked as a positive sample by experts and containing cracks in the image, an image content restoration model was invoked. Based on the location information of the cracks, the visual features of the cracks in the image were removed, generating an image of an intact dam surface. This image, together with its original text description, constituted a "counterfactual negative sample" as an augmented hazard record. For a record judged as a negative sample by experts, using the descriptive text "concrete spalling at the edge of the expansion joint" given by experts in another positive sample, a text-guided image editing model was invoked to synthesize the visual features of concrete spalling at the corresponding location in the image without a hazard record, thereby generating an "enhanced positive sample" as an augmented hazard record. In this way, a total of 2000 augmented hazard records were generated. The 2000 marked hazard records and the 2000 augmented hazard records were organized into 4000 structured data units containing positive, negative, and difficult samples.

[0050] The generative cross-modal augmentation method of the present invention improves the quality and utility of augmented hazard records through two complementary approaches, enhances the model's ability to distinguish subtle features, increases the diversity of training data, and helps to improve the classification performance and robustness of the final model.

[0051] This invention organizes data into structured data units containing positive, negative, and difficult samples, providing richer supervised information for the subsequent construction of domain knowledge indexes. Clearly defined negative samples help define clear classification boundaries, while quantified difficult sample scores provide weighted information on sample importance for subsequent comparative learning, thus improving the intrinsic quality of the domain knowledge index.

[0052] Furthermore, through sample comparison and instruction encoding, the structured data units are used to construct a domain knowledge index. The specific process is as follows:

[0053] Next, the domain knowledge index is constructed. A unified text encoder is used to generate 768-dimensional high-dimensional feature vectors for all 4000 structured data units. Through contrastive learning, the positions of these vectors in the feature vector space are adjusted: the vector of an original record labeled "structural safety hazard" is spatially brought closer to the vectors of its two augmented data sets; simultaneously, its vector is spatially distanced from the vector of a negative sample labeled "seepage safety hazard," which is explicitly excluded from that record. All adjusted vectors are stored in a vector database, forming the initial domain knowledge index.

[0054] By employing a contrastive learning approach, this invention "pushes away" positive and negative samples and "brings closer" positive and enhanced samples in the feature vector space. The resulting domain knowledge index is not merely a simple retrieval engine, but a "knowledge base" already encoded with domain knowledge discrimination logic. This structure ensures that the index itself contains discriminative information between categories. Converting external query commands into query vectors using the same encoder ensures alignment between the query intent and the index content within the same semantic space, thereby guaranteeing the efficiency, accuracy, and semantic relevance of subsequent retrieval processes.

[0055] Furthermore, the domain knowledge index is used as a new information retrieval tool. The closed-loop steps of multidimensional parsing and summarizing, record filtering, generative cross-modal expansion, and index optimization are repeatedly executed until the domain knowledge index performance reaches a preset indicator. The structured data units accumulated in the loop are then used as a dam hazard classification dataset. The specific process is as follows:

[0056] The second loop begins processing the next batch of 10,000 new unlabeled multimodal data entries. For each new record, before generating its multidimensional parsing summary, a search is performed using the existing domain knowledge index. This reveals that the three most similar labeled records are all classified as "seepage safety hazards," and their metadata features frequently include the entity "leakage." This information is extracted as context, and a context-guided temporary query is dynamically generated: "Please focus on analyzing the 'leakage' features related to 'seepage safety' and generate a multidimensional parsing summary." This more targeted temporary command is then used to invoke the dam safety analysis component, making its analysis and multidimensional parsing summary generation for this new record more accurate.

[0057] In the process of repeatedly executing the closed-loop steps, the strategy for screening multidimensional parsing summaries is divided into an exploration phase and a refinement phase. The initial phase is the exploration phase. In this phase, in order to prioritize expanding the breadth of knowledge, the screening strategy focuses on extracting information richness and identifying records with low confidence.

[0058] Specifically, the lower limit thresholds for information richness and recognition confidence are temporarily lowered; when the growth rate of the number of newly extracted high-frequency domain terms based on expert annotations is lower than the preset stable threshold in two consecutive cycles, the system automatically switches from the exploration stage to the refinement stage.

[0059] During the refinement stage, in order to prioritize improving knowledge depth, the screening strategy focuses on extracting records with contradictory content; specifically, it involves restoring and appropriately tightening the information richness and identification confidence thresholds, while lowering the upper limit threshold for content contradiction.

[0060] Dividing the selection strategy into two phases, exploration and refinement, introduces structured macro-control into the dataset construction process. This phased strategy maximizes the value of expert annotations at different stages, avoids ineffective investment in simple, repetitive knowledge in the later stages, and helps to build a dataset that balances breadth and depth of knowledge.

[0061] The process then repeats the closed-loop steps of multidimensional parsing and summarizing, record filtering, expert annotation (e.g., selecting 1500 records in this round), generative cross-modal augmentation, and index optimization. In each iteration, the domain knowledge index becomes more accurate and richer due to the newly added, expert-verified data. After five iterations, the retrieval accuracy of the domain knowledge index on the internal validation set reaches the preset target, and the closed-loop process of the entire dataset construction terminates.

[0062] The preset indicators are composite indicators and must simultaneously meet the following: First, the performance level indicator, that is, the retrieval accuracy of the domain knowledge index on a fixed validation set that does not participate in training must be higher than the preset high threshold for two consecutive cycles; Second, the performance stability indicator, that is, the growth rate of the retrieval accuracy in the most recent two cycles is lower than the preset low threshold.

[0063] The composite metric assesses both performance level and stability simultaneously, avoiding premature termination due to performance fluctuations, ensuring the reliability of the final results, and terminating the process when performance growth approaches saturation, thus achieving a balance between the final quality of the dataset and the construction cost.

[0064] Using the domain knowledge index built in the previous round as a "navigation tool" for the new round of processing is the key to the spiral-like improvement of the construction capability of this invention. By retrieving labeled hazard records similar to new data and using them as contextual information to dynamically generate query instructions, an intelligent processing mode of "using the known to guide the unknown" is realized. This means that when the dam safety analysis component processes new unlabeled multimodal data, it no longer has to "start from scratch," improving the efficiency of the entire closed loop and the quality of the final dataset.

[0065] Ultimately, after five rounds of iterations, with varying numbers of records selected for labeling in each round, a total of 9,000 expert-annotated hazard records and their corresponding 9,000 extended hazard records were accumulated, comprising 18,000 structured data units. These units were organized and stored as a graph database. In the graph database, each structured data unit is a node. For example, an extended hazard record is connected to its labeled hazard record node via a "derived from" relation edge; a difficult sample node, rated with 75% confidence by experts as a "structural safety hazard," is connected to a similar node labeled "seepage safety" via a "confusing with" relation edge; simultaneously, it is connected to explicitly excluded negative sample nodes via an "irrelevant" relation edge. This graph database constitutes the final dam hazard classification dataset, and its rich structure provides a solid foundation for subsequent complex model training and knowledge mining.

[0066] All accumulated structured data units are organized into a graph database structure, with each structured data unit as a node and the relationships between structured data units stored as edges; the relationships include: "derived from" relationships generated by data augmentation, "confusing with" relationships between difficult samples identified by experts and other samples, and "irrelevant" relationships between positive samples and negative samples.

[0067] This invention ultimately organizes the dataset into a graph database structure, which not only facilitates subsequent complex relationship analysis and visualization of the data, but also directly supports the training of more advanced graph neural networks and other models. This provides a data foundation for advanced applications such as in-depth analysis of the causes of dam hazards and prediction of associated risks, demonstrating the potential value of the dataset.

[0068] Example 2

[0069] This embodiment details the first part of the process in the method of the present invention, which involves generating structured data units from unlabeled multimodal data. This process is applied to processing a batch of 5,000 unlabeled dam inspection records, each record containing a site photograph and a text description.

[0070] At the start of the process, an unlabeled multimodal data point is processed. Its inspection report text reads "Minor cracks are visible on the water-facing side of the dam," accompanied by photos of the corresponding area. The dam safety analysis component is invoked, and based on the initial instruction set, a list of key entities and a semantic summary are generated from the text. Simultaneously, a list of target entities is generated from the image. By weighted averaging the confidence scores of the five identified entities, the record's recognition confidence score is 0.96, and due to the total number of entities being five, the information richness score is 5.0. This information collectively constitutes the multidimensional parsed summary of the record.

[0071] Next, the generated multidimensional parsed summary is filtered. The image description generation model is invoked, and the aforementioned target entity list is input to generate the image content description text: "The image shows cracks on the surface of the concrete dam." The Sentence-BERT model is used to calculate the feature vectors of this text and the aforementioned semantic summary; their semantic similarity is 0.98, therefore the content contradiction is determined to be 0.02. The filtering thresholds set in this method are: upper limit for content contradiction 0.15, lower limit for recognition confidence 0.90, and lower limit for information richness 4.0. Since the scores of this record (0.02, 0.96, 5.0) all meet the preset conditions, it is judged as high-quality automatically processed data, requiring no manual intervention. After filtering all 5000 records, 350 records were ultimately filtered out because one or more indicators did not meet the preset thresholds and were submitted to dam safety experts for manual annotation.

[0072] Experts annotated one of the 350 records, whose image showed a wide crack at the dam's joint. They identified the hazard type as "structural safety hazard - crack," a positive example. Simultaneously, they ruled out "flood control safety hazard," a negative example. Due to the crack's unusual morphology, they assigned it an 85% confidence score, classifying it as a difficult example. This step organized the record into a structured data unit containing positive, negative, and difficult information.

[0073] After all 350 records were annotated by experts, the descriptive text provided by the experts was analyzed to extract high-frequency domain terms such as "unprotected" and "scour pit." These terms were then filled into a preset instruction template to generate new record-derived instructions. These new instructions were incorporated into the instruction set and given higher priority. Using the updated instruction set, generative cross-modal expansion was performed on the 350 annotated structured data units to obtain expanded hazard records. This completed the first half of the process, yielding a total of 700 structured data units.

[0074] Example 3

[0075] This embodiment details the latter part of the process in the method of the present invention, which involves constructing and utilizing a domain knowledge index for closed-loop optimization after obtaining structured data units, and finally generating the dataset.

[0076] This embodiment begins with a prepared initial set containing 10,000 structured data units. First, a domain knowledge index is constructed for these 10,000 structured data units. A 768-dimensional high-dimensional feature vector is generated for each data unit. Next, the vector positions are adjusted using a comparison between data records: a positive sample vector labeled "seepage safety hazard" is brought closer to its corresponding expanded hazard record vector in the feature space; simultaneously, it is pushed further away from the negative sample vector of "structural safety hazard" that has been explicitly excluded. All 10,000 adjusted high-dimensional feature vectors are stored in a dedicated vector database, which constitutes the initial domain knowledge index.

[0077] Next, this domain knowledge index will be used as a new information retrieval tool, entering a closed-loop optimization process. When processing a new set of unlabeled multimodal data, its text description is "a masonry-lined tunnel with dampness around the tunnel and widespread, serious dripping leakage problems." First, the text is encoded into a query vector, and the domain knowledge index is used to retrieve the two nearest labeled hazard records. Analysis reveals that both of these neighboring records are labeled "seepage safety hazard - leakage," and the metadata features frequently include the entities "seepage point" and "humidity." Based on this contextual information, a temporary query command with contextual guidance is dynamically generated: "Task: Analyze this record. Contextual reference: Similar records are marked as 'seepage safety hazard - leakage,' with key features being 'seepage point' and 'humidity.' Please focus on analyzing and extracting these features." This highly targeted command is used to invoke the dam safety analysis component, making its metadata generation for new records more accurate.

[0078] This closed-loop process is repeated continuously. In each round, newly selected data, annotated by experts, is used to expand and optimize the domain knowledge index, making its retrieval capabilities increasingly stronger. The loop terminates based on preset indicators, which are composite indicators that must be met simultaneously: first, a performance level indicator, meaning the retrieval accuracy of the domain knowledge index on a fixed, non-training validation set must be higher than a preset high threshold for two consecutive loops; second, a performance stability indicator, meaning the growth rate of the retrieval accuracy in the most recent two loops must be lower than a preset low threshold. Once the preset indicators are met, the index performance is deemed satisfactory, and the closed-loop process terminates.

[0079] After the closed-loop step terminates, all the structured data units accumulated in the loop are organized into a final dataset. This dataset is stored in the form of a graph database, with each structured data unit as a node. The relationships between nodes are stored as edges. For example, a data augmentation-generated "seepage safety hazard" node is connected to its original labeled node via a "derived from" relationship edge; a "seepage safety hazard - severe seepage around the outlet of the water conveyance culvert" node, marked as a difficult sample by experts, is connected to a similar node labeled "structural safety hazard - aging and cracking of the water conveyance culvert body" via a "confusing with" relationship edge; simultaneously, the "seepage safety" node is connected to a negative sample node of the "metal structure safety" type, which was explicitly excluded by experts, via an "irrelevant to" relationship edge. This graph database containing rich relationships is the final dam hazard classification dataset produced by this method.

[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a dam hazard classification dataset based on semi-automatic active learning, characterized in that, include: Establish an instruction set that includes query and record-derived instructions; invoke the dam safety analysis component, process the unlabeled multimodal data according to the instruction set, and generate a multidimensional analytical summary; filter the multidimensional analytical summary, extract records with contradictory content and sparse information, and hand them over to experts for annotation to obtain annotated hazard records; Based on the marked hazard records, the instruction set is dynamically updated, and the optimized instruction set is used to perform generative cross-modal expansion on the marked hazard records to obtain expanded hazard records; the marked hazard records and expanded hazard records are organized into structured data units containing positive, negative and difficult samples; By comparing samples and encoding instructions, a domain knowledge index is constructed using the structured data units. The domain knowledge index is used as a new information retrieval tool. The closed-loop steps of multidimensional parsing summary generation, record filtering, generative cross-modal expansion and index optimization are repeatedly executed until the domain knowledge index performance reaches the preset index. The structured data units accumulated in the loop are used as the dam hazard classification dataset.

2. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The process of generating the multidimensional parsing summary specifically includes: for unlabeled multimodal data, the dam safety analysis component generates a key entity list and a semantic summary for the text portion of the record according to the instruction set, and generates a target entity list containing the identified objects and spatial locations for the image portion of the record; the dam safety analysis component performs preset statistical calculations on the recognition confidence scores of the entities contained in the key entity list and the target entity list to obtain a recognition confidence score, and counts the total number of entities to quantify the information richness score; finally, the key entity list, semantic summary, target entity list, recognition confidence score, and information richness score are combined together to form a structured record as the multidimensional parsing summary, and a one-to-one mapping relationship is established with the original data record.

3. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The process of filtering the multidimensional parsing summary specifically includes: calling a pre-trained image description generation model, inputting the target entity list of the image part into the image description generation model, and generating structured image content description text; using the Sentence-BERT model to calculate the corresponding feature vectors for the semantic summary of the text part and the image content description text, and determining the content contradiction degree based on these two feature vectors. The rule for determining the content contradiction degree is: the lower the semantic similarity between the two feature vectors, the higher the content contradiction degree; setting an upper threshold for content contradiction degree, a lower threshold for recognition confidence, and a lower threshold for information richness; and determining the original data records corresponding to the multidimensional parsing summary that fail to simultaneously meet all of the following conditions as records to be filtered and extracted for subsequent annotation processing by experts: the content contradiction degree is not higher than the upper threshold for content contradiction degree; the recognition confidence score is not lower than the lower threshold for recognition confidence; and the information richness score is not lower than the lower threshold for information richness.

4. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The process of dynamically updating the instruction set specifically includes: analyzing the classification tags and descriptive text given by experts to the marked hazard records, and extracting high-frequency domain terms and unique expression paradigms; pre-setting instruction templates containing keyword placeholders, wherein the instruction templates fixedly include sentence transformation templates and scene detail addition templates; filling the extracted high-frequency domain terms as keywords into the keyword placeholders of the instruction templates, and combining them to generate new record-derived instructions; incorporating the new record-derived instructions into the instruction set, and increasing the calling priority of the new record-derived instructions in subsequent generative cross-modal expansion tasks.

5. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The generative cross-modal augmentation process specifically includes: for marked hazard records identified as positive samples, an image content restoration model is invoked to remove hazard visual features from the image portion based on the hazard entity location information in the record, generating counterfactual negative samples, and using the counterfactual negative samples as augmented hazard records; for marked hazard records identified as negative samples, a text-guided image editing model is invoked to synthesize corresponding hazard visual features in the image portion of the negative samples using expert descriptive text from the positive samples as semantic guidance, generating enhanced positive samples, and using the enhanced positive samples as augmented hazard records; difficult samples are not augmented.

6. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The process of organizing the structured data unit specifically includes: creating a data structure containing multiple fields for the marked hazard records and expanded hazard records; the fields include a unique identifier for the data record, expert-marked correct classification labels as positive samples, a set of explicitly excluded incorrect classification labels as negative samples, and expert-marked confidence scores as difficult samples; wherein, the set of negative samples is provided by experts during the marking process, and the confidence score is the expert's assessment of the authenticity of the marking results.

7. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The construction of the domain knowledge index specifically includes: generating high-dimensional feature vectors for structured data units; adjusting vector positions in the feature vector space by comparing data records, with the adjustment rule being: minimizing the vector distance between positive samples and their corresponding expanded hazard records, while maximizing the vector distance between positive samples and negative samples; the domain knowledge index is constructed as a vector database supporting efficient vector retrieval, containing high-dimensional feature vectors of all structured data units; the instruction encoding refers to converting externally input text query instructions into query vectors using a text encoder identical to the data record encoder; subsequent information retrieval is accomplished by calculating the similarity between the query vector and the vectors of all data units in the vector database.

8. The method for constructing a dam hazard classification dataset based on semi-automatic active learning according to claim 1, characterized in that, The process of using the domain knowledge index as a new information retrieval tool specifically includes: in the next cycle, before generating a multidimensional analytical summary for new unlabeled multimodal data, using the domain knowledge index to retrieve the neighboring data records most similar to the unlabeled multimodal data in the labeled hazard records; extracting the classification labels and metadata features of the neighboring data records as context information, and dynamically generating temporary query instructions; using the temporary query instructions to call the dam safety analysis component to generate a new multidimensional analytical summary.

Images