Man-machine collaborative labeling disagreement arbitration method, system and device based on domain physical constraint rule base and storage medium

Abstract

The application discloses a kind of based on field physical constraint rule base Man-machine cooperation marking divergence arbitration method, system, equipment and storage medium, comprising: predefine defect type in hydraulic engineering, physical attribute name, constraint relationship and corresponding threshold parameter, to construct field physical constraint rule base;Obtain the artificial labeling result and AI Agent labeling result in defect area in hydraulic engineering image;According to artificial labeling result and AI Agent labeling result, the intersection-over-union between artificial labeling frame and AI Agent labeling frame is calculated;When the intersection-over-union is less than preset threshold, or category label is inconsistent, based on field physical constraint rule base, artificial labeling result and AI Agent labeling result are physically constrained arbitration.The application constructs the field physical constraint rule base containing four-tuple structure, carries out physical legality determination to the two parties of divergence respectively, realizes the automatic adjudication to divergence sample.

Description

Technical Field

[0001] This invention relates to the field of annotation disagreement arbitration technology, and in particular to a human-computer collaborative annotation disagreement arbitration method, system, device and storage medium based on a domain physical constraint rule library. Background Technology

[0002] With the deepening application of visual intelligent agent (AI) technology in the field of engineering infrastructure, the demand for large-scale annotation of professional images is becoming increasingly urgent. Taking the Yangtze River Diversion Project to Han River as a typical case, it is necessary to accurately annotate a large number of professional defect images such as cracks, seepage, rock spalling, and exposed steel bars in order to build a high-quality AI training dataset.

[0003] Currently, a three-way collaborative model is commonly adopted, consisting of crowdsourced annotation (by civil engineering students), automatic annotation, and manual review. However, this model suffers from several drawbacks. The varying levels of expertise among crowdsourced annotators and the uneven distribution of confidence levels in the AI-powered automatic annotation tools lead to discrepancies between the two annotation sources, making the handling of disagreements a core bottleneck restricting system efficiency. Therefore, how to automatically adjudicate discrepancies while maintaining accuracy and efficiency is a pressing issue that needs to be addressed. Summary of the Invention

[0004] The main objective of this invention is to provide a human-computer collaborative annotation disagreement arbitration method, system, device, and storage medium based on a domain physical constraint rule library, aiming to solve the technical problem of how to automatically adjudicate disagreement samples while ensuring accuracy and efficiency.

[0005] To achieve the above objectives, this invention provides a human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule base. The human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule base includes: Predefine the defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects to construct a domain physical constraint rule library; Obtain both manual and AI agent annotation results for defect areas in images of water conservancy projects; Calculate the intersection-union ratio (IU) of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results; When the intersection-union ratio of the labeled boxes is less than a preset threshold, or when the category labels of the manually labeled boxes and the category labels of the AI ​​Agent are inconsistent, physical constraint arbitration is performed on the manually labeled boxes and the AI ​​Agent labeled boxes based on the domain physical constraint rule base.

[0006] Optionally, calculating the intersection-union ratio (IUU) of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results includes: The coordinates of the manually labeled boxes are determined based on the manually labeled results, and the coordinates of the AI ​​Agent labeled boxes are determined based on the AI ​​Agent labeled results. Calculate the area of ​​the intersecting region based on the coordinates of the manually labeled bounding box and the coordinates of the AI ​​Agent labeled bounding box; The total union area is calculated using the union area formula based on the area of ​​the manually labeled box, the area of ​​the AI ​​Agent labeled box, and the area of ​​the intersecting region. The intersection-union ratio (IUR) between the manually labeled bounding boxes and the AI ​​Agent labeled bounding boxes is calculated using the IUR formula based on the area of ​​the intersecting region and the total union area.

[0007] Optionally, the formula for the area of ​​the union is: Area(Bh∪Ba)=Area(Bh)+Area(Ba)-Area(Bh∩Ba) In the formula, Area(Bh∪Ba) is the total union area, Area(Bh) is the area of ​​the manual annotation box, Area(Ba) is the area of ​​the AIAgent annotation box, and Area(Bh∩Ba) is the area of ​​the intersecting region. The formula for the crossover-union ratio is: IoU=Area(Bh∩Ba) / Area(Bh∪Ba) In the formula, IoU is the intersection-union ratio between manually labeled bounding boxes and AI agent labeled bounding boxes.

[0008] Optionally, after arbitrating the manual annotation results and the AI ​​Agent annotation results based on the domain physical constraint rule base, the method further includes: Obtain the category labels of the AI ​​Agent annotation results; When the category label is seepage or salting out, determine whether there is a preset interference scenario in the defect area of ​​the water conservancy project image; If so, the moisture index of the defective area is calculated using the moisture index formula; The AI ​​Agent labeling results are corrected based on the humidity index; A second physical constraint arbitration is performed on the manually labeled results and the corrected AI Agent labeled results based on the domain physical constraint rule base.

[0009] Optionally, calculating the moisture index of the defective region using the moisture index formula includes: Obtain the maximum brightness value of the pixels within the defective region; When the maximum brightness value is greater than the preset saturation threshold, the weights in the humidity index formula are adjusted. Based on the adjusted weights, the humidity index of the defective area in the water conservancy project image is calculated using the humidity index formula.

[0010] Optionally, the formula for the humidity index is: moisture_index=w1×saturation_mean+w2×value_std+w3×smoothness In the formula, w1, w2, and w3 are weights, moisture_index is the moisture index, saturation_mean is the mean saturation of the defect region, value_std is the standard deviation of the brightness of the defect, and smoothness is the texture smoothness of the defect region.

[0011] Optionally, after arbitrating the manual annotation results and the AI ​​Agent annotation results based on the domain physical constraint rule base, the method further includes: When both manual annotation and AI Agent annotation are deemed to be in violation of physical constraint arbitration, the proportion of the overlapping area between the manual annotation box and the AI ​​Agent annotation box to the area of ​​the manual annotation box and the area of ​​the AI ​​Agent annotation box, respectively, is calculated. When the proportion is greater than a preset threshold and the defective region meets the preset spatial physical associated law, the manual annotation results and the AI ​​Agent annotation results are pushed to experts for review.

[0012] Furthermore, to achieve the above objectives, this invention also proposes a human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule base, wherein the human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule base includes: The rule base management module is used to determine the defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects in order to build a domain physical constraint rule base. The divergence detection module is used to acquire both manual annotation results and AI agent annotation results within defect areas in water conservancy project images; The divergence detection module is also used to calculate the intersection-union ratio of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results; The physical legality determination module is used to arbitrate the physical constraints of the manual annotation results and the AI ​​Agent annotation results based on the domain physical constraint rule library when the intersection-union ratio of the annotation boxes is less than a preset threshold, or when the category labels of the manual annotations and the category labels of the AI ​​Agents are inconsistent. The arbitration award module is used to output the arbitration results of physical constraints.

[0013] Furthermore, to achieve the above objectives, the present invention also proposes a human-computer collaborative annotation disagreement arbitration device based on a domain physical constraint rule library. The device includes: a memory, a processor, and a human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library stored in the memory and executable on the processor. The human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library is configured to implement the steps of the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule library as described above.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library. When the human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library is executed by a processor, it implements the steps of the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule library as described above.

[0015] This invention first predefines defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects to construct a domain physical constraint rule base. Then, it acquires manual and AI agent annotation results within defect areas of water conservancy project images. Based on these results, it calculates the intersection-union ratio (IUR) of the manually and AI agent annotation boxes. When the IUR is less than a preset threshold, or when the category labels of the manually and AI agents are inconsistent, it arbitrates the physical constraints based on the domain physical constraint rule base. This invention constructs a domain physical constraint rule base containing a four-tuple structure, performs physical legitimacy determination on both sides of the disagreement, and outputs arbitration awards according to different disagreement scenarios. This achieves automatic adjudication of disagreement samples, significantly reducing the frequency of expert intervention while ensuring the physical legitimacy of the arbitration awards. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of a human-machine collaborative annotation disagreement arbitration device based on a domain physical constraint rule library in the hardware operating environment of the embodiment of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule library of the present invention. Figure 3 This is a structural block diagram of the first embodiment of the human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule library of the present invention.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a human-machine collaborative annotation disagreement arbitration device based on a domain physical constraint rule library in the hardware operating environment of the embodiment of the present invention.

[0020] like Figure 1 As shown, the human-machine collaborative annotation disagreement arbitration device based on a domain physical constraint rule base may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0021] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the human-computer collaborative annotation disagreement arbitration device based on the domain physical constraint rule base. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0022] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule base.

[0023] exist Figure 1 In the human-computer collaborative annotation disagreement arbitration device based on a domain physical constraint rule library shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the human-computer collaborative annotation disagreement arbitration device based on a domain physical constraint rule library of the present invention can be set in the human-computer collaborative annotation disagreement arbitration device based on a domain physical constraint rule library. The human-computer collaborative annotation disagreement arbitration device based on a domain physical constraint rule library calls the human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library stored in the memory 1005 through the processor 1001, and executes the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule library provided in the embodiment of the present invention.

[0024] This invention provides a human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule base, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule library of the present invention.

[0025] In this embodiment, the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule base includes the following steps: S1 predefines the defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects to construct a domain physical constraint rule library.

[0026] It is easy to understand that the executing entity of this embodiment can be a human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule base with functions such as data processing, network communication and program execution, or other computer devices with similar functions. This embodiment does not limit it.

[0027] In the specific implementation, a set of hard physical attribute constraint rules is constructed for the professional defect types in water conservancy engineering images. A domain physical constraint rule library is then built based on this set, employing a four-tuple structure: defect type (i.e., target object type), physical attribute name, constraint relationship, and threshold parameter. Refer to Table 1 below, which shows the set of hard physical attribute constraint rules:

[0028] The table shows the threshold parameters for crack width-to-length ratio, based on the following physical criteria: fine cracks <0.08 (stress cracks), coarse cracks 0.08-0.15 (stress / settlement), and >0.15 indicates peeling. The threshold parameter for minimum crack pixel area is based on the following physical criteria: at a 1280×720 resolution, 180px corresponds to approximately 3-5mm in actual size; values ​​below this are difficult to distinguish from texture noise. The threshold parameter for crack bounding box aspect ratio is based on the following physical criteria: cracks are linearly extending defects, and the bounding box length... The aspect ratio should be at least 3:1; the threshold parameter setting for the seepage wetting index is based on the following physical basis: 0.40-0.75 for true seepage areas, 0.18-0.38 for dry salting-out areas, with 0.45 being the optimal boundary; the threshold parameter setting for the number of seepage connected domains is based on the following physical basis: seepage areas should be relatively continuous, and more than 5 independent connected components are usually noise; the threshold parameter setting for the seepage / salting-out wetting index is based on the following physical basis: a wetting index below 0.35 indicates salting-out or dry rock, excluding mislabeling of seepage.

[0029] It should also be noted that each rule has an independent version number; online incremental updates driven by expert feedback are supported; the effect of a new rule does not affect the adjudication results of historical samples that have been arbitrated; and rule update records are recorded in a complete audit log.

[0030] S2, obtain the manual annotation results and AI Agent annotation results in the defect area of ​​the water conservancy project image.

[0031] It should be noted that the manual annotation results are the annotation results of the user on the defect area after receiving the water conservancy project image. The manual annotation results include the coordinates of the bounding box, the category label, and the physical attribute metadata. The AI ​​Agent annotation results are automatically annotated by the automatic annotation tool (i.e., it is built based on the mature deep object detection or semantic segmentation network framework in the existing technology. For example, the AI ​​Agent is built based on the YOLOv8 object detection network, the Faster R-CNN network, or the zero-shot weakly supervised segmentation framework based on Segment Anything Model (SAM). After receiving the image to be annotated (i.e., the water conservancy project image), its output terminal outputs the automatic annotation results containing the bounding box coordinates, the category label, the confidence score, and the target-related physical attribute metadata.

[0032] S3, calculate the intersection-union ratio of the annotation boxes between the manually annotated boxes and the AI ​​Agent annotation boxes based on the manually annotated results and the AI ​​Agent annotation results.

[0033] Furthermore, the method for calculating the intersection-union ratio (IUU) between the manually labeled boxes and the AI ​​Agent labeled boxes based on the manual annotation results and the AI ​​Agent annotation results is as follows: determine the coordinates of the manually labeled boxes based on the manual annotation results, and determine the coordinates of the AI ​​Agent labeled boxes based on the AI ​​Agent annotation results; calculate the area of ​​the intersecting region based on the coordinates of the manually labeled boxes and the AI ​​Agent labeled boxes; calculate the total union area using the union area formula based on the area of ​​the manually labeled boxes, the area of ​​the AI ​​Agent labeled boxes, and the area of ​​the intersecting region; and calculate the IUU between the manually labeled boxes and the AI ​​Agent labeled boxes using the intersection-union ratio formula based on the area of ​​the intersecting region and the total union area.

[0034] In the specific implementation, the Intersection over Union (IoU) of the two bounding boxes is calculated as follows: Let the manually labeled bounding box be Bh and the AI ​​Agent bounding box be Ba. With the top left corner of the image as the origin, the coordinates of the top left and bottom right corners of the manually labeled bounding box Bh are represented as (xh1, yh1, xh2, yh2), and the coordinates of the top left and bottom right corners of the AI ​​Agent bounding box Ba are represented as (xa1, ya1, xa2, ya2).

[0035] First, calculate the coordinates of the top-left corner (xi1, yi1) and the bottom-right corner (xi2, yi2) of the overlapping rectangle of the two annotation boxes: xi1 = max(xh1, xa1)(1) yi1 = max(yh1, ya1)(2) xi2 = min(xh2, xa2)(3) yi2 = min(yh2, ya2)(4) Therefore, the area of ​​the intersecting region, Area(Bh ∩ Ba), is derived: Area(Bh ∩ Ba) = max(0, xi2 - xi1) × max(0, yi2 - yi1) (Formula 5) Subsequently, the areas of the manually labeled bounding boxes (Area(Bh) and the AI ​​agent labeled bounding boxes (Area(Ba)) were calculated separately: Area(Bh) = (xh2 - xh1) × (yh2 - yh1)(6) Area(Ba) = (xa2 - xa1) × (ya2 - ya1)(7) Therefore, the total union area Area(Bh ∪ Ba) of the two label boxes can be calculated using the union area formula: Area(Bh∪Ba)=Area(Bh)+Area(Ba)-Area(Bh∩Ba)(8) wherein, Area(Bh∪Ba) is the total union area, Area(Bh) is the area of the manually labeled frame, Area(Ba) is the area of the AI Agent labeled frame, and Area(Bh∩Ba) is the intersection area; Finally, the calculation formula for Intersection over Union IoU is as follows: IoU=Area(Bh∩Ba) / Area(Bh∪Ba)(9) S4, when the Intersection over Union of the labeled frames is less than a preset threshold, or the manually labeled category label is inconsistent with the AI Agent labeled category label, performing physical constraint arbitration on the manual labeling result and the AI Agent labeling result based on the domain physical constraint rule base.

[0036] In a specific implementation, the preset threshold may be user-defined, and may be set to 0.3. The basis for determining the preset threshold T_iou=0.3 is: through three groups of comparative experiments on 200 simulated images of the Yangtze River Diversion to Hanjiang River Supplement Project (T_iou is taken as 0.3, 0.5, and 0.7 respectively), the arbitration coverage rate (weight 0.3), arbitration accuracy rate (weight 0.5) and expert intervention reduction (weight 0.2) are comprehensively evaluated, and T_iou=0.3 has the highest comprehensive score (0.766), which is the optimal disagreement triggering threshold.

[0037] It should also be noted that when the Intersection over Union of the labeled frames is less than the preset threshold, or the category labels are inconsistent, the arbitration process can be triggered. For the labeling results of both parties in the disagreement, the legality judgment of each rule in the physical constraint rule base is executed respectively, and a binary judgment result is output: LEGAL (pass all constraints) or ILLEGAL (violate at least one constraint), so as to determine the disagreement scenario, and record the specific violated rule entry for feedback at the same time, and corresponding subsequent processing can be performed according to the disagreement scenario. The arbitration rule table for the four disagreement scenarios is as follows:

[0038] Embodiment 1: Water Conservancy Engineering Crack Labeling Disagreement Arbitration Scenario description: For the same crack image area, the manual annotator labeled a crack frame with an aspect ratio of 0.12 (aspect_ratio=0.12, pixel_area=320px), and the AI Agent labeled a crack frame with an aspect ratio of 0.07 (aspect_ratio=0.07, pixel_area=280px). The IoU of the two is 0.25<T_iou=0.3, which triggers disagreement arbitration.

[0039] Physical legality determination: Manual annotation: aspect_ratio=0.12>R_crack_001 threshold 0.08, ILLEGAL (violation of crack width-to-length ratio constraint). Agent annotations: aspect_ratio=0.07 ≤ 0.08, pixel_area=280px ≥ 180px, bbox_ratio satisfies, LEGAL. Arbitration ruling: Trigger scenario one (human violation, agent legitimacy). The agent's annotation results are accepted, and the human annotator is given real-time feedback: the aspect ratio of the annotation box is 0.12, which exceeds the upper limit of the crack physical constraint of 0.08; and the credibility of the human annotator is updated (i.e., the historical credibility score of the corresponding defect type is reduced).

[0040] It should be noted that the strategy for lowering confidence is not a fixed penalty, but rather dynamically adjusted based on the type and severity of the violation, as well as the annotator's historical performance. This dynamic update is mathematically implemented using a lightweight Bayesian posterior probability negative update algorithm (as shown in Equation 11). The specific calculation process is as follows: Let C_old be the historical credibility score of the currently judged violator (which mathematically represents the prior probability distribution of the annotator in a "high credibility" state); the severity of the violation of this sample is represented by the severity coefficient K_v, which ranges from 1.0 to 3.0, and its value is determined by the magnitude of the deviation of the violation of the physical constraint rule; if a single rule is violated and the deviation is less than 10% of the preset threshold, then K_v is 1.0; if the deviation is between 10% and 30%, then K_v is 2.0; if the deviation exceeds 30% or multiple physical rules are violated at the same time, then K_v is 3.0.

[0041] Simultaneously, a historical credibility adjustment weight factor Alpha based on prior probability nonlinear adjustment is introduced, the calculation formula of which is as follows: Alpha=1.0-(1.0 / (1.0+Exp(-C_old)))(10) By adjusting the nonlinearity of Alpha, annotators with higher historical credibility have a higher tolerance for penalties for a single violation (the smaller the Alpha, the fewer the penalties), while annotators with lower historical credibility will be punished more severely (the larger the Alpha, the more penalties). This is physically consistent with the nonlinear suppression effect of prior probability on single noise interference in Bayesian posterior probability.

[0042] Finally, the updated dynamic posterior confidence score C_new is calculated using the following formula: C_new=C_old-0.05×K_v×Alpha(11) The above dynamic adjustment mechanism is illustrated below in conjunction with specific implementation examples: In one application scenario of the present invention, the historical credibility score C_old of human annotator A is 0.90. For a crack annotated by this annotator, the width-to-length ratio reaches 0.09 (exceeding the 0.08 threshold limit defined by the R_crack_001 rule), with a deviation amplitude of 12.5%, so the system determines that the severity coefficient K_v=2.0; substituting into Formula 10 for calculation gives Alpha approximately equal to 0.29; finally, the updated credibility score C_new calculated by the Formula 11 is 0.90-0.05×2.0×0.29=0.871.

[0043] At the same time, the historical credibility score C_old of another human annotator B is only 0.50, who made the same error on the same sample, and the severity coefficient K_v is also 2.0; due to its poor historical performance (that is, the prior credibility probability is low), the Alpha calculated by the system is approximately equal to 0.38; substituting into the Formula 10 for calculation gives the updated credibility score C_new=0.50-0.05×2.0×0.38=0.462.

[0044] It can be seen from the above examples that by non-linearly coupling Bayesian prior adjustment into the immediate deduction framework, the present solution not only realizes dynamic and precise negative feedback for different violation degrees and different annotator profiles, greatly improves the rationality of the quality control system, but also effectively eliminates mathematical conflicts caused by formula fragmentation.

[0045] Embodiment 2: Dispute arbitration for misannotation of water seepage areas and salt efflorescence Scenario description: For the same image area, the human annotator annotates it as water seepage (moisture_index=0.28), and the AI Agent annotates it as salt efflorescence (moisture_index=0.28). The two category labels are inconsistent, triggering dispute arbitration.

[0046] Physical legitimacy determination: Human annotation (water seepage): moisture_index=0.28 < the R_seep_001 threshold of 0.45, ILLEGAL (violating the lower limit constraint of the water seepage moisture index) Agent annotation (salt efflorescence): moisture_index=0.28 ≤ the R_seep_003 threshold of 0.35, satisfying the salt efflorescence exclusion rule, LEGAL Arbitration award: Scenario 1 is triggered. The salt efflorescence annotation of the Agent is adopted, and the human annotator is fed back that the moisture index of this area is insufficient to determine water seepage, and no expert intervention is required.

[0047] Example 3: Rule-level review of real divergent samples Scenario description: For a certain image region, it is manually labeled as a crack (aspect_ratio=0.07, pixel_area=200px), and the AI ​​Agent labels it as rock strata peeling. The IoU=0.15<0.3, the categories are inconsistent, and arbitration is triggered.

[0048] Physical legality determination: Manual annotation (cracks): aspect_ratio=0.07≤0.08, pixel_area=200px≥180px, LEGAL Agent annotation (rock layer exfoliation): satisfies R_layer_001 edge gradient constraint, LEGAL Arbitration Award: Triggered Scenario 3 (both parties have legitimate and genuine disagreements). Marked as a genuine disagreement sample, it is pushed to the expert rule-level review queue. After expert review, a priority rule for areas where rock spalling and cracks coexist may be added. This rule, once written to the rule base, automatically affects the processing of subsequent similar samples, eliminating the need for individual review.

[0049] It should also be noted that after Scenario 2 is triggered, Agent violation samples are recorded in the Agent bias log, affecting the Agent confidence weight on that type of sample.

[0050] Furthermore, during the actual image acquisition process at engineering sites such as dams and tunnels, the following extreme situations may exist (e.g., extreme lighting and strong specular reflection interference scenarios). The processing method is as follows: After performing a physical constraint arbitration on the manually labeled results and AI Agent labeled results based on the domain physical constraint rule library, the category label of the AI ​​Agent labeled results is obtained; when the category label is R_seep_001 seepage or R_seep_003 seepage / saltification, it is determined whether the defect area of ​​the water conservancy project image has a preset interference scenario (e.g., extreme lighting and strong specular reflection interference scenario); if so, the maximum brightness value of the pixels in the defect area is obtained; when the maximum brightness value is greater than the preset saturation threshold, the weights in the wetting index formula are adjusted; based on the adjusted weights, the wetting index of the defect area in the water conservancy project image is calculated using the wetting index formula; the AI ​​Agent labeled results are corrected according to the wetting index; a second physical constraint arbitration is performed on the manually labeled results and the corrected AI Agent labeled results based on the domain physical constraint rule library.

[0051] moisture_index=w1×saturation_mean+w2×value_std+w3×smoothness In the formula, w1, w2, and w3 are weights, moisture_index is the moisture index, saturation_mean is the mean saturation of the defect region, value_std is the standard deviation of the brightness of the defect, and smoothness is the texture smoothness of the defect region.

[0052] Saturation mean: Physical definition: Characterizes the average color purity of defective areas in an image within the HSV (Hue, Saturation, Luminance) color space. Due to the high moisture content in seepage areas of water conservancy or construction projects, their visual color saturation is typically significantly higher than that of dry, rough concrete or rock surfaces.

[0053] Acquisition method: First, the image defect area is converted from the original RGB color space to the HSV color space, and the S (Saturation) channel is separated and extracted; then, for all pixels inside the corresponding bounding box or segmentation mask of the image defect area, the arithmetic mean of the S channel values ​​is calculated to obtain the average saturation value.

[0054] Luminance standard deviation (value_std): Physical definition: Characterizes the degree of brightness fluctuation in defective areas of an image in the HSV color space, used to quantitatively describe the local light and shadow texture contrast within the area. Due to uneven wetting or the presence of weak reflections, truly water-soaked areas exhibit significantly different internal brightness contrast compared to dry, uniform areas.

[0055] Acquisition method: Separate and extract the V (Value, brightness) channel in the HSV color space; then, calculate the sample standard deviation of the V channel values ​​of all pixels within the defective area of ​​the image, and normalize the calculation result to the real number range of 0 to 1 to obtain the brightness standard deviation.

[0056] Texture smoothness: Physical definition: Characterizes the macroscopic smoothness of the surface of a defective area in an image. When water seeps into a concrete or rock surface, the flowing or accumulated water can fill the microscopic pores and rough particles of the substrate, reducing high-frequency rough details in visual representation and presenting a higher level of physical smoothness.

[0057] Acquisition method: The grayscale image of the defective region of the image is subjected to Laplacian operator filtering to extract the edge and high-frequency grayscale change information of the image; then, the variance value laplacian_var of the pixels in the filtered region is calculated (to quantitatively characterize the high-frequency texture intensity of the region); finally, the smoothness score is calculated by the negative correlation mapping function, the calculation formula is: smoothness=1 / (1+laplacian_var / 1000.0), and the smoothness result is restricted to the range of 0 to 1.

[0058] in: w1=0.5 (saturation weight, mean of S-channel in HSV space) w2=0.3 (luminance standard deviation weight, V channel standard deviation, normalized 0-1) w3=0.2 (texture smoothness weight, smoothness=1 / (1+laplacian_var / 1000.0)) Upon detecting overexposure (e.g., V channel maximum > 250), the system immediately activates 'Ambient Light Adaptive Property Correction'. The weight w1 for the saturation mean in the original wetting index formula is increased, while the weight w2 for the brightness standard deviation is temporarily decreased by 80% (e.g., from 0.3 to 0.06). Subsequently, the wetting index for the area is recalculated using this temporarily corrected adaptive weight. This recalculated wetting index is then re-entered into the 'Domain Physical Constraint Rule Base' for a secondary property constraint check. If the secondary check passes, the annotation is exempt from being judged as a violation, effectively eliminating false physical characteristics caused by reflection and significantly improving the accuracy of dispute arbitration in harsh, reflective tunnel / dam environments.

[0059] For example: When there is direct sunlight from strong spotlights inside the tunnel or exposure to the sun outdoors, the dry concrete or rock walls will produce strong specular reflections. This will cause the brightness standard deviation (value_std) of local areas of the image to spike abnormally, causing the moisture index (moisture_index) to exceed the judgment threshold, thus resulting in a systematic error of misclassifying "dry reflective surfaces" as "water seepage".

[0060] Countermeasures: The system incorporates an "adaptive property correction for ambient lighting" mechanism. During the physical attribute extraction in step one, the system first calculates the maximum brightness value of pixels within the labeled area. If the maximum brightness exceeds a preset saturation threshold (e.g., the maximum value of the V channel is greater than 250), the system automatically reduces the weight of the brightness standard deviation w2 by 80% temporarily and correspondingly increases the weight of the saturation mean w1. By weakening the weight of light and shadow textures and enhancing the weight of color purity, false physical features caused by reflections are effectively filtered out, ensuring the correctness of the arbitration results.

[0061] Furthermore, when both manual annotation and AI Agent annotation are deemed to be in violation of physical constraint arbitration, the overlapping area between the manual annotation box and the AI ​​Agent annotation box is calculated as the proportion of the area of ​​the manual annotation box and the area of ​​the AI ​​Agent annotation box, respectively. When the proportion is greater than a preset threshold and the defective area meets the preset spatial physical co-occurrence law, the manual annotation results and the AI ​​Agent annotation results are manually reviewed.

[0062] In practical implementation, in harsh engineering environments, cracks are often accompanied by water seepage, or salt precipitation may occur at the edges of the seepage area. This complex defect with multiple intertwined attributes can cause serious confusion between human annotators and AI agents when selecting bounding boxes and classifying data, triggering scenario four (both parties violate regulations). If directly judged as invalid annotation, a large number of high-value, complex samples will be mistakenly excluded.

[0063] Response strategy: Introduce a "composite defect dependency rule chain" into the domain physical constraint rule base. When the judgment result falls into scenario four, the system will not immediately reject it, but will calculate the ratio of the overlapping area of ​​the two bounding boxes to their respective total areas. If the overlap ratio is higher than 70%, and meets the spatial physical associated laws such as "cracks are inside seepage" or "saltification surrounds seepage", the system will automatically activate the multi-label tolerance mechanism, transform the sample into scenario three, and push it to experts for rule-level review, thereby protecting the annotation data of complex composite defect samples.

[0064] Conversely, if the overlap area of ​​the two calculated annotation boxes is less than 70% of their respective total areas, the system determines from a spatial topological perspective that the two bifurcation boxes are not overlapping representations of the same multi-attribute interwoven defect, but rather represent two spatially independent defects (such as an independent micro-crack and an independent slight seepage next to it).

[0065] At this point, the system directly exits the 'composite defect dependency rule chain' process, determines the two as two independent and non-conflicting single-target annotation items, and retains them independently in the annotation library at the same time, avoiding unnecessary arbitration intervention and ensuring the integrity of multi-target compact space annotation.

[0066] It should be understood that the rule base evolution interface, for Scenario 3: genuine disagreement samples where both parties are legitimate, is recorded as boundary cases not covered by the physical constraint rule base and pushed to experts for rule-level review (rather than sample-level review). After expert confirmation, incremental updates to the rule base can be triggered. This is a crucial leap in expert review efficiency from sample-level to rule-level—once a new rule passes review, it affects the processing of all subsequent similar samples.

[0067] In this embodiment, firstly, defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects are predefined to construct a domain physical constraint rule base. Then, manual annotation results and AI agent annotation results are obtained within defect areas of water conservancy project images. The intersection-union ratio (IUR) of the manually annotated boxes and the AI ​​agent annotated boxes is calculated based on these results. Subsequently, when the IUR is less than a preset threshold, physical constraint arbitration is performed on the manually annotated results and the AI ​​agent annotated results based on the domain physical constraint rule base. This embodiment constructs a domain physical constraint rule base containing a four-tuple structure, performs physical legality determination on both sides of the dispute, and outputs arbitration awards according to different dispute scenarios, achieving automatic adjudication of disputed samples, significantly reducing the frequency of expert intervention, while ensuring the physical legality of the arbitration awards. In this embodiment, firstly, defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects are predefined to construct a domain physical constraint rule base. Then, manual annotation results and AI Agent annotation results within defect areas in water conservancy project images are obtained. The intersection-union ratio (IUR) between the manually annotated boxes and the AI ​​Agent annotated boxes is calculated based on these results. Subsequently, when the IUR is less than a preset threshold, or when the category labels of the manually annotated boxes and the AI ​​Agent annotated boxes are inconsistent, physical constraint arbitration is performed on the manually annotated results and the AI ​​Agent annotated results based on the domain physical constraint rule base. This embodiment constructs a domain physical constraint rule base containing a four-tuple structure, performs physical legality determination on both sides of the disagreement, and outputs arbitration awards according to different disagreement scenarios, achieving automatic adjudication of disagreement samples, significantly reducing the frequency of expert intervention, while ensuring the physical legality of the arbitration awards.

[0068] Reference Figure 3 , Figure 3 This is a structural block diagram of the first embodiment of the human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule library of the present invention.

[0069] like Figure 3 As shown, the human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule base proposed in this embodiment of the invention includes: The rule base management module 3001 is used to determine the defect types, physical attribute names, constraint relationships and corresponding threshold parameters in water conservancy projects in order to build a domain physical constraint rule base. The divergence detection module 3002 is used to acquire the manual annotation results and AIAgent annotation results in the defect area of ​​the water conservancy project image; The divergence detection module 3002 is further configured to calculate the intersection-union ratio of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results; The physical legality determination module 3003 is used to perform physical constraint arbitration on the manual annotation result and the AI ​​Agent annotation result based on the domain physical constraint rule library when the intersection-union ratio of the annotation box is less than a preset threshold, or when the category label of the manual annotation and the category label of the AI ​​Agent are inconsistent. Arbitration award module 3004 is used to output the arbitration result of physical constraints.

[0070] Other embodiments or specific implementations of the human-computer collaborative annotation disagreement arbitration system based on the domain physical constraint rule library of the present invention can refer to the above-described method embodiments, and will not be repeated here.

[0071] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0072] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0074] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule base, characterized in that, The method includes the following steps: Predefine the defect types, physical attribute names, constraint relationships, and corresponding threshold parameters in water conservancy projects to construct a domain physical constraint rule library; Obtain both manual and AI agent annotation results for defect areas in images of water conservancy projects; Calculate the intersection-union ratio (IU) of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results; When the intersection-union ratio of the labeled boxes is less than a preset threshold, or when the category labels of the manually labeled boxes and the category labels of the AI ​​Agent are inconsistent, physical constraint arbitration is performed on the manually labeled boxes and the AI ​​Agent labeled boxes based on the domain physical constraint rule base.

2. The method as described in claim 1, characterized in that, The step of calculating the intersection-union ratio (IUU) of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results includes: The coordinates of the manually labeled boxes are determined based on the manually labeled results, and the coordinates of the AIAgent labeled boxes are determined based on the AIAgent labeled results. Calculate the area of ​​the intersecting region based on the coordinates of the manually labeled bounding box and the coordinates of the AI ​​Agent labeled bounding box; The total union area is calculated using the union area formula based on the area of ​​the manually labeled box, the area of ​​the AI ​​Agent labeled box, and the area of ​​the intersecting region. The intersection-union ratio (IUR) between the manually labeled bounding boxes and the AI ​​Agent labeled bounding boxes is calculated using the IUR formula based on the area of ​​the intersecting region and the total union area.

3. The method as described in claim 2, characterized in that, The formula for the area of ​​the union is: Area(Bh∪Ba)=Area(Bh)+Area(Ba)-Area(Bh∩Ba) In the formula, Area(Bh∪Ba) is the total union area, Area(Bh) is the area of ​​the manual annotation box, Area(Ba) is the area of ​​the AIAgent annotation box, and Area(Bh∩Ba) is the area of ​​the intersecting region. The formula for the crossover-union ratio is: IoU=Area(Bh∩Ba) / Area(Bh∪Ba) In the formula, IoU is the intersection-union ratio between manually labeled bounding boxes and AI agent labeled bounding boxes.

4. The method as described in claim 1, characterized in that, After arbitrating the manual annotation results and the AI ​​Agent annotation results based on the domain physical constraint rule base, the process further includes: Obtain the category labels of the AI ​​Agent annotation results; When the category label is seepage or salting out, determine whether there is a preset interference scenario in the defect area of ​​the water conservancy project image; If so, the moisture index of the defective area is calculated using the moisture index formula; The AI ​​Agent labeling results are corrected based on the humidity index; Based on the aforementioned domain physical constraint rule base, a second physical constraint arbitration is performed on the manually labeled results and the corrected AI Agent labeled results.

5. The method as described in claim 4, characterized in that, The calculation of the moisture index of the defective area using the moisture index formula includes: Obtain the maximum brightness value of the pixels within the defective region; When the maximum brightness value is greater than the preset saturation threshold, the weights in the humidity index formula are adjusted. Based on the adjusted weights, the humidity index of the defective area in the water conservancy project image is calculated using the humidity index formula.

6. The method as described in claim 4, characterized in that, The formula for the humidity index is: moisture_index=w1×saturation_mean+w2×value_std+w3×smoothness In the formula, w1, w2, and w3 are weights, moisture_index is the moisture index, saturation_mean is the mean saturation of the defect region, value_std is the standard deviation of the brightness of the defect, and smoothness is the texture smoothness of the defect region.

7. The method as described in claim 1, characterized in that, After arbitrating the manual annotation results and the AI ​​Agent annotation results based on the domain physical constraint rule base, the process further includes: When both manual annotation and AI Agent annotation are deemed to be in violation of physical constraint arbitration, the overlapping area between the manual annotation box and the AI ​​Agent annotation box is calculated as the proportion of the area of ​​the manual annotation box and the area of ​​the AI ​​Agent annotation box, respectively. When the proportion is greater than a preset threshold and the defective region meets the preset spatial physical associated law, the manual annotation results and the AI ​​Agent annotation results are pushed to experts for review.

8. A human-computer collaborative annotation disagreement arbitration system based on a domain physical constraint rule base, characterized in that, The system includes: The rule base management module is used to predefine the defect types, physical attribute names, constraint relationships and corresponding threshold parameters in water conservancy projects in order to build a domain physical constraint rule base. The divergence detection module is used to acquire both manual annotation results and AI agent annotation results within defect areas in water conservancy project images; The divergence detection module is also used to calculate the intersection-union ratio of the manually labeled boxes and the AI ​​Agent labeled boxes based on the manually labeled results and the AI ​​Agent labeled results; The physical legality determination module is used to arbitrate the physical constraints of the manual annotation results and the AI ​​Agent annotation results based on the domain physical constraint rule library when the intersection-union ratio of the annotation boxes is less than a preset threshold, or when the category labels of the manual annotations and the category labels of the AI ​​Agents are inconsistent. The arbitration award module is used to output the arbitration results of physical constraints.

9. A human-computer collaborative annotation disagreement arbitration device based on a domain physical constraint rule base, characterized in that, The device includes: a memory, a processor, and a human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule base stored on the memory and executable on the processor, wherein the human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule base is configured to implement the steps of the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule base as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library. When the human-computer collaborative annotation disagreement arbitration program based on a domain physical constraint rule library is executed by the processor, it implements the steps of the human-computer collaborative annotation disagreement arbitration method based on a domain physical constraint rule library as described in any one of claims 1 to 7.

Images