Transformer cable head quality detection method based on target detection and multi-modal large model

Abstract

The present application belongs to the technical field of transformer cable head construction quality, and particularly relates to a transformer cable head quality detection method based on target detection and a multi-modal large model. A transformer cable head quality detection component is set; a statement related to the installation quality of the transformer cable terminal is parsed and converted into a rule card executable by the multi-modal large model; the collected electric room construction site image is input into the multi-modal large model, the image is preliminarily globally semantically analyzed according to the predefined rule card, a multi-modal structured preliminary report is generated, if the overall confidence of the result of the target detection model is lower than the preset multi-modal semantic threshold, the image area is marked as a semantic uncertain area; based on the semantic uncertain area, the target detection model is used for directional visual detection on the semantic uncertain area, the result of the target detection model is obtained, and the result of the target detection model is fed back to the multi-modal large model for confidence reasoning review to obtain the final comprehensive judgment.

Description

Technical Field

[0001] This invention belongs to the technical field of transformer cable head construction quality, specifically relating to a transformer cable head quality inspection method based on target detection and multimodal large model. Background Technology

[0002] In the construction of power systems, transformer cable heads, as key connecting components of the power supply and distribution system, play a decisive role in the safety, stability, and reliability of the power grid. High-quality transformer cable head construction can effectively ensure smooth power transmission, reduce the probability of faults, and thus ensure the stable operation of the entire power grid; conversely, defects in construction quality may trigger a series of power failures, seriously affecting the operation of the power grid.

[0003] Currently, the acceptance of transformer cable head construction quality mainly relies on manual inspection. However, this traditional method has many drawbacks. On the one hand, manual inspection is inefficient, requiring significant manpower and time investment for large-scale power engineering projects, making it difficult to meet the demands of rapid project progress. On the other hand, human judgment is highly subjective; differences in experience and skill levels among inspection personnel lead to inconsistent standards for evaluating construction quality, resulting in biases and affecting the accuracy of the acceptance results. Furthermore, manual inspection has limited coverage; it often fails to conduct comprehensive and detailed inspections of hidden or hard-to-reach areas, causing some potential quality problems to go undetected. These issues make traditional manual inspection methods inadequate for meeting the urgent needs of modern smart grids for high-quality, high-efficiency project acceptance.

[0004] In conclusion, existing methods for quality acceptance of transformer cable head construction and target detection models cannot meet the needs of modern smart grid construction. Summary of the Invention

[0005] This invention aims to provide a transformer cable head quality inspection method based on target detection and multimodal large model to solve several problems existing in the prior art: on the one hand, traditional manual acceptance is inefficient and highly subjective, making it difficult to meet the quality management needs of large-scale construction sites; on the other hand, although a single visual detection model can achieve target recognition, it cannot perform semantic reasoning and structured analysis of complex scenes, making it difficult to accurately determine the connection relationship of cable heads.

[0006] To overcome the problems in the prior art, this invention proposes a method for quality inspection of transformer cable heads based on target detection and a multimodal large model.

[0007] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0008] This invention provides a method for quality inspection of transformer cable heads based on target detection and a multimodal large model, comprising the following steps:

[0009] A transformer cable head quality inspection component is set up, which includes the connection part, the three-phase identification of the cable terminal, and the grounding; statements related to the installation quality of transformer cable terminals are parsed and obtained, and then converted into multimodal large-scale executable rule cards;

[0010] The collected images of the power substation construction site are input into a multimodal large model. Preliminary global semantic analysis is performed on the images according to predefined rule cards to generate a preliminary multimodal structured report. The preliminary report includes the overall confidence score of the target detection model results. If the overall confidence score of the target detection model results is lower than the preset multimodal semantic threshold, the image region is marked as a semantically uncertain region.

[0011] Based on semantically uncertain regions, a target detection model is used to perform directional visual detection on semantically uncertain regions. The results of the target detection model are then fed back to a multimodal large model for confidence inference verification in order to obtain the final comprehensive judgment.

[0012] Furthermore, statements related to the installation quality of transformer cable terminals are parsed and transformed into executable rule cards for a multimodal large model, including:

[0013] This document analyzes the core standards for transformer cable head construction quality and outlines all clauses concerning the quality of transformer cable terminal installation.

[0014] By leveraging the semantic understanding, contextual reasoning, and structured generation capabilities of large language models, this approach, after automatically identifying, cleaning, and semantically reconstructing normative clauses, performs engineered analysis of key entities, attributes, and their constraints within the clauses, forming a unified and standardized semantic element library. Furthermore, based on this semantic element library, the originally qualitative and vague natural language descriptions are transformed into a logically clear, parameter-specific, and directly executable system of judgment rules.

[0015] The executable decision rules are transformed into a prompt word format that can be parsed by a multimodal large model, forming a rule card.

[0016] Furthermore, the inspection of the connection points includes: detecting looseness, fracture, and corrosion of the connection points. Looseness is determined by geometrically asymmetrical gaps and component displacement, fracture is identified by cracks and fracture edge features, and corrosion is determined by color changes and rust spots.

[0017] Furthermore, the three-phase markings of the cable terminals include: combining image information and predefined marking patterns to check whether the yellow, green, and red three-phase markings are correct and clear.

[0018] Furthermore, grounding inspection includes: checking the cable head for copper braided tape or grounding wire, and inspecting the integrity and connection position of the copper braided tape or grounding wire.

[0019] Furthermore, the multimodal large model is MiniCPM-V 4.5.

[0020] Furthermore, the structured preliminary report M:

[0021] ;

[0022] In the above formula, For predefined rule cards; Described in natural language; The confidence score; I represents a multimodal large model; I represents an image.

[0023] Furthermore, wherein the confidence level The formula for a fraction is as follows:

[0024] ;

[0025] In the above formula, Indicates a multimodal large model targeting k The classification prediction vector output by each detected target; This indicates that the vector is in the category dimension. j The probability is obtained by normalizing the above. This indicates that the highest probability value among all categories is taken as the confidence level of the target.

[0026] Furthermore, based on semantically uncertain regions, an object detection model is used to perform directional visual detection on these regions, yielding the results of the object detection model, including:

[0027] D =Detect( I, U ) ={( bj, cj, pj )}, j=1,2,…, m;

[0028] Where D represents the result of the target detection model; bj Indicates the first j The bounding box position of each detected object; cj ∈C represents the corresponding category label; pjPrecision score; Detect represents the detection function of the object detection model; U represents the semantically uncertain region.

[0029] Furthermore, the results of the object detection model are fed back to the multi-modal large model for confidence inference review, including:

[0030] The results of the object detection model and the multi-modal structured preliminary report are jointly input into the multi-modal large model for semantic-level joint inference review to generate the final comprehensive judgment result F:

[0031] F = Multimodal(M, D) = (r, s);

[0032] In the above formula, r ∈ (qualified, unqualified, require manual review), s ∈ [0,1] represents the comprehensive confidence score; M represents the multi-modal structured preliminary report.

[0033] Furthermore, the final comprehensive judgment includes:

[0034] According to the calculated comprehensive confidence s, output the final judgment result:

[0035] If s > the first threshold, output qualified; if s < the second threshold, output unqualified; if the second threshold < s < the first threshold, output require manual review.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] (1) The present invention integrates object detection and multi-modal large models, overcoming the disadvantages of low efficiency and strong subjectivity in traditional manual acceptance. The multi-modal large model first conducts a preliminary global semantic analysis on the images of the electrical room construction site, generates a structured preliminary report containing the overall confidence, and can screen out the semantically uncertain regions. Subsequently, the object detection model conducts targeted detection on these regions, and then feeds the results back to the multi-modal large model for confidence inference review. This multi-step and multi-model collaborative method greatly improves the detection efficiency, reduces the subjective error of manual judgment, significantly improves the accuracy of the results of the object detection model, and can better meet the quality management requirements of large-scale construction sites.

[0038] (2) The present invention transforms the power construction acceptance specification into an inference mapping method that can be understood by the multi-modal model. By performing semantic parsing and structured modeling on the specification clauses, the originally abstract text content is converted into an executable rule card (Rule Card). The rule card guides the multi-modal model to accurately understand the specification requirements and perform corresponding detection tasks with clear detection objectives, constraint conditions, and logical relationships.

[0039] (3) This invention introduces a multimodal semantic representation and reasoning mechanism, integrates visual information and semantic rules, and realizes a structured understanding and logical judgment of image content. Compared with traditional detection models that can only identify the existence and category of objects, this mechanism has semantic-level reasoning and scene understanding capabilities, and can perform complex detection tasks involving structural relationships and functional judgments, thereby achieving an upgrade from "recognition" to "understanding" in detection and improving the intelligence and automation level of detection. Attached Figure Description

[0040] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the process of the present invention;

[0042] Figure 2 This is a schematic diagram of the results of the active visual target detection model of the present invention. Detailed Implementation

[0043] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] This invention introduces a multimodal large model, which integrates image features with semantic information from rule cards to address the problem that visual models cannot perform semantic reasoning and structured analysis in complex scenes. However, in real-world scenarios, the multimodal large model is still susceptible to factors such as multimodal illusions, image occlusion, and changes in lighting, leading to insufficient confidence or biased results. Therefore, this invention further proposes a collaborative verification mechanism driven by multimodal semantic uncertainty. This mechanism uses the semantic uncertainty of the multimodal large model as a trigger signal, introduces a target detection model for targeted review, and then uses a multi-model cross-validation strategy to verify the consistency of the identification results in key areas, thereby significantly improving the accuracy, robustness, and interpretability of construction quality acceptance.

[0045] In this embodiment, refer to Figure 1This paper presents a method for quality inspection of transformer cable heads based on target detection and a multimodal large model, including the following steps:

[0046] The system parses and extracts statements related to the installation quality of transformer cable terminals, and converts them into executable rule cards for a multimodal large model; it also sets up a transformer cable head quality detection component, which includes the connection point, the three-phase identification of the cable terminal, and grounding.

[0047] The collected images of the substation construction site are input into a multimodal large model. Based on the predefined rule cards and detection range, preliminary global semantic analysis is performed on the images to generate a preliminary multimodal structured report. The preliminary report includes the overall confidence of the target detection model results. If the overall confidence of the target detection model results is lower than the preset multimodal semantic threshold, the image region is marked as a semantically uncertain region.

[0048] Based on semantically uncertain regions, a target detection model is used to perform directional visual detection on suspected regions to obtain the results of the target detection model. The results of the target detection model are then fed back to a multimodal large model for confidence inference verification to obtain the final comprehensive judgment.

[0049] The following is a detailed explanation of each of the above steps:

[0050] Step 100: Parse and obtain statements related to the installation quality of transformer cable terminals, and convert them into executable rule cards for a multimodal large model.

[0051] This step, based on automated acceptance procedures for the power construction industry, constructs a rule card-based system. This system aims to systematically transform the complex and abstract acceptance specifications within the power construction industry into a structured set of understandable and executable instructions. It also defines the logical thought process and judgment criteria, thus laying a solid foundation and providing a unified evaluation basis for the entire automated acceptance process. Within this methodological framework, all subsequent steps strictly adhere to the pre-defined logical paths and judgment criteria of the rule cards, ensuring the rigor and standardization of the acceptance process.

[0052] Step 110: Analyze the core standards for transformer cable head construction quality and review all clauses concerning the quality of transformer cable terminal installation.

[0053] First, leveraging the semantic understanding and long text processing capabilities of a large language model, a full scan and intelligent screening of the "Technical Specifications for Construction and Acceptance of Power Construction Projects" and the "Specifications for Construction and Acceptance of Distribution Equipment Installation Projects" were performed. Based on contextual information, the model automatically identified and extracted core clauses related to transformer cable terminal installation, forming an initial corpus.

[0054] Subsequently, based on engineering logic, the clauses are automatically categorized into multiple dimensions such as "connection reliability," "identification standardization," and "grounding safety," with precise annotations of the cited sources (e.g., "DL / T 5161.5-2018 3.2.4") and specific applicable scenarios to ensure clear traceability. Next, deep text cleaning and standardization are performed: leveraging the instruction-following capabilities of the large model, non-rigid modifiers such as "should" and "should" are intelligently removed, extracting core constraints; simultaneously, based on the built-in power industry knowledge graph, terminology normalization is performed, for example, automatically mapping "copper braided tape" and "soft connection" to "copper flexible conductor," eliminating semantic ambiguity. On this basis, a standardized clause-based corpus is constructed, no longer relying on traditional part-of-speech tagging tools, but directly using the model to perform semantic segmentation and intent recognition of the text.

[0055] Finally, the logical structure is reconstructed: the chain-of-thought reasoning ability of the large model is used to analyze the subject-verb-object logic in the clauses, identify key actions such as "connection" and "identification" and their corresponding modifying states such as "firm" and "clear"; the clause logic tree is automatically constructed to clarify the dependency relationship between each element (for example, to determine that "firm connection" must simultaneously satisfy the two sub-conditions of "physical contact" and "mechanical fastening"), laying the logical foundation for subsequent rule generation.

[0056] Step 120: Use semantic parsing technology to understand the meaning of the natural language descriptions in the clauses.

[0057] Leveraging the deep semantic reasoning capabilities of large language models, qualitative natural language descriptions in standards are decomposed and quantified in an engineering manner. For example, when the model reads "firm connection," it decodes it based on engineering common sense into detectable physical indicators such as "whether there is a gap at the contact surface" and "no macroscopic cracks." For "clear phase identification," the model derives specific parameters such as "color standard (A yellow / B green / C red)" and "geometric position." For "reliable grounding wire," it automatically associates it with quantitative constraints such as "grounding wire physical object (copper ≥ 25mm²)," "insulation integrity (whether there are cracks)," and "connection continuity."

[0058] Step 130: Construct a semantic element library of entities, attributes, and constraints using structured generation techniques.

[0059] Instead of traditional word embedding and syntactic analysis, this model directly leverages the structured output capabilities of a large model to accurately extract key information from text. Based on a predefined schema, the model identifies core entities such as "bolt," "grounding wire," and "color mark," and associates them with their physical attributes (material, specifications) and geometric features (shape, color). Simultaneously, the model can parse complex logical relationships within sentences, integrating the extracted entities, attributes, and constraints into structured semantic elements (such as JSON objects). This element library is stored in a standardized "entity-attribute-threshold" format, transforming fragmented document information into a computer-readable knowledge system, providing high-precision data support for automated rule generation.

[0060] Step 140: Based on the semantic element library, transform the natural language description into executable decision rules with clear logic and structure.

[0061] By leveraging the logical reasoning capabilities of large-scale models, structured semantic elements are transformed into judgment rules with rigorous logic. Specifically, key information in the clauses is accurately mapped and stored in the form of "entity-attribute-value" triples.

[0062] For example, the requirement of "secure cable head connection" is broken down into {cable head, contact surface flatness, ≤0.2mm}, specifying the upper limit value for the flatness of the cable head contact surface. Similarly, the requirement of "clear phase identification" is mapped to {phase identification, identification color, A phase yellow, B phase green, C phase red}, defining the color attributes corresponding to different phase identifications. This triplet format transforms the vague requirements described in natural language into a structured and precise data representation.

[0063] Structured elements are transformed into executable decision rules through a logical rule engine (such as Drools) or a decision tree model. For example, taking the determination of "cable head connection is secure" as an example, the logical rule engine sets the rule as follows: "If the flatness of the cable head contact surface is ≤0.2mm and there are no cracks or corrosion, then 'secure connection' is determined to be true; otherwise, it is false." This rule combines two key attribute conditions through a logical AND relationship. Only when both conditions are met simultaneously is "secure connection" determined to be true; otherwise, the determination is false.

[0064] To determine whether the phase identification is clear, a rule can be set: "If the color of the phase identification conforms to the regulations of yellow for phase A, green for phase B, and red for phase C, then the 'phase identification is clear' is determined to be true; otherwise, it is false." This rule directly judges the attribute of the identification color to ensure that the phase identification color meets the standard requirements.

[0065] Regarding the determination of "grounding reliability", the generation rule is: "If the presence of copper braided tape or grounding wire is detected, and there is no breakage or corrosion, then 'grounding reliability' is determined to be true; otherwise, it is false." Only when both meet the corresponding conditions is "grounding reliability" determined to be true.

[0066] Through the above steps, the clause requirements described in natural language are transformed into structured judgment rules that can be understood and executed by computers, thereby realizing the automated judgment and verification of clause requirements and improving the accuracy and efficiency of acceptance work.

[0067] Step 150: Convert the executable decision rules into a prompt word format that can be parsed by the multimodal large model to form a rule card.

[0068] The previously generated executable judgment rules are transformed into prompt word formats that can be accurately parsed by the multimodal large model, forming comprehensive, detailed, and highly operable rule cards. These rule cards must cover various installation quality requirements, clearly defining key information such as quantitative indicators, characteristic patterns, and judgment logic, ensuring that the multimodal large model can accurately judge installation quality based on these rules.

[0069] Cable head connection secure rule card:

[0070] (1) Rule name: Rule for judging the firmness of cable head connection.

[0071] (2) Applicable scenarios: During the construction and acceptance of power construction projects, the connection firmness of the cable head of the transformer cable terminal is tested.

[0072] (3) Feature extraction methods:

[0073] Connection appearance flatness: Using high-precision image acquisition equipment, images of the cable head connection area are acquired, the flatness deviation value is calculated, and the presence of cracks and corrosion is determined.

[0074] (4) Judgment logic: If the flatness deviation of the connection appearance is ≤0.2mm and there are no cracks or corrosion, then the "cable head connection is firm" is qualified; otherwise, it is unqualified.

[0075] (5) Tolerance threshold: Considering factors such as measurement error, the measurement error of the tightening torque is allowed to be within ±1 N·m, and the measurement error of the flatness deviation of the connection appearance is allowed to be within ±0.05 mm. If the measured value still does not meet the judgment logic condition after considering the tolerance threshold, it is judged as unqualified.

[0076] Clear phase identification rule card:

[0077] (1) Rule name: Phase identification clear determination rule.

[0078] (2) Applicable scenarios: During the installation and acceptance of transformer cable terminals, the clarity of phase markings is tested.

[0079] (3) Feature extraction methods:

[0080] Color identification: Analyze the colors of phases A, B, and C in the image, obtain their RGB values ​​or color parameters such as hue, saturation, and brightness, and compare them with standard colors (phase A: yellow, phase B: green, phase C: red).

[0081] Font size: The height of the logo font is measured using image measurement technology to obtain the actual font height value.

[0082] Position deviation: Using a preset standard position as a reference, image positioning technology is used to calculate the deviation distance between the actual position of the marker and the standard position.

[0083] (4) Judgment logic: Check whether the colors of phase A / B / C in the detection image meet the yellow / green / red standard and the color parameter deviation is within the allowable range; if all are met, the "phase identification is clear" is qualified; otherwise, it is unqualified.

[0084] (5) Tolerance threshold: When comparing colors, a certain degree of color deviation is allowed. The specific deviation range is set according to the actual application scenario and standard requirements. Generally, a color parameter deviation within ±5% can be considered as compliant with the standard. If the measured value still does not meet the judgment logic condition after considering the tolerance threshold, it is judged as unqualified.

[0085] Grounding Reliability Rule Card:

[0086] (1) Rule name: Grounding reliability judgment rule.

[0087] (2) Applicable scenarios: During the installation and acceptance of transformer cable terminals, the reliability of the grounding system is tested.

[0088] (3) Feature extraction methods:

[0089] Is there a copper braided strip or grounding wire present?

[0090] Insulation layer damage status: The insulation layer of the grounding conductor in the image is inspected to analyze whether there are cracks in the insulation layer and to measure the length of the cracks.

[0091] (4) Judgment logic: If the copper braided tape or grounding wire is present and the insulation layer is undamaged (crack length ≤ 5mm), then the grounding is deemed "reliable" and qualified; otherwise, it is deemed unqualified.

[0092] (5) Tolerance threshold: The measurement error of the insulation layer crack length is allowed to be within ±0.5mm. If the measured value still does not meet the judgment logic condition after considering the tolerance threshold, it is judged as unqualified.

[0093] In practical applications, the rule card is input into the multimodal large model in the form of prompt words. The multimodal large model obtains relevant data according to the feature extraction method in the rule card, and then judges the installation quality according to the judgment logic and fault tolerance threshold, and outputs the judgment result.

[0094] Based on the problems and feedback encountered during practical applications, the rule card is continuously optimized and improved. For example, the fault tolerance threshold is adjusted to adapt to different measurement environments and accuracy requirements, the feature extraction method is optimized to improve the accuracy and efficiency of data acquisition, and the decision logic is improved to cover more special cases, ensuring that the rule card can always provide accurate and reliable decision criteria for multimodal large models.

[0095] Step 200: Set up a transformer cable head quality inspection component, which includes defects at the cable head connection, three-phase markings at the cable terminal, and grounding.

[0096] As an example, step 200 specifically includes:

[0097] Step 210: Inspection of cable head connection points.

[0098] Based on the standardized semantic information provided by the input image and prompts, and according to the multimodal large model (MiniCPM-V4.5), defects such as loosening, fracture, and corrosion are identified. Loosening detection relies on geometric asymmetry in the image and the relative displacement of components to identify gaps or uneven contact surfaces at joints. Fracture is determined by analyzing the morphology of cracks. Corrosion is identified by color changes, rust spots, and surface damage to recognize corrosion characteristics of the metal surface.

[0099] Specifically, when given an input image, MiniCPM-V4.5 extracts image features through a visual encoder and uses a projection layer to translate them into embedded tokens (Visual Tokens) compatible with large language models. This enables the language model to directly understand visual information and output corresponding natural language descriptions based on semantic reasoning capabilities.

[0100] Step 220: Inspect the three-phase markings on the cable terminal.

[0101] Based on the standardized semantic information provided by the input image and prompts, the multimodal large model checks the yellow, green and red three-phase markings on the cable terminal one by one to ensure that they meet the strict testing standards.

[0102] Step 230: Grounding test.

[0103] Based on the standardized semantic information provided by the input image and prompts, the multimodal large model first identifies the presence of copper braided tape or grounding wire, and then detects for anomalies such as breakage or corrosion. Subsequently, by analyzing the connection status between the grounding wire and the grounding system, it determines whether the two have formed effective contact, thereby determining whether the grounding is reliable.

[0104] Step 300: Introduce a collaborative verification mechanism driven by the semantic uncertainty of a multimodal large model. Through active visual search and multi-model cross-validation strategies, the target detection model can be targeted for review and result enhancement.

[0105] While multimodal large models can perform semantic reasoning through image and text rules, thus supplementing the shortcomings of object detection in scene understanding, using a multimodal large model alone can still be affected by factors such as multimodal illusion and image occlusion, leading to insufficient judgment confidence. Therefore, this step proposes a collaborative verification mechanism driven by the semantic uncertainty of multimodal large models. Through active visual search and multi-model cross-validation strategies, it achieves targeted review and result enhancement of the object detection model, thereby improving the accuracy and robustness of the final acceptance decision.

[0106] Step 310: Input the high-definition images of the substation construction site into the multimodal large model, perform preliminary global semantic analysis on the images according to the predefined rule cards, and generate a preliminary multimodal structured report. The preliminary report includes the overall confidence of the target detection model results. If the overall confidence of the target detection model results is lower than the preset multimodal semantic threshold, mark the image region as a semantically uncertain region.

[0107] First, high-resolution images (I) of the power substation construction site are acquired and input into the multimodal large model MiniCPM-V 4.5. Global semantic parsing of the input images is performed based on predefined rule cards (R), outputting a preliminary structured report (M).

[0108]

[0109] in, This represents the multimodal large model MiniCPM-V 4.5; These are predefined rule cards used to guide the generation of structured descriptions that conform to construction specifications for multimodal large models; It is a natural language description containing fine-grained attribute information, such as "the cable head is not loose", "the phase marking is clear and the color corresponds to the standard", "the grounding copper braid is reliably connected", as well as the spatial relationship between components, such as "whether the grounding wire and the grounding system form an effective contact". The confidence score reflects the reliability of the multimodal results. In addition, the report M also includes semantic explanations U for the sources of uncertainty, such as "dim image lighting leads to unstable color judgment" or "occlusion areas lead to missing edge features".

[0110] Confidence The formula for a fraction is as follows:

[0111]

[0112] in, Indicates a multimodal large model targeting k The classification prediction vector output by each detected target; This indicates that the vector is in the category dimension. j The probability is obtained by normalizing the above. This indicates that the highest probability value among all categories is taken as the confidence level of the target.

[0113] When the overall confidence level of the target detection model is lower than the preset multimodal semantic threshold When the value is 0.6, the system automatically marks the image region as a semantically uncertain region and enters the active visual detection stage.

[0114] Step 320: Based on the semantically uncertain region, use the object detection model to perform directional visual detection on the suspected region and obtain the result of the object detection model.

[0115] For semantically uncertain regions, an active visual detection mechanism is triggered. An object detection model (such as GroundingDINO) is invoked to perform directional detection on the uncertain regions, yielding the results of the object detection model:

[0116]

[0117] in, bj Indicates the first j The bounding box position of each detected object. cj ∈C represents the corresponding category label (such as "cable head connection", "phase identifier", "grounding connection point", etc.); Detect represents the detection function of the target detection model; pj The accuracy score reflects the reliability of the target detection model's results, such as... Figure 2 As shown.

[0118] Step 330: Feed the results of the target detection model back to the multimodal large model for confidence inference verification in order to obtain the final comprehensive judgment.

[0119] The result D of the target detection model will be fed back to the multi-modal large model again for confidence inference review to obtain the final comprehensive judgment. Among them, when the confidence level is greater than 70%, the judgment result is qualified; when the confidence level is less than 30%, the judgment result is unqualified; if the confidence level is between 30% and 70%, it is determined that manual re-inspection is required to ensure the inspection accuracy of key scenarios.

[0120] After active vision detection and double-threshold verification, the system inputs the result D of the target detection model and the multi-modal report M into the multi-modal large model for semantic-level joint inference review, and generates the final comprehensive judgment result F:

[0121]

[0122] Among them, r ∈ (qualified, unqualified, require manual review), and s ∈ [0,1] represents the comprehensive confidence score.

[0123] Among them, the formula for the comprehensive confidence s is as follows:

[0124]

[0125] Among them, represents the weight coefficient, which determines the contribution ratio of target detection and multi-modal reasoning in the comprehensive judgment. Set . According to the calculated s, the system outputs the final judgment result: if s > 0.7, output qualified; if s < 0.3, output unqualified, if 0.3 < s < 0.7, output require manual review.

[0126] Through the collaborative work of target detection and multi-modal reasoning, a full-process closed-loop from active detection of semantically uncertain regions to confidence verification is achieved. It not only ensures the automation processing efficiency of high-confidence scenarios, but also ensures the inspection accuracy of low-confidence scenarios through the manual re-inspection channel. Finally, a digital acceptance report including defect type, location, severity, and specification basis is generated, realizing the intelligence and standardization of construction acceptance.

[0127] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for quality inspection of transformer cable heads based on target detection and a multimodal large model, characterized in that, Includes the following steps: A transformer cable head quality inspection component is established, comprising the connection points, three-phase markings at the cable termination, and grounding. Inspection of the connection points includes checking for looseness, breakage, and corrosion. Looseness is determined by geometrically asymmetrical gaps and component displacement; breakage is identified by cracks and fracture edge features; and corrosion is determined by color changes and rust spots. Inspection of the three-phase markings at the cable termination includes checking the correctness and clarity of the yellow, green, and red markings using image information and a predefined marking pattern. Grounding inspection includes checking for the presence of copper braided tape or grounding wire at the cable head, and verifying the integrity and connection position of the copper braided tape or grounding wire. The process involves parsing and extracting statements related to the quality of transformer cable terminal installation, and transforming them into executable rule cards for a multimodal large-scale model. This includes: parsing the core standards for transformer cable head construction quality and compiling all clauses related to the quality of transformer cable terminal installation; extracting entities, attributes, and relationships from the clauses using word embedding and dependency parsing techniques to form a structured semantic element library; transforming natural language descriptions into executable decision rules based on the semantic element library; and converting the executable decision rules into a prompt word format that can be parsed by the multimodal large-scale model to form rule cards. The collected images of the power substation construction site are input into a multimodal large model. Preliminary global semantic analysis is performed on the images according to predefined rule cards to generate a preliminary multimodal structured report. The preliminary report includes the overall confidence score of the target detection model results. If the overall confidence score of the target detection model results is lower than the preset multimodal semantic threshold, the image region is marked as a semantically uncertain region. Based on semantically uncertain regions, a target detection model is used to perform directional visual detection on semantically uncertain regions. The results of the target detection model are then fed back to a multimodal large model for confidence inference verification in order to obtain the final comprehensive judgment.

2. The method for quality inspection of transformer cable heads based on target detection and multimodal large model according to claim 1, characterized in that, The multimodal large model is MiniCPM-V 4.

5.

3. The method for quality inspection of transformer cable heads based on target detection and multimodal large model according to claim 1, characterized in that, The structured preliminary report M: ; In the above formula, For predefined rule cards; Described in natural language; The confidence score; I represents a multimodal large model; I represents an image; Wherein, the confidence level The formula for a fraction is as follows: In the above formula, This represents the classification prediction vector output by the multimodal large model for k detected targets; This indicates that the probability is obtained by normalizing the vector along the category dimension j. This indicates that the highest probability value among all categories is taken as the confidence level of the target.

4. The method for quality inspection of transformer cable heads based on target detection and multimodal large model according to claim 3, characterized in that, Based on semantically uncertain regions, an object detection model is used to perform directional visual detection on these regions, yielding the results of the object detection model, including: D =Detect( I, U ) ={( bj, cj, pj )},j=1,2,…, m; Where D represents the result of the target detection model; bj Indicates the first j The bounding box position of each detected object; cj ∈C represents the corresponding category label; pj is the accuracy score; Detect represents the detection function of the object detection model; U represents the semantically uncertain region.

5. The method for quality inspection of transformer cable heads based on target detection and multimodal large model according to claim 4, characterized in that, The results of the object detection model are fed back to the multimodal large model for confidence inference verification, including: The results of the target detection model and the preliminary multimodal structured report are input into the multimodal large model for semantic-level joint reasoning verification to generate the final comprehensive judgment result F: F = Multimodal(M, D) = (r, s); In the above formula, r ∈ (qualified, unqualified, requires manual review), s ∈ [0,1] represents the overall confidence score; M represents the multimodal structured preliminary report; The formula for the overall confidence level s is as follows: ； in, This represents the weighting coefficient.

6. The method for quality inspection of transformer cable heads based on target detection and multimodal large model according to claim 5, characterized in that, The final comprehensive judgment includes: Based on the calculated overall confidence level s, the final judgment result is output: If s > the first threshold, output "qualified"; If s < the second threshold, the output is unqualified; If the second threshold < s < the first threshold, output requires manual review.

Images