Lightweight power fitting detection method and system based on dual priori enhancement and medium

Abstract

The application belongs to the technical field of electric power intelligent inspection, and particularly relates to a lightweight electric power fitting detection method and system based on dual priori enhancement and a medium. The method comprises the following steps: acquiring an electric power fitting image of a power transmission line as input; constructing a target detection network; generating high-dimensional static semantic feature embeddings corresponding to various categories according to prompt words of electric power fitting category names and physical attributes; mapping the high-dimensional static semantic feature embeddings to a space matching the feature dimension of the target detection network; completely replacing text features extracted by a general CLIP text encoder by using semantic features output by a semantic projection layer after dimension reduction; directly inputting the semantic features after dimension reduction into a detection head, decoding and outputting the final detection result of the electric power fitting by the detection head; and converting the model into a lightweight neural network inference engine format and deploying the model on an edge device for real-time inference. The application significantly enhances the detection capability of small targets in a complex background.

Description

Technical Field

[0001] This invention belongs to the field of intelligent power inspection technology, specifically a lightweight power fitting detection method, system, and medium based on dual prior enhancement. Background Technology

[0002] In the field of power line inspection, timely and accurate detection of power fittings such as insulators and vibration dampers is crucial for ensuring the safe operation of the power grid. Power fittings in transmission lines often exhibit a long-tail distribution, meaning that data samples for a few common fittings are extremely abundant, while data for many special or rare fittings are extremely scarce. Furthermore, different categories, such as various types of suspension clamps and tension clamps, often appear very similar. In addition, many fittings appear as tiny targets in the wide field of view of drone aerial photography, easily concealed within complex tower structures or natural backgrounds. This makes traditional target detection models based on closed-set classification highly susceptible to confusion.

[0003] To address this issue, the introduction of Visual Language Models (VLMs) for zero-shot or open-vocabulary object detection has become a trend. However, the aforementioned technical solutions face two serious bottlenecks in their engineering implementation in vertical fields: YOLO-World relies excessively on the general CLIP model as a text encoder, and its general open vocabulary capabilities are difficult to transfer to the power industry vertical. When faced with complex and similar long-tail fittings, its zero-shot detection accuracy is less than 30%. While directly using large language models such as Qwen2.5-VL for inference provides fine-grained semantic understanding, the spatial target localization accuracy is extremely low, and the inference time per frame is often >500ms, which is simply insufficient to meet the stringent requirements of real-time UAV inspection.

[0004] In traditional multimodal detection networks, the Transformer and Attention structures of models like CLIP account for 67.5% and 70% of the total parameters, respectively, and consume over 70% of the GPU memory. Meanwhile, high-performance inference engines designed for mobile or embedded devices, such as Tencent YouTu's NCNN, generally do not support complete CLIP or large language model inference, preventing high-precision, large-scale models from being deployed on edge computing devices.

[0005] Therefore, there is an urgent need for a high-precision object detection method that can overcome the semantic bottleneck of general multimodal models in vertical domains and get rid of the constraints of large text encoders, so as to achieve lightweight deployment at the edge. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a lightweight power fitting testing method, system, and medium based on dual prior reinforcement.

[0007] This invention adopts the following technical solution: a lightweight power fitting testing method based on dual prior enhancement, comprising: S1: Obtain the image of the power fittings of the transmission line as input; S2: Construct an object detection network based on the YOLO-World architecture, and introduce a feature-level knowledge distillation mechanism in the Neck layer of the feature pyramid of the object detection network; S3: Using a large language model with fine-tuned parameters, generate high-dimensional static semantic feature embeddings for each category based on prompts for the category name and physical attributes of power fittings. S4: Construct and train a semantic projection layer to embed high-dimensional static semantic features into a space that matches the feature dimension of the object detection network. S5: For the general CLIP text encoder and detection head natively included in the target detection network constructed in step S2, the dimensionality-reduced semantic features output by the semantic projection layer in step S4 are used to completely replace the text features extracted by the general CLIP text encoder; the dimensionality-reduced semantic features are directly input into the detection head, and cross-modal fusion calculation is performed with the enhanced multi-scale image visual features finally output by the target detection network in step S2, and the detection head decodes and outputs the final detection result of the power fittings; S6: Remove the weights of the large language model in the network and convert the model into a lightweight neural network inference engine format for real-time inference on edge devices.

[0008] In some embodiments, in S3, the large language model adopts the Qwen2.5-VL model, and its fine-tuning process includes: Input the prompt word template, which includes the official standard name, national standard definition and core visual features of power fittings, into the Qwen2.5-VL model. The model generates high-dimensional static semantic feature embedding vectors for various power fittings offline in one go, and no large language model weights are loaded during the online inference stage.

[0009] In some embodiments, S2 includes: S21: Input the power fittings image obtained in step S1 into the constructed target detection network and output the student network visual features; S22: Synchronously input the same power fitting image obtained in step S1 into the pre-trained large vision model DINOv2 to extract high-resolution standard visual features including global semantic information. S23: Perform a two-step feature alignment strategy in the path aggregation network of the object detection network to align the visual features of the student network with the high-resolution standard visual features. S24: Based on the aligned student network visual features and high-resolution standard visual features, construct a hybrid distillation loss function; calculate the loss value according to the hybrid distillation loss function, and update the network weights of the object detection network through backpropagation.

[0010] In some embodiments, S23 specifically includes: Spatial resolution alignment: Bilinear interpolation is used to upsample the student network visual features in the spatial dimension so that their spatial size is consistent with the spatial size of high-resolution standard visual features. Channel dimension alignment: After the spatial resolution aligned features are passed through a 1×1 convolutional layer, the number of channels is precisely mapped to the same number of channels as the high-resolution standard visual features, resulting in aligned student network visual features.

[0011] In some embodiments, the hybrid distillation loss function : In the formula, This represents the visual features of the student network after processing using the above two-step feature alignment strategy. Visual features extracted by the large visual model DINOv2, representing the frozen parameters; The term on the right represents the mean squared error loss, used to constrain the absolute difference in feature pixel-level values; the term on the right is the cosine similarity loss, used to constrain the semantic orientation consistency of features in the vector space. and To balance the hyperparameter weights of the two losses.

[0012] In some embodiments, S4, a semantic projection layer is constructed and trained, and the construction and training process includes: S41: Using CLIP, a general text encoder included in the object detection network, extract 512-dimensional text feature vectors of various power fittings and use them as teacher supervision signals for training the semantic projection layer. S42: Obtain the 3584-dimensional high-dimensional static semantic feature vectors of various categories of power fittings generated offline by the Qwen2.5-VL large language model, and input them into the randomly initialized semantic projection layer; wherein, the semantic projection layer is an unbiased linear network layer, which linearly maps the input 3584-dimensional features through its internal weight matrix and outputs 512-dimensional reduced semantic features. S43: Based on dimensionality-reduced semantic features and teacher supervision signals, the absolute difference between the two in the feature space is calculated using the mean squared error loss function, enabling the semantic projection layer to learn the mapping and transformation rules across the modal feature space; S44: Calculate the gradient of the mean squared error loss function using the Adam optimizer, and update the weight matrix of the semantic projection layer through backpropagation, so that the randomly initialized semantic projection layer weights learn the correct feature space mapping transformation relationship.

[0013] In some embodiments, in S43, the semantic projection layer optimizes the mean squared error loss function during training. The definition is as follows: In the formula, N is the total number of categories of power fittings; The 3584-dimensional Qwen2.5-VL semantic feature vector of the i-th type of hardware generated offline; W Let be the weight matrix of the linear projection layer to be optimized; This represents the 512-dimensional text feature vector of the i-th type of power fittings extracted by the general CLIP text encoder natively included in the object detection network constructed in step S2, i.e., the teacher supervision signal.

[0014] In some embodiments, S6 includes: S61: Directly export the trained and weighted object detection network into the ONNX general intermediate neural network model format to eliminate the dependence on the underlying training framework. S62: Use a model conversion tool to convert the ONNX format model into the lightweight NCNN format supported by the mobile inference framework, generate a deployment file containing the network structure and weights, and perform real-time inference with pure convolutional computation on an edge device equipped with a mobile chip. S63: Specify the model input size and modify the number and name parameters in the mobile engineering interface, and finally package it into an application to perform real-time inference with pure convolutional computation on an edge device equipped with a mobile chip.

[0015] A system for implementing a lightweight power fitting inspection method based on dual prior enhancement, comprising: The image acquisition module is used to acquire images of power fittings inspection and perform standardized preprocessing. The visual distillation enhancement module has a built-in object detection network based on the YOLO-World architecture. It is used to introduce the DINOv2 large visual model into the Neck layer of the object detection network to perform feature-level knowledge distillation and output enhanced multi-scale image visual features. The offline semantic augmentation module is used to utilize the large language model fine-tuned by the power fittings dataset to generate high-dimensional static semantic features corresponding to each category offline, embed them, and store them in a fixed manner, based on the power fittings category and attribute prompt words. A lightweight dimensionality reduction and alignment module with a built-in semantic projection layer is used to embed high-dimensional static semantic features into a dimension that matches the target detection network. The dimensionality-reduced semantic features replace the CLIP text features of the original network. After being fused with the enhanced visual features, the input is fed into the detection head, and the power fitting detection results are output. The edge-decoupled deployment module is used to remove all large language models and Transformer text encoder weights from the system, convert the pure convolutional detection network into a lightweight inference engine format, and deploy it on edge devices to achieve real-time detection.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a lightweight power fitting detection method based on dual prior reinforcement.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention generates semantic vectors offline and employs a lightweight feature projection layer, completely eliminating the original CLIP model and large language model weights during the inference stage. This improvement allows the model to be completely converted into a pure convolutional and detection head structure that perfectly supports NCNN operation, significantly reducing hardware requirements. Taking the YOLO-World-s version as an example, the converted .bin weight file is only 18.7 MB, with approximately 11.2 M network parameters and a computational cost of approximately 28.7 GFLOPs, achieving high-speed real-time inference of approximately 30 FPS on devices equipped with mobile chips.

[0018] By replacing the general CLIP model with a finely tuned Qwen2.5-VL model on the power dataset to extract text embeddings, the semantic limitations of vertical domains are compensated for, making category relationships clearer. At the same time, the spatial localization branch fully retains the visual feature results after DINOv2 distillation. While achieving extreme lightweight design, the target detection accuracy mAP@0.5 is achieved without any loss, and the ability to detect small targets in complex backgrounds is significantly enhanced. Attached Figure Description

[0019] Figure 1 The overall architecture and processing flowchart of a lightweight power fitting detection method based on dual enhancement of vision and semantic prior are provided for embodiments of the present invention. Figure 2 This is a structural diagram of the module for visual feature distillation using DINOv2 in an embodiment of the present invention; Figure 3 This is a schematic diagram showing the detailed structure of the two-step feature alignment strategy upsampling and 1×1 convolution in an embodiment of the present invention; Figure 4 This is a data flow diagram for constructing an offline static semantic dictionary using a large language model in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the alignment training principle of offline semantic augmentation and lightweight semantic projection layer in an embodiment of the present invention. Figure 6 This is a flowchart illustrating the intermediate format export of the model and the deployment of NCNN on the edge in this embodiment of the invention. Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, a lightweight power fitting testing method based on dual prior enhancement includes: S1: Obtain the image of the power fittings of the transmission line as input; S2: Construct an object detection network based on the YOLO-World architecture, and introduce a feature-level knowledge distillation mechanism in the Neck layer of the feature pyramid of the object detection network; S3: Using a large language model with fine-tuned parameters, generate high-dimensional static semantic feature embeddings for each category based on prompts for the category name and physical attributes of power fittings. S4: Construct and train a semantic projection layer to embed high-dimensional static semantic features into a space that matches the feature dimension of the object detection network. S5: For the general CLIP text encoder and detection head natively included in the target detection network constructed in step S2, the dimensionality-reduced semantic features output by the semantic projection layer in step S4 are used to completely replace the text features extracted by the general CLIP text encoder; the dimensionality-reduced semantic features are directly input into the detection head, and cross-modal fusion calculation is performed with the enhanced multi-scale image visual features finally output in step S2, and the detection head decodes and outputs the final detection result of the power fittings; S6: Remove the weights of the large language model in the network and convert the model into a lightweight neural network inference engine format for real-time inference on edge devices.

[0021] In the specific implementation, step S1: Acquire image data of power fitting inspection and perform preprocessing. High-resolution images captured by a power transmission line inspection drone are acquired. To adapt to the input specifications of the lightweight target detection network, the original images are scaled, proportionally filled, and normalized to be uniformly processed into tensors of a specified size. The input size is 640×640 pixels, and the number of channels is 3. The dimensions of the input tensor are then represented as [B, C, H, W].

[0022] In specific implementation, an object detection network is constructed, and the large visual model DINOv2 is introduced into the Neck layer for feature-level knowledge distillation. Figure 1 and Figure 2 As shown, a target detection framework with a lightweight convolutional network as its backbone is constructed. Addressing the issues of weak global semantic modeling capabilities and high false negative rates for small power fittings in complex backgrounds in traditional backbone networks, this invention introduces a large visual model, DINOv2, as a teacher model for distillation into the feature pyramid network or path aggregation network (FPN / PAN, i.e., the Neck layer) of the target detection network. Using the large visual model DINOv2, whose parameters have been fine-tuned in advance on a dataset containing labeled images of power fittings, as the teacher model, a two-step feature alignment strategy combining upsampling and 1×1 convolution is employed to align the multi-scale image visual features extracted by the target detection network with the standard visual features extracted by the teacher model in terms of spatial resolution and channel dimension. Based on the aligned features, the distillation loss is calculated to complete the knowledge distillation of the visual backbone network and the Neck layer, ultimately extracting the enhanced multi-scale image visual features of the input image.

[0023] In specific implementation, step S2 includes: S21: Input the power fitting image obtained in step S1 into the constructed target detection network; wherein, the target detection network is based on the YOLO-World architecture, which natively includes a visual backbone network and a path aggregation network; after extracting preliminary features through the visual backbone network, the features are input into the path aggregation network, and the visual features of the target detection network are output.

[0024] S22: The same power fitting image obtained in step S1 is synchronously input into the pre-trained large vision model to extract high-resolution standard visual features including global semantic information; wherein, the large vision model is DINOv2, and its parameters have been fine-tuned and weights frozen in advance using a dedicated power fitting dataset containing real aerial images and fitting bounding box annotations.

[0025] S23: As Figure 3The detailed structural diagram shown illustrates that, due to the mismatch in spatial resolution and channel dimension between the high-fidelity visual features extracted by the large visual model DINOv2 and the feature maps output by the lightweight network, this invention designs a two-step feature alignment strategy of "upsampling + 1×1 convolution". This two-step feature alignment strategy is executed in the path aggregation network of the object detection network to align the visual features of the student network with the high-resolution standard visual features. The two-step feature alignment strategy specifically includes: Spatial resolution alignment: Bilinear interpolation is used to upsample the student network visual features in the spatial dimension so that their spatial size is consistent with the spatial size of high-resolution standard visual features. Channel dimension alignment: After the spatial resolution aligned features are passed through a 1×1 convolutional layer, the number of channels is precisely mapped to the same number of channels as the high-resolution standard visual features, resulting in aligned student network visual features.

[0026] S24: Based on the aligned student network visual features and high-resolution standard visual features, construct a hybrid distillation loss function; calculate the loss value according to the hybrid distillation loss function, and update the network weights of the object detection network through backpropagation.

[0027] Mixed distillation loss function : In the formula, This represents the visual features of the student network after processing using the above two-step feature alignment strategy. Visual features extracted by the large visual model DINOv2, representing the frozen parameters; The term on the right represents the mean squared error loss, used to constrain the absolute difference in feature pixel-level values; the term on the right is the cosine similarity loss, used to constrain the semantic orientation consistency of features in the vector space. and To balance the hyperparameter weights of the two losses, the hybrid distillation loss function includes a mean squared error loss for numerical alignment and a cosine similarity loss for semantic orientation alignment, ensuring consistency between feature details and semantic orientation.

[0028] In specific implementation, in step S3, the large language model adopts the Qwen2.5-VL model, and its fine-tuning process includes: The training consisted of 15 rounds with an initial learning rate of 1e-5. The dataset used contained 1600 images of electrical fittings, and the batch size during training was 16. Prompt word templates, including the official standard name, national standard definition, and core visual features of electrical fittings, were input into the Qwen2.5-VL model. High-dimensional static semantic feature embedding vectors for various types of electrical fittings were generated offline in one go, and no large language model weights were loaded during the online inference stage.

[0029] This invention utilizes a large language model to generate a static semantic dictionary for a vertical domain offline. Since the general CLIP model achieves less than 30% accuracy in zero-shot detection in the power industry, this invention employs a finely tuned Qwen2.5-VL large language model tailored to the power fittings dataset to extract specialized semantics. To avoid the significant memory consumption and latency exceeding 500ms per graph caused by online inference with the large language model, this step is performed entirely offline. Figure 4 The data flow diagram shown illustrates how the system automatically processes data in batches using scripts, generating high-dimensional features for 18 hardware categories at once and saving them as static semantic dictionary files in .npy format for direct reading by subsequent detection networks.

[0030] In a specific embodiment, in S4, a semantic projection layer is constructed and trained, and the construction and training process includes: S41: Using CLIP, a general text encoder included in the original object detection network, extract 512-dimensional text feature vectors of various power fittings, and use them as teacher supervision signals for training the semantic projection layer.

[0031] S42: Obtain the 3584-dimensional high-dimensional static semantic feature vectors of various categories of power fittings generated offline by the Qwen2.5-VL large language model, and input them into a randomly initialized semantic projection layer; wherein, the semantic projection layer is an unbiased linear network layer, which linearly maps the input 3584-dimensional features through its internal weight matrix and outputs 512-dimensional reduced semantic features.

[0032] S43: Based on dimensionality-reduced semantic features and teacher supervision signals, the absolute difference between the two in the feature space is calculated using the mean squared error loss function, enabling the semantic projection layer to learn the mapping and transformation rules across the modal feature space; In S43, the semantic projection layer optimizes the mean squared error loss function during training. The definition is as follows: In the formula, N is the total number of categories of power fittings; The 3584-dimensional Qwen2.5-VL semantic feature vector of the i-th type of hardware generated offline; W Let be the weight matrix of the linear projection layer to be optimized; This represents the 512-dimensional text feature vector of the i-th type of power fittings extracted by the general CLIP text encoder natively included in the object detection network constructed in step S2, i.e., the teacher supervision signal.

[0033] S44: Calculate the gradient of the mean squared error loss function using the Adam optimizer, and update the weight matrix of the semantic projection layer through backpropagation, so that the randomly initialized semantic projection layer weights learn the correct feature space mapping transformation relationship.

[0034] In a specific embodiment, S5: Network solidification and replacement of the forward propagation path. Combined with... Figure 1 and Figure 5 As shown, after the projection layer converges, its actual weights are solidified into the network. During the actual inference process, the extremely bloated CLIP text encoder module in the original network is completely removed. The model's forward propagation process is changed as follows: the input sparse view extracts strong visual features through the backbone network and the Neck layer; simultaneously, the network directly reads the Qwen2.5-VL features from the static semantic dictionary, reduces their dimensionality through a lightweight projection layer, and then directly feeds them, along with the visual features, into the detection head to calculate classification confidence and bounding box regression.

[0035] In a specific embodiment, S6 includes: S61: Directly export the trained and weighted target detection network model into the ONNX general intermediate neural network model format to eliminate the dependence on the underlying training framework. S62: Use a model conversion tool to convert the ONNX format model into the lightweight NCNN format supported by the mobile inference framework, generate a deployment file containing the network structure and weights, and perform real-time inference with pure convolutional computation on an edge device equipped with a mobile chip.

[0036] S63: Specify the model input size and modify the number and name parameters in the mobile engineering interface, and finally package it into an application to perform real-time inference with pure convolutional computation on an edge device equipped with a mobile chip.

[0037] Lightweight architecture intermediate format export and edge deployment. For example... Figure 6The deployment flowchart shown illustrates how this embodiment utilizes the NCNN high-performance neural network inference framework for edge deployment to enable operation on edge devices. First, the PyTorch model (.pt file) solidified in step S5 is exported to the ONNX general intermediate neural network model format. Second, the model conversion tool pnnx is used to further convert the ONNX model into an NCNN-specific intermediate model. During this process, the input size of the model is specified, and the generated Python script file is modified to adjust dimensions and post-processing operations. The conversion outputs a .param file containing the network graph structure and a .bin file containing the model weights. Thanks to the decoupling of the large model text encoder, taking the YOLO-World-s network structure as an example, the final generated .bin weight file is only 18.7MB in size, with approximately 11.2M model parameters and approximately 28.7 GFLOPs of computation, truly achieving extreme model slimming. Finally, in the Android Studio mobile development environment, the converted model files were integrated into the project, basic parameters such as category and network nodes were configured, and then compiled and packaged into an application. This application was then deployed on a target terminal device equipped with a Dimensity 9300+ series mobile chip to perform real-time edge inference. This invention successfully achieved high-speed, real-time, and high-precision power fitting detection at 30 FPS on edge devices with limited computing power.

[0038] A system includes: an image acquisition module for acquiring and standardizing preprocessing images of power fittings inspections; a visual distillation enhancement module, which incorporates a target detection network based on the YOLO-World architecture, and introduces the DINOv2 large visual model into the Neck layer of the target detection network for feature-level knowledge distillation, outputting enhanced multi-scale image visual features; an offline semantic expansion module, which uses a large language model fine-tuned from the power fittings dataset to generate and embed high-dimensional static semantic features corresponding to each category offline, based on the category and attribute prompts of the power fittings, and stores them; a lightweight dimensionality reduction and alignment module, which incorporates a semantic projection layer, and embeds and reduces the high-dimensional static semantic features to a dimension matching the target detection network, replacing the CLIP text features of the original network with the dimensionality-reduced semantic features, fusing them with the enhanced visual features, inputting them into the detection head, and outputting the power fitting detection results; and an edge-decoupled deployment module, which removes all large language model and Transformer text encoder weights from the system, converts the pure convolutional detection network into a lightweight inference engine format, and deploys it on edge devices to achieve real-time detection.

[0039] A computer-readable storage medium having a computer program stored thereon, the method of which is implemented when the computer program is executed by a processor.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight power fitting detection method based on double priori enhancement, characterized in that, include: S1: Obtain the image of the power fittings of the transmission line as input; S2: Construct an object detection network based on the YOLO-World architecture, and introduce a feature-level knowledge distillation mechanism in the Neck layer of the feature pyramid of the object detection network; S3: Using a large language model with fine-tuned parameters, generate high-dimensional static semantic feature embeddings for each category based on prompts for the category name and physical attributes of power fittings. S4: Construct and train a semantic projection layer to embed high-dimensional static semantic features into a space that matches the feature dimension of the object detection network. S5: For the general CLIP text encoder and detection head natively included in the target detection network constructed in step S2, the dimensionality-reduced semantic features output by the semantic projection layer in step S4 are used to completely replace the text features extracted by the general CLIP text encoder; the dimensionality-reduced semantic features are directly input into the detection head, and cross-modal fusion calculation is performed with the enhanced multi-scale image visual features finally output by the target detection network in step S2, and the detection head decodes and outputs the final detection result of the power fittings; S6: Remove the weights of the large language model in the network and convert the model into a lightweight neural network inference engine format for real-time inference on edge devices.

2. The lightweight power fitting detection method based on dual-priori enhancement according to claim 1, characterized in that, In S3, the large language model adopts the Qwen2.5-VL model, and its fine-tuning process includes: Input the prompt word template, which includes the official standard name, national standard definition and core visual features of power fittings, into the Qwen2.5-VL model. The model generates high-dimensional static semantic feature embedding vectors for various power fittings offline in one go, and no large language model weights are loaded during the online inference stage.

3. The lightweight power fitting detection method based on dual-priori enhancement according to claim 1, characterized in that S2 include: S21: Input the power fittings image obtained in step S1 into the constructed target detection network and output the student network visual features; S22: Synchronously input the same power fitting image obtained in step S1 into the pre-trained large vision model DINOv2 to extract high-resolution standard visual features including global semantic information. S23: Perform a two-step feature alignment strategy in the path aggregation network of the object detection network to align the visual features of the student network with the high-resolution standard visual features. S24: Based on the aligned student network visual features and high-resolution standard visual features, construct a hybrid distillation loss function; calculate the loss value according to the hybrid distillation loss function, and update the network weights of the object detection network through backpropagation.

4. The lightweight power fittings testing method based on dual prior reinforcement according to claim 3, characterized in that, In S23, the two-step feature alignment strategy specifically includes: Spatial resolution alignment: Bilinear interpolation is used to upsample the student network visual features in the spatial dimension so that their spatial size is consistent with the spatial size of high-resolution standard visual features. Channel dimension alignment: After the spatial resolution aligned features are passed through a 1×1 convolutional layer, the number of channels is precisely mapped to the same number of channels as the high-resolution standard visual features, resulting in aligned student network visual features.

5. The lightweight power fitting detection method based on dual-priori enhancement according to claim 3, characterized in that, Mixed distillation loss function : In the formula, This represents the visual features of the student network after processing using a two-step feature alignment strategy. Visual features extracted by the large visual model DINOv2, representing the frozen parameters; The term on the right represents the mean squared error loss, used to constrain the absolute difference in feature pixel-level values; the term on the right is the cosine similarity loss, used to constrain the semantic orientation consistency of features in the vector space. and To balance the hyperparameter weights of the two losses.

6. The lightweight power fitting based on dual-priori enhancement detection method according to claim 1, characterized in that, In S4, a semantic projection layer is constructed and trained. The construction and training process includes: S41: Using CLIP, a general text encoder included in the object detection network, extract 512-dimensional text feature vectors of various power fittings and use them as teacher supervision signals for training the semantic projection layer. S42: Obtain the 3584-dimensional high-dimensional static semantic feature vectors of various categories of power fittings generated offline by the Qwen2.5-VL large language model, and input them into the randomly initialized semantic projection layer; wherein, the semantic projection layer is an unbiased linear network layer, which linearly maps the input 3584-dimensional features through its internal weight matrix and outputs 512-dimensional reduced semantic features. S43: Based on dimensionality-reduced semantic features and teacher supervision signals, the absolute difference between the two in the feature space is calculated using the mean squared error loss function, enabling the semantic projection layer to learn the mapping and transformation rules across the modal feature space; S44: Calculate the gradient of the mean squared error loss function using the Adam optimizer, and update the weight matrix of the semantic projection layer through backpropagation, so that the randomly initialized semantic projection layer weights learn the correct feature space mapping transformation relationship.

7. The lightweight power fitting detection method based on dual-prior augmentation according to claim 6, characterized in that, In S43, the semantic projection layer optimizes the mean square error loss function in the training process, and the optimized mean square error loss function of the semantic projection layer is is defined as follows: In the formula, N is the total number of categories of power fittings; The 3584-dimensional Qwen2.5-VL semantic feature vector of the i-th type of hardware generated offline; W Let be the weight matrix of the linear projection layer to be optimized; This represents the 512-dimensional text feature vector of the i-th type of power fittings extracted by the general CLIP text encoder natively included in the object detection network constructed in step S2, i.e., the teacher supervision signal.

8. The lightweight power fitting based on dual-priori enhancement detection method according to claim 1, characterized in that S6 include: S61: Directly export the trained and weighted object detection network into the ONNX general intermediate neural network model format to eliminate the dependence on the underlying training framework. S62: Use a model conversion tool to convert the ONNX format model into the lightweight NCNN format supported by the mobile inference framework, generate a deployment file containing the network structure and weights, and perform real-time inference with pure convolutional computation on an edge device equipped with a mobile chip. S63: Specify the model input size and modify the number and name parameters in the mobile engineering interface, and finally package it into an application to perform real-time inference with pure convolutional computation on an edge device equipped with a mobile chip.

9. A system for implementing the lightweight power fitting detection method based on dual-prior enhancement according to any one of claims 1-8, characterized in that, include: The image acquisition module is used to acquire images of power fittings inspection and perform standardized preprocessing. The visual distillation enhancement module has a built-in object detection network based on the YOLO-World architecture. It is used to introduce the large visual model DINOv2 into the Neck layer of the object detection network to perform feature-level knowledge distillation and output enhanced multi-scale image visual features. The offline semantic augmentation module is used to utilize the large language model fine-tuned by the power fittings dataset to generate high-dimensional static semantic features corresponding to each category offline, embed them, and store them in a fixed manner, based on the power fittings category and attribute prompt words. A lightweight dimensionality reduction and alignment module with a built-in semantic projection layer is used to embed high-dimensional static semantic features into a dimension that matches the target detection network. The dimensionality-reduced semantic features replace the CLIP text features of the original network. After being fused with the enhanced visual features, the input is fed into the detection head, and the power fitting detection results are output. The edge-decoupled deployment module is used to remove all large language models and Transformer text encoder weights from the system, convert the pure convolutional detection network into a lightweight inference engine format, and deploy it on edge devices to achieve real-time detection.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: When executed by a processor, the computer program implements the lightweight power fitting detection method based on dual prior enhancement as described in any one of claims 1 to 8.

Images