Method, device and storage medium for determining the operating state of an ultra-high voltage disconnector

Abstract

The present application discloses a method, device and storage medium for determining the operating status of an ultra-ultra-high voltage disconnector, which relates to the technical field of power systems and solves the problem that the current method of manual periodic inspection cannot detect in time when the operating status of an ultra-ultra-high voltage disconnector is abnormal. The method includes: constructing a target ultra-ultra-high voltage disconnector data set, determining a visual language detection model, training the visual language detection model using the ultra-ultra-high voltage disconnector data set, updating the model parameters, obtaining a target visual language detection model for indicating the interactive relationship between images and languages, obtaining the current ultra-ultra-high voltage disconnector image, inputting the current ultra-ultra-high voltage disconnector image into the target visual language detection model, obtaining the target description text output by the target visual language detection model, and determining the target operating status of the current ultra-ultra-high voltage disconnector based on the target description text.

Description

Technical Field

[0001] The present application relates to the technical field of power systems, and in particular to a method, device, storage medium and computer program product for determining the operating status of an ultra-high voltage disconnector. Background Art

[0002] With the development of UHV transmission and transformation technology, equipment operation status monitoring and fault warning have become increasingly important. Among them, the operation status detection of key equipment such as UHV disconnectors is directly related to the safe and stable operation of the entire power grid. At present, the manual inspection method is mainly used to conduct regular inspections of UHV disconnectors, for example, once a week. However, through the manual regular inspection method, when the operation status of the UHV disconnectors is abnormal, it cannot be detected in time, and the real-time detection is poor, which is not conducive to the safe and stable operation of the entire power grid. Summary of the invention

[0003] The embodiments of the present application provide a method, device, storage medium and computer program product for determining the operating status of an ultra-high voltage disconnector. The main purpose is to solve the problem that the current method of regular manual inspection cannot detect abnormalities in the operating status of the ultra-high voltage disconnector in a timely manner, resulting in poor real-time detection.

[0004] According to the first aspect of the present application, a method for determining the operating state of an ultra-high voltage disconnector is provided, comprising:

[0005] Constructing a target ultra-high voltage disconnector data set, wherein the target ultra-high voltage disconnector data set includes multiple ultra-high voltage disconnector images and description text annotated with each ultra-high voltage disconnector image, and the description text includes scene description information related to the switch operation state, bounding box coordinate information, and target segmentation mask information;

[0006] Determine a visual language detection model, train the visual language detection model using the ultra-high voltage disconnector data set, update model parameters, and obtain a target visual language detection model for indicating the interactive relationship between images and language;

[0007] Acquire a current ultra-high voltage disconnector image, input the current ultra-high voltage disconnector image into the target visual language detection model, and obtain a target description text output by the target visual language detection model;

[0008] The target operating state of the current ultra-high voltage disconnector is determined based on the target description text.

[0009] Optionally, constructing a target ultra-high voltage disconnector data set includes:

[0010] Constructing an ultra-ultra-high voltage disconnector data set, wherein the ultra-ultra-high voltage disconnector data set includes a plurality of training sample pairs, each training sample pair includes a positive training sample and a negative training sample, wherein the positive training sample is composed of an ultra-ultra-high voltage disconnector image and at least one high-similarity scene description information, frame coordinate information, and target segmentation mask information annotated by the ultra-ultra-high voltage disconnector image, and the negative training sample is composed of the ultra-ultra-high voltage disconnector image and the frame coordinate information, target segmentation mask information, and at least one missing scene description information annotated by the ultra-ultra-high voltage disconnector image;

[0011] Removing at least one training sample pair whose image and description information do not match from the ultra-high voltage disconnector data set to obtain an intermediate ultra-high voltage disconnector data set, wherein the intermediate ultra-high voltage disconnector data set includes a plurality of intermediate training sample pairs, each of which includes an intermediate positive training sample and an intermediate negative training sample;

[0012] Determine a multimodal large model, and train the multimodal large model based on the intermediate ultra-high voltage disconnector data set, update model parameters, and obtain a target multimodal large model for quantifying the similarity between the image and the description text;

[0013] The multiple intermediate positive training samples are extracted from the intermediate ultra-high voltage disconnector data set, and the multiple intermediate positive training samples are input into the target multimodal large model to determine the scene description text with the highest similarity corresponding to each ultra-high voltage disconnector image, so as to obtain the target ultra-high voltage disconnector data set, wherein the scene description text with the highest similarity includes the scene description information with the highest similarity, the bounding box coordinate information and the target segmentation mask information.

[0014] Optionally, constructing an ultra-high voltage disconnector data set includes:

[0015] Acquire multiple ultra-high voltage disconnector images at a substation site, each ultra-high voltage disconnector image includes an ultra-high voltage disconnector and components related to the ultra-high voltage disconnector, and the ultra-high voltage disconnector corresponds to an angle and an operating state;

[0016] Preprocessing the plurality of ultra-high voltage disconnector images to obtain a plurality of processed ultra-high voltage disconnector images;

[0017] Obtain a language description set for an ultra-high voltage disconnector, wherein the language description set includes multiple scenario description information, wherein the scenario description information is obtained by inputting switch professional terms and scenario information into a language description function, and the switch professional terms include switch model specifications, switch voltage level, switch operating status, switch operation mode, switch structure type, accessory device type, and installation location;

[0018] The following operations are performed for each ultra-high voltage disconnector image: using a multimodal large model to determine a similarity score between the ultra-high voltage disconnector image and each scene description information in the multiple scene description information to obtain multiple similarity scores, and sorting the multiple similarity scores in descending order to obtain multiple similarity scores with a sequential relationship, extracting at least one similarity score whose similarity score meets a target threshold from the multiple similarity scores with a sequential relationship, and using at least one scene description information corresponding to the at least one similarity score as at least one high-similarity scene description information;

[0019] Determine the bounding box coordinate information and target segmentation mask information of each ultra-high voltage disconnector image, and obtain a positive training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one high-similarity scene description information of each ultra-high voltage disconnector image;

[0020] Acquire at least one missing description information corresponding to each ultra-high voltage disconnector image, and obtain a negative training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, the target segmentation mask information and the at least one missing description information of each ultra-high voltage disconnector image;

[0021] Based on the positive training samples and negative training samples corresponding to each ultra-high voltage disconnector image, a training sample pair corresponding to each ultra-high voltage disconnector image is obtained, and based on the training sample pair corresponding to each ultra-high voltage disconnector image, the ultra-high voltage disconnector data set is obtained.

[0022] Optionally, preprocessing the multiple ultra-high voltage disconnector images to obtain the processed multiple ultra-high voltage disconnector images includes:

[0023] Unifying the sizes of the multiple ultra-high voltage disconnector images to obtain multiple first intermediate ultra-high voltage disconnector images with target sizes;

[0024] Performing quality screening on the plurality of first intermediate ultra-high voltage disconnector images to obtain a plurality of second intermediate ultra-high voltage disconnector images;

[0025] Data enhancement is performed on the multiple second intermediate ultra-high voltage disconnector images to obtain the multiple processed ultra-high voltage disconnector images.

[0026] Optionally, removing at least one training sample pair whose image and description information do not match from the ultra-high voltage disconnector data set to obtain an intermediate ultra-high voltage disconnector data set includes:

[0027] Determining that the ultra-high voltage disconnector data set includes a plurality of training sample pairs;

[0028] Using a target image encoder ResNet to perform feature extraction on the ultra-high voltage disconnector image corresponding to the training sample pair to obtain an image feature vector corresponding to the training sample pair, and using a target text encoder BERT to convert high-similarity scene description information, bounding box coordinate information, and segmentation mask information corresponding to the training sample pair into a first text embedding vector, and calculating a first cosine similarity between the image feature vector and the first text embedding vector. If the first cosine similarity is lower than a first target threshold, removing the training sample pair from the multiple training sample pairs; and,

[0029] Using the target image encoder ResNet to perform feature extraction on the ultra-high voltage disconnector image corresponding to the training sample pair to obtain the image feature vector corresponding to the training sample pair, and using the target text encoder BERT to convert the missing scene description information, border coordinate information and segmentation mask information corresponding to the training sample pair into a second text embedding vector, and calculating the second cosine similarity between the image feature vector and the second text embedding vector, and if the second cosine similarity is higher than a second target threshold, removing the training sample pair from the multiple training sample pairs;

[0030] The intermediate ultra-high voltage disconnector data set is obtained.

[0031] Optionally, the using the ultra-high voltage disconnector data set to train the visual language detection model, updating model parameters, and obtaining a target visual language detection model for indicating the interactive relationship between images and languages ​​includes:

[0032] Divide the ultra-high voltage disconnector data set into multiple non-overlapping sample subsets, select a sample subset from the multiple sample subsets as a first validation subset, use the remaining sample subsets in the multiple sample subsets except the first validation subset as a first training subset, use the first training subset to train the visual language detection model, update model parameters, and obtain a first visual language detection model, input the first validation subset into the first visual language detection model to obtain a first description text set, obtain a first actual description text set from the first validation subset, calculate the matching degree between the first description text set and the first actual description text set, continue to select another sample subset from the multiple sample subsets as a second validation subset, use the remaining sample subsets in the multiple sample subsets except the second validation subset as a second training subset, repeat the above operation until all sample subsets in the multiple sample subsets complete cross-validation, obtain multiple matching degrees, determine a maximum matching degree among the multiple matching degrees, and use the visual language detection model corresponding to the maximum matching degree as an intermediate visual language detection model;

[0033] The intermediate visual language detection model is trained using the ultra-high voltage disconnector data set, and the model parameters are updated to obtain the target visual language detection model for indicating the interactive relationship between image and language.

[0034] Optionally, after determining the target operating state of the current ultra-high voltage disconnector based on the target description text, the method further includes:

[0035] Acquire the actual operating state of the current ultra-high voltage disconnector, compare the actual operating state with the target operating state, and obtain a comparison result;

[0036] When the comparison result indicates that the actual operating state is consistent with the target operating state, determining that the image detection result is reliable;

[0037] When the comparison result indicates that the actual operating status is inconsistent with the target operating status, the image detection result is determined to be unreliable, and a warning message indicating that the image detection result is unreliable is generated, and the warning message is sent to the terminal held by the technician, so that the technician can perform problem troubleshooting based on the terminal when receiving the warning message.

[0038] According to a second aspect of the present application, a device for determining an operating state of an ultra-high voltage disconnector is provided, comprising:

[0039] A construction module is used to construct a target ultra-high voltage disconnector data set, wherein the target ultra-high voltage disconnector data set includes multiple ultra-high voltage disconnector images and description text annotated with each ultra-high voltage disconnector image, and the description text includes scene description information related to the switch operation state, bounding box coordinate information and target segmentation mask information;

[0040] A training module, used to determine a visual language detection model, train the visual language detection model using the ultra-high voltage disconnector data set, update model parameters, and obtain a target visual language detection model for indicating the interactive relationship between image and language;

[0041] A detection module, used to obtain a current ultra-high voltage disconnector image, input the current ultra-high voltage disconnector image into the target visual language detection model, and obtain a target description text output by the target visual language detection model;

[0042] A determination module is used to determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

[0043] According to a third aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0044] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0045] Through the above technical scheme, the present application provides a method, device, storage medium and computer program product for determining the operating status of an ultra-ultra-high voltage disconnector. The present application uses a high-quality multimodal dataset that combines images and language descriptions, namely, a target ultra-ultra-high voltage disconnector dataset, to train a visual language detection model to obtain an optimal target visual language detection model. Since the target ultra-ultra-high voltage disconnector dataset contains detailed image annotations and flexible language descriptions, the trained target visual language detection model can accurately identify various states of the disconnector. Subsequently, the current ultra-ultra-high voltage disconnector image is input into the target visual language detection model. The target visual language detection model can accurately identify the operating status of the disconnector, output a target description text, and then determine the target operating status of the current ultra-ultra-high voltage disconnector based on the target description text. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of a method for determining the operating state of an ultra-high voltage disconnector provided in an embodiment of the present application is shown;

[0047] Figure 2 A flow chart showing another method for determining the operating state of an ultra-high voltage disconnector provided in an embodiment of the present application is shown;

[0048] Figure 3 A schematic diagram showing the structure of a CLIP model of another method for determining the operating state of an ultra-high voltage disconnector provided in an embodiment of the present application is shown;

[0049] Figure 4A A schematic diagram of the structure of a device for determining the operating state of an ultra-high voltage disconnector provided in an embodiment of the present application is shown;

[0050] Figure 4B A schematic structural diagram of a device for determining the operating status of an ultra-high voltage disconnector provided in another embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0052] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0053] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0054] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0055] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0056] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0057] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0058] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0059] The present application embodiment provides a method for determining the operating state of an ultra-high voltage disconnector, such as Figure 1 As shown, including:

[0060] 101. Build the target UHV disconnector dataset.

[0061] In an embodiment of the present application, the executing entity is a system that can implement a method for determining the operating status of an ultra-high voltage disconnector. The system needs to first construct a target ultra-high voltage disconnector dataset. The target ultra-high voltage disconnector dataset here includes multiple ultra-high voltage disconnector images and description text annotated with each ultra-high voltage disconnector image, and the description text includes scene description information related to the operating status of the switch, border coordinate information, and target segmentation mask information.

[0062] 102. Determine a visual language detection model, train the visual language detection model using the ultra-high voltage disconnector data set, update the model parameters, and obtain a target visual language detection model for indicating the interactive relationship between images and languages;

[0063] In an embodiment of the present application, after obtaining the target ultra-ultra-high voltage disconnector data set, a visual language detection model can be determined, exemplarily, the Grounding-DINO model, which is a new SOTA zero-sample object detection model, is a prior art, and is easily obtained by those skilled in the art, so it will not be described in detail here. The visual language detection model is trained using the ultra-ultra-high voltage disconnector data set, and the model parameters are updated so that the model can learn the rich features of the switch operating status, and describe the switch operating status results in combination with the scene information. By training the visual language detection model using the ultra-ultra-high voltage disconnector data set and updating the model parameters, a target visual language detection model for indicating the interactive relationship between images and language can be obtained.

[0064] Exemplarily, the target ultra-high voltage disconnector data set is divided into a training set and a validation set in a ratio of 7:3. The Grounding-DINO model is initialized, the learning rate is set to 0.001, the Adam optimizer is used for parameter update, the batch size is set to 32, and the network weights are assigned random values ​​through the He initialization method. The training set is input into the Grounding-DINO model, and 100 rounds of iterative training are performed. Each round of iteration traverses the entire training set once, and then the validation set is used for verification to obtain the verification results. Through these steps, the model can learn the rich features of the equipment status and realize efficient and accurate equipment status recognition in practical applications.

[0065] 103. Obtain the current ultra-high voltage disconnector image, input the current ultra-high voltage disconnector image into the target visual language detection model, and obtain the target description text output by the target visual language detection model.

[0066] In an embodiment of the present application, a trained target visual language detection model is obtained through model training in step 102, and the current ultra-high voltage disconnector image is input into the target visual language detection model. The target visual language detection model automatically extracts key features in the image, such as the disconnecting state of the disconnector, through its deep learning architecture, and identifies the switch operating state represented in the current ultra-high voltage disconnector image. Exemplarily, the target visual language detection model can distinguish between the open and closed states of the disconnector, and describe the results in combination with the surrounding environment to obtain a target description text.

[0067] Furthermore, after the target description text is determined, it can be displayed through an intuitive visual interface, where different states of the equipment are represented by different color icons, which is easy for operators to understand. At the same time, the target description text can also be output through a standardized interface, so that it can be directly used by other systems or applications to achieve real-time monitoring and automated management of the status of UHV substation equipment.

[0068] 104. Determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

[0069] In an embodiment of the present application, the target description text contains descriptive information for indicating the operating status of the switch and descriptive information for indicating the scene in which it is located. Therefore, the target operating status of the current ultra-high voltage disconnector can be determined through the target description text. For example, the disconnector is in the closed state and the open state, or in the open-closed state.

[0070] The method provided in the embodiment of the present application utilizes a high-quality multimodal dataset that combines images and language descriptions, namely, a target ultra-high voltage disconnector dataset, to train a visual language detection model to obtain an optimal target visual language detection model. Since the target ultra-high voltage disconnector dataset contains detailed image annotations and flexible language descriptions, the trained target visual language detection model can accurately identify various states of the disconnector. That is to say, the current ultra-high voltage disconnector image is input into the target visual language detection model, and the target visual language detection model can accurately identify the operating state of the disconnector, output a target description text, and then determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

[0071] Furthermore, by providing flexible and diverse language expression samples, the dataset can improve the adaptability of the model in new scenarios. The target visual language detection model trained with the dataset can also handle complex descriptions that have never been encountered.

[0072] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation of this embodiment, the embodiment of the present invention provides another method for determining the operating state of an ultra-high voltage disconnector, such as Figure 2 As shown, including:

[0073] 201. Construct the target UHV disconnector dataset.

[0074] In the embodiment of the present application, the specific process of constructing the target ultra-high voltage disconnector data set is as follows:

[0075] 2011. Acquire multiple ultra-high voltage disconnector images at a substation site, each ultra-high voltage disconnector image includes an ultra-high voltage disconnector and components related to the ultra-high voltage disconnector, and the ultra-high voltage disconnector corresponds to an angle and an operating state.

[0076] For example, a collection of multi-angle images of ultra-high voltage disconnectors and their components is provided. , Multi-state image.

[0077] 2012. Preprocessing a plurality of ultra-high voltage disconnector images to obtain a plurality of processed ultra-high voltage disconnector images.

[0078] In an embodiment of the present application, the size of multiple ultra-high voltage disconnector images can be first unified to obtain multiple first intermediate ultra-high voltage disconnector images with a target size; then, the multiple first intermediate ultra-high voltage disconnector images can be quality screened to obtain multiple second intermediate ultra-high voltage disconnector images; finally, the multiple second intermediate ultra-high voltage disconnector images can be data enhanced to obtain multiple processed ultra-high voltage disconnector images.

[0079] For example, the image size can be unified by using geometric transformation and image scaling techniques in image processing. ,Right now,

[0080] Image acquisition: Use on-site monitoring equipment and handheld cameras to capture images of the disconnectors at the substation site, and try to make the collected images evenly distributed in the number of three states: closed, open, and not closed or open.

[0081] Constructing an image set: Assume a finite set of angles and a finite set of states , then the image set under all angles and state combinations can be represented in the form of a multidimensional array or matrix:

[0082] ,

[0083] in, Indicated in angle The switch is in the state Here, is a dimensional array or matrix, each item is an image vector or matrix, representing the image information at a specific angle and state.

[0084] In other words, each It is an image data, corresponding to the angle and status This method makes it easy to manage and process image data acquired at different angles and in different states.

[0085] Image size uniformity: through scaling function Each of the different sizes collected The UHV disconnector image is converted to the same size , which is responsible for mapping the original coordinates to the target coordinates and calculating the value of the new pixel based on the values ​​of the surrounding pixels.

[0086] Assume there is an original image , the goal is to create a new resized image ,in represents the horizontal coordinate (column), Indicates the vertical coordinate (row). The range is from 0 to , The range is from 0 to , here and are the width and height of the new image respectively. The image size transformation process can be written as:

[0087] ,

[0088] here is a transformation function that maps pixel values ​​in the original image to corresponding positions in the new image. Depending on the interpolation method, Can have different forms:

[0089] Bilinear interpolation: In this method, By calculating the new coordinates The result is obtained by weighted average of the four nearest neighboring pixels. These four pixels are ,as well as .

[0090] Nearest neighbor interpolation: In this method, By finding the coordinates of the target The nearest pixel is used to determine the value of the new pixel.

[0091] For any target coordinates , whose integer and decimal parts are:

[0092] Integer part:

[0093] Fractional part:

[0094] here, Represents a floor operation. These integer and fractional parts are used to determine the specific value of each pixel in the new image.

[0095] Image quality screening: Filter the converted images and discard images that are not suitable for labeling, ambiguous, unrecognizable, and low-resolution.

[0096] Data augmentation: Perform data augmentation on the filtered images to increase the diversity and robustness of the training set. Data augmentation methods include but are not limited to:

[0097] (a) Spatial transformation: illustratively, operations such as rotation, translation, scaling, and flipping are designed to simulate image changes under different viewpoints.

[0098] (b) Color transformation: adjust the brightness, contrast, and saturation of the image to adapt to different lighting conditions.

[0099] (c) Noise addition: Add an appropriate amount of random noise, such as Gaussian noise, to the image to improve the model’s ability to resist various noises in the real world.

[0100] Specifically, the added Gaussian noise can be described by the following formula:

[0101] ,

[0102] in, is a random variable, representing the noise value; is the standard deviation of the Gaussian distribution, which controls the intensity of the noise.

[0103] 2013. Obtain a set of language descriptions for ultra-high voltage disconnectors.

[0104] In an embodiment of the present application, the language description set includes multiple scenario description information, which is obtained by inputting switch professional terms and scenario information into a language description function, and the switch professional terms include switch model specifications, switch voltage levels, switch operating status, switch operation modes, switch structure types, accessory equipment types, and installation locations.

[0105] It should be noted that a series of language description sets for ultra-high voltage disconnectors can be generated by design, covering short category names To detailed feature description , specifically:

[0106] Professional terminology collection and organization: A well-organized image dataset of on-site ultra-high voltage disconnectors , clarify the key components and core technologies of ultra-high voltage disconnectors, refine the characteristics and application scenarios of various switch models, and comprehensively collect and organize professional terms in related fields, expressed as , as shown in Table 1:

[0107] Table 1

[0108]

[0109] Description template design: Based on a list of collected professional terms , creating a set of flexible and comprehensive description templates , ensuring that it can adapt to different scenarios and needs, including:

[0110] Short Category Name Template Ultra-high voltage isolating switch Status", "Rated voltage is of Ultra-high voltage isolating switch Status", etc. and Placeholders representing the type, voltage level, and structure type to be replaced;

[0111] Detailed functional feature description template : "Rated voltage is , whose structure type is ,use The operating mode is suitable for Environment Installation Model UHV disconnector is in Status", etc. They represent placeholders for the professional terms to be replaced.

[0112] Description generation: Using a list of specialized terms As a parameter filling template, it generates diversified descriptions according to different application scenarios and requirements.

[0113] Considering the template Include placeholders, respectively , and the professional term list Include term, denoted as . Mapping function Defines the correspondence between placeholders and terms. Value function ,in is the set of all possible values, giving the actual value of each term. The mathematical expression that generates the final description It can be expressed as:

[0114] ,

[0115] here, Representation Template All placeholders in Replaced by The string after.

[0116] It should be noted that if Represented as a string function ,in Is a placeholder The value of can generate the function described can be written as:

[0117] ,

[0118] in, Indicates application first Re-application .

[0119] In order to make the above process more intuitive, a specific example is used to illustrate that a simple template is determined =“ Ultra-high voltage isolating switch Status", a list of specialized terms , mapping function , and the value function , The resulting description is for: The ultra-high voltage disconnector is in the state of not opening or closing properly.

[0120] Furthermore, an example of complex scenario description is: further combining the actual situation to generate a detailed description containing complex attributes and specific scenarios "The double-column vertical rotary UHV disconnector located on the east side of the substation tower has a rated voltage of 110kV and is currently in a state of incomplete opening and closing. The red LED warning light on the top is flashing, indicating that the equipment is responding to the instructions of the remote dispatching center for emergency processing. At the same time, its intelligent diagnosis system is providing real-time feedback on the mechanical performance indicators and electrical parameters of the disconnector action."

[0121] Diversity enhancement: In order to improve the diversity of descriptions, the present invention adopts a method based on a pre-trained language model. Specifically, the BART (Bidirectional and Auto-Regressive Transformers) model in the Transformers library of Hugging Face is selected as the core component to perform the synonym replacement task. By fine-tuning a large number of corpora in the field of power engineering, especially optimizing the relevant terms and descriptive language of ultra-high voltage disconnectors, it is ensured that the model can understand and generate professional expressions in this field. In the fine-tuning process, special emphasis is placed on the model's contextual understanding and synonym expression capabilities of professional terms to ensure the accuracy and naturalness of the replacement. When implementing synonym replacement, the switch description text in the original data set is input into the fine-tuned BART model, and the model is guided to generate multiple synonymous expressions for each description through prompt instructions. These expressions retain the original intent while being replaced with appropriate synonyms in the power field, thereby increasing the diversity and accuracy of the description. For example, to describe "the double-column vertical rotary ultra-high voltage disconnector with a rated voltage of 110kV is in the open state", the model may generate synonymous expressions such as "the double-column vertical rotary ultra-high voltage disconnector with a voltage level of 110kV is in the disconnected state" etc.

[0122] Furthermore, in order to ensure that the text descriptions generated for the UHV disconnector equipment are collected S p Highly consistent with the actual image content, manual screening is used for strict quality control. In order to ensure the accuracy and professionalism of the description, all descriptions automatically generated by the model are reviewed one by one by personnel with professional knowledge background in this process. The purpose of the review is to confirm that each description can accurately reflect the key information such as the device type, model, material, voltage level, operation mode, safety level, installation location and appearance characteristics, and accessory device status in the image.

[0123] 2014. Determine at least one highly similar scene description information corresponding to each ultra-high voltage disconnector image.

[0124] In an embodiment of the present application, the following operations are performed for each ultra-high voltage disconnector image: a multimodal large model is used to determine a similarity score between the ultra-high voltage disconnector image and each scene description information in a plurality of scene description information to obtain a plurality of similarity scores, and the plurality of similarity scores are sorted in descending order to obtain a plurality of similarity scores having a sequential relationship, at least one similarity score whose similarity score meets a target threshold is extracted from the plurality of similarity scores having a sequential relationship, and at least one scene description information corresponding to the at least one similarity score is used as at least one high-similarity scene description information.

[0125] 2015. Obtain the training sample pairs corresponding to each ultra-high voltage disconnector image.

[0126] In an embodiment of the present application, the frame coordinate information and target segmentation mask information of each ultra-high voltage disconnector image can be first determined, and the positive training sample corresponding to each ultra-high voltage disconnector image can be obtained based on the frame coordinate information, target segmentation mask information and at least one high-similarity scene description information of each ultra-high voltage disconnector image. Then, at least one missing description information corresponding to each ultra-high voltage disconnector image is obtained, and the negative training sample corresponding to each ultra-high voltage disconnector image is obtained based on the frame coordinate information, target segmentation mask information and at least one missing description information of each ultra-high voltage disconnector image. Finally, the training sample pair corresponding to each ultra-high voltage disconnector image is obtained based on the positive training samples and negative training samples corresponding to each ultra-high voltage disconnector image. It should be noted that the image recognition algorithm is a prior art, which can be easily obtained by those skilled in the art and will not be described in detail here.

[0127] It should be noted that positive training samples can be understood as switch instances that match the description, and negative training samples can be understood as switch instances that do not match the description.

[0128] Given an ultra-high voltage disconnector image and a set of language descriptions , using the CLIP model to calculate the similarity score between each pair of images and descriptions ,in, Representing images With description Then, for each image , calculate its correlation with each description Similarity , and set a threshold , keeping only those The descriptions of each image are selected as candidate positive samples. The descriptions with the highest similarity are taken as positive training samples and are denoted as ,in is with the image Next, for each image , based on its description Design 1-2 "absent" descriptions that contain elements or features that are not present in the image as references for negative training samples. ,in is with the image Unmatched descriptions. Finally, for each image Construct positive and negative training sample pairs ,in is a positive sample pair, is a negative sample pair. In this way, the number of positive and negative samples is balanced and the dataset can effectively guide the model to learn the correspondence between language and images. Finally, the fully labeled positive and negative training samples and their related descriptions are integrated into pairs for further training.

[0129] 2016. Based on the training sample pairs corresponding to each ultra-high voltage disconnector image, an ultra-high voltage disconnector dataset is obtained.

[0130] 2017. At least one training sample pair whose image and description information do not match is removed from the ultra-high voltage disconnector dataset to obtain an intermediate ultra-high voltage disconnector dataset.

[0131] In the example of the present application, it can be determined that the ultra-ultra-high voltage disconnector data set includes multiple training sample pairs. For each training sample pair, the target image encoder ResNet can be used to perform feature extraction on the ultra-ultra-high voltage disconnector image corresponding to the training sample pair to obtain the image feature vector corresponding to the training sample pair, and the target text encoder BERT can be used to convert the high-similarity scene description information, border coordinate information, and segmentation mask information corresponding to the training sample pair into a first text embedding vector, and calculate the first cosine similarity between the image feature vector and the first text embedding vector. If the first cosine similarity is lower than the first target threshold, the training sample pair is removed from the multiple training sample pairs; and, the target image encoder ResNet is used to perform feature extraction on the ultra-ultra-high voltage disconnector image corresponding to the training sample pair to obtain the image feature vector corresponding to the training sample pair, and the target text encoder BERT is used to convert the missing scene description information, border coordinate information, and segmentation mask information corresponding to the training sample pair into a second text embedding vector, and calculate the second cosine similarity between the image feature vector and the second text embedding vector. If the second cosine similarity is higher than the second target threshold, the training sample pair is removed from the multiple training sample pairs. Through the above process, the language description can be further optimized, and inaccurate language descriptions can be discarded, thereby obtaining an intermediate ultra-high voltage disconnector data set with an accurate correlation between the image and the language description. It should be noted that, in order to distinguish it from the above training samples, the intermediate ultra-high voltage disconnector data set includes multiple intermediate training sample pairs, and each intermediate training sample pair includes an intermediate positive training sample and an intermediate negative training sample.

[0132] 2018. Based on the intermediate ultra-high voltage disconnector dataset, the multimodal big model is trained, the model parameters are updated, and the target multimodal big model for quantifying the similarity between images and description texts is obtained.

[0133] In the embodiment of the present application, the multimodal large model is a CLIP model, and its specific structure is as follows Figure 3 As shown in the figure, the joint representation space of images and texts can be learned simultaneously. First, the intermediate ultra-high voltage disconnector dataset training set and validation set , train the CLIP model on the training set so that it maximizes the similarity between image and text pairs during the training process, while minimizing the similarity of unmatched pairs. Then use the validation set to check the high similarity pairs of the trained CLIP model. That is, for image-text pairs with high similarity matching, the accuracy of the labels can be further checked. If it is determined that a text description fails to accurately capture the key features of the image, for example, structure, action, state, etc., it can be manually modified or detailed descriptions can be added. If necessary, a new label system can be designed or existing labels can be refined for special categories or difficult-to-identify situations. Through the above process, an updated intermediate ultra-ultra-high voltage disconnector dataset can be obtained, and then the updated intermediate ultra-ultra-high voltage disconnector dataset is re-divided to obtain an updated training set. and update the validation set , using the updated training set Retrain the CLIP model to get an updated CLIP model, and use the updated validation set Verify the updated CLIP model and re-evaluate the performance of the similarity matrix and the model on the validation set. If there is still room for improvement in the performance of the updated CLIP model, repeat the above steps until the generated data quality reaches the preset data quality, and finally obtain the optimal target multimodal large model.

[0134] It should be noted that after obtaining the updated intermediate ultra-high voltage disconnector dataset, we can continue to remove low-quality labeled data, such as mislabeling, missing labels, etc. from the updated intermediate ultra-high voltage disconnector dataset; for those samples with missing key information, appropriate descriptive information can be filled in by combining expert knowledge or other auxiliary means.

[0135] 2019. Extract multiple intermediate positive training samples from the intermediate ultra-high voltage disconnector data set, and input the multiple intermediate positive training samples into the target multimodal large model to determine the scene description text with the highest similarity corresponding to each ultra-high voltage disconnector image, and obtain the target ultra-high voltage disconnector data set.

[0136] In an embodiment of the present application, the trained target multimodal large model can accurately capture the key features of the image, exemplarily, structure, action, state, etc., and multiple intermediate positive training samples can be extracted from the intermediate ultra-high voltage disconnector data set. By inputting the multiple intermediate positive training samples into the target multimodal large model, the scene description text with the highest similarity corresponding to each ultra-high voltage disconnector image can be determined, and then the target ultra-high voltage disconnector data set is obtained. It should be noted that the scene description text with the highest similarity includes the scene description information with the highest similarity, border coordinate information and target segmentation mask information. It should be noted that the scene description information includes at least one of the switch operation status description information, environment description information, switch structure description information, electrical parameter description information, switch installation position description information and switch basic description information.

[0137] The visual language detection model is trained using the ultra-high voltage disconnector dataset, and the model parameters are updated to obtain a target visual language detection model for indicating the interactive relationship between image and language.

[0138] In an embodiment of the present application, the ultra-high voltage disconnector data set can be divided into multiple non-overlapping sample subsets, a sample subset is selected from the multiple sample subsets as a first verification subset, and the remaining sample subsets of the multiple sample subsets except the first verification subset are used as a first training subset. The visual language detection model is trained using the first training subset, and the model parameters are updated to obtain a first visual language detection model. The first verification subset is input into the first visual language detection model to obtain a first description text set, and a first actual description text set is obtained from the first verification subset. The matching degree between the first description text set and the first actual description text set is calculated. Continue to select another sample subset from the multiple sample subsets as the second validation subset, use the remaining sample subsets in the multiple sample subsets except the second validation subset as the second training subset, repeat the above operation until all sample subsets in the multiple sample subsets complete cross validation, obtain multiple matching degrees, determine the maximum matching degree among the multiple matching degrees, and use the visual language detection model corresponding to the maximum matching degree as the intermediate visual language detection model; then use the ultra-high voltage disconnector data set to train the intermediate visual language detection model, update the model parameters, and obtain the target visual language detection model for indicating the interactive relationship between the image and the language. By training the visual language detection model by the above method, the overfitting of the visual language detection model can be prevented.

[0139] It should be noted that the first description text set is a set of first description texts corresponding to multiple ultra-high voltage disconnector images, and the first actual description text set is a set of first actual description texts corresponding to multiple ultra-high voltage disconnector images, and a first description text in the first description text set corresponds to a first actual description text in the first actual description text set. Therefore, the specific method for calculating the matching degree between the first description text set and the first actual description text set is: for each ultra-high voltage disconnector image, the text matching algorithm is used to determine the matching degree between its corresponding first description text and its corresponding first actual description text, and finally the matching degree corresponding to all ultra-high voltage disconnector images is obtained. The text matching algorithm here is a prior art and can be used directly, and will not be described in detail here.

[0140] 203. Input the current ultra-high voltage disconnector image into the target visual language detection model to obtain the target description text output by the target visual language detection model.

[0141] 204. Determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

[0142] 205. Check the image detection result.

[0143] In an embodiment of the present application, the reliability of the image detection result can also be verified, that is, first, the actual operating status of the current ultra-high voltage disconnector is obtained, and the actual operating status is compared with the target operating status to obtain a comparison result; when the comparison result indicates that the actual operating status is consistent with the target operating status, the image detection result is determined to be reliable; when the comparison result indicates that the actual operating status is inconsistent with the target operating status, the image detection result is determined to be unreliable, and a warning message indicating that the image detection result is unreliable is generated, and the warning message is sent to the terminal held by the technician, so that the technician can perform problem troubleshooting based on the terminal when receiving the warning message.

[0144] The method provided in the embodiment of the present application utilizes a high-quality multimodal dataset that combines images and language descriptions, namely, a target ultra-high voltage disconnector dataset, to train a visual language detection model to obtain an optimal target visual language detection model. Since the target ultra-high voltage disconnector dataset contains detailed image annotations and flexible language descriptions, the trained target visual language detection model can accurately identify various states of the disconnector. Subsequently, the current ultra-high voltage disconnector image is input into the target visual language detection model. The target visual language detection model can accurately identify the operating state of the disconnector, output a target description text, and then determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

[0145] Furthermore, the target ultra-high voltage disconnector dataset of this application contains detailed image annotations and flexible language descriptions, that is, the dataset covers various switch working states, environmental changes, and internal component details, and each target is accompanied by a detailed text description. Through the training of this dataset, the visual language detection model can understand and associate the connection between images and professional descriptions, that is, the visual language detection model can accurately identify and understand the various states of the switch, regardless of the position, state, or switch with specific characteristics, it can be accurately identified and located, which improves the accuracy and reliability of the equipment operation state detection. Through the refined annotation and reasonable ratio of positive and negative samples, the number of positive and negative samples is balanced, overfitting or underfitting is avoided, the learning effect of the model in the training process and the recognition accuracy during the test are guaranteed, and the model's understanding of abstract concepts and missing situations and its ability to cope with complex scenes are improved. Therefore, using this dataset to train the visual language detection model can achieve accurate identification and understanding of ultra-high voltage disconnector equipment and its state, effectively improve the safety, reliability and automation level of intelligent monitoring and operation and maintenance of power facilities, and lay a solid foundation for building an efficient and intelligent power management system and future power scientific research.

[0146] Further, as Figure 1 The specific implementation of the method is as follows: Figure 4A As shown, an embodiment of the present invention provides a device for determining the operating status of an ultra-high voltage disconnector, including: a construction module 401, a training module 402, a detection module 403 and a determination module 404.

[0147] The construction module 401 is used to construct a target ultra-high voltage disconnector data set, wherein the target ultra-high voltage disconnector data set includes multiple ultra-high voltage disconnector images and description text annotated with each ultra-high voltage disconnector image, and the description text includes scene description information related to the switch operation state, bounding box coordinate information and target segmentation mask information;

[0148] The training module 402 is used to determine a visual language detection model, train the visual language detection model using the ultra-high voltage disconnector data set, update model parameters, and obtain a target visual language detection model for indicating the interactive relationship between image and language;

[0149] The detection module 403 is used to obtain the current ultra-high voltage disconnector image, input the current ultra-high voltage disconnector image into the target visual language detection model, and obtain the target description text output by the target visual language detection model;

[0150] The 404 determination module is used to determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

[0151] In a specific application scenario, the construction module 401 is also used to construct an ultra-ultra-high voltage disconnector data set, wherein the ultra-ultra-high voltage disconnector data set includes multiple training sample pairs, each training sample pair includes a positive training sample and a negative training sample, wherein the positive training sample is composed of an ultra-ultra-high voltage disconnector image and at least one high-similarity scene description information, border coordinate information and target segmentation mask information annotated by the ultra-ultra-high voltage disconnector image, and the negative training sample is composed of the ultra-ultra-high voltage disconnector image and the border coordinate information, target segmentation mask information and at least one missing scene description information annotated by the ultra-ultra-high voltage disconnector image; at least one training sample pair whose image and description information do not match is removed from the ultra-ultra-high voltage disconnector data set to obtain an intermediate ultra-ultra-high voltage disconnector data set, wherein the intermediate ultra-ultra-high voltage disconnector data set is The switch data set includes multiple intermediate training sample pairs, each intermediate training sample pair includes an intermediate positive training sample and an intermediate negative training sample; a multimodal large model is determined, and the multimodal large model is trained based on the intermediate ultra-high voltage disconnector data set, and the model parameters are updated to obtain a target multimodal large model for quantifying the similarity between an image and a description text; the multiple intermediate positive training samples are extracted from the intermediate ultra-high voltage disconnector data set, and the multiple intermediate positive training samples are input into the target multimodal large model, and the scene description text with the highest similarity corresponding to each ultra-high voltage disconnector image is determined to obtain the target ultra-high voltage disconnector data set, wherein the scene description text with the highest similarity includes the scene description information with the highest similarity, the border coordinate information and the target segmentation mask information.

[0152] In a specific application scenario, the construction module 401 is also used to obtain multiple ultra-high voltage disconnector images at the substation site, each ultra-high voltage disconnector image includes an ultra-high voltage disconnector and components related to the ultra-high voltage disconnector, and the ultra-high voltage disconnector corresponds to an angle and an operating state; the multiple ultra-high voltage disconnector images are preprocessed to obtain multiple processed ultra-high voltage disconnector images; a language description set for the ultra-high voltage disconnector is obtained, the language description set includes multiple scene description information, the scene description information is obtained by inputting switch professional terms and scene information into a language description function, and the switch professional terms include switch model specifications, switch voltage level, switch operating state, switch operation mode, switch structure type, accessory equipment type, and installation location; for each ultra-high voltage disconnector image, the following operations are performed: a multimodal large model is used to determine the similarity score between the ultra-high voltage disconnector image and each scene description information in the multiple scene description information to obtain multiple similarity scores, and the multiple similarity scores are sorted in order from high to low to obtain to multiple similarity scores with a sequential relationship, extract at least one similarity score whose similarity score meets the target threshold from the multiple similarity scores with a sequential relationship, and use at least one scene description information corresponding to the at least one similarity score as at least one high-similarity scene description information; determine the bounding box coordinate information and target segmentation mask information of each ultra-high voltage disconnector image, and obtain the positive training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one high-similarity scene description information of each ultra-high voltage disconnector image; obtain at least one missing description information corresponding to each ultra-high voltage disconnector image, and obtain the negative training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one missing description information of each ultra-high voltage disconnector image; obtain the training sample pair corresponding to each ultra-high voltage disconnector image based on the positive training sample and the negative training sample corresponding to each ultra-high voltage disconnector image, and obtain the ultra-high voltage disconnector data set based on the training sample pair corresponding to each ultra-high voltage disconnector image.

[0153] In a specific application scenario, the construction module 401 is also used to unify the size of the multiple ultra-high voltage disconnector images to obtain multiple first intermediate ultra-high voltage disconnector images with a target size; perform quality screening on the multiple first intermediate ultra-high voltage disconnector images to obtain multiple second intermediate ultra-high voltage disconnector images; perform data enhancement on the multiple second intermediate ultra-high voltage disconnector images to obtain the processed multiple ultra-high voltage disconnector images.

[0154] In a specific application scenario, the construction module 401 is also used to determine that the ultra-high voltage disconnector data set includes multiple training sample pairs; use the target image encoder ResNet to extract features of the ultra-high voltage disconnector image corresponding to the training sample pair to obtain the image feature vector corresponding to the training sample pair; and use the target text encoder BERT to convert the high-similarity scene description information, border coordinate information, and segmentation mask information corresponding to the training sample pair into a first text embedding vector, calculate the first cosine similarity between the image feature vector and the first text embedding vector, and if the first cosine similarity is lower than the first target threshold, remove the training sample pair from the image feature vector. The method comprises the following steps: removing the training sample pairs from the plurality of training sample pairs; and extracting features of the ultra-high voltage disconnector images corresponding to the training sample pairs using the target image encoder ResNet to obtain image feature vectors corresponding to the training sample pairs, and converting the missing scene description information, bounding box coordinate information and segmentation mask information corresponding to the training sample pairs into second text embedding vectors using the target text encoder BERT, and calculating the second cosine similarity between the image feature vector and the second text embedding vector. If the second cosine similarity is higher than a second target threshold, removing the training sample pairs from the plurality of training sample pairs; and obtaining the intermediate ultra-high voltage disconnector data set.

[0155] In a specific application scenario, the training module 402 is further used to divide the ultra-high voltage disconnector data set into multiple non-overlapping sample subsets, select a sample subset from the multiple sample subsets as a first verification subset, and use the remaining sample subsets of the multiple sample subsets except the first verification subset as a first training subset, use the first training subset to train the visual language detection model, update the model parameters, obtain the first visual language detection model, input the first verification subset into the first visual language detection model, obtain the first description text set, obtain the first actual description text set from the first verification subset, and calculate the first description text set and the first actual description text set. The matching degree between the sets is determined, and another sample subset is continuously selected from the multiple sample subsets as the second validation subset, and the remaining sample subsets except the second validation subset in the multiple sample subsets are used as the second training subset, and the above operation is repeated until all sample subsets in the multiple sample subsets complete cross-validation to obtain multiple matching degrees, and the maximum matching degree is determined among the multiple matching degrees, and the visual language detection model corresponding to the maximum matching degree is used as the intermediate visual language detection model; the intermediate visual language detection model is trained using the ultra-high voltage disconnector data set, and the model parameters are updated to obtain the target visual language detection model for indicating the interactive relationship between image and language.

[0156] In specific application scenarios, such as Figure 4BAs shown, the device also includes: a verification module 405.

[0157] The verification module 405 is used to obtain the actual operating status of the current ultra-high voltage disconnector, compare the actual operating status with the target operating status, and obtain a comparison result; when the comparison result indicates that the actual operating status is consistent with the target operating status, the image detection result is determined to be reliable; when the comparison result indicates that the actual operating status is inconsistent with the target operating status, the image detection result is determined to be unreliable, and a warning message indicating that the image detection result is unreliable is generated, and the warning message is sent to the terminal held by the technician, so that the technician can perform problem troubleshooting based on the terminal when receiving the warning message.

[0158] The device provided in the embodiment of the present application constructs a high-quality multimodal dataset combining images and language descriptions, namely, a target ultra-ultra-high voltage disconnector dataset, through a construction module. Then, the visual language detection model is trained using the target ultra-ultra-high voltage disconnector dataset through a training module to obtain an optimal target visual language detection model. Since the target ultra-ultra-high voltage disconnector dataset contains detailed image annotations and flexible language descriptions, the trained target visual language detection model can accurately identify various states of the disconnector. Subsequently, the current ultra-ultra-high voltage disconnector image is input into the target visual language detection model through the detection module. The target visual language detection model can accurately identify the operating state of the disconnector and output a target description text. Then, the target operating state of the current ultra-ultra-high voltage disconnector can be determined based on the target description text through the determination module.

[0159] It should be noted that for other corresponding descriptions of the functional units involved in the device for determining the operating state of an ultra-high voltage disconnector provided in an embodiment of the present invention, reference can be made to Figure 1 and Figure 4A , 4B The corresponding description in will not be repeated here.

[0160] In an exemplary embodiment, the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for determining the operating status of the ultra-high voltage disconnector are implemented.

[0161] If the module / unit integrated in the computer unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0162] The present application also provides a computer program product, comprising a computer program, which implements the steps of the method described above when executed by a processor.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by means of software plus necessary general hardware platforms. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0164] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present application.

[0165] Those skilled in the art will appreciate that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0166] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0167] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for determining the operating state of an ultra-high voltage disconnector, characterized in that: include: Constructing a target ultra-high voltage disconnector data set, wherein the target ultra-high voltage disconnector data set includes multiple ultra-high voltage disconnector images and description text annotated with each ultra-high voltage disconnector image, and the description text includes scene description information related to the switch operation state, bounding box coordinate information, and target segmentation mask information; Determine a visual language detection model, train the visual language detection model using the ultra-high voltage disconnector data set, update model parameters, and obtain a target visual language detection model for indicating the interactive relationship between images and language; Acquire a current ultra-high voltage disconnector image, input the current ultra-high voltage disconnector image into the target visual language detection model, and obtain a target description text output by the target visual language detection model; Determine the target operating state of the current ultra-high voltage disconnector based on the target description text; The step of constructing a target ultra-high voltage disconnector data set includes: Constructing an ultra-ultra-high voltage disconnector data set includes: obtaining multiple ultra-ultra-high voltage disconnector images at a substation site, each ultra-ultra-high voltage disconnector image includes an ultra-ultra-high voltage disconnector and components related to the ultra-ultra-high voltage disconnector, and the ultra-ultra-high voltage disconnector corresponds to an angle and an operating state; preprocessing the multiple ultra-ultra-high voltage disconnector images to obtain multiple processed ultra-ultra-high voltage disconnector images; obtaining a language description set for the ultra-ultra-high voltage disconnector, the language description set includes multiple scene description information, the scene description information is obtained by inputting switch professional terms and scene information into a language description function, and the switch professional terms include switch model specifications, switch voltage level, switch operating state, switch operation mode, switch structure type, accessory equipment type, and installation location; performing the following operations for each ultra-ultra-high voltage disconnector image: using a multimodal large model to determine a similarity score between the ultra-ultra-high voltage disconnector image and each scene description information in the multiple scene description information, obtaining multiple similarity scores, and sorting the multiple similarity scores in descending order to obtain a specific scene description information. A plurality of similarity scores having a sequential relationship, extracting at least one similarity score whose similarity score meets a target threshold from the plurality of similarity scores having a sequential relationship, and using at least one scene description information corresponding to the at least one similarity score as at least one high-similarity scene description information; determining the bounding box coordinate information and target segmentation mask information of each ultra-high voltage disconnector image, and obtaining a positive training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one high-similarity scene description information of each ultra-high voltage disconnector image; obtaining at least one missing description information corresponding to each ultra-high voltage disconnector image, and obtaining a negative training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one missing description information of each ultra-high voltage disconnector image; obtaining a training sample pair corresponding to each ultra-high voltage disconnector image based on the positive training sample and the negative training sample corresponding to each ultra-high voltage disconnector image, and obtaining the ultra-high voltage disconnector data set based on the training sample pair corresponding to each ultra-high voltage disconnector image; Removing at least one training sample pair whose image and description information do not match from the ultra-high voltage disconnector data set to obtain an intermediate ultra-high voltage disconnector data set, wherein the intermediate ultra-high voltage disconnector data set includes a plurality of intermediate training sample pairs, each of which includes an intermediate positive training sample and an intermediate negative training sample; Determine a multimodal large model, and train the multimodal large model based on the intermediate ultra-high voltage disconnector data set, update model parameters, and obtain a target multimodal large model for quantifying the similarity between the image and the description text; Extracting the plurality of intermediate positive training samples from the intermediate ultra-high voltage disconnector data set, and inputting the plurality of intermediate positive training samples into the target multimodal large model, determining the scene description text with the highest similarity corresponding to each ultra-high voltage disconnector image, and obtaining the target ultra-high voltage disconnector data set, wherein the scene description text with the highest similarity includes the scene description information with the highest similarity, the bounding box coordinate information, and the target segmentation mask information; Wherein, removing at least one training sample pair whose image and description information do not match from the ultra-high voltage disconnector data set to obtain an intermediate ultra-high voltage disconnector data set includes: Determining a plurality of training sample pairs included in the ultra-high voltage disconnector data set; Using a target image encoder ResNet to perform feature extraction on the ultra-high voltage disconnector image corresponding to the training sample pair to obtain an image feature vector corresponding to the training sample pair, and using a target text encoder BERT to convert high-similarity scene description information, bounding box coordinate information, and segmentation mask information corresponding to the training sample pair into a first text embedding vector, and calculating a first cosine similarity between the image feature vector and the first text embedding vector. If the first cosine similarity is lower than a first target threshold, removing the training sample pair from the multiple training sample pairs; and, Using the target image encoder ResNet to perform feature extraction on the ultra-high voltage disconnector image corresponding to the training sample pair to obtain the image feature vector corresponding to the training sample pair, and using the target text encoder BERT to convert the missing scene description information, border coordinate information and segmentation mask information corresponding to the training sample pair into a second text embedding vector, and calculating the second cosine similarity between the image feature vector and the second text embedding vector, and if the second cosine similarity is higher than a second target threshold, removing the training sample pair from the multiple training sample pairs; The intermediate ultra-high voltage disconnector data set is obtained.

2. The method for determining the operating status of an ultra-high voltage disconnector according to claim 1, characterized in that: The preprocessing of the plurality of ultra-high voltage disconnector images to obtain the processed plurality of ultra-high voltage disconnector images comprises: Unifying the sizes of the multiple ultra-high voltage disconnector images to obtain multiple first intermediate ultra-high voltage disconnector images with target sizes; Performing quality screening on the plurality of first intermediate ultra-high voltage disconnector images to obtain a plurality of second intermediate ultra-high voltage disconnector images; Data enhancement is performed on the multiple second intermediate ultra-high voltage disconnector images to obtain the multiple processed ultra-high voltage disconnector images.

3. The method for determining the operating status of an ultra-high voltage disconnector according to claim 1, characterized in that: The method of training the visual language detection model by using the ultra-high voltage disconnector data set, updating the model parameters, and obtaining a target visual language detection model for indicating the interactive relationship between the image and the language includes: Divide the ultra-high voltage disconnector data set into multiple non-overlapping sample subsets, select a sample subset from the multiple sample subsets as a first validation subset, use the remaining sample subsets in the multiple sample subsets except the first validation subset as a first training subset, use the first training subset to train the visual language detection model, update model parameters, and obtain a first visual language detection model, input the first validation subset into the first visual language detection model to obtain a first description text set, obtain a first actual description text set from the first validation subset, calculate the matching degree between the first description text set and the first actual description text set, continue to select another sample subset from the multiple sample subsets as a second validation subset, use the remaining sample subsets in the multiple sample subsets except the second validation subset as a second training subset, repeat the above operation until all sample subsets in the multiple sample subsets complete cross-validation, obtain multiple matching degrees, determine a maximum matching degree among the multiple matching degrees, and use the visual language detection model corresponding to the maximum matching degree as an intermediate visual language detection model; The intermediate visual language detection model is trained using the ultra-high voltage disconnector data set, and the model parameters are updated to obtain the target visual language detection model for indicating the interactive relationship between image and language.

4. The method for determining the operating status of an ultra-high voltage disconnector according to claim 1, characterized in that: After determining the target operating state of the current ultra-high voltage disconnector based on the target description text, the method further includes: Acquire the actual operating state of the current ultra-high voltage disconnector, compare the actual operating state with the target operating state, and obtain a comparison result; When the comparison result indicates that the actual operating state is consistent with the target operating state, determining that the image detection result is reliable; When the comparison result indicates that the actual operating status is inconsistent with the target operating status, the image detection result is determined to be unreliable, and a warning message indicating that the image detection result is unreliable is generated, and the warning message is sent to the terminal held by the technician, so that the technician can perform problem troubleshooting based on the terminal when receiving the warning message.

5. A device for determining the operating state of an ultra-high voltage disconnector, characterized in that: include: A construction module is used to construct a target ultra-high voltage disconnector data set, wherein the target ultra-high voltage disconnector data set includes multiple ultra-high voltage disconnector images and description text annotated with each ultra-high voltage disconnector image, and the description text includes scene description information related to the switch operation state, bounding box coordinate information and target segmentation mask information; The step of constructing a target ultra-high voltage disconnector data set includes: Constructing an ultra-ultra-high voltage disconnector data set includes: obtaining multiple ultra-ultra-high voltage disconnector images at a substation site, each ultra-ultra-high voltage disconnector image includes an ultra-ultra-high voltage disconnector and components related to the ultra-ultra-high voltage disconnector, and the ultra-ultra-high voltage disconnector corresponds to an angle and an operating state; preprocessing the multiple ultra-ultra-high voltage disconnector images to obtain multiple processed ultra-ultra-high voltage disconnector images; obtaining a language description set for the ultra-ultra-high voltage disconnector, the language description set includes multiple scene description information, the scene description information is obtained by inputting switch professional terms and scene information into a language description function, and the switch professional terms include switch model specifications, switch voltage level, switch operating state, switch operation mode, switch structure type, accessory equipment type, and installation location; performing the following operations for each ultra-ultra-high voltage disconnector image: using a multimodal large model to determine a similarity score between the ultra-ultra-high voltage disconnector image and each scene description information in the multiple scene description information, obtaining multiple similarity scores, and sorting the multiple similarity scores in descending order to obtain a specific scene description information. A plurality of similarity scores having a sequential relationship, extracting at least one similarity score whose similarity score meets a target threshold from the plurality of similarity scores having a sequential relationship, and using at least one scene description information corresponding to the at least one similarity score as at least one high-similarity scene description information; determining the bounding box coordinate information and target segmentation mask information of each ultra-high voltage disconnector image, and obtaining a positive training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one high-similarity scene description information of each ultra-high voltage disconnector image; obtaining at least one missing description information corresponding to each ultra-high voltage disconnector image, and obtaining a negative training sample corresponding to each ultra-high voltage disconnector image based on the bounding box coordinate information, target segmentation mask information and at least one missing description information of each ultra-high voltage disconnector image; obtaining a training sample pair corresponding to each ultra-high voltage disconnector image based on the positive training sample and the negative training sample corresponding to each ultra-high voltage disconnector image, and obtaining the ultra-high voltage disconnector data set based on the training sample pair corresponding to each ultra-high voltage disconnector image; Removing at least one training sample pair whose image and description information do not match from the ultra-high voltage disconnector data set to obtain an intermediate ultra-high voltage disconnector data set, wherein the intermediate ultra-high voltage disconnector data set includes a plurality of intermediate training sample pairs, each of which includes an intermediate positive training sample and an intermediate negative training sample; Determine a multimodal large model, and train the multimodal large model based on the intermediate ultra-high voltage disconnector data set, update model parameters, and obtain a target multimodal large model for quantifying the similarity between the image and the description text; Extracting the plurality of intermediate positive training samples from the intermediate ultra-high voltage disconnector data set, and inputting the plurality of intermediate positive training samples into the target multimodal large model, determining the scene description text with the highest similarity corresponding to each ultra-high voltage disconnector image, and obtaining the target ultra-high voltage disconnector data set, wherein the scene description text with the highest similarity includes the scene description information with the highest similarity, the bounding box coordinate information, and the target segmentation mask information; Wherein, removing at least one training sample pair whose image and description information do not match from the ultra-high voltage disconnector data set to obtain an intermediate ultra-high voltage disconnector data set includes: Determining a plurality of training sample pairs included in the ultra-high voltage disconnector data set; Using a target image encoder ResNet to perform feature extraction on the ultra-high voltage disconnector image corresponding to the training sample pair to obtain an image feature vector corresponding to the training sample pair, and using a target text encoder BERT to convert high-similarity scene description information, bounding box coordinate information, and segmentation mask information corresponding to the training sample pair into a first text embedding vector, and calculating a first cosine similarity between the image feature vector and the first text embedding vector. If the first cosine similarity is lower than a first target threshold, removing the training sample pair from the multiple training sample pairs; and, Using the target image encoder ResNet to perform feature extraction on the ultra-high voltage disconnector image corresponding to the training sample pair to obtain the image feature vector corresponding to the training sample pair, and using the target text encoder BERT to convert the missing scene description information, border coordinate information and segmentation mask information corresponding to the training sample pair into a second text embedding vector, and calculating the second cosine similarity between the image feature vector and the second text embedding vector, and if the second cosine similarity is higher than a second target threshold, removing the training sample pair from the multiple training sample pairs; Obtaining the intermediate ultra-high voltage disconnector data set; A training module, used to determine a visual language detection model, train the visual language detection model using the ultra-high voltage disconnector data set, update model parameters, and obtain a target visual language detection model for indicating the interactive relationship between image and language; A detection module, used to obtain a current ultra-high voltage disconnector image, input the current ultra-high voltage disconnector image into the target visual language detection model, and obtain a target description text output by the target visual language detection model; A determination module is used to determine the target operating state of the current ultra-high voltage disconnector based on the target description text.

6. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images