Method, device and equipment for constructing image data set of ultra-high voltage circuit breaker and medium

Abstract

The invention relates to the technical field of data processing, and provides an ultra-high voltage circuit breaker image data set construction method, device and equipment and a storage medium, and the method comprises the steps: obtaining multi-angle and multi-state images of an ultra-high voltage circuit breaker and an assembly thereof, and carrying out the preprocessing, and obtaining a processed image; generating an image description corresponding to each processed image; carrying out fine labeling on the calibrated ultra-high voltage circuit breaker instance in each processed image; creating corresponding positive and negative sample pairs according to the image description; performing supplementary labeling and scene expansion on the positive and negative sample pairs according to the multi-target scene to obtain labeled and expanded positive and negative sample pairs; and constructing an ultra-high voltage circuit breaker data set according to the image description, the annotation and the expanded positive and negative sample pairs. Through the technical scheme, the learning effect of the model in the training process and the recognition accuracy during testing are ensured, and the understanding of the model for abstract concepts and missing situations and the ability of coping with complex scenes are improved.

Description

Technical field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and medium for constructing an ultra-high voltage circuit breaker image dataset. [Background Technology]

[0002] With the advancement of ultra-high voltage (UHV) transmission and transformation technology, equipment status monitoring and fault warning have become increasingly important. Monitoring the status of key equipment, such as switchgear, is directly related to the safe and stable operation of the entire power grid. However, traditional monitoring methods often rely on manual inspections and static threshold judgments, resulting in low efficiency and poor real-time performance. Computer vision technology, particularly deep learning object detection algorithms, has made significant progress, driven by large-scale datasets such as MS COCO. These algorithms can effectively identify and locate complex objects in images. Applying these technologies to high-precision object detection and fine-grained equipment status recognition for power equipment will help achieve intelligent remote monitoring and automated operation and maintenance.

[0003] In today's era, data governance has become a key factor in improving decision-making efficiency and service quality across all industries, and the power industry is no exception. However, faced with the challenges of growing complexity and diversity, existing power system datasets have exposed significant limitations in descriptive detection of key components such as circuit breakers. Traditional object detection datasets, such as general object detection datasets, have played an important role in the past, but they are no longer able to meet the urgent needs for accurate identification and condition monitoring of ultra-high voltage (UHV) circuit breakers in modern power grids. These high-end devices operate in diverse environments, requiring not only extremely high safety standards but also complex operation and maintenance scenarios.

[0004] Therefore, in order to cope with the complex and changeable equipment status recognition challenges in power systems, especially for key equipment in ultra-high voltage substations such as circuit breakers, it is urgent to build a descriptive target detection dataset specifically for ultra-high voltage circuit breaker equipment. [Summary of the invention]

[0005] The embodiments of the present application provide a method, apparatus, device and medium for constructing an ultra-high voltage circuit breaker image dataset, aiming to solve technical problems such as data problems for automatic identification of opening and closing information of key equipment such as circuit breakers in ultra-high voltage substations in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a method for constructing an ultra-high voltage circuit breaker image dataset, comprising:

[0007] Acquire multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components, and perform preprocessing to obtain processed images;

[0008] generating an image description corresponding to each processed image, wherein the image description includes a short category name and a detailed functional description;

[0009] Finely label the UHV circuit breaker instances identified in each processed image;

[0010] Creating corresponding positive and negative sample pairs according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description;

[0011] According to the multi-target scenario, the positive and negative sample pairs are supplementally labeled and the scenario is expanded to obtain labeled and expanded positive and negative sample pairs;

[0012] An ultra-high voltage circuit breaker dataset is constructed based on the image description and the annotated and expanded positive and negative sample pairs.

[0013] In one embodiment, optionally, acquiring multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components and performing preprocessing to obtain processed images includes:

[0014] Acquire multiple multi-angle and multi-state images of UHV circuit breakers and their components;

[0015] Use bilinear interpolation to scale each image to the target size to obtain the target image;

[0016] Data enhancement is performed on the target image to obtain a processed image.

[0017] In one embodiment, optionally, generating an image description corresponding to each processed image includes:

[0018] Obtain the preset categories and classifications corresponding to ultra-high voltage circuit breakers;

[0019] Creating image description templates suitable for various scenarios and needs based on the preset categories and classifications;

[0020] The image description template is parameterized using the preset categories and classifications to generate an image description corresponding to each processed image.

[0021] In one embodiment, optionally, fine-marking is performed on the UHV circuit breaker instances calibrated in each processed image, including:

[0022] Detect each UHV circuit breaker instance in each processed image and accurately calibrate the bounding box location;

[0023] Perform instance segmentation masking on each UHV circuit breaker instance to obtain the annotation result;

[0024] The annotation results are stored in a preset format.

[0025] In one embodiment, optionally, creating corresponding positive and negative sample pairs according to the image description includes:

[0026] Calculating CLIP similarity between the processed image and the image description to obtain a calculation result;

[0027] According to the calculation result, determining the switch instance that matches the image description as a positive sample and the switch instance that does not match the image description as a negative sample;

[0028] All positive and negative samples are associated and integrated with their corresponding images and image descriptions to form positive and negative sample pairs.

[0029] In one embodiment, optionally, supplementary annotation and scene expansion are performed on the positive and negative sample pairs according to a multi-target scenario to obtain annotated and expanded positive and negative sample pairs, including:

[0030] Creating target annotation information for the positive and negative sample pairs according to the switch instance and the image corresponding thereto;

[0031] Supplement the target scene samples not included in the current positive and negative sample pairs to obtain the labeled and expanded positive and negative sample pairs.

[0032] In one embodiment, optionally, the method further includes:

[0033] Extracting image feature vectors from the ultra-high voltage circuit breaker dataset using a pre-trained image encoder;

[0034] Using a pre-trained text encoder to convert image descriptions in the ultra-high voltage circuit breaker dataset into text embedding vectors;

[0035] Calculating the similarity between the image feature vector and the text embedding vector, and when the similarity is less than a preset threshold, regenerating an image description for the corresponding image to optimize the ultra-high voltage circuit breaker dataset;

[0036] The optimized UHV circuit breaker dataset is randomly divided into training set and validation set;

[0037] Training the CLIP model on the training set so that during the training process the model maximizes the similarity between matching pairs of images and texts while minimizing the similarity between non-matching pairs of images and texts;

[0038] Analyze and optimize image descriptions and annotations until the ultra-high voltage circuit breaker dataset meets preset data quality requirements.

[0039] In a second aspect, an embodiment of the present application provides a device for constructing an ultra-high voltage circuit breaker image dataset, comprising:

[0040] An acquisition module is used to acquire multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components, and perform preprocessing to obtain processed images;

[0041] a generating module, configured to generate an image description corresponding to each processed image, wherein the image description includes a short category name and a detailed function description;

[0042] Annotation module, used to finely annotate the UHV circuit breaker instances identified in each processed image;

[0043] a sample pair creation module, configured to create corresponding positive and negative sample pairs according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description;

[0044] An expansion module, configured to perform additional annotation and scene expansion on the positive and negative sample pairs according to a multi-target scenario, so as to obtain annotated and expanded positive and negative sample pairs;

[0045] A construction module is used to construct an ultra-high voltage circuit breaker data set based on the image description and the annotated and expanded positive and negative sample pairs.

[0046] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned method for constructing an ultra-high voltage circuit breaker image dataset are implemented.

[0047] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned method for constructing an ultra-high voltage circuit breaker image dataset are implemented.

[0048] In the scheme implemented by the above-mentioned ultra-ultra-high voltage circuit breaker image dataset construction method, device, system, equipment and storage medium, multi-angle and multi-state images of the ultra-ultra-high voltage circuit breaker and its components are obtained and preprocessed to obtain processed images; an image description corresponding to each processed image is generated, wherein the image description includes a brief category name and a detailed function description; the ultra-ultra-high voltage circuit breaker instance calibrated in each processed image is finely annotated; corresponding positive and negative sample pairs are created according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description; according to a multi-target scenario, the positive and negative sample pairs are supplementally annotated and the scenario is expanded to obtain annotated and expanded positive and negative sample pairs; an ultra-ultra-high voltage circuit breaker dataset is constructed according to the image description and the annotated and expanded positive and negative sample pairs. In this invention, the dataset includes detailed image annotations and flexible language descriptions, enabling machine learning models to accurately identify and understand the various states of switchgear. Regardless of position, state, or switches with specific characteristics, these switches can be accurately identified and located, improving the accuracy and reliability of device operating status detection. By meticulously annotating and properly proportioning positive and negative samples, balancing the number of positive and negative samples and avoiding overfitting or underfitting, the model's learning effect during training and recognition accuracy during testing are ensured, improving the model's understanding of abstract concepts and missing contexts and its ability to cope with complex scenarios.

Brief Description of the Drawings

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A schematic flowchart of a method for constructing an ultra-high voltage circuit breaker image dataset according to an embodiment of the present application is shown.

[0051] Figure 2 A schematic flowchart of step S101 in a method for constructing an ultra-high voltage circuit breaker image dataset according to an embodiment of the present application is shown.

[0052] Figure 3 A schematic flowchart of step S102 in a method for constructing an ultra-high voltage circuit breaker image dataset according to an embodiment of the present application is shown.

[0053] Figure 4 A schematic flowchart of step S103 in a method for constructing an ultra-high voltage circuit breaker image dataset according to an embodiment of the present application is shown.

[0054] Figure 5A A schematic flowchart of step S104 in a method for constructing an ultra-high voltage circuit breaker image dataset according to an embodiment of the present application is shown.

[0055] Figure 5B An example diagram of a dataset after pairing positive and negative samples according to an embodiment of the present application is shown.

[0056] Figure 5C FIG2 shows an example diagram of a dataset fully annotated with positive and negative examples according to an embodiment of the present application.

[0057] Figure 5D An example diagram of a data set with a free and unrestricted description format according to an embodiment of the present application is shown.

[0058] Figure 5E An example diagram of a dataset of presence and absence descriptions according to one embodiment of the present application is shown.

[0059] Figure 6 A schematic flowchart of a method for constructing an ultra-high voltage circuit breaker image dataset according to another embodiment of the present application is shown.

[0060] Figure 7 A schematic diagram of a CLIP model framework according to an embodiment of the present application is shown.

[0061] Figure 8 A block diagram of an ultra-high voltage circuit breaker image dataset construction device according to an embodiment of the present application is shown.

[0062] Figure 9 A structural diagram of a computer device according to an embodiment of the present application is shown.

[0063] Figure 10 Another structural schematic diagram of a computer device according to an embodiment of the present application is shown. [Specific implementation method]

[0064] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0065] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0066] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0067] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0068] See also Figure 1 , Figure 1 A schematic flow chart of a method for constructing an ultra-high voltage (UHV) circuit breaker image dataset according to one embodiment of the present application is provided. This method is used to address technical issues related to automated recognition of tripping and closing information for key equipment such as circuit breakers in UHV substations.

[0069] like Figure 1 As shown, according to an embodiment of the present application, a method for constructing an ultra-high voltage circuit breaker image dataset includes the following steps:

[0070] Step S101, acquiring multi-angle and multi-state images of an ultra-high voltage circuit breaker and its components, and performing pre-processing to obtain processed images;

[0071] like Figure 2 As shown, in one embodiment, optionally, step S101 includes:

[0072] Step S201, acquiring a plurality of multi-angle and multi-state images of an ultra-high voltage circuit breaker and its components;

[0073] Acquire a large number of images containing the circuit breaker open / close indicator color codes. The circuit breaker open / close states in these images should be distributed as evenly as possible, that is, the number of images showing the indicator color codes in both closed and open states should be as even as possible. Specifically, images of the circuit breakers at the substation site can be captured using on-site monitoring equipment and handheld cameras.

[0074] Construct an image set: suppose there is a finite set of angles A = θ1, θ2,…, θ m and a finite set of states S = s1, s2, ..., s n , then all multi-angle and multi-state image sets can be represented in the form of a multi-dimensional array or matrix:

[0075]

[0076] Among them, θ i represents the i-th observation angle, s i Indicates the jth switch state, I(θi ,s j ) represents the image captured at the i-th angle and when the switch is in the j-th state. Images is an m×n dimensional array or matrix. Each item is an image vector or matrix that represents the information of the image at a specific angle and state.

[0077] A carefully designed image acquisition strategy ensures that the dataset contains rich images of ultra-high voltage circuit breakers and their components from multiple angles and positions, covering all critical operating and environmental conditions. The application of image processing techniques, such as geometric transformation and dimensional normalization, ensures data consistency and standardization, providing a solid foundation for model learning.

[0078] Step S202, using bilinear interpolation method, scaling each image to a target size to obtain a target image;

[0079] In this embodiment, each of the collected images of different sizes W can be scaled by a scaling function S(·) original ×H original The UHV circuit breaker image is converted to the same size W target ×H target , which is responsible for mapping the original coordinates to the target coordinates and calculating the value of the new pixel based on the values ​​of the surrounding pixels.

[0080] Given the original image I original (u,v), the goal is to generate a new image I resized (u, v), where u∈[0,W-1] represents the horizontal axis (column) coordinate, that is, the position index of the pixel in the width direction of the image, and v∈[0,H-1] represents the vertical axis (row) coordinate, that is, the position index of the pixel in the height direction of the image. The conversion process is expressed as:

[0081] I resized (u,v)=S(I original (u,v),W original ,H original ,W target ,H target )

[0082] For bilinear interpolation, the specific calculation method of S involves the weighted average of four neighboring pixels, while for nearest neighbor interpolation, the pixel value closest to the target coordinate is directly taken. S can be expressed as:

[0083] S(x,y)=(1-x)(1-y)×I(x ′ ,y ′ )+x×(1-y)×I(x ′ +1,y ′ )+(1-x)×y

[0084] ×I(x ′ ,y ′ +1)+x×y×I(x ′ +1,y ′ +1)

[0085] Among them, x and y represent the fractional part of the target coordinates, that is, x' and y' represent the integer parts of the target coordinates, that is,

[0086] Furthermore, the converted images can be filtered to discard images that are not suitable for labeling, are ambiguous, unrecognizable, or have low resolution.

[0087] Step S203: performing data enhancement on the target image to obtain a processed image.

[0088] Perform data enhancement on the filtered images, including spatial changes (rotation, translation, scaling, flipping), adjusting the color and hue of the image (brightness, contrast, saturation), and adding moderate random noise (Gaussian noise) to the image;

[0089] The mathematical description of the Gaussian noise added by data enhancement is shown in the following formula:

[0090]

[0091] Here, x is a random variable and σ is the standard deviation.

[0092] Step S102, generating an image description corresponding to each processed image, wherein the image description includes a short category name and a detailed function description;

[0093] like Figure 3 As shown, in one embodiment, optionally, step S102 includes:

[0094] Step S301, obtaining the preset category and classification corresponding to the ultra-high voltage circuit breaker;

[0095] In this embodiment, professional terms related to ultra-high voltage circuit breakers, namely preset categories and classifications, can be obtained, including but not limited to model, voltage level, operating mode, safety level, installation location, appearance characteristics, accessory equipment and various operating states.

[0096] Specifically, based on the compiled image dataset Images of on-site ultra-high voltage circuit breakers, we clarify the key components and core technologies of ultra-high voltage circuit breakers, refine the characteristics and application scenarios of various switch models, and comprehensively collect and organize professional terminology in related fields, represented as T. This includes but is not limited to the following aspects, as shown in Table 1.

[0097] Table 1

[0098]

[0099] Step S302: creating image description templates applicable to various scenarios and needs based on the preset categories and classifications;

[0100] Based on the collected professional term list T, a set of flexible and comprehensive description templates M is created to ensure that they can adapt to different scenarios and needs. The templates should cover everything from short category names M_brief (such as "UHV circuit breaker") to detailed functional descriptions M_detailed (such as "enclosed type, rated voltage 35kV").

[0101] Short Category Name Template M brief "The operating state of the ${type} ultra-high voltage circuit breaker is ${state}", "The operating state of the ${type} ultra-high voltage circuit breaker with a rated voltage of ${voltage} is ${state}", etc., where ${type} and ${voltage} are placeholders for the type and voltage level to be replaced, respectively;

[0102] Detailed functional characteristics description template M detailed "The rated voltage is ${voltage}, it is made of ${material}, it complies with the ${standard} operating mode of ${operation}, it has a ${Security} level of security, and it is suitable for installation in ${surroundings} environments. The operating state of the ${type} ultra-high voltage circuit breaker is ${state}", etc., where ${·} represents a placeholder for the professional term to be replaced.

[0103] Step S303 : Filling parameters of the image description template using the preset categories and classifications to generate an image description corresponding to each processed image.

[0104] Use the professional term list T as a parameter to fill in the template and generate diversified descriptions according to different application scenarios and requirements. For example, replace different adjectives, quantifiers, status words, etc. to reflect the differences of circuit breakers in different environments, perspectives, and states.

[0105] Consider a template M, which contains n placeholders, denoted as p1, p2, ..., p n , and the term list T contains m terms, denoted as t1, t2, ..., t m The mapping function f:{1,2,…,n}→{1,2,…,m} defines the correspondence between placeholders and terms, that is, the placeholder p i will be term t i In addition, define a value function v:{1,2,…,m}→V, where V is the set of all possible values, which gives the actual value of each term, that is, v(j) = actual t j value.

[0106] The mathematical expression D that generates the final description can be expressed as:

[0107] p i =M(p1 / v(f(1)),p2 / v(f(2)),…,p n / v(f(n)))

[0108] Among them, M(p1 / v(f(1)),p2 / v(f(2)),…,p n / v(f(n))) means that all placeholders M in template M are replaced by the string after v(f(i)).

[0109] More specifically, if we represent M as a string function M(x1,x2,…,x n ), where x i is a placeholder p i The value of , then the function G that generates the description can be written as:

[0110] G(v°f)=M(v(f(1),v(f(2)),…,v(f(n)))

[0111] Here, v°f represents function composition, that is, first applying f and then applying v.

[0112] Instantiated description generation: To instantiate this process, consider a simple template M(x1,x2) = "$x1 type ultra-high voltage circuit breaker operating state is x2", professional term list T = {t1 = "model specification", t9 = "operating state"}, mapping f(1) = 1, f(2) = 7, and value function v(1) = "TGF-500KV", v(7) = "normally closed". The final generated description D is:

[0113] D=M("Model Specifications", "Operating Status")="The operating status of the TGF-500KV ultra-high voltage circuit breaker is normally closed."

[0114] Further combined with the actual situation, generate a detailed description M containing complex attributes and specific scenarios advanced "The open-type CB3000 ultra-high voltage circuit breaker, rated at 1100kV, located on the east side of the substation tower, is currently in the opening state. The red LED warning light on its upper part flashes, indicating that the device is responding to instructions from the remote dispatching center for emergency disconnection. Simultaneously, its intelligent diagnostic system is providing real-time feedback on the mechanical performance indicators and electrical parameters of the circuit breaker's operation."

[0115] Use a large language model for synonym replacement to increase description diversity. The T5 (Text-to-Text Transfer Transformer) large model from the Transformers library is selected as the core component for the synonym replacement task.

[0116] First, leveraging the T5 model's pre-trained capabilities, we fine-tuned it to optimize terminology and descriptive language specifically for the power sector, particularly ultra-high voltage (UHV) circuit breakers. This involved utilizing a large corpus of power engineering data to ensure the model's ability to fully understand and generate specialized expressions within this field. During the fine-tuning process, we emphasized the model's contextual understanding of specialized terminology and its ability to express synonyms, ensuring accurate and natural replacements.

[0117] To implement synonym replacement, the switch description text from the original dataset was fed into the fine-tuned T5 model. Prompt instructions guided the model to generate multiple synonyms for each description. These expressions retained the original intent while replacing it with appropriate synonyms within the power sector, increasing description diversity.

[0118] In order to ensure that the text descriptions generated for the UHV circuit breaker equipment are collected p Highly consistent with the actual image content, manual screening can be used for strict quality control. During this process, personnel with professional knowledge review all descriptions automatically generated by the model to ensure that each description accurately reflects key information such as the device type, model, material, voltage level, operating mode, safety level, installation location and appearance characteristics, and the status of attached devices in the image.

[0119] Through the above technical solution, a multi-level language description system ranging from short categories to detailed functional characteristics was constructed. This system not only includes basic information such as the type, model, and voltage level of the switch, but also includes complex descriptions such as installation location, appearance characteristics, and operating status, providing precise language guidance for the model to understand image content.

[0120] Step S103 , finely labeling the UHV circuit breaker instances calibrated in each processed image.

[0121] like Figure 4 As shown, in one embodiment, optionally, step S103 includes:

[0122] Step S401 , detecting each UHV circuit breaker instance in each processed image and accurately calibrating the bounding box position;

[0123] For each image, all instances of ultra-high voltage circuit breakers are identified. For each switch instance, a precise bounding box is drawn to define its exact location in the image.

[0124] Step S402: performing corresponding instance segmentation masking on each ultra-high voltage circuit breaker instance to obtain a labeling result;

[0125] Create pixel-level annotations corresponding to each circuit breaker instance, also known as instance segmentation masks, which are usually represented as black and white binary images, where white areas represent pixels where the switch is located and black areas are background or other objects.

[0126] If necessary, perform additional detailed annotations on specific parts or features of the switch, such as the location and status of important components such as the switch's operating handle, status indicator, and nameplate. After completing the initial annotations, perform multiple rounds of cross-checks to ensure consistency and accuracy of all annotations, especially in complex scenarios to confirm that no omissions or errors have been made.

[0127] Step S403: storing the annotation results in a preset format.

[0128] The annotation results are organized in the specified format, including bounding box coordinates, instance segmentation masks, and other possible additional information (such as switch status, etc.), so that they can be subsequently integrated with the corresponding image and description information.

[0129] In the above example, the switch instances in the image were meticulously annotated, including bounding box location and instance segmentation, to ensure the model's accurate recognition of the target. By creating pairs of positive and negative examples—those that match and don't match the description—the model's discriminative capabilities were enhanced, effectively preventing overfitting and underfitting, and improving recognition accuracy and robustness.

[0130] Step S104: creating corresponding positive and negative sample pairs according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description;

[0131] like Figure 5A As shown, in one embodiment, optionally, step S104 includes:

[0132] Step S501, calculating the CLIP similarity between the processed image and the image description to obtain a calculation result;

[0133] In this example, given existing UHV circuit breaker images and corresponding image descriptions, the CLIP similarity between the image and each description is used to identify switch instances that perfectly match the description as positive samples. For example, if the description is "a green circuit breaker located below a tower," the image should contain circuit breaker instances with matching location and color features.

[0134] Step S502: determining, based on the calculation result, that a switch instance that matches the image description is a positive sample, and a switch instance that does not match the image description is a negative sample;

[0135] Mark the switch instances that match the language description in the image, record their bounding box coordinates and instance segmentation mask information, and ensure that the model can accurately learn the mapping relationship between language descriptions and image regions during training;

[0136] Negative samples are searched for switch instances or other objects in the same or other images that do not match the given language description. For example, if the description is "circuit breaker with nameplate," other electrical equipment without nameplates or that is not a circuit breaker can serve as negative samples. These negative samples are also annotated to clearly indicate that they do not match the current language description. Bounding box positioning and instance segmentation information are also provided so that the model can distinguish between positive and negative samples, improving the model's recognition accuracy and anti-interference ability.

[0137] Considering the training effect, the number of positive and negative samples should be relatively balanced. The ratio of positive and negative samples can be adjusted according to actual conditions to prevent the model from overfitting or underfitting during training.

[0138] The specific similarity calculation method includes:

[0139] Given an image I and a set of language descriptions D = {d1, d2, ..., d n}, the CLIP model outputs the similarity score s for each pair ij =CLIP(I,l j ), where s ij Represents image I and description d j similarity between

[0140] For each image I, calculate its correlation with each description d j The similarity s ij , set the threshold threshold, and only keep those s ij>threshold descriptions are taken as candidate positive samples. To ensure diversity, top-k (e.g. 4 to 6) descriptions with the highest similarity are selected from the candidate positive samples of each image as positive samples, denoted as P I ={p I1 ,p I2 ,...,p Ik}, where p Ij is a positive description that matches I;

[0141] For each image I, design 1 to 2 “absent” descriptions d based on its description D j ′ ,These descriptions contain elements or features that do not appear in the image, which serve as references for negative samples. The negative sample set N I ={n I1 ,n I2}, where n Ij Indicates a description that does not match I.

[0142] In step S503 , all positive samples and negative samples are associated and integrated with their corresponding images and image descriptions to form positive and negative sample pairs.

[0143] Finally, construct positive and negative sample pairs V for each image I P ,pairs N ), where pairs P ={(I,p I1 ),(I,p I2 ),...,(I,p Ik ) are positive sample pairs, pairs N ={(I,n I1 ),(I,n I2 ) is a negative sample pair.

[0144] In order to optimize the model performance, balance the number of positive and negative samples, and ensure that the dataset can effectively guide the model to learn the correspondence between language and image regions, the positive and negative samples with complete annotations and their related descriptions are finally integrated into pairs, such as Figure 5B shown.

[0145] Step S105, performing supplementary annotation and scene expansion on the positive and negative sample pairs according to the multi-target scenario to obtain annotated and expanded positive and negative sample pairs;

[0146] In one embodiment, optionally, step S105 includes:

[0147] 1) creating target annotation information for the positive and negative sample pairs based on the switch instance and its corresponding image;

[0148] In this embodiment, when there are multiple UHV circuit breaker instances in the image, each instance is independently and accurately annotated, including its own bounding box positioning and instance segmentation mask information; a unique description is generated for each instance based on the characteristics of each instance and the correspondence with the language description, and it is ensured that the dataset contains positive sample annotations for these multi-target scenes; for specific conditions mentioned in the description, such as "circuit breaker on the left", it is verified whether there are instances consistent with the description in the image, and they are accurately annotated; for UHV circuit breaker instances mentioned in the description but not actually existing in the image (such as "circuit breaker without nameplate"), that is, negative samples, they are also annotated, and it is clearly pointed out which areas or image spaces do not conform to the description, such as Figures 5C to 5E shown.

[0149] Record the association between each image description and the instance annotated in the image, regardless of whether the instance actually exists in the image, and ensure that the dataset contains enough positive and negative sample pairs so that the model can learn how to correctly identify and reject the target object in the description.

[0150] 2) Supplement the target scene samples not included in the current positive and negative sample pairs to obtain the labeled and expanded positive and negative sample pairs.

[0151] In some cases, an image may contain elements that are not explicitly mentioned in the description but are common in real-world environments, or a feature mentioned in the description may not be present in the current image. To this end, we actively design and construct samples that include missing concepts to enhance the model's understanding and generalization capabilities for complex real-world situations.

[0152] In this embodiment, scene samples that may appear in reality but are not shown in the current image can be added to enrich the diversity of the training data set and improve the generalization ability of the model, so that it can not only accurately identify the target object, but also adapt to the target detection needs in various complex environments.

[0153] Step S106: constructing an ultra-high voltage circuit breaker dataset based on the image description and the annotated and expanded positive and negative sample pairs.

[0154] Integrate the results of the above steps S101-S105 and ensure that a high-quality ultra-high voltage circuit breaker dataset DS is constructed that comprehensively covers multiple scenarios and language description complexity.

[0155] From the previous steps S101-S105, the following results are obtained:

[0156] 1. All collected UHV circuit breaker equipment image set ImgSet = {I1, I2, ..., I N}, where I iEach is an image file;

[0157] 2. Equipment term list T = {t1, t2, ..., t n}, where each t i It is a professional term, such as voltage level, operating mode, etc.

[0158] 3. Description template set M = {m1,m2,...,m k}, each template m j Contains some variable fields to hold terms selected from a term list;

[0159] 4. Parameterized Description Set D p ={(d1,v1),(d2,v2),...,(d m ,v m )}, where d i is filled with the corresponding variable value v i Descriptive example, v i from lists of terms and other quantitative parameters;

[0160] For each ultra-high voltage circuit breaker equipment image I i , find the parameterized description set D that matches it p Description instance d ij , which fully displays the image I i The state or characteristics of the switch device are displayed to form image-text pairs, ensuring that each set of samples contains an image and its corresponding positive and negative sample language descriptions;

[0161] Organize the integrated image-text data pairs into a unified multimodal training sample set dataset I i is with description d ij The sample set features rich language expressions and a variety of complex scenarios, meeting the requirements of image-to-text conversion tasks and model training. Through rigorous data cleaning and integrity checks, the constructed dataset ensures good balance and breadth in terms of sample distribution, scenario coverage, and level of abstraction of language descriptions, which facilitates the model's generalization capabilities in new scenarios and improves its understanding of complex language.

[0162] Specifically, a quality check is performed on each set of image-text pairs. If the error is greater than 2%, it is returned for re-annotation; otherwise, the dataset construction is completed.

[0163] like Figure 6 As shown, in one embodiment, optionally, the method further includes:

[0164] Step S601, using a pre-trained image encoder to extract image feature vectors from the ultra-high voltage circuit breaker dataset;

[0165] Step S602, using a pre-trained text encoder to convert the image descriptions in the ultra-high voltage circuit breaker dataset into text embedding vectors;

[0166] In the above embodiment, the image feature vector V is extracted using the pre-trained image encoder ResNet. I ; For the corresponding text description, the pre-trained text encoder BERT is used to convert it into a text embedding vector V T , the vector dimensions are all D-dimensional;

[0167] Step S603: calculating the similarity between the image feature vector and the text embedding vector; when the similarity is less than a preset threshold, regenerating an image description for the corresponding image to optimize the ultra-high voltage circuit breaker dataset;

[0168] Calculate the cosine similarity between the image feature vector and the text embedding vector: Set a threshold θ. If the similarity is lower than this threshold, the image-text pair is considered mismatched and needs to be discarded or a more accurate image description needs to be regenerated.

[0169] Step S604, randomly dividing the optimized ultra-high voltage circuit breaker data set into a training set and a validation set;

[0170] The optimized dataset is randomly divided into a training set TrainSet and a validation set ValSet, using the CLIP model framework, such as Figure 7 As shown in the figure, it simultaneously learns the joint representation space of images and texts. In this way, the CLIP model is trained on the training set to maximize the similarity between image and text pairs during training, while minimizing the similarity of unmatched pairs. The model performance is tested on the validation set to observe whether the model can produce accurate descriptions or matching results for new samples that have not been seen, so as to measure the general generalization of image and text annotations in the current dataset.

[0171] Step S605 , training the CLIP model on the training set so that the similarity between matching pairs of images and texts is maximized while minimizing the similarity between non-matching pairs of images and texts during the training process;

[0172] Step S606: Analyze and optimize the image description and annotation until the ultra-high voltage circuit breaker dataset meets the preset data quality requirements.

[0173] For image-text pairs with high similarity matching, further check the accuracy of the labels; if it is found that a text description fails to accurately capture the key features of the image, such as structure, action, state, etc., manually modify or add a detailed description. If necessary, design a new labeling system or refine the existing labels for special categories or difficult-to-identify situations; based on the analysis results, remove low-quality annotated data from the dataset, such as mislabeled or missing labels; for those samples missing key information, combine expert knowledge or other auxiliary means to fill in appropriate descriptive information; retrain the CLIP model based on the optimized dataset, and re-evaluate the similarity matrix and the model performance on the validation set. If there is still room for improvement in model performance, repeat the above steps until the preset data quality requirements are met.

[0174] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0175] Figure 8 A block diagram of an ultra-high voltage circuit breaker image dataset construction device according to an embodiment of the present application is shown.

[0176] like Figure 8 As shown, in a second aspect, an embodiment of the present application provides an ultra-high voltage circuit breaker image dataset construction device 80, comprising:

[0177] An acquisition module 81 is used to acquire multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components, and perform pre-processing to obtain processed images;

[0178] a generating module 82 for generating an image description corresponding to each processed image, wherein the image description includes a short category name and a detailed function description;

[0179] a labeling module 83 for finely labeling the UHV circuit breaker instances calibrated in each processed image;

[0180] a sample pair creation module 84 for creating corresponding positive and negative sample pairs according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description;

[0181] An expansion module 85 is used to perform additional annotation and scene expansion on the positive and negative sample pairs according to a multi-target scenario to obtain annotated and expanded positive and negative sample pairs;

[0182] The construction module 86 is configured to construct an ultra-high voltage circuit breaker dataset based on the image description and the annotated and expanded positive and negative sample pairs.

[0183] In one embodiment, optionally, the acquisition module includes:

[0184] An image acquisition unit, configured to acquire a plurality of multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components;

[0185] An image processing unit, configured to scale each image to a target size using a bilinear interpolation method to obtain a target image;

[0186] The data enhancement unit is used to perform data enhancement on the target image to obtain a processed image.

[0187] In one embodiment, optionally, the generating module includes:

[0188] A category acquisition unit, used to acquire the preset category and classification corresponding to the ultra-high voltage circuit breaker;

[0189] A template creation unit, configured to create image description templates applicable to various scenarios and requirements based on the preset categories and classifications;

[0190] A filling unit is used to fill parameters of the image description template using the preset categories and classifications to generate an image description corresponding to each processed image.

[0191] In one embodiment, optionally, the annotation module includes:

[0192] A detection unit, which detects each UHV circuit breaker instance in each processed image and accurately calibrates the bounding box position;

[0193] A labeling unit, configured to perform instance segmentation masking on each ultra-high voltage circuit breaker instance to obtain a labeling result;

[0194] The storage unit is used to store the annotation results in a preset format.

[0195] In one embodiment, optionally, the sample pair creation module includes:

[0196] a calculation unit, configured to calculate the CLIP similarity between the processed image and the image description to obtain a calculation result;

[0197] a sample determination unit, configured to determine, based on the calculation result, a switch instance that matches the image description as a positive sample and a switch instance that does not match the image description as a negative sample;

[0198] The integration unit is used to associate and integrate all positive samples and negative samples with their corresponding images and image descriptions to form positive and negative sample pairs.

[0199] In one embodiment, optionally, the expansion module includes:

[0200] an information creation unit, configured to create target annotation information for the positive and negative sample pairs according to the switch instance and the image corresponding thereto;

[0201] The supplementary annotation unit is used to supplement the target scene samples not included in the current positive and negative sample pairs to obtain annotated and expanded positive and negative sample pairs.

[0202] In one embodiment, optionally, the device further comprises:

[0203] An extraction module, configured to extract an image feature vector from the ultra-high voltage circuit breaker dataset using a pre-trained image encoder;

[0204] A conversion module, configured to convert image descriptions in the ultra-high voltage circuit breaker dataset into text embedding vectors using a pre-trained text encoder;

[0205] a similarity calculation module, configured to calculate the similarity between the image feature vector and the text embedding vector, and when the similarity is less than a preset threshold, regenerate an image description for the corresponding image to optimize the ultra-high voltage circuit breaker dataset;

[0206] A partitioning module is used to randomly divide the optimized UHV circuit breaker dataset into a training set and a validation set;

[0207] a training module for training the CLIP model on the training set so as to maximize the similarity between matching pairs of images and texts while minimizing the similarity between non-matching pairs of images and texts during the training process;

[0208] The optimization module is used to analyze and optimize the image description and annotation until the ultra-high voltage circuit breaker data set meets the preset data quality requirements.

[0209] The specific limitations of the ultra-high voltage circuit breaker image dataset construction device can be found in the limitations of the ultra-high voltage circuit breaker image dataset construction method described above and will not be repeated here. Each module in the ultra-high voltage circuit breaker image dataset construction device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0210] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a method for constructing an ultra-high voltage circuit breaker image dataset.

[0211] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a method for constructing an ultra-high voltage circuit breaker image dataset.

[0212] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0213] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in the first aspect of the embodiment is implemented when the processor executes the computer program.

[0214] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or electronic device can refer to the relevant description in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0215] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0216] It should be understood that although the terms "first," "second," etc. may be used to describe the setting units in the embodiments of the present application, these setting units should not be limited to these terms. These terms are merely used to distinguish the setting units from each other. For example, without departing from the scope of the embodiments of the present application, the first setting unit may also be referred to as the second setting unit, and similarly, the second setting unit may also be referred to as the first setting unit.

[0217] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0218] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, indirect coupling or communication connection of the device or unit, which may be electrical, mechanical or other forms.

[0219] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0220] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0221] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for constructing an ultra-high voltage circuit breaker image dataset, characterized in that: include: Acquire multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components, and perform preprocessing to obtain processed images; generating an image description corresponding to each processed image, wherein the image description includes a brief category name and a detailed functional description; Finely annotate the UHV circuit breaker instances identified in each processed image; Creating corresponding positive and negative sample pairs according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description; According to the multi-target scenario, the positive and negative sample pairs are supplementally labeled and the scenario is expanded to obtain labeled and expanded positive and negative sample pairs; An ultra-high voltage circuit breaker dataset is constructed according to the image description and the annotated and expanded positive and negative sample pairs.

2. The method according to claim 1, characterized in that The method of acquiring multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components and performing preprocessing to obtain processed images includes: Acquire multiple multi-angle and multi-state images of UHV circuit breakers and their components; Use bilinear interpolation to scale each image to the target size to obtain the target image; Data enhancement is performed on the target image to obtain a processed image.

3. The method according to claim 1, characterized in that: The generating an image description corresponding to each processed image includes: Obtain the preset categories and classifications corresponding to the ultra-high voltage circuit breakers; Creating image description templates suitable for various scenarios and needs based on the preset categories and classifications; The image description template is parameterized using the preset categories and classifications to generate an image description corresponding to each processed image.

4. The method according to claim 1, characterized in that: Finely annotate the UHV circuit breaker instances identified in each processed image, including: Detect each UHV circuit breaker instance in each processed image and accurately calibrate the bounding box location; Perform a corresponding instance segmentation mask on each ultra-high voltage circuit breaker instance to obtain a labeling result; The annotation results are stored in a preset format.

5. The method according to claim 1, characterized in that Creating corresponding positive and negative sample pairs according to the image description includes: Calculating CLIP similarity between the processed image and the image description to obtain a calculation result; According to the calculation result, determining that the switch instance matching the image description is a positive sample, and the switch instance not matching the image description is a negative sample; All positive and negative samples are associated and integrated with their corresponding images and image descriptions to form positive and negative sample pairs.

6. The method according to claim 1, characterized in that According to the multi-target scenario, the positive and negative sample pairs are supplementally labeled and the scenario is expanded to obtain labeled and expanded positive and negative sample pairs, including: Creating target annotation information for the positive and negative sample pairs according to the switch instance and the image corresponding thereto; Supplement the target scene samples not included in the current positive and negative sample pairs to obtain labeled and expanded positive and negative sample pairs.

7. The method according to claim 1, characterized in that The method further comprises: Extracting an image feature vector from the ultra-high voltage circuit breaker dataset using a pre-trained image encoder; Using a pre-trained text encoder to convert image descriptions in the ultra-high voltage circuit breaker dataset into text embedding vectors; Calculating the similarity between the image feature vector and the text embedding vector, and when the similarity is less than a preset threshold, regenerating an image description for the corresponding image to optimize the ultra-high voltage circuit breaker data set; The optimized UHV circuit breaker dataset is randomly divided into a training set and a validation set; Training the CLIP model on the training set so that during the training process, the model maximizes the similarity between matching pairs of images and texts while minimizing the similarity between non-matching pairs of images and texts; The image description and annotation are analyzed and optimized until the ultra-high voltage circuit breaker data set meets the preset data quality requirements.

8. A device for constructing an ultra-high voltage circuit breaker image data set, characterized in that: The device comprises: An acquisition module is used to acquire multi-angle and multi-state images of the ultra-high voltage circuit breaker and its components, and perform preprocessing to obtain processed images; A generating module, configured to generate an image description corresponding to each processed image, wherein the image description includes a short category name and a detailed function description; A labeling module, used to finely label the UHV circuit breaker instances calibrated in each processed image; A sample pair creation module, configured to create corresponding positive and negative sample pairs according to the image description, wherein the positive sample includes a switch instance that matches the image description, and the negative sample includes a switch instance that does not match the image description; An expansion module, used for supplementing the annotation and scene expansion of the positive and negative sample pairs according to a multi-target scene, so as to obtain annotated and expanded positive and negative sample pairs; A construction module is used to construct an ultra-high voltage circuit breaker data set according to the image description and the annotated and expanded positive and negative sample pairs.

9. A computer device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, wherein the instructions are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the method according to any one of claims 1 to 7.

Images