Method and system for constructing power visual model trust evaluation dataset

Abstract

This invention provides a method and system for constructing a reliable evaluation dataset for power vision models, belonging to the field of intelligent recognition technology for power vision models. The construction method includes: acquiring collected on-site images and obstacle images; constructing virtual 3D models of power equipment and virtual obstacle models; marking the virtual 3D models and virtual obstacle models to obtain corresponding labels; acquiring multiple images from a virtual perspective on the composite model of the virtual 3D models and virtual obstacle models; determining the positional relationship between the virtual 3D models and virtual obstacles in the images based on the images; and generating a composite image and corresponding composite labels based on the positional relationship, the on-site images, and the obstacle images to form a dataset.

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology for power vision models, and more specifically to a method and system for constructing a reliable evaluation dataset for power vision models. Background Technology

[0002] Artificial intelligence (AI) credibility metrics have become a hot topic in AI research in recent years. The International Organization for Standardization (ISO) and the International Electrotechnical Commission (ISO / IEC) were among the earliest to develop research on AI credibility metrics, covering topics such as AI system bias, risk management, AI system quality models, and neural network robustness.

[0003] Commonly used datasets in existing technologies include the Pascal-VOC dataset, the MS COCO dataset, and the ImageNet dataset. These datasets offer high performance and relatively broad applicability. However, in power systems, due to the uniform color and similar shape of components, directly using these datasets to train the network results in low recognition rates, necessitating the acquisition of new images from the power system field. However, the complex environment of power system fields makes direct image acquisition difficult. Furthermore, the acquired images require extensive manual calibration, significantly hindering the efficiency of this step. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for constructing a reliable evaluation dataset for power visual models, which can improve the efficiency of dataset construction.

[0005] To achieve the above objectives, embodiments of the present invention provide a method for constructing a reliable evaluation dataset for power visual models, comprising:

[0006] Acquire images of the scene and obstacles;

[0007] Construct virtual 3D models of power equipment and virtual obstacle models;

[0008] Mark the virtual 3D model and the virtual obstacle model;

[0009] Multiple images were captured from a virtual perspective on the composite model of the virtual 3D model and the virtual obstacle model.

[0010] Determine the positional relationship between the virtual 3D model and the virtual obstacle in the image based on the image;

[0011] Based on the location association, on-site images, and obstacle images, a composite image and a composite label corresponding to the composite image are generated to form a dataset.

[0012] Optionally, markings are made on the virtual 3D model and the virtual obstacle model, including:

[0013] The various positions of the virtual 3D model are represented by a combination of black and white blocks;

[0014] The positions of the virtual obstacles are represented by combinations of blue-green blocks.

[0015] Optionally, determining the positional relationship between the virtual 3D model and the virtual obstacle in the image based on the image includes:

[0016] The image is divided into multiple pixel grids;

[0017] Traverse each type of obstacle. For the same type of obstacle, in the image, the cell containing the obstacle is represented by the number 1, and the cell without the obstacle is represented by the number 0, so as to obtain the fuzzy matrix corresponding to the image.

[0018] The similarity between each pair of fuzzy matrices is calculated to obtain the corresponding set of clustering matrices;

[0019] Determine the center matrix of the clustering matrix set;

[0020] The central matrix is ​​used as a representation of the positional relationship of obstacles in the image.

[0021] Optionally, generating a composite image based on the location correlation, the scene image, and the obstacle image includes:

[0022] Based on each obtained center matrix, the corresponding obstacle image is sequentially placed into the scene image to obtain the composite image.

[0023] On the other hand, the present invention also provides a system for constructing a reliable evaluation dataset for power visual models, the system comprising a processor configured to:

[0024] Acquire images of the scene and obstacles;

[0025] Construct virtual 3D models of power equipment and virtual obstacle models;

[0026] Mark the virtual 3D model and the virtual obstacle model to obtain corresponding labels;

[0027] Multiple images were captured from a virtual perspective on the composite model of the virtual 3D model and the virtual obstacle model.

[0028] Determine the positional relationship between the virtual 3D model and the virtual obstacle in the image based on the image;

[0029] Based on the location association, on-site images, and obstacle images, a composite image and a composite label corresponding to the composite image are generated to form a dataset.

[0030] Optionally, the processor is used to:

[0031] The various positions of the virtual 3D model are represented by a combination of black and white blocks;

[0032] The positions of the virtual obstacles are represented by combinations of blue-green blocks.

[0033] Optionally, the processor is used to:

[0034] The image is divided into multiple pixel grids;

[0035] Traverse each type of obstacle. For the same type of obstacle, in the image, the cell containing the obstacle is represented by the number 1, and the cell without the obstacle is represented by the number 0, so as to obtain the fuzzy matrix corresponding to the image.

[0036] The similarity between each pair of fuzzy matrices is calculated to obtain the corresponding set of clustering matrices;

[0037] Determine the center matrix of the clustering matrix set;

[0038] The central matrix is ​​used as a representation of the positional relationship of obstacles in the image.

[0039] Optionally, the processor is used to:

[0040] Based on each obtained center matrix, the corresponding obstacle image is sequentially placed into the scene image to obtain the composite image.

[0041] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the construction methods described above.

[0042] Through the above technical solution, the power vision model credibility assessment dataset construction method and system provided by the present invention determines the expected position of the obstacle in the field image by setting and constructing a virtual three-dimensional model and a virtual obstacle model, and then generates a composite image for the expected position. With only a few field images to be collected, a complete dataset can be generated, and a large number of manual calibration steps are omitted, thereby improving the efficiency of dataset construction.

[0043] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a flowchart of a method for constructing a reliable evaluation dataset for a power visual model according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart of a method for constructing a reliable evaluation dataset for a power visual model according to an embodiment of the present invention. Detailed Implementation

[0047] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0048] like Figure 1 The diagram shows a flowchart of a method for constructing a reliable evaluation dataset for a power visual model according to an embodiment of the present invention. Figure 1 In this context, the construction method may include the following steps:

[0049] In step S10, the collected scene images and obstacle images are acquired;

[0050] In step S11, a virtual 3D model of the power equipment and a virtual obstacle model are constructed;

[0051] In step S12, marks are made on the virtual 3D model and the virtual obstacle model to obtain corresponding labels;

[0052] In step S13, multiple images are captured on the composite model of the virtual 3D model and the virtual obstacle model from a virtual perspective;

[0053] In step S14, the positional relationship between the virtual 3D model and the virtual obstacle in the image is determined based on the image;

[0054] In step S15, a composite image and a composite label corresponding to the composite image are generated based on the location association, the scene image, and the obstacle image to form a dataset.

[0055] In such Figure 1In the construction method shown, step S10 is used to acquire the collected scene images and obstacle images. The scene images can be acquired in conjunction with a desired acquisition viewpoint. This desired acquisition viewpoint can be determined by the movement path of the drone or inspection robot.

[0056] Step S11 can be used to construct a virtual 3D model of the power equipment and a virtual obstacle model. Both the virtual 3D model and the virtual obstacle model can be constructed by scaling down the actual power equipment.

[0057] Step S12 can be used to mark the virtual 3D model and virtual obstacles. In this embodiment, to improve the efficiency of subsequent machine recognition, different colors can be used for marking. Considering that machine recognition generally uses a binary recognition method, different electrical equipment or obstacles can be represented by relative combinations of multiple color blocks based on the color differences. Specifically, for electrical equipment, for example, the various positions of the virtual 3D model can be represented by combinations of black and white blocks. For obstacles, for example, the various positions of the virtual obstacles can be represented by combinations of blue and green blocks.

[0058] Step S13 can be used to acquire multiple images from a composite model of a virtual 3D model and a virtual obstacle model using a virtual perspective. This virtual perspective can be combined with a predetermined acquisition perspective, which can be determined by the movement path of a drone or inspection robot.

[0059] Step S14 can be used to determine the positional relationship between the virtual 3D model and virtual obstacles in the image based on the image. This positional relationship can represent the position and type of the obstacle in any image. In this embodiment, considering that the type has already been labeled when constructing the virtual 3D model and virtual obstacles, and that obstacles are generally a continuous region in the image, this saves the step of extracting non-connected regions in conventional image recognition algorithms. Therefore, in step S14, the following can be used: Figure 2 The method shown in the diagram. Figure 2 In this context, step S14 may further include the following steps:

[0060] In step S20, the image is divided into multiple pixel grids;

[0061] In step S21, each type of obstacle is traversed. For the same type of obstacle, the cell containing the obstacle is represented as the number 1 and the cell without the obstacle is represented as the number 0 in the image, so as to obtain the fuzzy matrix corresponding to the image.

[0062] In step S22, the similarity between each pair of fuzzy matrices is calculated to obtain the corresponding set of clustering matrices;

[0063] In step S23, the center matrix of the cluster matrix set is determined;

[0064] In step S24, the center matrix is ​​used as a representation of the positional relationship of obstacles in the image.

[0065] In such Figure 2 In the method shown, step S20 is used to divide the image into multiple pixel grids, thereby reducing the amount of computation required by the machine.

[0066] Step S21 can be used to traverse each type of obstacle. For the same type of obstacle, the cell containing the obstacle is represented by the number 1, and the cell without obstacles is represented by the number 0, to obtain the corresponding blur matrix of the image. It can be represented, for example, by the following formula (1).

[0067] , (1).

[0068] Since the virtual obstacles in images captured from different perspectives are not entirely in the same position, directly treating all obstacle positions as positional relationships would result in a large amount of redundant data when generating subsequent images. Therefore, in this implementation, steps S22 to S23 are used to first aggregate the various fuzzy matrices based on similarity, and then use the center matrix of each aggregated set to represent the obstacle position, thereby saving subsequent computation and avoiding the generation of excessive redundant data.

[0069] Step S15 can be used to generate a composite image and a corresponding composite label based on the location association, the scene image, and the obstacle image, to form a dataset. In this embodiment, given the center matrix determined in step S24, the corresponding obstacle image can be sequentially placed into the scene image according to each obtained center matrix to obtain the composite image.

[0070] On the other hand, the present invention also provides a system for constructing a reliable evaluation dataset for power visual models. The system includes a processor that can be configured to perform tasks such as... Figure 1 and Figure 2 The construction method shown is described in this example. Figure 1 In this context, the processor can be used for:

[0071] In step S10, the collected scene images and obstacle images are acquired;

[0072] In step S11, a virtual 3D model of the power equipment and a virtual obstacle model are constructed;

[0073] In step S12, marks are made on the virtual 3D model and the virtual obstacle model;

[0074] In step S13, multiple images are captured on the composite model of the virtual 3D model and the virtual obstacle model from a virtual perspective;

[0075] In step S14, the positional relationship between the virtual 3D model and the virtual obstacle in the image is determined based on the image;

[0076] In step S15, a composite image and a composite label corresponding to the composite image are generated based on the location association, the scene image, and the obstacle image to form a dataset.

[0077] In such Figure 1 In the construction method shown, step S10 is used to acquire the collected scene images and obstacle images. The scene images can be acquired in conjunction with a desired acquisition viewpoint. This desired acquisition viewpoint can be determined by the movement path of the drone or inspection robot.

[0078] Step S11 can be used to construct a virtual 3D model of the power equipment and a virtual obstacle model. Both the virtual 3D model and the virtual obstacle model can be constructed by scaling down the actual power equipment.

[0079] Step S12 can be used to mark the virtual 3D model and virtual obstacles. In this embodiment, to improve the efficiency of subsequent machine recognition, different colors can be used for marking. Considering that machine recognition generally uses a binary recognition method, different electrical equipment or obstacles can be represented by relative combinations of multiple color blocks based on the color differences. Specifically, for electrical equipment, for example, the various positions of the virtual 3D model can be represented by combinations of black and white blocks. For obstacles, for example, the various positions of the virtual obstacles can be represented by combinations of blue and green blocks.

[0080] Step S13 can be used to acquire multiple images from a composite model of a virtual 3D model and a virtual obstacle model using a virtual perspective. This virtual perspective can be combined with a predetermined acquisition perspective, which can be determined by the movement path of a drone or inspection robot.

[0081] Step S14 can be used to determine the positional relationship between the virtual 3D model and virtual obstacles in the image based on the image. This positional relationship can represent the position and type of the obstacle in any image. In this embodiment, considering that the type has already been labeled when constructing the virtual 3D model and virtual obstacles, and that obstacles are generally a continuous region in the image, this saves the step of extracting non-connected regions in conventional image recognition algorithms. Therefore, in step S14, the following can be used: Figure 2 The method shown in the diagram. Figure 2 In this context, step S14 may further include the following steps:

[0082] In step S20, the image is divided into multiple pixel grids;

[0083] In step S21, each type of obstacle is traversed. For the same type of obstacle, the cell containing the obstacle is represented as the number 1 and the cell without the obstacle is represented as the number 0 in the image, so as to obtain the fuzzy matrix corresponding to the image.

[0084] In step S22, the similarity between each pair of fuzzy matrices is calculated to obtain the corresponding set of clustering matrices;

[0085] In step S23, the center matrix of the cluster matrix set is determined;

[0086] In step S24, the center matrix is ​​used as a representation of the positional relationship of obstacles in the image.

[0087] In such Figure 2 In the method shown, step S20 is used to divide the image into multiple pixel grids, thereby reducing the amount of computation required by the machine.

[0088] Step S21 can be used to traverse each type of obstacle. For the same type of obstacle, the cell containing the obstacle is represented by the number 1, and the cell without obstacles is represented by the number 0, to obtain the corresponding blur matrix of the image. It can be represented, for example, by the following formula (1).

[0089] , (1).

[0090] Since the virtual obstacles in images captured from different perspectives are not entirely in the same position, directly treating all obstacle positions as positional relationships would result in a large amount of redundant data when generating subsequent images. Therefore, in this implementation, steps S22 to S23 are used to first aggregate the various fuzzy matrices based on similarity, and then use the center matrix of each aggregated set to represent the obstacle position, thereby saving subsequent computation and avoiding the generation of excessive redundant data.

[0091] Step S15 can be used to generate a composite image and a corresponding composite label based on the location association, the scene image, and the obstacle image, to form a dataset. In this embodiment, given the center matrix determined in step S24, the corresponding obstacle image can be sequentially placed into the scene image according to each obtained center matrix to obtain the composite image.

[0092] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the construction methods described above.

[0093] Through the above technical solution, the power vision model credibility assessment dataset construction method and system provided by the present invention determines the expected position of the obstacle in the field image by setting and constructing a virtual three-dimensional model and a virtual obstacle model, and then generates a composite image for the expected position. With only a few field images to be collected, a complete dataset can be generated, and a large number of manual calibration steps are omitted, thereby improving the efficiency of dataset construction.

[0094] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0096] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0097] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0098] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0099] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0100] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0101] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0102] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for constructing a reliable evaluation dataset for power visual models, characterized in that, The construction method includes: Acquire images of the scene and obstacles; Construct virtual 3D models of power equipment and virtual obstacle models; Mark the virtual 3D model and the virtual obstacle model to obtain corresponding labels; Multiple images were captured from a virtual perspective on the composite model of the virtual 3D model and the virtual obstacle model. Determine the positional relationship between the virtual 3D model and the virtual obstacle in the image based on the image; Based on the location association, on-site images, and obstacle images, a composite image and a composite label corresponding to the composite image are generated to form a dataset; Determining the positional relationship between the virtual 3D model and the virtual obstacle in the image based on the image includes: The image is divided into multiple pixel grids; Traverse each type of obstacle. For the same type of obstacle, in the image, the cell containing the obstacle is represented by the number 1, and the cell without the obstacle is represented by the number 0, so as to obtain the fuzzy matrix corresponding to the image. The similarity between each pair of fuzzy matrices is calculated to obtain the corresponding set of clustering matrices; Determine the center matrix of the clustering matrix set; The central matrix is ​​used as a representation of the positional relationship of obstacles in the image.

2. The construction method according to claim 1, characterized in that, Marking on the virtual 3D model and the virtual obstacle model includes: The various positions of the virtual 3D model are represented by a combination of black and white blocks; The positions of the virtual obstacles are represented by combinations of blue-green blocks.

3. The construction method according to claim 1, characterized in that, Generating a composite image based on the location correlation, on-site images, and obstacle images includes: Based on each obtained center matrix, the corresponding obstacle image is sequentially placed into the scene image to obtain the composite image.

4. A system for constructing a reliable evaluation dataset for power visual models, characterized in that, The construction system includes a processor, which is configured to: Acquire images of the scene and obstacles; Construct virtual 3D models of power equipment and virtual obstacle models; Mark the virtual 3D model and the virtual obstacle model; Multiple images were captured from a virtual perspective on the composite model of the virtual 3D model and the virtual obstacle model. Determine the positional relationship between the virtual 3D model and the virtual obstacle in the image based on the image; Based on the location association, on-site images, and obstacle images, a composite image and a composite label corresponding to the composite image are generated to form a dataset; The processor is used for: The image is divided into multiple pixel grids; Traverse each type of obstacle. For the same type of obstacle, in the image, the cell containing the obstacle is represented by the number 1, and the cell without the obstacle is represented by the number 0, so as to obtain the fuzzy matrix corresponding to the image. The similarity between each pair of fuzzy matrices is calculated to obtain the corresponding set of clustering matrices; Determine the center matrix of the clustering matrix set; The central matrix is ​​used as a representation of the positional relationship of obstacles in the image.

5. The construction system according to claim 4, characterized in that, The processor is used for: The various positions of the virtual 3D model are represented by a combination of black and white blocks; The positions of the virtual obstacles are represented by combinations of blue-green blocks.

6. The construction system according to claim 4, characterized in that, The processor is used for: Based on each obtained center matrix, the corresponding obstacle image is sequentially placed into the scene image to obtain the composite image.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that are read by a machine to cause the machine to perform the construction method as described in any one of claims 1 to 3.

Images