Power vision dataset augmentation method and system based on physical system characteristics

Abstract

A power vision dataset augmentation method and system based on physical system characteristics. The method comprises: acquiring a three-dimensional structure point cloud of a power device, and determining point clouds of vertices and edge lines of the power device on the basis of the three-dimensional structure point cloud; partitioning the point clouds of non-salient vertices and edge lines in the three-dimensional structure point cloud, and replacing the partitions with geometric planar primitives to obtain a three-dimensional structure represented on the basis of the geometric planar primitives; acquiring a real-shot image of the power device, and determining lighting and shadow information of the power device on the basis of the real-shot image; matching a three-dimensional structure with the real-shot image by means of transforming the angle and scale of the three-dimensional structure; after the three-dimensional structure is matched with the real-shot image, determining a three-dimensional true-color structure of the power device on the basis of the lighting and shadow information of the power device; and, according to the three-dimensional true-color structure and the real-shot image, constructing a dataset.

Description

Method and system for augmenting electric power visual data sets based on physical system characteristics

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 5, 2023, with application number 202311655113.0. The entire contents of the above application are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of data set augmentation technology, for example, to a method and system for augmenting an electric power vision data set based on physical system characteristics. Background Art

[0003] The current digital and intelligent transformation of power grids has driven the development and application of a large number of power visual recognition models. These models recognize visual data such as visible light and infrared. These models identify specific equipment targets or components, equipment defects and faults, and environmental anomalies and risks in visual data for typical scenarios such as transmission, substation, distribution, and safety management. Performance testing of general-purpose recognition models is typically conducted and validated based on large-scale public datasets (such as ImageNet). The model's testability and performance are inferred and evaluated based on the recognition results. Recognition models in specialized fields are also rapidly advancing. However, unlike general-purpose fields, public datasets are scarce, often difficult to obtain, or lack flexibility and standards. This creates the need for standardization of validation datasets before model validation. Model testing and validation often face challenges with the independence and validity of test datasets, resulting in a lack of objective reflection on model robustness. Visual data generation methods based on mainstream generative models often fail to conform to physical system characteristics such as lighting and materials in real-world applications. However, the visual data generation method based on physics engines requires high computing power. Moreover, because the parameter settings of physics engines involve dynamic processes, they require continuous debugging and training. Their use on site or at the terminal is limited, and they have high requirements on the technical level of front-line personnel. Summary of the Invention

[0004] To this end, the technical problem to be solved by this application is to overcome the problem that the professional fields in related technologies lack corresponding data sets as support, resulting in the subsequent model robustness test results being not objective enough.

[0005] To solve the above technical problems, the present application provides a method for augmenting an electric power visual data set based on physical system characteristics, including: obtaining an initial three-dimensional structural point cloud of the electric power equipment and preprocessing it to obtain a three-dimensional structural point cloud, and determining the point cloud of the vertices and edges of the electric power equipment based on the three-dimensional structural point cloud; partitioning the point cloud of non-significant vertices and edges in the three-dimensional structure point cloud, replacing the partitions with geometric plane primitives, and obtaining a three-dimensional structure represented by geometric plane primitives, wherein the point cloud of non-significant vertices and edges is a point cloud with a feature degree lower than a preset percentage; obtaining a real-shot image of the electric power equipment, and determining the light and shadow information of the electric power equipment based on the real-shot image; matching the three-dimensional structure with the real-shot image by changing the angle and scale of the three-dimensional structure; when the three-dimensional structure matches the real-shot image, determining the three-dimensional true-color structure of the electric power equipment based on the light and shadow information of the electric power equipment; and constructing a data set based on the three-dimensional true-color structure and the real-shot image.

[0006] In one embodiment of the present application, the initial three-dimensional structural point cloud of the power equipment is obtained and preprocessed to obtain a three-dimensional structural point cloud, and the point cloud of the vertices and edges of the power equipment is determined based on the three-dimensional structural point cloud, including: obtaining the initial three-dimensional structural point cloud of the power equipment, performing point cloud registration and point cloud filtering on the initial three-dimensional structural point cloud to obtain a three-dimensional structural point cloud, and at the same time performing voxel gradient solution on the three-dimensional structural point cloud based on point cloud voxels, calculating the maximum value of the gradient change rate, and obtaining the coordinates of the vertices and edges of the power equipment in the three-dimensional structural point cloud; for the edge coordinates of the power equipment in the three-dimensional structural point cloud, the point cloud at the location of the edge coordinates is segmented based on a clustering algorithm to obtain several community subsets, detecting the curvature changes and normal direction changes of the local spatial point sets in the several community subsets, and combining the RANSAC algorithm to obtain the edge representation of the three-dimensional structure point cloud.

[0007] In one embodiment of the present application, the point cloud of non-significant vertices and edges in the three-dimensional structure point cloud is partitioned, and the partitions are replaced by geometric plane primitives to obtain a three-dimensional structure represented by geometric plane primitives, including: partitioning the area of ​​the non-significant vertex and edge point cloud in the three-dimensional structure point cloud by a clustering algorithm, wherein the equivalent diameter of the partition is less than 0.5 times the minimum length of the point cloud detection line, and replacing the partitions obtained by clustering with triangular plane primitives to obtain a three-dimensional structure represented by triangular plane primitives.

[0008] In one embodiment of the present application, the method of obtaining a real-shot image of the power equipment and determining the light and shadow information of the power equipment based on the real-shot image includes: obtaining a real-shot image of the power equipment, and separating the lighting information of the power equipment in the real-shot image based on a surface reflection method: setting different areas of the power equipment to mirror reflection or diffuse reflection according to material properties, wherein the mirror reflection is generated by the smooth resin material of the power equipment, and the diffuse reflection is generated by the paint or metal surface of the power equipment; obtaining the lighting information of the power equipment in the corresponding area according to the lighting model of the mirror reflection or diffuse reflection, and obtaining a lighting-material key-value pair; performing regional segmentation on the color information in the real-shot image based on the K-means algorithm, pairing and recording the material and color information of the power equipment to form a material-color key-value pair, and setting the same material but different color areas to a shadow-color key-value pair; obtaining the texture information of the real-shot image through Gabor filter detection, and constructing a texture-material key-value pair by comparing it with the material distribution of the power equipment; the lighting-material key-value pair, material-color key-value pair, shadow-color key-value pair and texture-material key-value pair constitute the light and shadow information of the power equipment.

[0009] In one embodiment of the present application, the three-dimensional structure is matched with the real-shot image by transforming the angle and scale of the three-dimensional structure, including: calculating the degree of matching between the corner points and edge features in the real-shot image and the three-dimensional structure by transforming the angle and scale of the three-dimensional structure represented by geometric plane primitives, and obtaining the angle and scale corresponding to the three-dimensional structure when the error is minimized by adjusting the matching error. At this time, the real-shot image is regarded as the projection of the three-dimensional structure at this angle and scale, and the lighting, color, and texture information of the real-shot image is used as the two-dimensional image information after the orthogonal projection of the three-dimensional structure at this angle and scale.

[0010] In one embodiment of the present application, after the three-dimensional structure matches the real-shot image, the three-dimensional true-color structure of the power equipment is determined according to the light and shadow information of the power equipment, including: after the three-dimensional structure represented by the geometric plane primitives matches the real-shot image, the inverse orthogonal projection is used, and the projection mapping of the shadow under the depth information is considered to obtain the lighting, color, and texture information of the three-dimensional structure at the angle and scale; based on the extracted lighting-material key-value pairs, material-color key-value pairs, shadow-color key-value pairs, and texture-material key-value pairs, under the constraints of geometric plane primitives, the lighting, color, and texture information of the remaining parts of the three-dimensional structure are fitted and completed to obtain a three-dimensional true-color structure with lighting, color, and texture information.

[0011] In one embodiment of the present application, the data set is constructed based on the three-dimensional true-color structure and the real-shot images, including: performing orthogonal projection on the three-dimensional true-color structure at different angles and scales to obtain two-dimensional image data of arbitrary direction and size; obtaining an image set A of the power equipment under partial occlusion based on angle transformation; adjusting the illumination to obtain an image set B under strong light and weak light conditions; performing a preset amplitude angle transformation on the designated components or defect locations of the power equipment to obtain an image set C of targets with different perspectives; performing scale enlargement and reduction on the designated targets of the power equipment to obtain a local image set D and an image set E with a small proportion of target pixels; and performing combined transformations or other transformations according to other specific needs to obtain an image set Pi; the real-shot images, image set A, image set B, image set C, image set D, image set E and image set Pi are combined into a data set.

[0012] To solve the above technical problems, the present application provides an electric power visual dataset augmentation system based on physical system characteristics, comprising: an acquisition and determination module: configured to acquire an initial three-dimensional structure point cloud of an electric power device and preprocess it to obtain a three-dimensional structure point cloud, and determine the point clouds of vertices and edges of the electric power device based on the three-dimensional structure point cloud; a three-dimensional structure construction module: configured to partition the point clouds of non-significant vertices and edges in the three-dimensional structure point cloud, and replace the partitions with geometric plane primitives to obtain a three-dimensional structure represented by geometric plane primitives, wherein the point clouds of non-significant vertices and edges are point clouds with a feature degree lower than a preset percentage; an acquisition and matching module: configured to acquire a real-shot image of the electric power device and determine the light and shadow information of the electric power device based on the real-shot image; and further configured to match the three-dimensional structure with the real-shot image by transforming the angle and scale of the three-dimensional structure; a three-dimensional true-color structure construction module: configured to determine the three-dimensional true-color structure of the electric power device based on the light and shadow information of the electric power device after the three-dimensional structure matches the real-shot image; and a dataset construction module: configured to construct a dataset based on the three-dimensional true-color structure and the real-shot image.

[0013] In one embodiment of the present application, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for augmenting a power visual dataset based on physical system characteristics as described above are implemented.

[0014] In one embodiment of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method for augmenting a power visual data set based on physical system characteristics are implemented.

[0015] The above-mentioned technical solution of the present application has the following advantages compared with the related technology: the present application constructs an image dataset based on the three-dimensional structural point cloud and real-shot images of the power equipment, and the image dataset can be used for the robustness test of the power visual recognition model; the present application realizes the flexible expansion of the image dataset of specific power scenes, and can provide effective data support for the robustness test of the power visual recognition model in the professional field (power field). BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to make the contents of this application easier to understand, the application is further described in detail below based on the specific embodiments of this application and in conjunction with the accompanying drawings.

[0017] FIG1 is a flow chart of the method of the present application. DETAILED DESCRIPTION

[0018] The present application will be further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present application and implement it, but the embodiments are not intended to limit the present application.

[0019] Example 1

[0020] 1 , the present application relates to a method for augmenting a power visual dataset based on physical system characteristics, which can be used in a power visual recognition model. The method includes:

[0021] Step S1: obtaining an initial three-dimensional structural point cloud of the power equipment and performing preprocessing to obtain a three-dimensional structural point cloud, and determining point clouds of vertices and edges of the power equipment based on the three-dimensional structural point cloud;

[0022] Step S2: Partitioning the point cloud of non-significant vertices and edges in the three-dimensional structure point cloud, and replacing the partitions with geometric plane primitives to obtain a three-dimensional structure represented by geometric plane primitives, wherein the point cloud of non-significant vertices and edges is a point cloud with a feature degree lower than a preset percentage;

[0023] Step S3: Acquire a real-shot image of the power equipment, and determine light and shadow information of the power equipment based on the real-shot image;

[0024] Matching the three-dimensional structure with the real-shot image by changing the angle and scale of the three-dimensional structure;

[0025] Step S4: After the three-dimensional structure matches the real-shot image, the three-dimensional true-color structure of the power equipment is determined according to the light and shadow information of the power equipment;

[0026] Step S5: constructing a data set based on the three-dimensional true color structure and the real-shot image.

[0027] The dataset augmentation method of this embodiment can achieve flexible augmentation of a specific power scene image dataset.

[0028] The following is a detailed introduction to this embodiment:

[0029] The method of this embodiment mainly includes six parts: (1) acquisition of three-dimensional structure point cloud, (2) primitiveization of three-dimensional structure point cloud, (3) extraction of light and shadow information of real-shot images, (4) transformation of three-dimensional structure at different angles and scales, (5) construction of three-dimensional true color structure through light and shadow information mapping, and (6) acquisition of data set.

[0030] (1) Acquisition of 3D structure point cloud

[0031] The acquisition of the three-dimensional structure of the power equipment is based on laser radar or structured light to obtain the initial three-dimensional structure point cloud of the power equipment, which specifically includes static information such as the coordinates and external shape of the power equipment. The initial three-dimensional structure point cloud is processed by point cloud registration and point cloud filtering to obtain a three-dimensional structure point cloud with high precision. At the same time, the voxel gradient is solved based on the point cloud voxels, and the maximum value of the gradient change rate is calculated to obtain the coordinate attributes of the vertices and edges (i.e., boundary segments) in the three-dimensional structure point cloud. So far, this embodiment believes that the vertex representation of the three-dimensional structure point cloud has been obtained. After the point cloud of the edge coordinate position is segmented based on the clustering algorithm, several community subsets are obtained, and the normal direction, curvature, etc. of the local space point set in the several community subsets are detected. According to the detected curvature change and normal direction change, the edge representation of the three-dimensional structure point cloud is obtained in combination with the point cloud line segment detection algorithm RANSAC.

[0032] (2) Primitiveization of 3D Structure Point Cloud

[0033] For the areas of non-significant vertices and edges in the three-dimensional structure point cloud (point clouds with a feature level lower than a preset percentage obtained based on the clustering algorithm in part (1)), the point cloud is partitioned using a clustering algorithm. The partitions obtained by clustering are replaced by geometric plane primitives (in this embodiment, triangular plane primitives are used). This embodiment also processes the size of the triangular plane primitives to make the triangles as small as possible so that the subsequent three-dimensional structure is sufficiently smooth. The three points of the triangular plane primitive satisfy the requirement that there is no overlap or gap between them in the structure. At this time, a three-dimensional structure represented by the triangular plane primitives is obtained.

[0034] Furthermore, the three-dimensional structure can be transformed at different angles and scales based on rotation and translation. The vertices in the three-dimensional structure are orthogonally projected at a certain angle to obtain the two-dimensional projection coordinates of the vertices at that angle. The triangular plane primitives at the intersection with the normal plane of the projection plane are decomposed into smaller triangles along the intersection line to facilitate projection processing.

[0035] (3) Extraction of light and shadow information from real-shot images

[0036] Based on the real-shot images of power equipment, especially the image data actually obtained when the power equipment has defects, this type of image data is generally not reproducible. This embodiment separates the lighting information of the power equipment in the real-shot images based on the surface reflection method. This embodiment assumes that the materials of the various parts of the power equipment are known, and sets different areas for mirror reflection (smooth resin materials) and diffuse reflection (paint and metal surfaces, etc.) according to the material properties. The lighting information of the equipment in the corresponding area is obtained based on the two reflection illumination models to obtain the lighting-material key-value pairs; this embodiment uses the K-means algorithm to segment the color information in the real-shot images, and pairs and records the equipment material and color information to form material-color key-value pairs, and sets the same material but different color partitions as shadow-color key-value pairs; this embodiment uses the Gabor filter to detect the texture information of the real-shot images, and also constructs the texture-material key-value pairs by comparing the distribution of the equipment materials.

[0037] Light-material key-value pairs, material-color key-value pairs, shadow-color key-value pairs, and texture-material key-value pairs constitute the light and shadow information of power equipment.

[0038] (4) Transformation of three-dimensional structures at different angles and scales

[0039] By transforming the angle and scale of the three-dimensional structure, the degree of matching between the corner points and edge features of the real-shot image equipment and the three-dimensional structure is calculated. By adjusting the matching error, the angle and scale corresponding to the three-dimensional structure are obtained when the error is minimized. At this time, the real-shot image can be regarded as the projection of the three-dimensional structure at this angle and scale, and the lighting, color, texture and other information of the image target (i.e., the power equipment) extracted from the real-shot image is the two-dimensional image information after the orthogonal projection of the three-dimensional structure at this angle and scale.

[0040] (5) Through light and shadow information mapping, a three-dimensional true color structure is constructed

[0041] After the three-dimensional structure is matched with the real-shot image, inverse orthogonal projection is used, and the projection mapping of the shadow under the depth information is considered to obtain the lighting, color, and texture information at the angle and scale of the three-dimensional structure. The lighting-material key-value pairs, material-color key-value pairs, shadow-color key-value pairs, and texture-material key-value pairs extracted above are used to fit and complete the lighting, color, and texture information of the remaining parts of the three-dimensional structure, considering the constraints of the triangle primitives. At this time, a three-dimensional true color structure with lighting, color, and texture information is obtained.

[0042] (6) Dataset acquisition

[0043] By controlling the three-dimensional true color structure at different angles and scales and performing orthogonal projection at different angles and scales, two-dimensional image data of arbitrary direction and size can be obtained.

[0044] Based on the angle transformation, we can obtain the image set A under the conditions of occlusion of the target part (i.e., power equipment);

[0045] Adjusting the lighting can obtain image set B under strong light, weak light and other conditions;

[0046] Perform a small angle transformation on the location of a specific component or defect to obtain an image set C of the target at different viewing angles;

[0047] Zoom in and out on a specific target to obtain a local image set D and an image set E with a small percentage of target pixels;

[0048] and other combined transformations or other transformations according to specific requirements to obtain the image set Pi;

[0049] The real-world images, Image Sets A, B, C, D, E, and Pi are combined to create an augmented dataset suitable for robustness testing of power vision recognition models. This robustness testing stems from data augmentation through flexible transformations of three-dimensional structures. The image sets are batch-classified according to test requirements, and this batch data is used to validate and test power vision algorithm models.

[0050] At the same time, the augmented dataset can also be used to improve the capabilities of power vision algorithm models, and by improving the recognition capabilities, feedback can be provided on the effective collection of real-shot image data.

[0051] Example 2

[0052] This embodiment provides a system for augmenting a power visual dataset based on physical system characteristics, including:

[0053] An acquisition and determination module is configured to acquire an initial three-dimensional structural point cloud of the power equipment and perform preprocessing to obtain a three-dimensional structural point cloud, and determine point clouds of vertices and edges of the power equipment based on the three-dimensional structural point cloud;

[0054] a three-dimensional structure construction module configured to partition a point cloud of non-significant vertices and edges in the three-dimensional structure point cloud, and replace the partitions with geometric plane primitives to obtain a three-dimensional structure represented by the geometric plane primitives, wherein the point cloud of the non-significant vertices and edges is a point cloud having a feature degree lower than a preset percentage;

[0055] An acquisition and matching module is configured to acquire a real-shot image of the power equipment and determine light and shadow information of the power equipment based on the real-shot image;

[0056] It is also configured to match the three-dimensional structure with the real-shot image by changing the angle and scale of the three-dimensional structure;

[0057] A three-dimensional true color structure construction module: configured to determine the three-dimensional true color structure of the power equipment according to the light and shadow information of the power equipment after the three-dimensional structure matches the real-shot image;

[0058] Dataset construction module: configured to construct a dataset based on the three-dimensional true color structure and real-shot images.

[0059] Example 3

[0060] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for augmenting a power visual dataset based on physical system characteristics described in Example 1 are implemented.

[0061] Example 4

[0062] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for augmenting a power visual dataset based on physical system characteristics described in the first embodiment are implemented.

[0063] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0064] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0065] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0066] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

Claims

1. A method for augmenting electric power visual data sets based on physical system characteristics, comprising: Acquire an initial three-dimensional structure point cloud of the power equipment and perform preprocessing to obtain a three-dimensional structure point cloud, and determine point clouds of vertices and edges of the power equipment according to the three-dimensional structure point cloud; Partitioning the point clouds of non-significant vertices and edges in the three-dimensional structure point cloud, and replacing the partitions with geometric plane primitives to obtain a three-dimensional structure represented by geometric plane primitives, wherein the point clouds of non-significant vertices and edges are point clouds with a feature degree lower than a preset percentage; Acquire a real-shot image of the electric power equipment, and determine light and shadow information of the electric power equipment according to the real-shot image; Matching the three-dimensional structure with the real-shot image by changing the angle and scale of the three-dimensional structure; When the three-dimensional structure matches the real-shot image, determining the three-dimensional true-color structure of the power equipment according to the light and shadow information of the power equipment; A data set is constructed according to the three-dimensional true color structure and the real-shot image.

2. The method for augmenting electric power visual data set based on physical system characteristics according to claim 1, wherein: The step of obtaining an initial three-dimensional structure point cloud of the power equipment and performing preprocessing to obtain a three-dimensional structure point cloud, and determining point clouds of vertices and edges of the power equipment according to the three-dimensional structure point cloud, comprises: Acquire an initial three-dimensional structure point cloud of the power equipment, perform point cloud registration and point cloud filtering on the initial three-dimensional structure point cloud to obtain a three-dimensional structure point cloud, and simultaneously perform voxel gradient solution on the three-dimensional structure point cloud based on point cloud voxels, calculate the maximum value of the gradient change rate, and obtain the coordinates of the vertices and edges of the power equipment in the three-dimensional structure point cloud; For the edge coordinates of the power equipment in the three-dimensional structure point cloud, the point cloud where the edge coordinates are located is segmented based on the clustering algorithm to obtain several community subsets. The curvature changes and normal direction changes of the local space point sets in several community subsets are detected, and the edge representation of the three-dimensional structure point cloud is obtained by combining the RANSAC algorithm.

3. The method for augmenting electric power visual data set based on physical system characteristics according to claim 1, wherein: Partitioning the point cloud of non-significant vertices and edges in the three-dimensional structure point cloud, and replacing the partitions with geometric plane primitives to obtain a three-dimensional structure represented by geometric plane primitives, includes: The areas of non-significant vertices and edge point clouds in the three-dimensional structure point cloud are partitioned by a clustering algorithm. The equivalent diameter of the partition is less than 0.5 times the minimum length of the point cloud detection line. The partitions obtained by clustering are replaced by triangular plane primitives to obtain a three-dimensional structure represented by triangular plane primitives.

4. The method for augmenting electric power visual data set based on physical system characteristics according to claim 1, wherein: The step of acquiring a real-shot image of the electric power equipment and determining light and shadow information of the electric power equipment according to the real-shot image includes: Acquire a real-shot image of the power equipment, and separate the lighting information of the power equipment in the real-shot image based on the surface reflection method: set different areas of the power equipment to specular reflection or diffuse reflection according to material properties, wherein the specular reflection is generated by the smooth resin material of the power equipment, and the diffuse reflection is generated by the paint or metal surface of the power equipment; acquire the lighting information of the power equipment in the corresponding area according to the lighting model of the specular reflection or diffuse reflection, and obtain the lighting-material key-value pair; Based on the K-means algorithm, the color information in the real-shot image is segmented, the material and color information of the power equipment are paired and recorded to form material-color key-value pairs, and the same material but different color areas are set as shadow-color key-value pairs; The texture information of the real-shot image is obtained through Gabor filter detection, and the texture-material key-value pairs are constructed by comparing the material distribution of the power equipment; The illumination-material key-value pairs, material-color key-value pairs, shadow-color key-value pairs and texture-material key-value pairs constitute the light and shadow information of the electric power equipment.

5. The method for augmenting electric power visual data set based on physical system characteristics according to claim 4, wherein: The matching of the three-dimensional structure with the real-shot image by changing the angle and scale of the three-dimensional structure includes: By transforming the angle and scale of the three-dimensional structure, the degree of matching between the corner points and edge line features in the real image and the three-dimensional structure is calculated, and by adjusting the matching error, the angle and scale corresponding to the three-dimensional structure are obtained when the error is minimized. At this time, the real image is regarded as the projection of the three-dimensional structure at this angle and scale, and the lighting, color, and texture information of the real image is used as the two-dimensional image information after the orthogonal projection of the three-dimensional structure at this angle and scale.

6. The method for augmenting electric power visual data set based on physical system characteristics according to claim 5, wherein: After the three-dimensional structure matches the real-shot image, determining the three-dimensional true-color structure of the power equipment according to the light and shadow information of the power equipment includes: After the three-dimensional structure is matched with the real-shot image, the illumination, color, and texture information of the three-dimensional structure at the angle and scale are obtained by using inverse orthogonal projection and taking into account the projection mapping of the shadow under the depth information; Based on the extracted lighting-material key-value pairs, material-color key-value pairs, shadow-color key-value pairs, and texture-material key-value pairs, under the constraints of geometric plane primitives, the lighting, color, and texture information of the rest of the three-dimensional structure are fitted and completed to obtain a three-dimensional true color structure with lighting, color, and texture information.

7. The method for augmenting electric power visual data set based on physical system characteristics according to any one of claims 1 to 6, wherein: Constructing a data set according to the three-dimensional true color structure and the real-shot image, including: Orthogonal projection is performed on the three-dimensional true color structure at different angles and scales to obtain two-dimensional image data of any direction and size; Based on the angle transformation, the image set A of the power equipment under partial occlusion is obtained; Adjust the illumination to obtain image set B under strong light and weak light conditions; Perform preset amplitude and angle transformation on the designated parts or defect locations of the power equipment to obtain an image set C of targets with different viewing angles; The designated targets of the power equipment are enlarged and reduced in scale to obtain the local image set D and the target pixel small proportion image set E; The real-shot images, image set A, image set B, image set C, image set D, and image set E form a data set.

8. A power visual data set augmentation system based on physical system characteristics, comprising: An acquisition and determination module: configured to acquire an initial three-dimensional structural point cloud of the power equipment and perform preprocessing to obtain a three-dimensional structural point cloud, and determine point clouds of vertices and edges of the power equipment according to the three-dimensional structural point cloud; A three-dimensional structure construction module: configured to partition the point clouds of non-significant vertices and edges in the three-dimensional structure point cloud, and replace the partitions with geometric plane primitives to obtain a three-dimensional structure represented by geometric plane primitives, wherein the point clouds of non-significant vertices and edges are point clouds with a feature degree lower than a preset percentage; An acquisition and matching module: configured to acquire a real-shot image of the electric power equipment and determine light and shadow information of the electric power equipment according to the real-shot image; It is also configured to match the three-dimensional structure with the real-shot image by changing the angle and scale of the three-dimensional structure; A three-dimensional true color structure building module: configured to determine the three-dimensional true color structure of the power equipment according to the light and shadow information of the power equipment after the three-dimensional structure matches the real-shot image; A data set construction module: configured to construct a data set according to the three-dimensional true color structure and the real shot image.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for augmenting a power visual data set based on physical system characteristics as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for augmenting a power visual data set based on physical system characteristics as claimed in any one of claims 1 to 7 are implemented.

Images