Local feature generation method applied to power scene virtual data synthesis

Abstract

The invention discloses a local feature generation method applied to power scene virtual data synthesis, and the method comprises the steps: extracting basic information of a target three-dimensional model, and setting the number of needed patches; a search algorithm is adopted, patches are searched from the starting point of the target three-dimensional model to the periphery, searching is stopped until the number of the searched patches meets the requirement, and the searched patches and vertex information are stored; deleting the searched surface patches in the target three-dimensional model, and creating a new three-dimensional model based on the searched surface patches and vertex information; performing texture mapping on the new three-dimensional model to obtain a local feature model; and returning the cut target three-dimensional model and the local feature model together. According to the method, the local defect features required by the power scene are automatically and randomly generated based on the original three-dimensional model, and the annotation file is generated, so that the workload of constructing the three-dimensional model of the power scene is remarkably reduced, and the efficiency of feature construction can be improved while data imbalance can be effectively made up.

Description

Technical Field

[0001] The present application relates to the technical field of substation inspection, and in particular to a local feature generation method for synthesizing virtual data of power scenarios. Background Art

[0002] In the substation inspection scenario, the deep learning algorithm is used to identify and locate power defects. The performance of the algorithm model is related to the data quality. Due to the complexity of power scenarios, the collected data cannot meet the needs of algorithm training. In the face of the problem of scarce samples or even zero samples that cannot be collected in some power scenarios, 3D virtual synthesis technology can be used to supplement and augment the data. Based on advanced rendering, material, ray tracing and other technologies, the virtual data is very realistic and automatically generates data and annotation files. However, there are still the following problems in the power field:

[0003] 1. Virtual data needs to generate corresponding three-dimensional models in advance. Different defect characteristics of different devices need to be designed separately. In order to increase the volume and diversity of data, it is usually necessary to create as many defect models as possible, which brings a huge workload in the early stage.

[0004] 2. Currently, the annotation files used in virtual data synthesis are based on the entire 3D model, and annotation information cannot be automatically generated for defect features in local areas of the model surface. Summary of the invention

[0005] In order to solve the problem that data for some scenes in the power field are difficult to construct, the present application proposes a local feature generation method for virtual data synthesis of power scenes. The present application automatically and randomly generates local target features required for power scenes based on the original three-dimensional model and generates a labeling file. There is no need to separately construct a three-dimensional model for each local feature to generate data, which significantly reduces the workload of constructing three-dimensional models of power scenes, can effectively make up for data imbalance and improve the efficiency of feature construction.

[0006] This application is implemented through the following technical solutions:

[0007] A local feature generation method for power scene virtual data synthesis, the local target feature generation method comprising:

[0008] Determine the target three-dimensional model and model feature scheme;

[0009] Extract the basic information of the target 3D model and set the required number of facets; the basic information includes facets, number and coordinates of vertices, normal vectors and UV coordinates;

[0010] Using a search algorithm, searching for facets from the starting point of the target three-dimensional model in all directions until the number of facets searched reaches the required number of facets, and then stopping the search, and saving the searched facets and vertex information;

[0011] Deleting the searched face pieces in the target three-dimensional model, and creating a new three-dimensional model based on the searched face pieces and vertex information;

[0012] Performing texture mapping on the new three-dimensional model based on the model feature scheme to obtain a local feature model;

[0013] The cropped target three-dimensional model and the local feature model are returned together.

[0014] In some embodiments, the search algorithm is used to search for faces from the starting point in all directions until the number of faces searched reaches the required number of faces, and the searched faces and vertex information are saved, specifically including:

[0015] Create an adjacency list, traverse each face in the target three-dimensional model, add it to the adjacency list, and construct a mapping relationship between a vertex and a set of its adjacent vertices and faces;

[0016] Initialize a candidate vertex queue, and add a random vertex as a starting point to the initialized candidate vertex queue;

[0017] Taking the starting point from the candidate vertex queue and marking it as visited, searching for vertices adjacent to the starting point, adding the searched adjacent vertices to the candidate vertex queue, searching for all facets containing the starting point and its adjacent vertices and marking them as visited;

[0018] When the candidate vertex queue is not empty and the number of visited facets is less than the required number of facets, a vertex is sequentially taken out from the candidate vertex queue and the vertex is marked as visited, and the vertex adjacent to the vertex is searched. If the searched adjacent vertex is not in the candidate vertex queue and has not been visited, it is added to the candidate vertex queue, and all facets containing the vertex and its adjacent vertices are searched and the unmarked facets therein are marked as visited;

[0019] If the candidate vertex queue is empty or the number of visited facets reaches the required number of facets, stop searching, otherwise return to the previous step to continue searching;

[0020] Save the visited vertex and face information.

[0021] In some embodiments, the target three-dimensional model is a three-dimensional model of equipment and / or environment determined based on the mission requirements of the current power scenario;

[0022] The model feature scheme is determined according to the task that needs to be processed corresponding to the three-dimensional model.

[0023] In some implementations, creating a new three-dimensional model based on the searched face pieces and vertex information specifically includes:

[0024] A new three-dimensional model is created in the same coordinate system as the target three-dimensional model in the same directory, and the searched face and vertex information are assigned to the new three-dimensional model.

[0025] In some implementations, the performing texture mapping on the new three-dimensional model based on the model feature scheme to obtain a local feature model specifically includes:

[0026] According to the model feature scheme, the required target feature texture is selected from the texture library, and the target feature texture is applied to the new three-dimensional model. According to the requirements, it is completely covered or merged with the previous texture to obtain a local feature model.

[0027] In a second aspect, the present application proposes a method for synthesizing virtual data of an electric power scene, and the method for synthesizing virtual data includes:

[0028] Construct a virtual scene with random background, random camera angle, random lighting, and random poses and positions of multiple target 3D models under certain conditions;

[0029] For each virtual scene, the local target feature generation method is executed on the target three-dimensional model for which local features need to be generated, and updated in the virtual scene as required;

[0030] According to task requirements, the data generated based on the local feature model and the corresponding annotation information are saved as training data for subsequent model training;

[0031] Repeat the above steps until enough training data is obtained.

[0032] In some implementations, the data generated based on the local feature model and the corresponding annotation information are saved according to the task requirements, specifically including:

[0033] According to the specific task requirements, if it is a target detection task, the minimum circumscribed rectangular annotation box of the local feature model under the camera's perspective is returned; if it is a segmentation type task, the mask image of the local feature model patch area under the camera's perspective is returned.

[0034] In a third aspect, the present application proposes a local feature generation system for power scene virtual data synthesis, the local feature generation system comprising:

[0035] A determination module, the determination module is used to determine a target three-dimensional model and a model feature scheme;

[0036] An extraction module, the extraction module is used to extract basic information of the target three-dimensional model and set the required number of facets; the basic information includes facets, number and coordinates of vertices, normal vectors and UV coordinates;

[0037] A search module, wherein the search module uses a search algorithm to search for facets from the starting point of the target three-dimensional model in all directions until the number of facets searched reaches the required number of facets, and stops searching, and saves the searched facets and vertex information;

[0038] A creation module, wherein the creation module deletes the searched face pieces in the target three-dimensional model and creates a new three-dimensional model based on the searched face pieces and vertex information;

[0039] A mapping module, wherein the mapping module performs texture mapping on the new three-dimensional model based on the model feature scheme to obtain a local feature model;

[0040] And, an output module, wherein the output module is used to return the cropped target three-dimensional model and the local feature model together.

[0041] In a fourth aspect, the present application proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0042] In a fifth aspect, the present application proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0043] The present application proposes a local feature generation method for power scene virtual data synthesis, which automatically generates a local feature model based on the original three-dimensional model, without the need to construct a three-dimensional model separately for the local features, significantly reducing the workload of constructing the three-dimensional model of the power scene, and improving the efficiency of feature construction and data feature diversity;

[0044] The present application proposes a local feature generation method for virtual data synthesis of power scenes. Based on the data generated by the local feature model and the corresponding annotation information, the local feature data can be constructed, which reduces the workload of three-dimensional model construction and improves the efficiency of feature construction and the diversity of data features. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the embodiments of the present application, constitute a part of the present application, and do not constitute a limitation on the embodiments of the present application. In the drawings:

[0046] Figure 1 A flow chart of a local feature generation method proposed in an embodiment of the present application;

[0047] Figure 2 A schematic diagram for searching a target three-dimensional model;

[0048] Figure 3 This is a principle block diagram of the local feature generation system proposed in the embodiment of the present application;

[0049] Figure 4 This is a flow chart of the virtual data synthesis method proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with examples and drawings. The illustrative implementation scheme of the present application and its description are only used to explain the present application and are not intended to limit the present application.

[0051] Example:

[0052] The existing virtual data generation technology mainly generates target annotation files by acquiring the entire three-dimensional model, such as generating the minimum circumscribed rectangular frame under the camera's perspective in target detection. However, many features faced in the power field are local feature targets, such as rust, dust, burns, oil leakage, defects, etc. If each target feature of each device in each power scene is separately generated with a corresponding three-dimensional model and a corresponding texture, the workload is undoubtedly huge; and the texture is mostly local information and cannot be marked with a specific location. In view of this, this embodiment proposes a local feature generation method for virtual data synthesis in power scenes.

[0053] like Figure 1 As shown, the local target feature generation method proposed in this embodiment includes the following steps:

[0054] Step 100, determine the target three-dimensional model and model feature solution.

[0055] In this step, firstly, based on the task requirements of the current power scenario, determine which 3D models of equipment and / or environment to use, that is, as the target 3D model, and determine the tasks that need to be processed for each model, that is, the model feature scheme. For example, common scenarios include but are not limited to: oil leakage on the surface of the oil storage cabinet, discoloration of the silicone of the respirator, blurred marking dials, dirt on insulators, cable defects, etc.

[0056] Step 200, extract the basic information of the target 3D model and set the required number of facets, wherein the basic information includes facets, number and coordinates of vertices, normal vectors and UV coordinates.

[0057] In this step, the target 3D model path is selected, the specific information of the model is loaded, and the model structure is read including the specific coordinates of the vertices, the number of facets, the UV coordinates for rendering, etc. Depending on the actual situation, the number of facets required can be determined randomly or under certain conditions.

[0058] Step 300, using a search algorithm, searching for faces from the starting point of the target three-dimensional model in all directions until the number of faces searched reaches the required number of faces, then stopping the search, and saving the information of the searched faces and vertices.

[0059] In this step, a breadth-first search algorithm may be used to search for connected faces from the starting point in all directions, and the searched vertices, faces, and other contents may be copied and saved.

[0060] Specifically, the search process in this step is as follows:

[0061] Step 301, create an adjacency list, traverse each face in the model, add it to the adjacency list, and build a mapping relationship between a vertex and its adjacent vertices and a set of faces.

[0062] Step 302: Initialize a candidate vertex queue and add a random vertex as a starting point to the candidate vertex queue.

[0063] Step 303, taking out vertices from the candidate vertex queue in order as starting points respectively and marking the starting points as visited, searching for vertices adjacent to the starting points, adding the searched adjacent vertices to the candidate vertex queue, searching for all facets containing the starting point and its adjacent vertices and marking them as visited;

[0064] Step 304, when the candidate vertex queue is not empty and the number of visited facets is less than the required number of facets, vertices are sequentially taken out from the candidate vertex queue and marked as visited, and vertices adjacent to the vertex are searched. If the searched adjacent vertex is not in the candidate vertex queue and has not been visited, it is added to the candidate vertex queue, and all facets containing the vertex and its adjacent vertices are searched and unmarked facets are marked as visited;

[0065] Step 305: If the candidate vertex queue is empty or the number of visited facets reaches the required number of facets, stop searching; otherwise, return to the previous step to continue searching.

[0066] Step 306, saving the visited vertex and face information.

[0067] by Figure 2 The target three-dimensional model shown in the figure is taken as an example to illustrate the above search process. Vertex 0 is taken as the starting point and marked as visited. The vertices adjacent to the starting point, namely, vertices 1, 2, 3, 4, 5, and 6, are searched. These six vertices are saved in the candidate vertex queue, and all the facets containing the starting point and its adjacent vertices are searched and marked as visited, namely, facets a, b, c, d, e, and f. Then vertex 1 is taken out and marked as visited. The vertices adjacent to vertex 1 are searched, including vertex 2 and vertex 6 already in the candidate vertex queue, and vertex 0 which has not been visited in the candidate vertex queue. These are no longer added to the candidate vertex queue, while vertices which are not in the candidate vertex pair and have not been visited need to be added to the candidate vertex queue. All facets containing vertex 1 and its adjacent vertices are searched, including visited facets a and f, and other unvisited facets are marked as visited. Similarly, vertex 2, vertex 3, etc. are taken out from the candidate vertex queue in turn and searched according to the above search method until the search end condition is reached and the search ends.

[0068] Step 400: Delete the searched facets in the target three-dimensional model, and create a new three-dimensional model based on the searched information.

[0069] In this step, a new 3D model is created in the same directory and coordinate system as the original 3D model, and the searched information such as faces and vertices is assigned to the new 3D model.

[0070] Step 500: texture mapping is performed on the new three-dimensional model based on the model feature solution to obtain a local feature model.

[0071] In this step, according to the model feature scheme, the required texture can be selected from a large number of candidate texture libraries and applied to the newly created 3D model. According to the needs, it can be fully covered or merged with the previous texture to obtain a local feature model.

[0072] Step 600, returning the cropped target 3D model and the newly constructed local feature model together.

[0073] The method proposed in this embodiment automatically generates a local feature model based on the original three-dimensional model, without the need to construct a three-dimensional model separately for local features, which significantly reduces the workload of constructing the three-dimensional model of the power scene and improves the efficiency of feature construction and the diversity of data features.

[0074] Based on the same technical concept as above, this embodiment also proposes a local feature generation system for power scene virtual data synthesis, such as Figure 3 As shown, the local feature generation system proposed in this embodiment includes:

[0075] A determination module is used to determine a target three-dimensional model and a model feature solution.

[0076] The extraction module is used to extract basic information of the target three-dimensional model and set the required number of facets.

[0077] The search module adopts a search algorithm to search for faces in all directions starting from the starting point of the target three-dimensional model until the number of faces searched reaches the required number of faces, and then stops searching and saves the information of the searched faces and vertices.

[0078] A creation module deletes the searched facets in the target three-dimensional model and creates a new three-dimensional model based on the searched information.

[0079] A mapping module performs texture mapping on the new three-dimensional model based on a model feature scheme to obtain a local feature model.

[0080] And, an output module, which is used to return the cropped target three-dimensional model and the newly constructed local feature model together.

[0081] It should be noted that the specific implementation process of each functional module of the local feature generation system proposed in this embodiment is as described in the above steps 100 to 600, and will not be repeated here.

[0082] Furthermore, this embodiment also proposes a method for synthesizing virtual data of power scenarios, such as Figure 4 As shown, the virtual data synthesis method proposed in this embodiment specifically includes the following steps:

[0083] Step 10, construct a virtual scene with random background, random camera perspective, random lighting, random poses and positions of multiple target three-dimensional models under certain conditions.

[0084] Step 20, for each scene, the above-mentioned local target feature generation method is executed on the target three-dimensional model that needs to generate local features, and updated in the virtual scene according to the requirements.

[0085] Step 30, according to task requirements, save the data generated based on the local feature model and the corresponding annotation information for subsequent model training.

[0086] According to the specific task scenario requirements, if it is a target detection task, the minimum circumscribed rectangular annotation box of the local feature model under the camera's perspective is returned; if it is a segmentation type task, the mask image of the local feature model patch area under the camera's perspective is returned.

[0087] Step 40, repeat the above steps until sufficient training data is obtained.

[0088] This embodiment can complete the construction of local feature data based on the data generated by the local feature model and the corresponding annotation information, thereby reducing the workload of three-dimensional model construction and improving the efficiency of feature construction and data feature diversity.

[0089] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage devices, CD-ROMs, optical storage devices, etc.) that contain computer-usable program codes.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable storage device that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable storage device produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A local feature generation method for power scene virtual data synthesis, characterized in that: The local target feature generation method comprises: Determine the target three-dimensional model and model feature scheme; Extract the basic information of the target 3D model and set the required number of facets; the basic information includes facets, number and coordinates of vertices, normal vectors and UV coordinates; Using a search algorithm, searching for facets from the starting point of the target three-dimensional model in all directions until the number of facets searched reaches the required number of facets, and then stopping the search, and saving the searched facets and vertex information; Deleting the searched face pieces in the target three-dimensional model, and creating a new three-dimensional model based on the searched face pieces and vertex information; Performing texture mapping on the new three-dimensional model based on the model feature scheme to obtain a local feature model; The cropped target three-dimensional model and the local feature model are returned together.

2. According to claim 1, a local feature generation method for power scene virtual data synthesis is characterized in that: The search algorithm described herein starts from the starting point and searches for faces in all directions until the number of faces searched reaches the required number of faces, and stops searching, and saves the searched faces and vertex information, specifically including: Create an adjacency list, traverse each face in the target three-dimensional model, add it to the adjacency list, and construct a mapping relationship between a vertex and a set of its adjacent vertices and faces; Initialize a candidate vertex queue, and add a random vertex as a starting point to the initialized candidate vertex queue; Taking the starting point from the candidate vertex queue and marking it as visited, searching for vertices adjacent to the starting point, adding the searched adjacent vertices to the candidate vertex queue, searching for all facets containing the starting point and its adjacent vertices and marking them as visited; When the candidate vertex queue is not empty and the number of visited facets is less than the required number of facets, a vertex is sequentially taken out from the candidate vertex queue and the vertex is marked as visited, and the vertex adjacent to the vertex is searched. If the searched adjacent vertex is not in the candidate vertex queue and has not been visited, it is added to the candidate vertex queue, and all facets containing the vertex and its adjacent vertices are searched and the unmarked facets therein are marked as visited; If the candidate vertex queue is empty or the number of visited facets reaches the required number of facets, stop searching, otherwise return to the previous step to continue searching; Save the visited vertex and face information.

3. A local feature generation method for power scene virtual data synthesis according to claim 1 or 2, characterized in that: The target three-dimensional model is a three-dimensional model of equipment and / or environment determined based on the task requirements of the current power scenario; The model feature scheme is determined according to the task that needs to be processed corresponding to the three-dimensional model.

4. A local feature generation method for power scene virtual data synthesis according to claim 1 or 2, characterized in that: The step of creating a new three-dimensional model based on the searched face and vertex information specifically includes: A new three-dimensional model is created in the same coordinate system as the target three-dimensional model in the same directory, and the searched face and vertex information are assigned to the new three-dimensional model.

5. A local feature generation method for power scene virtual data synthesis according to claim 1 or 2, characterized in that: The method of performing texture mapping on the new three-dimensional model based on the model feature scheme to obtain a local feature model specifically includes: According to the model feature scheme, the required target feature texture is selected from the texture library, and the target feature texture is applied to the new three-dimensional model. According to the requirements, it is completely covered or merged with the previous texture to obtain a local feature model.

6. A method for synthesizing virtual data of power scenes, characterized in that: The virtual data synthesis method comprises: Construct a virtual scene with random background, random camera angle, random lighting, and random poses and positions of multiple target 3D models under certain conditions; For each virtual scene, the local feature generation method according to any one of claims 1 to 5 is executed on the target three-dimensional model for which local features need to be generated, and updated in the virtual scene as required; According to task requirements, the data generated based on the local feature model and the corresponding annotation information are saved as training data for subsequent model training; Repeat the above steps until enough training data is obtained.

7. A method for synthesizing virtual data of electric power scenarios according to claim 6, characterized in that: The data generated based on the local feature model and the corresponding annotation information are saved according to the task requirements, specifically including: According to the specific task requirements, if it is a target detection task, the minimum circumscribed rectangular annotation box of the local feature model under the camera's perspective is returned; if it is a segmentation type task, the mask image of the local feature model patch area under the camera's perspective is returned.

8. A local feature generation system for power scene virtual data synthesis, characterized in that: The local feature generation system comprises: A determination module, the determination module is used to determine a target three-dimensional model and a model feature scheme; An extraction module, the extraction module is used to extract basic information of the target three-dimensional model and set the required number of facets; the basic information includes facets, number and coordinates of vertices, normal vectors and UV coordinates; A search module, wherein the search module uses a search algorithm to search for facets from the starting point of the target three-dimensional model in all directions until the number of facets searched reaches the required number of facets, and stops searching, and saves the searched facets and vertex information; A creation module, wherein the creation module deletes the searched face pieces in the target three-dimensional model and creates a new three-dimensional model based on the searched face pieces and vertex information; A mapping module, wherein the mapping module performs texture mapping on the new three-dimensional model based on the model feature scheme to obtain a local feature model; And, an output module, wherein the output module is used to return the cropped target three-dimensional model and the local feature model together.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images