Three-dimensional model semantic monomer method and system for target object around power grid

Abstract

The invention relates to the technical field of computer vision, and discloses a three-dimensional model semantic monomer method for a target object around a power grid, which comprises the following steps: performing inverse rendering on an initial three-dimensional model of the target object based on a first image to obtain a first intermediate three-dimensional model; projecting and rendering the first intermediate three-dimensional model at a specified visual angle to obtain a second image; performing inverse rendering on the first intermediate three-dimensional model by using the second image to obtain a second intermediate three-dimensional model; updating the initial prompt point set by using the prompt point with the highest confidence score in the second intermediate three-dimensional model; taking the second intermediate three-dimensional model as a first intermediate three-dimensional model, and repeatedly executing the step of updating the initial prompt point set until an end condition is met to obtain a target prompt point set; and generating a three-dimensional mask and a three-dimensional model of the target object according to the target prompt point set of each first image. The method has the beneficial effects that the precision of the monomer model is improved, and the fitting degree of the mask and the three-dimensional model is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and system for semantic monomerization of three-dimensional models of target objects around a power grid. Background Art

[0002] As the global energy transformation and digitalization process accelerates, the traditional two-dimensional power grid model can no longer meet the increasingly complex needs of power grid management and optimization. The three-dimensional power grid model can more accurately reflect the physical structure and spatial relationship of the power grid. Through three-dimensional visualization technology, engineers can intuitively carry out power grid planning, fault diagnosis, risk assessment and asset management.

[0003] In the application of 3D power grid model, the power grid 3D model monomer technology is a key technology, which refers to the separate 3D modeling and digital processing of key buildings within the power grid during the construction of the 3D power grid model. This technology ensures that the geometric shape, size, material and other information of the building can be reflected in the power grid model with high accuracy and high authenticity.

[0004] The individual model not only improves the professionalism and accuracy of grid planning and design, but also helps to better evaluate the potential impact of buildings on grid facilities during the implementation of grid projects, such as obstruction of transmission lines and impact on substation layout.

[0005] Among the related technologies, the accuracy and automation capabilities of three-dimensional model monomerization technology need to be improved. Summary of the invention

[0006] The present application provides a method and system for semantic singulation of three-dimensional models of target objects around a power grid, which solves the technical problem that the three-dimensional model singulation process requires manual intervention and supervision, and achieves the technical effect of improving the accuracy of the singulated model and enhancing the fit between the mask and the three-dimensional model.

[0007] In order to achieve the above objectives, the main technical solutions adopted in this application include:

[0008] In a first aspect, an embodiment of the present application provides a method for semantically monomerizing a three-dimensional model of a target object around a power grid, wherein the target object has an image data set in advance; the image data set includes a plurality of first images, and the first images correspond to an initial prompt point set; the method includes:

[0009] Performing inverse rendering on the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image;

[0010] Projecting and rendering the first intermediate 3D model at a specified viewing angle to obtain a second image at the specified viewing angle; inversely rendering the first intermediate 3D model using the second image to obtain a second intermediate 3D model; and updating the initial cue point set using the cue point with the highest confidence score in the second intermediate 3D model; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set;

[0011] The second intermediate 3D model is used as the first intermediate 3D model, and the step of updating the initial cue point set is repeated until an end condition is satisfied, thereby obtaining a target cue point set for the first image;

[0012] A three-dimensional mask and a three-dimensional model of the target object are generated according to the target cue point set of each first image in the image data set.

[0013] The method for semantic monomerization of the three-dimensional model of the target object around the power grid proposed in the embodiment of the present application is to project the three-dimensional model of the target object onto a specified viewing angle, and inversely render the three-dimensional model according to the mask and image at the specified viewing angle, thereby improving the confidence of each voxel in the three-dimensional model and improving the degree of matching between the three-dimensional model and the target object. The initial prompt point set is updated according to the point with the highest confidence score, and the target prompt point set is obtained after repeated updates, wherein the target prompt point set includes multiple prompt points about the target object, which has an optimization effect on the three-dimensional model of the target object. Therefore, the target object is reconstructed in three dimensions according to the target prompt point set, which improves the precision of the three-dimensional model of the target object, and at the same time improves the degree of fit between the mask and the three-dimensional model, thereby enhancing the display effect of the target object.

[0014] In a second aspect, an embodiment of the present application provides a system for semantically monomerizing a three-dimensional model of a target object around a power grid, wherein the target object has an image data set in advance; the image data set includes a plurality of first images, and the first images correspond to an initial prompt point set; the system includes:

[0015] an initial inverse rendering module, configured to inversely render the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image;

[0016] a cue point set updating module, configured to project the first intermediate 3D model at a specified viewing angle to obtain a second image at the specified viewing angle; perform inverse rendering on the first intermediate 3D model using the second image to obtain a second intermediate 3D model; and update the initial cue point set using the cue point with the highest confidence score in the second intermediate 3D model; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set;

[0017] an iterative updating module, configured to use the second intermediate 3D model as the first intermediate 3D model, and repeatedly execute the above steps of updating the initial cue point set until an end condition is met, thereby obtaining a target cue point set for the first image;

[0018] The model generation module is used to generate a three-dimensional mask and a three-dimensional model of the target object according to the target cue point set of each first image in the image data set.

[0019] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for semantic monomerization of three-dimensional models of target objects around a power grid as described in any one of the above embodiments by executing the computer instructions.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to enable a computer to execute the method for semantic monomerization of a three-dimensional model of a target object surrounding a power grid as described in any one of the above embodiments.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer instructions, wherein the computer instructions are used to enable a computer to execute the method for semantic monomerization of a three-dimensional model of a target object surrounding a power grid described in any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A schematic diagram of a process for a method provided in an embodiment of the present application;

[0024] Figure 2 A step diagram of a method for semantically monomerizing a three-dimensional model of a target object surrounding a power grid provided in an embodiment of the present application;

[0025] Figure 3 A diagram showing the steps of obtaining a second image in an embodiment of the present application;

[0026] Figure 4 A diagram showing the steps of obtaining a second intermediate three-dimensional model in an embodiment of the present application;

[0027] Figure 5A diagram showing the steps of updating the initial prompt point set according to the second intermediate three-dimensional model in an embodiment of the present application;

[0028] Figure 6 This is a method framework diagram in an embodiment of the present application;

[0029] Figure 7 This is a SAM network framework diagram in the embodiment of the present application;

[0030] Figure 8 Schematic diagram of a three-dimensional model and a three-dimensional mask obtained in an embodiment of the present application;

[0031] Fig. 9 A module diagram of a system for semantic monomerization of three-dimensional models of target objects around a power grid provided in an embodiment of the present application.

[0032] Among them, the figure numbers of the drawings in the specification are as follows: 100. Initial inverse rendering module, 200. Prompt point set update module, 300. Iterative update module, 400. Model generation module. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0034] With the acceleration of global energy transformation and digitalization, traditional two-dimensional power grid models can no longer meet the increasingly complex needs of power grid management and optimization. Three-dimensional power grid models can more accurately reflect the physical structure and spatial relationship of the power grid, including all power grid components such as transmission lines and towers. Through three-dimensional visualization technology, engineers can intuitively carry out power grid planning, fault diagnosis, risk assessment and asset management. The three-dimensional model of the power grid is a key technology. It refers to the separate three-dimensional modeling and digital processing of key buildings within the power grid during the construction of the three-dimensional power grid model. This technology ensures that the geometric shape, size, material and other information of the building can be reflected in the power grid model with high precision and high authenticity. The single model not only improves the professionalism and accuracy of power grid planning and design, but also helps to better evaluate the potential impact of buildings on power grid facilities during the implementation of power grid projects, such as the obstruction of transmission lines and the impact on the layout of substations.

[0035] At present, the existing singulation methods are mostly aimed at regular buildings, such as houses and buildings, to achieve the singulation of urban buildings. Currently, there are three main methods for singulation extraction: manual method, semi-automatic method and fully automatic method. The singulation methods also mainly analyze and extract spatial features and texture features. The main methods for extracting this information are: (1) singulation methods based on image segmentation, mathematical morphology, and building contours; (2) singulation methods based on spectral and shape features; (3) singulation methods that combine depth and color buffers to track the edge line of the target.

[0036] The building singulation methods based on oblique photography mainly include three methods: logical singulation, physical cutting singulation, and tag ID singulation. (1) Logical singulation: Logical singulation is divided into two forms: vector cutting singulation and vector tagging singulation. The former refers to using the vector surface of the single object target to physically cut the overall model to achieve singulation; the latter refers to overlaying the two-dimensional vector surface data on the surface of the corresponding real-life three-dimensional model, and achieving the effect of selecting the single three-dimensional model by clicking the attributes of the vector layer. However, this method does not really separate the target building object from the overall model, which limits the application space of the model data. (2) Physical cutting singulation: The main technical method to achieve physical cutting singulation is to form different single objects into a selectable and separable entity through three-dimensional reconstruction. At present, there are many studies on physical cutting singulation models. The main commercial application software includes DP-Modeler software, OSketch software, and SVS Intelligent Space. The basic data source of these three software is continuous oblique multi-view images, and the building singulation is carried out in a semi-automatic way with manual intervention. (3) Marking ID singulation: Marking ID singulation is to assign the same ID value to the vertices of the triangular faces contained in the target building, so that when the target model is selected alone, the entire model is highlighted. However, this singulation method does not segment the model and does not achieve true singulation of the building model.

[0037] In summary, there are problems with current technologies: (1) The current building unitization method is only applicable to simple regular urban buildings such as houses and buildings, while power grid projects may involve more complex and irregular buildings, whose geometric features are difficult to accurately capture using existing algorithms; (2) Current technologies do not achieve true automation or separation between the unit model and the overall model during the building unitization process.

[0038] Therefore, in order to realize the semantic individualization of models of key buildings in power grid engineering survey, it is necessary to develop new technologies that can adapt to complex geometric shapes, understand and extract semantic information, and have high automation and robustness.

[0039] In order to solve the above problems, the present application provides a method and system for semantic singulation of three-dimensional models of target objects around a power grid, wherein the method includes: inversely rendering the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model; projecting and rendering the first intermediate three-dimensional model at a specified viewing angle to obtain a second image; inversely rendering the first intermediate three-dimensional model using the second image to obtain a second intermediate three-dimensional model; updating the initial prompt point set using the prompt point with the highest confidence score in the second intermediate three-dimensional model; using the second intermediate three-dimensional model as the first intermediate three-dimensional model, repeating the above steps of updating the initial prompt point set until the end condition is met to obtain the target prompt point set; generating a three-dimensional mask and a three-dimensional model of the target object according to the target prompt point set of each first image. The present application solves the technical problem that the three-dimensional model singulation process requires manual intervention and supervision, and achieves the technical effect of improving the accuracy of the singulated model and improving the fit between the mask and the three-dimensional model.

[0040] Reference Figure 1 , Figure 1 A flow chart of the method provided in the embodiment of the present application, as shown in the figure, in the method provided in the embodiment of the present application, firstly, image acquisition is performed on the target object to obtain a multi-view image sequence of the target object and camera pose information corresponding to the view image, wherein the initial three-dimensional model of the target object can be obtained according to the multi-view image sequence of the target object. According to the initial three-dimensional model and the camera pose information, any view in the multi-view image sequence is rendered by the neural radiation field to obtain a single-view rendering result; the single-view rendering result is image analyzed, and the corresponding mask prompt points are determined according to the features in the single-view rendering result. The single-view rendering result and the mask prompt points are input into the segmentation model of any object, and the single-view rendering result is semantically segmented by segmenting the any object model to obtain a single-view image semantic mask.

[0041] It should be noted that Neural Radiance Fields (NeRF) is a deep learning model that uses neural networks to represent and render 3D scenes. The core of NeRF is to train a neural network to learn the volume information and color information when observing a scene from different perspectives, so as to generate new perspectives of the scene. Segment Anything Model (SAM) is an image segmentation model based on prompt learning. It uses prompts to assist training and obtain the desired segmentation effect. It aims to achieve accurate segmentation of any object. The SAM network has the characteristics of versatility and efficiency.

[0042] The initial three-dimensional model is masked and inversely rendered according to the single-view rendering result and the single-view image semantic mask, thereby updating the initial three-dimensional model. The updated initial three-dimensional model is projected at multiple angles, and the updated initial three-dimensional model is repeatedly updated with the view images at multiple angles, and prompt points are extracted from the view images at multiple angles according to the updated initial three-dimensional model, thereby forming a target prompt point set that can be used for cross-view self-prompting. A three-dimensional mask of the target object is formed according to the target prompt point set, and the world coordinate information of the surface contour of the target object is inversely calculated according to the three-dimensional mask, wherein the world coordinate information is related to the position of the target object in the three-dimensional power grid model. According to the world coordinate information of the surface contour of the target object, the three-dimensional mask is fitted with the three-dimensional model, thereby obtaining a high-precision three-dimensional model as a monomer model of the target object.

[0043] The method for semantic monomerization of three-dimensional models of target objects around the power grid provided in this specification can be used to identify target objects in the digital model base during the power grid engineering survey, and separate the model of the target object into monomers, so as to obtain a three-dimensional power grid model containing the monomer model of the target object, and provide a data basis for power grid planning, fault diagnosis, risk assessment and asset management. Among them, the target objects include power grid components and important buildings. The power grid components include transmission lines and towers, etc. The important buildings are buildings that have potential impact on power grid facilities within the scope of the power grid. It should be noted that after adaptive modification of this application, the method provided in this application can also be used to monomerize the model of the target object in other fields.

[0044] According to an embodiment of the present application, an embodiment of a method for semantic monomerization of a three-dimensional model of a target object around a power grid is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] In this embodiment, a method for semantically monomerizing a three-dimensional model of a target object around a power grid is provided, which can be used to generate a three-dimensional model of a power grid during a power grid engineering survey. Figure 2 As shown, Figure 2 A step diagram of a method for semantically singulating a three-dimensional model of a target object around a power grid provided in an embodiment of the present application, as shown in the figure, the target object has an image data set in advance; the image data set includes a plurality of first images, and the first images correspond to an initial prompt point set; the method includes:

[0046] S100. Perform inverse rendering on the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image.

[0047] S200. Project and render the first intermediate three-dimensional model at a specified viewing angle to obtain a second image at the specified viewing angle; use the second image to inversely render the first intermediate three-dimensional model to obtain a second intermediate three-dimensional model; use the cue point with the highest confidence score in the second intermediate three-dimensional model to update the initial cue point set; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set.

[0048] S300. Use the second intermediate 3D model as the first intermediate 3D model, and repeat the above steps of updating the initial cue point set until the end condition is met, thereby obtaining the target cue point set of the first image.

[0049] S400. Generate a three-dimensional mask and a three-dimensional model of a target object according to a target cue point set of each first image in the image data set.

[0050] Specifically, the first images in the image data set are images of the target object at a plurality of viewing angles, and the number of the first images is not specifically limited. It is understandable that the more the number of first images is, the more complete the information about the target object that can be obtained in advance from the image data set is, and the fewer times the step of updating the initial cue point set needs to be repeated in step S300.

[0051] Furthermore, before inverse rendering the initial three-dimensional model of the target object, the first image is rendered according to the initial three-dimensional model, and the rendering step includes: inputting the image data set into the tensor radiation field for training to obtain a rendering function, which is in the following form:

[0052] f θ :(x,d)→(c,σ)

[0053] Among them, f θ represents the rendering function, usually represented by a multi-layer perceptron (MLP); θ is a learnable parameter; x∈R 3 represents the spatial coordinates in the target space containing the initial 3D model; d∈S 2 represents the viewing angle of the first image; c∈R 3 represents the color of the initial three-dimensional model; σ∈R represents the density of the initial three-dimensional model.

[0054] Among them, Tensor Radiance Field (TensorRF) is a novel method for modeling and reconstructing radiation fields. It models the scene as a four-dimensional tensor. Each item in the tensor represents a voxel, which contains volume density and multi-dimensional feature information.

[0055] Furthermore, in the rendering process, a plurality of rendering rays are obtained according to the mapping relationship between the initial three-dimensional model and the first image, and the form thereof is as follows:

[0056] r(t)=x o +td

[0057] Among them, x o represents the camera origin of the first image; t represents the distance from the pixel point in the first image to the camera origin along the rendering ray; d represents the direction of the rendering ray. The color of the first image can be updated according to the rendering ray, and its form is as follows:

[0058]

[0059] Among them, I θ (r) indicates color, in RGB format; t f Indicates the far bound of the rendering light; t n represents the near bound of the rendering ray; ω represents the density of voxels used for rendering, which is in the following form:

[0060]

[0061] Among them, r(s) represents the rendering light in a certain direction; s represents the position on the rendering light; σ represents the differentiable volume rendering function.

[0062] Furthermore, the initial three-dimensional model of the target object is inversely rendered by using the rendered first image, so as to update the initial three-dimensional model according to the feature information in the first image, improve the matching degree between the initial three-dimensional model and the target object, and thus obtain a first intermediate three-dimensional model. The matching degree can be described by the confidence score of each voxel in the three-dimensional model, and the confidence score is expressed as a soft mask for the three-dimensional model, which ranges between [0, 1] and can be obtained by the voxel attribute of the voxel. During the inverse rendering process, the confidence score will be updated to reflect the confidence that each voxel belongs to the target object, so whether the voxel belongs to the target object can be judged according to the confidence score.

[0063] Furthermore, after obtaining the 3D mask of the target object, the coordinate information of the model surface of the target object is inversely calculated according to the 3D mask to obtain the coordinate information of the surface of the target object, thereby generating a 3D model of the target object by cutting in the target space containing the target object. Finally, the 3D mask is attached to the surface of the 3D model to obtain a high-precision single body model of the target object.

[0064] The method for semantic monomerization of the three-dimensional model of the target object around the power grid provided in this embodiment projects the three-dimensional model of the target object onto a specified viewing angle, and inversely renders the three-dimensional model according to the mask and image at the specified viewing angle, thereby improving the confidence of each voxel in the three-dimensional model and improving the matching degree between the three-dimensional model and the target object. The initial prompt point set is updated according to the point with the highest confidence score, and the target prompt point set is obtained after repeated updates, wherein the target prompt point set includes multiple prompt points about the target object, which has an optimization effect on the three-dimensional model of the target object. Therefore, the target object is reconstructed in three dimensions according to the target prompt point set, which improves the precision of the three-dimensional model of the target object, and at the same time improves the fit between the mask and the three-dimensional model, thereby enhancing the display effect of the target object.

[0065] Reference Figure 3 , Figure 3 This is a step diagram for obtaining a second image in an embodiment of the present application. As shown in the figure, as an embodiment of the present application, the first intermediate three-dimensional model is projected and rendered at a specified viewing angle to obtain a second image at the specified viewing angle, including:

[0066] S202. Project the first intermediate three-dimensional model at a specified viewing angle to obtain a second mask corresponding to the second image.

[0067] S204. Render the second mask according to the model features of the first intermediate three-dimensional model to obtain a second image.

[0068] Specifically, after the first intermediate 3D model is inversely rendered by the first image or the second image, the voxels contained therein have different confidence scores. According to these confidence scores, a soft mask about the target object can be obtained, and the second mask is projected at a specified viewing angle according to the soft mask, wherein the second mask represents the range of the target object in the second image. Further, the second mask is rendered according to the first intermediate 3D model at this time, and the model features in the first intermediate 3D model are reflected in the second mask, thereby obtaining the second image. It can be understood that the second image and the first image are target objects at different viewing angles, and can reflect the feature information of the target object at different viewing angles.

[0069] Furthermore, the second mask is in the form of:

[0070]

[0071] Among them, M(r) is the confidence score corresponding to the pixel in the second image; r(t) is the light projected on the second mask; ω(r(t)) inherits the density value of the pre-trained TensoRF network; V(r(t)) is the confidence score obtained from the position r(t) in the first intermediate 3D model. It can be seen that M(r) is a soft mask, and M(r) is compared with the set threshold, and the part greater than the set threshold is used as the second mask.

[0072] It should be noted that the specified perspective is obtained according to the actual scene requirements. In this embodiment, according to the needs of the power grid project, the specified perspective can be the perspective required for power grid planning or analysis. These perspectives reflect the impact of different objects within the power grid on the power grid project, including the shielding of buildings on transmission lines and the impact on the layout of substations. These impacts need to be evaluated during the planning process to improve the accuracy of power grid planning and design.

[0073] Reference Figure 4 , Figure 4 This is a step diagram of obtaining a second intermediate three-dimensional model in an embodiment of the present application. As shown in the figure, as an embodiment of the present application, the first intermediate three-dimensional model is inversely rendered using the second image to obtain the second intermediate three-dimensional model, including:

[0074] S210. Obtain mapping light according to the viewing angle of the second image and the second mask; wherein the mapping light is located in the target space containing the first intermediate three-dimensional model.

[0075] S212. Determine the inverse rendering voxel passed by the mapped light according to the position and direction of the mapped light in the target space.

[0076] S214. Based on the mapped light, the voxel attributes and confidence scores of the inversely rendered voxels are updated using the neural radiation field to obtain a second intermediate three-dimensional model; wherein the voxel attributes include color and density, and the confidence score is obtained based on the voxel attributes.

[0077] Specifically, according to the second image and the second mask, the NeRF network is used to perform mask inverse rendering on the first intermediate three-dimensional model. Specifically, the method includes: voxelizing the first intermediate three-dimensional model in the target space to obtain a plurality of voxels discretized into a grid; obtaining mapping rays according to the viewing angle of the second image and the second mask, wherein the mapping rays are emitted from the camera position corresponding to the second image and projected to the target space through the second mask, and the mapping rays have different positions and directions; determining the inverse rendering voxels that the mapping rays pass through in the target space along the direction of any mapping ray; accumulating the voxel attributes of each inverse rendering voxel according to the mapping ray to update the voxel attributes and confidence scores of each inverse rendering voxel.

[0078] Furthermore, voxel attributes include color and density, which are in the following form:

[0079] C new =Σ voxel C voxel ·exp(-ω·r(s) voxel )

[0080] Among them, C new Indicates the color of the voxel after inverse rendering; C voxel is the color of the voxel; ω is the density of the voxel; r(s) voxel is the location where the mapping ray intersects the voxel target space.

[0081] Furthermore, the confidence score of the corresponding voxel can be obtained according to the updated voxel attribute, and the voxel attribute can be mapped to the confidence score through a mapping function. The mapping function can be selected according to the actual scene requirements, and can be linear or nonlinear.

[0082] As an embodiment of the present application, the confidence score includes an attenuation term, and the attenuation term is obtained according to the distance between each voxel in the second intermediate three-dimensional model and each cue point in the current initial cue point set.

[0083] Specifically, in order to make the new cue point obtained from the second intermediate 3D model have a high confidence score and be close to the cue point in the initial cue point set, an attenuation term is added to the confidence score of the voxel, wherein the attenuation term is in the following form:

[0084]

[0085] Among them, ΔM (n) (p) represents the attenuation term; is the confidence score of the cue point in the initial cue point set; is the minimum-maximum normalized Euclidean distance, which represents the distance between voxel p and the cue points in the initial cue point set; P s Represents the current initial set of cue points.

[0086] Furthermore, the confidence score of each voxel can be obtained according to the attenuation term, which is in the following form:

[0087]

[0088] Among them, M (n) (p) represents the confidence score of voxel p; represents the confidence score of voxel p after attenuation.

[0089] Reference Figure 5 , Figure 5This is a step diagram of updating the initial cue point set according to the second intermediate three-dimensional model in an embodiment of the present application. As shown in the figure, as an embodiment of the present application, the initial cue point set is updated using the cue point with the highest confidence score in the second intermediate three-dimensional model, including:

[0090] S220. Determine a target pixel in the second image belonging to the target object according to the voxel attribute size of each voxel in the second intermediate three-dimensional model; wherein the value of the voxel attribute corresponding to the target pixel is greater than the voxel attribute threshold.

[0091] S222. According to the confidence scores of the target pixels, determine the target pixel with the highest confidence score as the prompt pixel.

[0092] S224. Based on the mapping relationship between each voxel in the second intermediate three-dimensional model and each pixel in the second image, a mapping conversion is performed in the second image using the hint pixel to obtain the hint voxel.

[0093] S226. Update the initial cue point set using the cue voxel as a new cue point.

[0094] Specifically, according to the projection of the second intermediate three-dimensional model at the viewing angle of the second image, each voxel and its voxel attribute in the second intermediate three-dimensional model is mapped to the second image in the following form:

[0095] C new (u,v)=Σ s C voxel (r(s) voxel )·ρ(r(s) voxel )

[0096] Among them, C new (u,v) represents the voxel attribute corresponding to the pixel with coordinates (u,v); s is the position parameter of the mapping light, which represents the distance from the camera position corresponding to the second image to the voxel; C voxel (r(s) voxel ) is at position r(s) voxel The color of the voxel at ; ρ(r(s) voxel ) is at position r(s) voxel The density of the voxel at ; r(s) voxel is the position where the mapping ray s intersects the target space at position s.

[0097] Furthermore, according to the voxel attributes corresponding to each pixel, the target pixel belonging to the target object is determined, which is in the following form:

[0098]

[0099] Among them, M binary A simple 2D mask representing the second image, Mbinary (u,v)=1 means the pixel with coordinates (u,v) belongs to the target object, M binary (u, v) = 0 means that the pixel with coordinates (u, v) does not belong to the target object; T is the voxel attribute threshold. According to the simple two-dimensional mask of the second image, the target pixel is determined in the second image.

[0100] Furthermore, based on the voxel attributes corresponding to the target pixel, the confidence score of the target pixel can be obtained through the conversion function, which is in the following form:

[0101] S(u,v)=g(C new (u,v)

[0102] Where S(u,v) represents the confidence score of the target pixel with coordinates (u,v), S(u,v)∈[0,1]; g is a mapping function used to map voxel attributes to confidence scores. The mapping function g can be selected according to the actual scene requirements and can be linear or nonlinear.

[0103] Furthermore, according to the confidence scores of the target pixels, the target pixel with the highest confidence score is determined, which is in the following form:

[0104] p0=argmax p M (n) (p)

[0105] Where p0 represents the target pixel with the highest confidence score. According to the mapping relationship between each voxel in the second intermediate three-dimensional model and each pixel in the second image, the voxel coordinates corresponding to the pixel in the second image in the second intermediate three-dimensional model can be obtained, which is in the following form:

[0106]

[0107] in, represents the voxel p in the second intermediate 3D model, and the coordinate form of voxel p is K -1 represents the mapping relationship between each voxel in the second intermediate 3D model and each pixel in the second image; z(p) represents the depth corresponding to the pixel in the second image in the second intermediate 3D model, which can be estimated by the NeRF network. The cue voxel is obtained by the above formula, and the cue voxel is used as a new cue point to update the initial cue point set.

[0108] As an embodiment of the present application, before inverse rendering the initial three-dimensional model of the target object based on the first image to obtain the first intermediate three-dimensional model corresponding to the first image, the method further includes:

[0109] S110. Render the first image according to the initial three-dimensional model to obtain a rendered first image.

[0110] S120. Obtain a first mask prompt corresponding to the first image by performing image analysis on the first image.

[0111] S130. Input the rendered first image and the first mask prompt into the segmentation model, segment the first image by segmenting the object model, and obtain a first mask corresponding to the first image.

[0112] Specifically, the image analysis includes edge detection and corner point detection, etc., which are used to identify key features of the target object. For example, when the target object is a house, its key features include eaves and column points, etc. According to the target features representing the boundary of the target object, the first image is segmented to separate the outline of the target object from the background.

[0113] Furthermore, based on the obtained target object outline, the algorithm conditional random field of structured prediction is used to improve the accuracy and consistency of target object extraction on the basis of considering spatial relationships, thereby further identifying the structural features of the target object. For example, when the target object is a house, its structural features include roofs, walls, doors and windows, etc.

[0114] Furthermore, according to the pixels where the key features and structural features are located, some pixels are extracted as the first mask prompt.

[0115] As an embodiment of the present application, the rendered first image and the first mask prompt are input into a segmentation model of any object, and the first image is segmented by segmenting the any object model to obtain a first mask corresponding to the first image, including:

[0116] S132. Use a convolutional network to extract features of the first image to obtain first image features.

[0117] S134. Use the attention mechanism to assign an attention weight to the first image feature, and perform weighted calculation on the first image feature according to the attention weight to obtain an image enhancement feature.

[0118] S136. Perform target detection based on the image enhancement feature to obtain the range of the target object, and perform image segmentation on the first image according to the first mask prompt to obtain a first mask.

[0119] Specifically, the first image feature is in the following form:

[0120] F = CNN(I in )

[0121] Where, F represents the first image feature; I in The first image after rendering includes the color, texture and shape features of the target object.

[0122] Furthermore, the first image feature is weighted by using the attention mechanism to obtain the image enhancement feature. In some embodiments, the first image feature can also be enhanced by using Gaussian weights, which are in the following form:

[0123]

[0124] Where F' represents the Gaussian weight of the first image feature; P in(1) represents the first mask prompt; p represents the position of the pixel in the first image feature; σ is the standard deviation of the Gaussian distribution.

[0125] Furthermore, the first image feature is weighted according to the weight of the first image feature, where the weight includes the attention weight and the Gaussian weight, and the form is as follows:

[0126] F intergrated =Σ i ω i F i '

[0127] Among them, F intergrated represents the image enhancement feature; ω i is the weight of the i-th first image feature.

[0128] As an embodiment of the present application, the initial three-dimensional model is obtained according to the following method:

[0129] S102. Iteratively train the tensor radiation field according to the image dataset.

[0130] S104. Perform three-dimensional reconstruction of the target object using the iteratively trained tensor radiation field to obtain an initial three-dimensional model.

[0131] Specifically, the image dataset is learned using the TensoRF network to obtain a complex mapping relationship between the pixels representing the target object in the first image in the image dataset and the three-dimensional model of the target object. Exemplarily, the number of iterative training times may be 30,000. Further, the target object is three-dimensionally reconstructed based on the image dataset according to the obtained complex mapping relationship using the iteratively trained TensoRF network to obtain an initial three-dimensional model.

[0132] As an embodiment of the present application, the end condition includes an iteration number condition and a confidence limit condition, wherein:

[0133] The iteration number condition is that the step is repeated a certain number of times.

[0134] The confidence limit condition is that in multiple consecutive step repetitions, the improvement of the confidence score of the second intermediate three-dimensional model relative to the first intermediate three-dimensional model is less than a preset amplitude threshold.

[0135] Specifically, the number of iterations can be set according to the device used to execute the method to meet the computing capacity of the device and avoid the decrease in computing efficiency due to too many iterations. max , when the number of iterations reaches N max The repetitive process ends when .

[0136] Furthermore, when the confidence limit condition is reached, it means that the initial prompt point set is stable enough, the accuracy of the generated 3D model is high enough, and the iteration can be ended. Exemplarily, the confidence limit condition is in the form of:

[0137]

[0138] Among them, S new represents the updated confidence score; S previous represents the confidence score before the update; δ represents the preset amplitude threshold. If the improvement of the updated confidence score is less than the preset amplitude threshold δ during k consecutive iterations, it is considered that further iterations will have limited improvement on the confidence score and the iteration can be terminated.

[0139] Reference Figure 6 , Figure 6 This is a method framework diagram in an embodiment of the present application. As shown in the figure, this embodiment also provides a method for semantically singulating a three-dimensional model of a target object around a power grid, which specifically includes the following steps:

[0140] S502. The target object has an image data set in advance; the image data set includes a plurality of first images; and the tensor radiation field is iteratively trained according to the image data set.

[0141] S504. Perform three-dimensional reconstruction of the target object using the iteratively trained tensor radiation field to obtain an initial three-dimensional model.

[0142] S506. Render the first image according to the initial three-dimensional model to obtain a rendered first image.

[0143] S508. Obtain a first mask prompt corresponding to the first image by performing image analysis on the first image.

[0144] S510. Input the first image and the first mask prompt to segment any object model.

[0145] S512 uses a convolutional network to extract features from the first image to obtain first image features.

[0146] S514. Use the attention mechanism to assign an attention weight to the first image feature, and perform weighted calculation on the first image feature according to the attention weight to obtain an image enhancement feature.

[0147] S516. Perform target detection based on the image enhancement feature to obtain the range of the target object, and obtain a first mask about the target object in the first image through image segmentation.

[0148] Reference Figure 7 , Figure 7 This is a SAM network framework diagram in an embodiment of the present application. As shown in the figure, the SAM network includes a prompt encoder and an image encoder. The image encoder is used to encode the input first image to extract features in the first image; and the first image is compressed to obtain an image format that is easier to process.

[0149] Furthermore, the input first mask prompt is encoded by the prompt encoder to obtain the coordinates corresponding to the first mask prompt, and the information of the first mask prompt is compressed into a format that is easier to process.

[0150] Further, the encoded first image and the first mask prompt are input into a lightweight mask decoder, the first image is segmented according to the first mask prompt to obtain a first mask, and the first mask is decoded and output to obtain a first mask M SAM .

[0151] S518. Project the first intermediate three-dimensional model at a specified viewing angle to obtain a second mask corresponding to the second image; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set.

[0152] S520. Render the second mask according to the model features of the first intermediate three-dimensional model to obtain a second image.

[0153] S522. Obtain mapping light according to the viewing angle of the second image and the second mask; wherein the mapping light is located in the target space containing the first intermediate three-dimensional model.

[0154] S524. Determine the inverse rendering voxel passed by the mapped light according to the position and direction of the mapped light in the target space.

[0155] S526. According to the mapping light, the voxel attributes and confidence scores of the inverse rendering voxels are updated using the neural radiation field to obtain a second intermediate three-dimensional model; wherein the voxel attributes include color and density, the confidence score is obtained based on the voxel attributes, and the confidence score includes an attenuation term, which is obtained based on the distance between each voxel in the second intermediate three-dimensional model and each cue point in the current initial cue point set.

[0156] S528. Determine the target pixel in the second image belonging to the target object according to the voxel attribute size of each voxel in the second intermediate three-dimensional model; wherein the value of the voxel attribute corresponding to the target pixel is greater than the voxel attribute threshold.

[0157] S530. According to the confidence scores of the target pixels, determine the target pixel with the highest confidence score as the prompt pixel.

[0158] S532. Based on the mapping relationship between each voxel in the second intermediate three-dimensional model and each pixel in the second image, a mapping conversion is performed in the second image using the hint pixel to obtain the hint voxel.

[0159] S534. Update the initial cue point set using the cue voxel as a new cue point.

[0160] S536. Use the second intermediate 3D model as the first intermediate 3D model, and repeat the above steps of updating the initial cue point set until the end condition is met, thereby obtaining the target cue point set of the first image.

[0161] Among them, the termination conditions include the number of iterations condition and the confidence limit condition, among which: the number of iterations condition is that the number of step repetitions reaches a certain number; the confidence limit condition is that in multiple consecutive step repetitions, the improvement of the confidence score of the second intermediate three-dimensional model relative to the first intermediate three-dimensional model is less than a preset amplitude threshold.

[0162] S538. Generate a three-dimensional mask and a three-dimensional model of the target object according to the target cue point set of each first image in the image data set.

[0163] Reference Figure 8 , Figure 8 This is a schematic diagram of the three-dimensional model and the three-dimensional mask obtained in the embodiment of the present application. As shown in the figure, in the three-dimensional model of the power grid, a three-dimensional mask about the target object can be obtained, and presented as a two-dimensional mask covering the target object in the two-dimensional image; and it can be seen that the two-dimensional mask has a high degree of fit with the target object, realizing the individualization of the target object, and providing an accurate data basis for power grid engineering planning.

[0164] Reference Fig. 9 , Fig. 9 A module diagram of a system for semantically singulating a three-dimensional model of a target object around a power grid provided in an embodiment of the present application. As shown in the figure, a system for semantically singulating a three-dimensional model of a target object around a power grid is also provided in this embodiment. The target object has an image data set in advance; the image data set includes multiple first images, and the first images correspond to an initial prompt point set; the system includes:

[0165] The initial inverse rendering module 100 is used to perform inverse rendering on the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image.

[0166] The prompt point set updating module 200 is used to project the first intermediate three-dimensional model at a specified viewing angle to obtain a second image at the specified viewing angle; use the second image to inversely render the first intermediate three-dimensional model to obtain a second intermediate three-dimensional model; and use the prompt point with the highest confidence score in the second intermediate three-dimensional model to update the initial prompt point set; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set.

[0167] The iterative updating module 300 is used to use the second intermediate 3D model as the first intermediate 3D model and repeatedly execute the above steps of updating the initial cue point set until the end condition is met to obtain the target cue point set of the first image.

[0168] The model generation module 400 is used to generate a 3D mask and a 3D model of a target object according to a target cue point set of each first image in the image data set.

[0169] In some optional implementations, the cue point set update module 200 includes:

[0170] The second mask acquisition unit is used to perform projection on the specified viewing angle according to the first intermediate three-dimensional model to obtain a second mask corresponding to the second image.

[0171] The second image acquisition unit is used to render the second mask according to the model features of the first intermediate three-dimensional model to obtain a second image.

[0172] In some optional implementations, the cue point set update module 200 further includes:

[0173] The mapping light acquisition unit is used to obtain the mapping light according to the viewing angle of the second image and the second mask; wherein the mapping light is located in the target space containing the first intermediate three-dimensional model.

[0174] The inverse rendering voxel determination unit is used to determine the inverse rendering voxel passed by the mapping light according to the position and direction of the mapping light in the target space.

[0175] The image inverse rendering unit is used to update the voxel attributes and confidence scores of the inversely rendered voxels according to the mapped light and using the neural radiation field to obtain a second intermediate three-dimensional model; wherein the voxel attributes include color and density, and the confidence score is obtained based on the voxel attributes.

[0176] In some optional implementations, the confidence score in the image inverse rendering unit includes an attenuation term, where the attenuation term is obtained based on the distance between each voxel in the second intermediate three-dimensional model and each cue point in the current initial cue point set.

[0177] In some optional implementations, the cue point set update module 200 further includes:

[0178] A target pixel determination unit, used to determine a target pixel belonging to the target object in the second image according to the voxel attribute size of each voxel in the second intermediate three-dimensional model; wherein the value of the voxel attribute corresponding to the target pixel is greater than the voxel attribute threshold;

[0179] A hint pixel determination unit, used to determine the target pixel with the highest confidence score as the hint pixel according to the confidence scores of the target pixels;

[0180] A hint voxel mapping unit, configured to perform mapping conversion in the second image using hint pixels based on a mapping relationship between each voxel in the second intermediate three-dimensional model and each pixel in the second image, to obtain hint voxels;

[0181] The cue point set updating unit is used to update the initial cue point set by taking the cue voxel as a new cue point.

[0182] In some optional implementations, the initial inverse rendering module 100 includes:

[0183] The first image acquisition unit is used to render the first image according to the initial three-dimensional model to obtain a rendered first image.

[0184] The first mask prompt unit is used to obtain a first mask prompt corresponding to the first image by performing image analysis on the first image.

[0185] The first mask acquisition unit is used to input the rendered first image and the first mask prompt into the segmentation any object model, segment the first image by segmenting the any object model, and obtain a first mask corresponding to the first image.

[0186] In some optional implementations, the initial inverse rendering module 100 further includes:

[0187] The feature extraction unit is used to extract features from the first image using a convolutional network to obtain first image features.

[0188] The feature enhancement unit is used to assign an attention weight to the first image feature by using an attention mechanism, and perform weighted calculation on the first image feature according to the attention weight to obtain an image enhancement feature.

[0189] The image segmentation unit is used to perform target detection based on the image enhancement feature to obtain the range of the target object, and obtain a first mask about the target object in the first image through image segmentation.

[0190] In some optional embodiments, the system further includes an initial model reconstruction module, which includes:

[0191] A network training unit is used to iteratively train the tensor radiation field according to the image dataset.

[0192] The model reconstruction unit is used to perform three-dimensional reconstruction of the target object using the tensor radiation field after iterative training to obtain an initial three-dimensional model.

[0193] In some optional implementations, the termination condition in the iterative update module 300 includes an iteration number condition and a confidence limit condition, wherein:

[0194] The iteration number condition is that the step is repeated a certain number of times.

[0195] The confidence limit condition is that in multiple consecutive step repetitions, the improvement of the confidence score of the second intermediate three-dimensional model relative to the first intermediate three-dimensional model is less than a preset amplitude threshold.

[0196] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0197] The three-dimensional model semantic monomer system of the target objects around the power grid in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0198] A computer device is also provided in the present embodiment, and the computer device includes: one or more processors, memory, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0199] The processor may be a central processing unit, a network processor or a combination thereof. The processor may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0200] The memory stores instructions executable by at least one processor, so that the at least one processor executes the method shown in the above embodiment.

[0201] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0202] The memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid state drive; the memory may also include a combination of the above types of memory.

[0203] The computer device also includes a communication interface, which is used for the computer device to communicate with other devices or a communication network.

[0204] The embodiment of the present application also provides a computer-readable storage medium. The above method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0205] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.

[0206] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

[0207] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0208] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0209] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0210] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0213] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0214] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0215] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

[0216] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for semantic monomerization of a three-dimensional model of a target object around a power grid, characterized in that: The target object has an image data set in advance; the image data set includes a plurality of first images, and the first images correspond to an initial prompt point set; The method comprises: Performing inverse rendering on the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image; Projecting and rendering the first intermediate 3D model at a specified viewing angle to obtain a second image at the specified viewing angle; inversely rendering the first intermediate 3D model using the second image to obtain a second intermediate 3D model; and updating the initial cue point set using the cue point with the highest confidence score in the second intermediate 3D model; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set; The second intermediate 3D model is used as the first intermediate 3D model, and the step of updating the initial cue point set is repeated until an end condition is satisfied, thereby obtaining a target cue point set for the first image; A three-dimensional mask and a three-dimensional model of the target object are generated according to the target cue point set of each first image in the image data set.

2. The method according to claim 1, characterized in that The projecting and rendering the first intermediate three-dimensional model at a specified viewing angle to obtain a second image at the specified viewing angle includes: Projecting the first intermediate three-dimensional model at the specified viewing angle, obtaining a second mask corresponding to the second image; The second mask is rendered according to the model features of the first intermediate three-dimensional model to obtain the second image.

3. The method according to claim 2, characterized in that The inverse rendering of the first intermediate 3D model by using the second image to obtain a second intermediate 3D model includes: Obtaining a mapping light according to the viewing angle of the second image and the second mask; wherein the mapping light is located in a target space including the first intermediate three-dimensional model; Determining, according to the position and direction of the mapping light in the target space, an inverse rendering voxel passed by the mapping light; According to the mapping light, the voxel attributes and confidence scores of the inversely rendered voxels are updated using the neural radiation field to obtain the second intermediate three-dimensional model; wherein the voxel attributes include color and density, and the confidence score is obtained based on the voxel attributes.

4. The method according to claim 3, characterized in that: The confidence score includes an attenuation term, and the attenuation term is obtained according to the distance between each voxel in the second intermediate three-dimensional model and each cue point in the current initial cue point set.

5. The method according to claim 1, characterized in that The updating of the initial cue point set by using the cue point with the highest confidence score in the second intermediate three-dimensional model comprises: Determining a target pixel in the second image belonging to the target object according to the voxel attribute size of each voxel in the second intermediate three-dimensional model; wherein the value of the voxel attribute corresponding to the target pixel is greater than a voxel attribute threshold; According to the confidence scores of the target pixels, determining the target pixel with the highest confidence score as a prompt pixel; Based on the mapping relationship between each voxel in the second intermediate three-dimensional model and each pixel in the second image, the hint pixel is used to perform mapping conversion in the second image to obtain the hint voxel; The initial cue point set is updated by taking the cue voxel as a new cue point.

6. The method according to claim 1, characterized in that Before performing inverse rendering on the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image, the method further includes: Rendering the first image according to the initial three-dimensional model to obtain the rendered first image; Obtaining a first mask prompt corresponding to the first image by performing image analysis on the first image; The rendered first image and the first mask prompt are input into a segmentation model for any object, and the first image is segmented by segmenting the segmentation model for any object to obtain a first mask corresponding to the first image.

7. The method according to claim 6, characterized in that The step of inputting the rendered first image and the first mask prompt into a segmentation model for any object, segmenting the first image by segmenting the any object model, and obtaining a first mask corresponding to the first image includes: Using a convolutional network to extract features from the first image to obtain first image features; Using an attention mechanism to assign an attention weight to the first image feature, and performing weighted calculation on the first image feature according to the attention weight to obtain an image enhancement feature; Target detection is performed based on the image enhancement feature to obtain the range of the target object, and image segmentation is performed on the first image according to the first mask prompt to obtain the first mask.

8. The method according to any one of claims 1 to 7, characterized in that The initial three-dimensional model is obtained according to the following method: Iteratively training the tensor radiation field according to the image data set; The target object is three-dimensionally reconstructed using the iteratively trained tensor radiation field to obtain the initial three-dimensional model.

9. The method according to any one of claims 1 to 7, characterized in that The end condition includes an iteration number condition and a confidence limit condition, wherein: The iteration number condition is that the step is repeated a certain number of times; The confidence limit condition is that in a plurality of consecutive steps, the improvement of the confidence score of the second intermediate three-dimensional model relative to the first intermediate three-dimensional model is less than a preset amplitude threshold.

10. A three-dimensional model semantic monomer system for target objects around a power grid, characterized in that: The target object has an image data set in advance; the image data set includes a plurality of first images, and the first images correspond to an initial prompt point set; The system comprises: an initial inverse rendering module, configured to inversely render the initial three-dimensional model of the target object based on the first image to obtain a first intermediate three-dimensional model corresponding to the first image; a cue point set updating module, configured to project the first intermediate 3D model at a specified viewing angle to obtain a second image at the specified viewing angle; perform inverse rendering on the first intermediate 3D model using the second image to obtain a second intermediate 3D model; and update the initial cue point set using the cue point with the highest confidence score in the second intermediate 3D model; wherein the specified viewing angle is different from the imaging viewing angle of any image in the image data set; an iterative updating module, configured to use the second intermediate 3D model as the first intermediate 3D model, and repeatedly execute the above steps of updating the initial cue point set until an end condition is met, thereby obtaining a target cue point set for the first image; The model generation module is used to generate a three-dimensional mask and a three-dimensional model of the target object according to the target cue point set of each first image in the image data set.

11. The system according to claim 10, characterized in that The cue point set update module comprises: a second mask acquiring unit, configured to obtain a second mask corresponding to the second image by projecting the first intermediate three-dimensional model at the specified viewing angle; The second image acquisition unit is used to render the second mask according to the model features of the first intermediate three-dimensional model to obtain the second image.

12. The system according to claim 11, characterized in that The cue point set update module also includes: a mapping light acquisition unit, configured to obtain mapping light according to the viewing angle of the second image and the second mask; wherein the mapping light is located in a target space including the first intermediate three-dimensional model; An inverse rendering voxel determining unit, used to determine the inverse rendering voxel passed by the mapping light according to the position and direction of the mapping light in the target space; An image inverse rendering unit is used to update the voxel attributes and confidence scores of the inversely rendered voxels according to the mapping light and using the neural radiation field to obtain the second intermediate three-dimensional model; wherein the voxel attributes include color and density, and the confidence score is obtained based on the voxel attributes.

13. The system according to claim 12, characterized in that The confidence score includes an attenuation term, and the attenuation term is obtained according to the distance between each voxel in the second intermediate three-dimensional model and each cue point in the current initial cue point set.

14. The system according to claim 10, characterized in that The cue point set update module also includes: a target pixel determination unit, configured to determine a target pixel in the second image belonging to the target object according to a voxel attribute value of each voxel in the second intermediate three-dimensional model; wherein a value of a voxel attribute corresponding to the target pixel is greater than a voxel attribute threshold; a hint pixel determination unit, configured to determine, according to the confidence scores of the target pixels, a target pixel with the highest confidence score as a hint pixel; a hint voxel mapping unit, configured to perform mapping conversion in the second image using the hint pixel based on a mapping relationship between each voxel in the second intermediate three-dimensional model and each pixel in the second image, so as to obtain a hint voxel; The cue point set updating unit is used to update the initial cue point set by taking the cue voxel as a new cue point.

15. The system according to claim 10, characterized in that The initial inverse rendering module includes: A first image acquisition unit, configured to render the first image according to the initial three-dimensional model to obtain the rendered first image; A first mask prompt unit, configured to obtain a first mask prompt corresponding to the first image by performing image analysis on the first image; The first mask acquisition unit is used to input the rendered first image and the first mask prompt into a segmentation any object model, segment the first image by segmenting the any object model, and obtain a first mask corresponding to the first image.

16. The system according to claim 15, characterized in that The initial inverse rendering module further includes: A feature extraction unit, configured to extract features from the first image using a convolutional network to obtain first image features; a feature enhancement unit, configured to assign an attention weight to the first image feature by using an attention mechanism, and perform weighted calculation on the first image feature according to the attention weight to obtain an image enhancement feature; An image segmentation unit is used to perform target detection based on the image enhancement feature to obtain the range of the target object, and perform image segmentation on the first image according to the first mask prompt to obtain the first mask.

17. The system according to any one of claims 10 to 16, characterized in that The system also includes an initial model reconstruction module, which includes: A network training unit, used for iteratively training the tensor radiation field according to the image data set; The model reconstruction unit is used to perform three-dimensional reconstruction of the target object using the tensor radiation field after iterative training to obtain the initial three-dimensional model.

18. The system according to any one of claims 10 to 16, characterized in that The termination conditions in the iterative update module include the number of iterations condition and the confidence limit condition, where: The iteration number condition is that the step is repeated a certain number of times; The confidence limit condition is that in a plurality of consecutive steps, the improvement of the confidence score of the second intermediate three-dimensional model relative to the first intermediate three-dimensional model is less than a preset amplitude threshold.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for semantic monomerization of a three-dimensional model of a target object surrounding a power grid according to any one of claims 1 to 9.

Images