Converter station digital twin model construction method and device and computer equipment

Abstract

The invention relates to a digital twin model construction method and device of a converter station and computer equipment. The method comprises the following steps: processing a visible light image of a converter station by adopting a motion recovery structure model and a multi-view stereoscopic model to obtain dense point cloud data of the converter station; fusing the dense point cloud data and the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station; constructing a grid model of the converter station according to the geometric point cloud data; according to the visible light image and the grid model, obtaining a visible light texture mapping relation corresponding to the visible light image, and according to the visible light image, performing registration processing on other multi-source images of the converter station to obtain other texture mapping relations corresponding to the other multi-source images; and obtaining a multi-source texture digital twin model of the converter station according to the grid model, the visible light texture mapping relation and other texture mapping relations. By adopting the method, the precision of the digital twin model of the converter station can be improved.

Description

Technical Field

[0001] The present application relates to the field of digital twin technology, and in particular to a method, device, computer equipment, computer-readable storage medium, and computer program product for constructing a digital twin model of a converter station. Background Art

[0002] In the historical context of the rapid development of digital technology and the intersection of the "dual carbon" strategic goals, power companies, as important energy hubs, should actively use digital tools to achieve green transformation from multiple dimensions such as technology, business, and strategy. As an emerging technology, digital twin technology empowers smart grids and provides more real-time, efficient, and intelligent service value.

[0003] However, at present, in complex environments and when the surface features of equipment are not obvious, it is impossible to accurately and comprehensively capture the three-dimensional structure and surface features of converter station equipment. As a result, the digital twin model is insufficiently accurate in some cases and cannot meet the needs of high-precision and full-coverage modeling. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for constructing a digital twin model of a converter station, which can improve the accuracy of the digital twin model of the converter station in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for constructing a digital twin model of a converter station, comprising:

[0006] Using a structure-from-motion model and a multi-view stereo model, the visible light image of the converter station is processed to obtain dense point cloud data of the converter station;

[0007] fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station;

[0008] Constructing a grid model of the converter station according to the geometric point cloud data;

[0009] Obtaining a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and performing registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images;

[0010] According to the grid model, the visible light texture mapping relationship and the other texture mapping relationships, a multi-source texture digital twin model of the converter station is obtained.

[0011] In one embodiment, the visible light image of the converter station is processed using the motion recovery structure model and the multi-view stereo model to obtain dense point cloud data of the converter station, including:

[0012] Using the motion recovery structure model, perform feature matching processing on the visible light image to obtain shooting posture information corresponding to the visible light image and sparse point cloud data of the converter station;

[0013] The visible light image, the shooting posture information and the sparse point cloud data are input into the multi-view stereo model to obtain dense point cloud data of the converter station.

[0014] In one embodiment, constructing a grid model of the converter station according to the geometric point cloud data includes:

[0015] Performing surface reconstruction processing on the geometric point cloud data to obtain an initial grid model of the converter station;

[0016] The quadratic error metric simplified model is adopted to simplify the vertices in the initial mesh model to obtain a simplified mesh model as the mesh model of the converter station.

[0017] In one embodiment, the method of simplifying the model using a quadratic error metric to simplify the vertices in the initial mesh model to obtain a simplified mesh model includes:

[0018] Taking the initial grid model as the current iterative grid model, calculating the quadratic error metric value of each edge of the current iterative grid model;

[0019] Determine a target edge of the quadratic error metric value, and merge two vertices corresponding to the target edge into one vertex to obtain a new current iterative mesh model;

[0020] Return to the step of calculating the quadratic error metric value of each edge of the current iterative mesh model until the new current iterative mesh model reaches a preset complexity target, and use the new current iterative mesh model as the simplified mesh model.

[0021] In one embodiment, the registering process is performed on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images, including:

[0022] Extracting image feature points from the visible light image and the other multi-source images, and performing feature point matching processing on the image feature points using a scale-invariant feature transformation model to obtain matching feature points;

[0023] Calculating a spatial transformation matrix between the other multi-source images and the visible light image according to the matching feature points;

[0024] According to the spatial transformation matrix, the textures in the other multi-source images are mapped onto the grid model to obtain other texture mapping relationships corresponding to the other multi-source images.

[0025] In one of the embodiments, the other multi-source images include a plurality of other multi-source images, each of which includes a plurality of candidate images;

[0026] Mapping the textures in the other multi-source images onto the grid model according to the spatial transformation matrix to obtain other texture mapping relationships corresponding to the other multi-source images includes:

[0027] For any other multi-source image, determining the matching degree between each candidate image and the visible light image according to the spatial transformation matrix of each candidate image in the any other multi-source image;

[0028] Determining a target matching image of any other multi-source image from candidate images of any other multi-source image according to the matching degree;

[0029] According to the spatial transformation matrix of the target matching image, the texture in the target matching image is mapped onto the grid model to obtain other texture mapping relationships corresponding to any other multi-source image.

[0030] In a second aspect, the present application also provides a device for constructing a digital twin model of a converter station, comprising:

[0031] A dense point cloud acquisition module is used to process the visible light image of the converter station by using a motion recovery structure model and a multi-view stereo model to obtain dense point cloud data of the converter station;

[0032] A point cloud data fusion module, used for fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station;

[0033] A grid model building module, used to build a grid model of the converter station according to the geometric point cloud data;

[0034] A texture mapping acquisition module, configured to obtain a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and to perform registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images;

[0035] A digital twin construction module is used to obtain a multi-source texture digital twin model of the converter station based on the grid model, the visible light texture mapping relationship and the other texture mapping relationships.

[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] Using a structure-from-motion model and a multi-view stereo model, the visible light image of the converter station is processed to obtain dense point cloud data of the converter station;

[0038] fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station;

[0039] Constructing a grid model of the converter station according to the geometric point cloud data;

[0040] Obtaining a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and performing registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images;

[0041] According to the grid model, the visible light texture mapping relationship and the other texture mapping relationships, a multi-source texture digital twin model of the converter station is obtained.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0043] Using a structure-from-motion model and a multi-view stereo model, the visible light image of the converter station is processed to obtain dense point cloud data of the converter station;

[0044] fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station;

[0045] Constructing a grid model of the converter station according to the geometric point cloud data;

[0046] Obtaining a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and performing registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images;

[0047] According to the grid model, the visible light texture mapping relationship and the other texture mapping relationships, a multi-source texture digital twin model of the converter station is obtained.

[0048] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0049] Using a structure-from-motion model and a multi-view stereo model, the visible light image of the converter station is processed to obtain dense point cloud data of the converter station;

[0050] fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station;

[0051] Constructing a grid model of the converter station according to the geometric point cloud data;

[0052] Obtaining a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and performing registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images;

[0053] According to the grid model, the visible light texture mapping relationship and the other texture mapping relationships, a multi-source texture digital twin model of the converter station is obtained.

[0054] The digital twin model construction method, device, computer equipment, computer-readable storage medium and computer program product of the converter station first use the motion recovery structure model and the multi-view stereo model to process the visible light image of the converter station to obtain dense point cloud data of the converter station. The motion recovery structure model and the multi-view stereo model are used to generate dense point cloud data from the multi-view visible light images, which makes up for the shortcomings of laser scanning technology in capturing details in weak texture areas, and provides high-resolution geometric structure and surface texture information, laying a solid foundation for the subsequent fusion of point cloud data. More three-dimensional information is captured in complex environments; then, the dense point cloud data and the laser scanning point cloud data of the converter station are fused to obtain the geometric point cloud data of the converter station. By fusing the dense point cloud data and the laser scanning point cloud data, the advantages of the two data are complemented. The high-precision geometric information of the laser scanning point cloud is used, and the surface texture details provided by the dense point cloud are combined to generate geometric point cloud data with complete geometric shapes and precise surface features, providing high-quality input for the construction of the grid model; then, the grid model of the converter station is constructed based on the geometric point cloud data. The surface is reconstructed to generate the grid model of the converter station, and the discrete points are converted into continuous surfaces, so that the geometric information of the model is more structured and smooth, which is convenient for subsequent texture mapping and visualization, and at the same time improves the availability and rendering efficiency of the model; then, according to the visible light image and the grid model, the visible light texture mapping relationship corresponding to the visible light image is obtained, and according to the visible light image, other multi-source images of the converter station are registered to obtain other texture mapping relationships corresponding to other multi-source images. By generating the texture mapping relationship of the visible light image, the grid model is endowed with real texture information. At the same time, by registering the multi-source images, the spatial alignment and consistency of the multi-source texture information are ensured, so that the digital twin model has the ability to display multi-dimensional information and meet the application needs of more scenarios; finally, according to the grid model, the visible light texture mapping relationship and other texture mapping relationships, the multi-source texture digital twin model of the converter station is obtained, and the grid model, visible light texture and other multi-source textures are fused to generate a multi-source texture digital twin model with high-precision geometric shape and multi-dimensional information, which provides precise support for the operation monitoring, fault diagnosis and intelligent management of the converter station, and realizes the multi-dimensional visualization of the global information of the converter station. In the above method, a multi-source texture digital twin model of the converter station is constructed by integrating the motion recovery structure model, multi-view stereo model and laser scanning technology, combined with point cloud data processing, surface reconstruction and multi-source image texture mapping methods. This model has high-precision geometric structure and multi-dimensional attribute display capabilities, and can fully capture the three-dimensional form and surface features of the converter station equipment, solving the problem of insufficient accuracy of the digital twin model in complex environments and when the surface features of the equipment are not obvious.At the same time, the comprehensive use of multi-source data enhances the authenticity of the model and its ability to integrate multi-dimensional information, providing precise technical support for applications such as operation monitoring, fault diagnosis, and predictive maintenance of converter stations, and significantly improving the digitalization and intelligence level of smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A schematic diagram of a process of constructing a digital twin model of a converter station in an embodiment;

[0057] Figure 2 A schematic flow chart of steps for constructing a grid model of a converter station in one embodiment;

[0058] Figure 3 A schematic diagram of the distance from a vertex to a plane in a mesh model in one embodiment;

[0059] Figure 4 A simplified schematic diagram of mesh model vertices in one embodiment;

[0060] Figure 5 A schematic diagram of a three-dimensional laser scanner measuring a spatial structure in one embodiment;

[0061] Figure 6 A schematic diagram of a flow chart of a method for spatial synthesis and three-dimensional display of multi-source data of a converter station in one embodiment;

[0062] Figure 7 A schematic diagram of a process of reconstructing a grid model of a converter station in one embodiment;

[0063] Figure 8 is a schematic diagram of a camera epipolar geometry model in one embodiment;

[0064] Fig. 9 A schematic diagram of multi-source texture mapping of a converter station model in one embodiment;

[0065] Fig.10 A structural block diagram of a digital twin model building device for a converter station in one embodiment;

[0066] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] In one embodiment, Figure 1 As shown, a method for constructing a digital twin model of a converter station is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, and tablet computers. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. In this embodiment, the method includes the following steps:

[0069] Step S101, using a structure-from-motion model and a multi-view stereo model to process a visible light image of a converter station to obtain dense point cloud data of the converter station.

[0070] Among them, Structure from Motion (SfM) is a computer vision technology that recovers the three-dimensional structure of an object by extracting key feature points from images of different perspectives and calculating the camera's pose; Multi-View Stereo (MVS) is a technology that generates dense point clouds by using images from multiple perspectives. It calculates the parallax between images to achieve high-precision three-dimensional point cloud reconstruction.

[0071] Exemplarily, first, the terminal collects multiple visible light images of the converter station. Through the motion recovery structure model, feature points are extracted from visible light images of multiple perspectives, and the camera pose information of each image is calculated by matching these feature points. Using this pose information, the terminal can determine the spatial position of each image relative to the object, and infer the three-dimensional structure of the converter station through geometric relationships. Next, the terminal uses a multi-view stereo model, uses image information obtained from different perspectives, and calculates the parallax between images to generate denser point cloud data. These dense point cloud data accurately reflect the geometric features of the converter station and provide rich three-dimensional data as the basis for subsequent geometric modeling.

[0072] Step S102, fusing the dense point cloud data and the laser scanning point cloud data of the converter station to obtain the geometric point cloud data of the converter station.

[0073] Among them, dense point cloud data is three-dimensional point cloud data generated from visible light images through a multi-view stereo model. It usually contains a large number of points, each of which represents a physical location in the image; laser scanning point cloud data is point cloud data collected directly from the surface of the converter station by a laser scanner. It has the characteristics of high precision and high density, and can accurately capture the geometric shape of the equipment.

[0074] Exemplarily, first, the terminal performs denoising and cropping on the dense point cloud data and the laser scanning point cloud data, removes redundant points, noise points and point cloud data irrelevant to the converter station equipment in the two data, and ensures the purity of the point cloud data. The laser scanning point cloud data is converted into a format (for example, the data is converted into a universal point cloud format) so as to be compatible with the dense point cloud data. Next, the terminal uses an initial alignment method based on feature points to extract feature points (such as corner points and edge points on the surface) in the dense point cloud and the laser scanning point cloud, and uses the point-to-point matching relationship to preliminarily align the two groups of point clouds. Then, the dense point cloud and the laser scanning point cloud are accurately aligned by an iterative nearest point algorithm, so that the two data can overlap with high precision in the same coordinate system. The terminal performs weighted fusion of the dense point cloud and the laser scanning point cloud according to the spatial overlapping area of ​​the point cloud. The laser scanning point cloud is used to provide high-precision geometric information, while the dense point cloud supplements the surface details and texture features. In the non-overlapping area, the terminal directly retains the respective information of the two point cloud data to ensure the integrity and coverage of the point cloud. After the fusion is completed, the terminal generates unified geometric point cloud data. This data not only has the high-precision geometric characteristics of the laser scanning point cloud, but also combines the surface texture details of the dense point cloud data to provide richer three-dimensional information. Finally, the terminal simplifies and optimizes the fused point cloud, removes redundant point clouds and readjusts the point cloud density to provide lightweight and high-quality point cloud data for subsequent steps.

[0075] Step S103: constructing a grid model of the converter station according to the geometric point cloud data.

[0076] Among them, the mesh model is a three-dimensional model composed of a series of adjacent triangular faces, which can accurately represent the shape characteristics of the object and is often used in applications such as computer graphics, visualization, and 3D printing.

[0077] Exemplarily, the terminal first uses geometric point cloud data to reconstruct the surface. Through triangulation technology based on point cloud data (such as Delaunay triangulation), the discrete point cloud data is converted into a continuous three-dimensional mesh. During the surface reconstruction process, the terminal generates triangular facets for each group of adjacent points in the point cloud, thereby constructing a rough mesh model of the device. The mesh model can roughly reflect the geometric shape of the device surface. After generating the preliminary mesh model, the terminal refines it through a mesh optimization algorithm. This includes removing unnecessary facets and vertices, reducing the number of facets, and eliminating the roughness caused by point cloud noise or irregularities through mesh smoothing. The optimization process uses algorithms such as edge simplification and vertex merging to ensure that the structure of the mesh is not affected while reducing the computational burden and improving subsequent rendering efficiency. For the surface details of the device, the terminal combines the point cloud density information and surface texture to enhance the details of the mesh model, retaining areas with higher surface complexity and simplifying simpler areas. In this way, the mesh model maintains the rich details of the device shape while ensuring geometric accuracy, which is suitable for high-precision three-dimensional modeling needs. The generated mesh model is closed and repaired to eliminate possible holes, overlapping surfaces or disconnected areas to ensure the integrity of the mesh. During the mesh repair process, the terminal uses local smoothing and topology optimization methods to ensure that the mesh model can adapt to subsequent texture mapping and physical simulation requirements while maintaining its original shape.

[0078] Step S104, obtaining a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and performing registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to other multi-source images.

[0079] Among them, the visible light texture mapping relationship refers to the relationship of mapping the texture information of the visible light image to the three-dimensional mesh model, which is usually achieved by calculating the spatial transformation matrix between the UV coordinates of the mesh model and the image; other multi-source images refer to other image types besides visible light images, such as infrared images, ultraviolet images or depth images, which provide additional physical information (such as temperature distribution, electrical state, etc.), and the texture mapping relationship is the process of spatially aligning the texture information of these multi-source images with the three-dimensional model.

[0080] Exemplarily, the terminal first maps the texture information of the visible light image to the mesh model using the spatial transformation matrix of the image based on the registration of the mesh model and the visible light image. Specifically, the terminal calculates the corresponding UV coordinates for each mesh vertex and uses these coordinates to project the texture information in the image onto the three-dimensional mesh to form a texture mapping relationship corresponding to the visible light image. Through texture mapping, the terminal can give the mesh model more realistic surface details to ensure that the surface color and texture of the device match the actual situation. The terminal then uses image registration technology to register other multi-source images (such as infrared, ultraviolet images, etc.) based on the texture mapping relationship between the visible light image and the mesh model. The registration process first extracts feature points in the multi-source image and matches them with the corresponding feature points in the visible light image, and achieves alignment between images by calculating the spatial transformation matrix (such as the perspective transformation matrix). During this process, the terminal may use feature matching algorithms such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), or Oriented FAST and Rotated BRIEF (ORB) to accurately align multi-source images and visible light images to ensure spatial consistency of different types of images. After the registration is completed, the terminal maps the texture information of other multi-source images to the grid model through the calculated spatial transformation matrix. These images may contain different types of physical data (such as temperature distribution, discharge area, etc.). Through texture mapping, the terminal can superimpose this information on the corresponding area of ​​the grid model to generate other texture mapping relationships. During the texture mapping process, the terminal ensures the alignment accuracy between different types of images, avoids texture overlap, distortion or seam problems, and enables the texture information of multi-source images to be seamlessly integrated into the three-dimensional grid model.

[0081] Step S105, obtaining a multi-source texture digital twin model of the converter station according to the grid model, the visible light texture mapping relationship and other texture mapping relationships.

[0082] Exemplarily, the terminal determines which types of texture information need to be mapped to the grid model according to the needs of the user. For example, if the user is concerned about the temperature distribution of the equipment, the texture mapping relationship of the infrared image is selected; if the surface of the equipment or electrical activity needs to be checked, the texture mapping relationship of the visible light image or ultraviolet image is selected. The user can select the appropriate texture type and mapping area according to the specific application scenario or monitoring requirements. The terminal maps the selected texture image to the grid model through the spatial transformation matrix according to the texture type and corresponding area specified by the user. The texture mapping of each area is independent, and only the corresponding image is selected for mapping according to the user's requirements to avoid unnecessary texture redundancy. In this process, the terminal will register each image to ensure the consistency of different texture images and the grid model in spatial coordinates, and complete the accurate mapping of textures in the selected area. The terminal combines the generated texture mapping relationship with the grid model to obtain a target texture model. For example, the user only needs to map the temperature distribution map to the electrical equipment area of ​​the converter station, or apply the visible light image texture to the surface of the external structure, without merging all textures into a single model. Through this on-demand mapping, the generated digital twin model is more in line with the actual application needs and can provide customized texture data display for different users. The terminal optimizes the texture mapping results to ensure the alignment accuracy of the selected texture on the mesh model, and adjusts the lighting effect and contrast of the texture to improve the visual effect of the final model. In addition, the terminal will also perform necessary texture gap repair to avoid seams and distortion caused by different texture mappings, ensuring the realism and accuracy of the model.

[0083] In the digital twin model construction method of the above converter station, firstly, the visible light image of the converter station is processed by using the motion recovery structure model and the multi-view stereo model to obtain the dense point cloud data of the converter station. The dense point cloud data is generated from the visible light image of multiple viewpoints by using the motion recovery structure model and the multi-view stereo model, which makes up for the deficiency of laser scanning technology in capturing details in weak texture areas, and provides high-resolution geometric structure and surface texture information, laying a solid foundation for the subsequent fusion of point cloud data, and also capturing more three-dimensional information in complex environments; then , the dense point cloud data and the laser scanning point cloud data of the converter station are integrated to obtain the geometric point cloud data of the converter station. By integrating the dense point cloud data and the laser scanning point cloud data, the advantages of the two data are complemented. The high-precision geometric information of the laser scanning point cloud is used, and the surface texture details provided by the dense point cloud are combined to generate geometric point cloud data with complete geometric shapes and precise surface features, providing high-quality input for the construction of the mesh model; then, the mesh model of the converter station is constructed according to the geometric point cloud data, and the mesh model of the converter station is generated by surface reconstruction of the geometric point cloud data. The model is constructed by converting discrete points into continuous surfaces, making the geometric information of the model more structured and smooth, which is convenient for subsequent texture mapping and visualization, and at the same time improves the usability and rendering efficiency of the model; then, according to the visible light image and the mesh model, the visible light texture mapping relationship corresponding to the visible light image is obtained, and according to the visible light image, other multi-source images of the converter station are registered to obtain other texture mapping relationships corresponding to other multi-source images. By generating the texture mapping relationship of the visible light image, the mesh model is endowed with real texture information. At the same time, the spatial alignment and consistency of the multi-source texture information are ensured through the registration processing of the multi-source images, so that the digital twin model has the ability to display multi-dimensional information and meets the application requirements of more scenarios; finally, according to the mesh model, the visible light texture mapping relationship and other texture mapping relationships, the multi-source texture digital twin model of the converter station is obtained, and the mesh model, visible light texture and other multi-source textures are fused to generate a multi-source texture digital twin model with high-precision geometric shape and multi-dimensional information, which provides precise support for the operation monitoring, fault diagnosis and intelligent management of the converter station, and realizes the multi-dimensional visualization of the global information of the converter station. In the above method, a multi-source texture digital twin model of the converter station is constructed by integrating the motion recovery structure model, multi-view stereo model and laser scanning technology, combined with point cloud data processing, surface reconstruction and multi-source image texture mapping methods. This model has high-precision geometric structure and multi-dimensional attribute display capabilities, and can fully capture the three-dimensional form and surface features of the converter station equipment, solving the problem of insufficient accuracy of the digital twin model in complex environments and when the surface features of the equipment are not obvious.At the same time, the comprehensive use of multi-source data enhances the authenticity of the model and its ability to integrate multi-dimensional information, providing precise technical support for applications such as operation monitoring, fault diagnosis, and predictive maintenance of converter stations, and significantly improving the digitalization and intelligence level of smart grids.

[0084] In an exemplary embodiment, the above-mentioned step S101 uses a motion recovery structure model and a multi-view stereo model to process the visible light image of the converter station to obtain dense point cloud data of the converter station, and also includes: using the motion recovery structure model to perform feature matching processing on the visible light image to obtain the shooting posture information corresponding to the visible light image and the sparse point cloud data of the converter station; inputting the visible light image, the shooting posture information and the sparse point cloud data into the multi-view stereo model to obtain the dense point cloud data of the converter station.

[0085] Exemplarily, the terminal first uses a motion recovery structure model to process the visible light image. By extracting feature points from images of multiple perspectives and matching these feature points, the terminal calculates the shooting pose information of each image (i.e., the spatial position of each image relative to the object). In this process, the terminal estimates the camera pose of the image, generates the internal and external parameters of the camera, and uses these parameters to calculate the position of the feature points in the image in three-dimensional space to obtain sparse point cloud data. These sparse point cloud data are three-dimensional point sets calculated based on the matching of feature points between images, representing the basic geometric shape of the object. Then, the terminal inputs the above-mentioned visible light image, shooting pose information and sparse point cloud data into a multi-view stereo model (MVS), and the terminal further calculates the disparity between images through the model to generate more accurate dense point cloud data. Through multi-view stereo technology, the terminal not only uses the overlapping areas between different perspectives to obtain depth information, but also obtains the three-dimensional coordinates of each pixel through disparity calculation to generate a more detailed three-dimensional point cloud data set, which represents the three-dimensional structure of the converter station equipment. The terminal post-processes the generated dense point cloud data, removes noise points and smoothes the point cloud, thereby improving the accuracy and integrity of the point cloud data.

[0086] In this embodiment, the visible light image is processed by using a motion recovery structure model and a multi-view stereo model, the image features are extracted, the shooting posture is calculated, and sparse point cloud data is generated. Subsequently, the multi-view stereo model is used for further processing to obtain fine dense point cloud data. The image data and camera posture information from different perspectives are effectively combined to improve the accuracy and integrity of the point cloud data. As a result, the three-dimensional structure of the converter station can be reconstructed with high precision, solving the problem of modeling accuracy when the surface features of the equipment are not obvious in complex environments, and providing reliable data support for the construction of subsequent mesh models and the generation of digital twin models.

[0087] In an exemplary embodiment, Figure 2 As shown, the above step S103 constructs a grid model of the converter station according to the geometric point cloud data, which can also be achieved by the following steps:

[0088] Step S201, performing surface reconstruction processing on the geometric point cloud data to obtain an initial grid model of the converter station;

[0089] Step S202, using a quadratic error metric simplified model to simplify the vertices in the initial mesh model, and obtaining a simplified mesh model as the mesh model of the converter station.

[0090] Among them, the quadratic error metric simplification model is a mesh simplification technology that selects vertices to be simplified by calculating the impact of mesh simplification on the geometric shape.

[0091] Exemplarily, the terminal first performs surface reconstruction on the geometric point cloud data. Specifically, the terminal uses the Delaunay triangulation algorithm or other suitable triangulation methods to convert the point cloud data into a preliminary three-dimensional mesh model. This process converts the sparse point cloud data into a preliminary mesh structure by connecting adjacent points in the point cloud to form triangular facets. The initially generated mesh model reflects the geometric structure and surface morphology of the device well through the surface reconstruction method, but due to the density and noise problems of the point cloud data, the initial mesh model may have redundant facets or irregular structures, which are suitable for subsequent optimization processing. The terminal then uses the quadratic error metric simplification model (Quadric Error Metrics, QEM) to simplify the vertices in the initial mesh model. The QEM model calculates the simplification error of each mesh vertex to evaluate the impact of the simplification operation on the mesh geometry. In this process, the terminal calculates the quadratic error metric value of each mesh vertex to indicate the degree of change in the mesh geometry when simplifying the vertex. By selecting the vertex with the smallest error metric for simplification, merging adjacent vertices, and reducing unnecessary vertices and facets, the structure of the mesh is optimized. During the simplification process, the terminal also repairs and optimizes the mesh model to eliminate problems caused by point cloud noise and simplification operations. For example, it repairs possible mesh holes, corrects disconnected parts, smoothes mesh surfaces, etc., to ensure the continuity and integrity of the mesh. After surface reconstruction and quadratic error metric simplification, the mesh model finally generated by the terminal achieves a good balance between geometric accuracy, computational efficiency, and visualization effects. This mesh model not only accurately reflects the geometric shape and surface features of the converter station, but also has higher rendering efficiency and smaller storage space due to simplification and optimization.

[0092] In this embodiment, a preliminary mesh model is generated using point cloud data through surface reconstruction processing, which can accurately represent the three-dimensional geometric shape of the converter station. Subsequently, the quadratic error metric simplification model is applied to optimize the vertices in the initial mesh model, and the vertices with the least impact on the geometric shape are selected for simplification, which greatly reduces redundant data and improves the storage efficiency and computing performance of the mesh.

[0093] In an exemplary embodiment, the above-mentioned step S202 adopts a quadratic error metric simplification model to simplify the vertices in the initial mesh model to obtain a simplified mesh model, and also includes: taking the initial mesh model as the current iterative mesh model, calculating the quadratic error metric value of each edge of the current iterative mesh model; determining the target edge of the quadratic error metric value, and merging the two vertices corresponding to the target edge into one vertex to obtain a new current iterative mesh model; returning to execute the step of calculating the quadratic error metric value of each edge of the current iterative mesh model until the new current iterative mesh model reaches the preset complexity target, and taking the new current iterative mesh model as the simplified mesh model.

[0094] Exemplarily, the terminal uses the initial mesh model as the current iterative mesh model and starts to simplify it. The initial mesh model is obtained by surface reconstruction and contains the geometric shape and structural information of the converter station equipment, but it has not been simplified and optimized, and there are problems of redundant data and high computational complexity. In each iteration, the terminal calculates the quadratic error metric value of each edge in the current iterative mesh model. The quadratic error metric value is used to measure the impact of simplifying a certain edge on the mesh geometry. The smaller the error metric value, the smaller the geometric change after the edge is simplified, and the smaller the impact on the overall model. The terminal traverses each edge of the current mesh model, calculates the quadratic error metric value of each edge, and obtains the measure of the impact of each edge on the model. The terminal determines the target edge based on the calculated quadratic error metric value. The target edge refers to the edge that can bring the minimum error when simplified. Usually, the edge with the smallest error metric value is selected for simplification. For the target edge, the terminal merges its two vertices into one vertex, reduces redundant points and faces by merging vertices, and optimizes the structure of the mesh model. After merging the vertices, the surrounding patches are recalculated and the positions of the vertices are adjusted to ensure that the simplified mesh maintains a good geometric shape. After completing the simplification of the target edge, the terminal reconnects the merged new vertices with other vertices to generate a new current iterative mesh model. The new mesh model is optimized in terms of the number of vertices and patches, and its complexity is reduced compared to the initial mesh model. The terminal continues to perform quadratic error metric value calculation and simplification operations on the new iterative mesh model until the new mesh model reaches the predetermined complexity target. This target may be a preset number of vertices, number of patches, or a certain standard for computational efficiency and rendering effect. When the current iterative mesh model meets the preset complexity target, the terminal stops iterating and uses the new current iterative mesh model as the final simplified mesh model. The simplified mesh model has fewer vertices and patches, and the storage and computational efficiency are significantly improved, while maintaining a high geometric accuracy, which is suitable for subsequent texture mapping, digital twin modeling, simulation and other applications.

[0095] Specifically, by calculating a metric for each edge on the grid, these edges are placed in a minimum heap. Each iteration removes the value with the smallest metric in the heap, then recalculates the metric values ​​of other edges directly connected to the vertex of this edge, and updates the minimum heap until the pre-set threshold is reached. For example, when the number of vertices reaches a certain number, how to determine the metric value, thereby determining which edge to choose for collapse, is a key part of the QEM simplification technology. When the collapsed edge is shrunk to a certain point, the smaller the difference between the new face and the original face, the better the effect. This difference can be defined as the sum of the squares of the distances from the vertex to the original adjacent faces. If the collapsed edge does not shrink, the distances from the two vertices to the adjacent faces of the two vertices are both 0, and the difference is also the smallest. After shrinkage, the two points become one point, and the distance from the new point to the old face is used instead of calculating the difference between the new face and the old face. The difference is the sum of the squares of the distances from the new point to the adjacent faces of the original two points. Figure 3 As shown in the figure, it is a schematic diagram of the distance from the vertex to the plane in the mesh model. Therefore, this metric value can be defined as the minimum sum of the squares of the distances from a new point to the original adjacent faces after an edge is contracted. An edge can be contracted to many points on this edge, and we use the minimum value as the metric value of this edge. Figure 4 The simplified schematic diagram of the mesh model vertices shown in the figure shows that there is an edge with two vertices v 1 , v 2 , the vertex after contraction is v 0 , and its measurement error Δ is:

[0096]

[0097] Where p = [a, b, c, d] T Represents a triangle collection, planes (v i ) indicates that the edge (v i , v j ) associated with a set of triangles, and there exists:

[0098]

[0099] From the above formula, we can see that this measurement error is mainly determined by two parts: the new point v 0 The position of , and the sum of the quadratic basic error matrices of the adjacent planes of the two vertices of the edge, is defined as Q: .

[0100] The Q of each vertex can be calculated first, and the new point v 0 The position is at v 1 , v 2 and (v 1 +v 2 ) / 2 and select one that makes the contraction cost Δ(v0 ) minimum position.

[0101] In this way, the number of vertices of the model can be significantly reduced while keeping the appearance of the model almost unchanged, thereby achieving the goal of lightweight 3D models and improving rendering efficiency and loading speed.

[0102] In this embodiment, by adopting the quadratic error metric simplification model to simplify the mesh, the redundant data in the mesh model is effectively reduced, and the storage and computing efficiency is improved. By gradually simplifying each edge and merging redundant vertices, the optimized mesh model not only reduces the computational burden, but also maintains good geometric shape and structural accuracy. On the basis of ensuring accuracy, the rendering efficiency of the mesh is significantly improved, which is suitable for subsequent texture mapping, digital twin model construction, physical simulation, intelligent monitoring and other application scenarios, providing strong support for the digital modeling and intelligent management of the converter station.

[0103] In an exemplary embodiment, the above-mentioned step S104 performs registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images, and also includes: extracting image feature points from the visible light image and the other multi-source images, and using a scale-invariant feature transformation model to perform feature point matching processing on the image feature points to obtain matching feature points; based on the matching feature points, calculating the spatial transformation matrix between the other multi-source images and the visible light image; based on the spatial transformation matrix, mapping the textures in the other multi-source images to the grid model to obtain other texture mapping relationships corresponding to the other multi-source images.

[0104] Among them, image feature points refer to key points identified from images through feature extraction algorithms. These key points have unique local features and can remain consistent under different viewing angles or scales; the scale-invariant feature transform model (SIFT) is an algorithm used to extract local feature points of images and generate descriptors to ensure that the feature points are robust under changes in scale, rotation, etc.; the spatial transformation matrix is ​​a transformation matrix that describes the spatial correspondence between two images. The perspective transformation matrix is ​​commonly used to achieve image alignment.

[0105] Exemplarily, the terminal first extracts image feature points from the visible light image and other multi-source images. Each image is processed by a scale-invariant feature transformation model to extract significant feature points such as corners and edges in the image, and generate corresponding feature descriptors. Subsequently, the terminal uses the descriptors of the feature points for matching processing, and identifies the corresponding feature point pairs in the two images by calculating the similarity between the feature descriptors to obtain matching feature points. Matching feature points are corresponding point sets of the same object surface area in the two images, which are the basis for achieving spatial alignment. Based on the coordinate information of the matching feature points, the terminal calculates the spatial transformation matrix between the other multi-source images and the visible light image. An algorithm based on a homography matrix or a perspective transformation matrix is ​​usually used, and the least squares method or the random sample consensus (RANSAC) method is used to optimize the calculation results and eliminate mismatched points to ensure the accuracy of the transformation matrix. The transformation matrix describes how the other multi-source images are spatially aligned with the visible light image, including translation, rotation, scaling, and perspective transformation. Then, the terminal uses the transformation matrix to match the pixel coordinates in other multi-source images with the three-dimensional coordinates in the grid model, and assigns the texture values ​​in the multi-source images to the corresponding areas on the grid model. Finally, the terminal stores the texture mapping relationships of other multi-source images as a data structure for subsequent use. Each multi-source image texture mapping relationship represents the spatial distribution of a physical attribute (such as temperature, fault area, electrical activity, etc.) on the three-dimensional grid model.

[0106] Specifically, firstly, feature points on the device contour in other multi-source images are extracted in the curvature scale space, and auxiliary feature points are constructed according to the positions of the curvature minimum points on the left and right sides of each feature point. Then, the feature direction vector of the feature point is constructed by the following formula:

[0107] ,

[0108] Among them, (x f ,y f ) is the coordinate of the feature point; (x fL ,y fL ) is the coordinate of the auxiliary feature point on the left; (x fR ,y fR ) is the coordinate of the auxiliary feature point on the right; (x fm ,y fm ) is the feature vector coordinate. The main direction angle φ(Pf) of the feature point’s contour feature is calculated by the above formula:

[0109]

[0110] Then, the improved Scale Invariant Feature Transform (SIFT) descriptors of other multi-source images at multiple scales are extracted respectively, and the initial matching points are obtained using the bilateral matching method. Since there are a large number of incorrect matching points in the initial matching points, the Random Sample Consensus (RANSAC) method is further used to iteratively screen the matching points to obtain matching results without incorrect matching points.

[0111] Finally, the perspective transformation model is used to solve the perspective transformation relationship matrix between other multi-source images and the visible light image from the matching point pairs, and the original detection data corresponding to the other multi-source images are interpolated and transformed to the same scale and perspective as the visible light image to establish the spatial correlation relationship of other multi-source images.

[0112] In this embodiment, by extracting feature points from visible light images and other multi-source images, and using a scale-invariant feature transformation model to match feature points, the spatial transformation matrix between other multi-source images and visible light images is calculated. Using this transformation matrix, the terminal realizes texture mapping of other multi-source images on a three-dimensional grid model, and generates texture mapping relationships corresponding to other multi-source images. The precise spatial alignment of multi-source images and the seamless fusion of texture mapping are ensured, so that the multi-source texture digital twin model of the converter station has the information capability to display multi-dimensional physical properties such as temperature and electrical state. The comprehensive display capability and practical application value of the digital twin model have been greatly improved, providing reliable technical support for intelligent monitoring, fault diagnosis and predictive maintenance of converter stations.

[0113] In an exemplary embodiment, the other multi-source images include a plurality of other multi-source images, and each other multi-source image includes a plurality of candidate images.

[0114] The above-mentioned method maps the textures in other multi-source images to the grid model according to the spatial transformation matrix to obtain other texture mapping relationships corresponding to the other multi-source images, and also includes: for any other multi-source images, according to the spatial transformation matrix of each candidate image in any other multi-source images, determining the matching degree between each candidate image and the visible light image; according to the matching degree, determining the target matching image of any other multi-source image from the candidate images of any other multi-source images; according to the spatial transformation matrix of the target matching image, mapping the texture in the target matching image to the grid model to obtain other texture mapping relationships corresponding to any other multi-source images.

[0115] Among them, other multi-source images refer to other image types besides visible light images, such as infrared and ultraviolet images, etc. These images are usually used to provide different physical property information of objects, such as temperature, electrical activity, etc.; candidate images refer to multiple images selected from each other multi-source image that may be used for mapping; the degree of matching refers to the spatial alignment between the candidate image and the visible light image, which is usually evaluated by calculating the accuracy of image feature point matching.

[0116] Exemplarily, the terminal selects multiple candidate images from any other multi-source image for processing. First, by extracting and matching feature points of each candidate image and the visible light image, the spatial transformation matrix between the two is calculated, and the degree of matching between each candidate image and the visible light image is evaluated according to the matching of the feature points. The degree of matching is calculated based on the accuracy of the feature points, that is, the spatial error between the matching point pairs. Through the RANSAC algorithm or other algorithms with strong robustness, the terminal screens out images with high matching accuracy to ensure the accuracy of subsequent texture mapping. According to the calculated degree of matching between each candidate image and the visible light image, the terminal selects the candidate image with the highest matching degree as the target matching image. This target image is the most suitable image for texture mapping with the grid model, usually an image with a high degree of overlap with the visible light image and good matching accuracy. The terminal ensures that the selected target image can provide the best texture mapping quality and maximize the spatial alignment between the grid model and the visible light image by analyzing the matching results of all candidate images. The terminal then uses the spatial transformation matrix of the target matching image to accurately map the texture information in the image to the grid model. Specifically, the terminal calculates the corresponding position of each pixel of the target image in the grid model through the spatial transformation matrix, and passes the texture data of the target image to the patch of the corresponding area of ​​the grid model. This process ensures that the texture of the target image can be seamlessly mapped to the three-dimensional grid surface without distortion or dislocation, maintaining the authenticity and accuracy of the texture information. After the mapping is completed, the terminal generates other texture mapping relationships corresponding to the target matching image. This mapping relationship represents the spatial distribution of the physical properties (such as temperature, radiation, etc.) in the image in the three-dimensional grid model. These mapping relationships provide the basis for subsequent multi-source data fusion, so that the digital twin model of the converter station not only has visible light texture information, but also can display other types of physical features, further enhancing the model's multi-dimensional information display capabilities.

[0117] Specifically, after the registration of other multi-source image data is completed, there is a situation where a visible light image is successfully registered with multiple infrared or ultraviolet images; similarly, after the homologous image data registration is completed using the SIFT method, there is also a situation where a single image is successfully registered with multiple homologous images. To this end, it is necessary to select the optimal single image to establish a one-to-one image matching relationship. Specifically, suppose the single image V 1 The corresponding matching set of homologous or other multi-source images is {I 1 , I 2 ,…,I n}, the screening process is as follows:

[0118] 1) Calculate {I 1 , I 2 ,…,I n} and V 1 The registration error ε is used to reflect the alignment accuracy of other multi-source images after registration:

[0119]

[0120] Where: (u o i , v o i ) represent V 1 The pixel coordinates of the i-th pair of original matching points in (u f i , v f i ) represent V 1 The coordinates of the i-th pair of original matching points after perspective transformation; N c Indicates the number of matching point pairs.

[0121] According to the following formula, {I 1 , I 2 ,…,I n The information entropy value E of the grayscale image after perspective transformation is used to reflect the clarity of a single image and the richness of texture information: . Where p(i) represents the frequency of pixels with gray value i in the gray image; the range of i is [0, 255].

[0122] 2) Calculate {I 1 , I 2 ,…,I n}After perspective transformation, all pixel coordinates are located at V 1 The ratio S of the number of pixels in the pixel coordinate index range to the total number of pixels, which reflects the I i With V 1 The percentage of valid data after registration:

[0123]

[0124] Among them, Counter() represents the counting function; M and N represent V 1 horizontal and vertical resolution.

[0125] 3) Calculate {I 1 , I 2 ,…,I n The weighted ratio and R of the indicators for each image in}:

[0126]

[0127] Among them, wε, w e 、w s are the registration errors ε corresponding to the i-th matching image. i , information entropy value E i , effective data ratio S i The weight of , and wε+w e +w s =1.

[0128] R is used to reflect {I 1 , I 2 ,…,I n The comprehensive matching quality of each image in the image set {I 1 , I 2 ,…,I n The image with the largest R value among them is taken as the single input image V 1 The optimal matching image is obtained, and the spatial transformation relationship between the matching images is obtained.

[0129] In this embodiment, multiple candidate images are extracted from other multi-source images, and the matching degree between them and the visible light image is calculated to select the best target matching image, and the texture information is mapped to the three-dimensional mesh model according to its spatial transformation matrix. This ensures high-precision alignment of different types of images (such as infrared and ultraviolet images) with the mesh model, and generates an accurate texture mapping relationship.

[0130] In another exemplary embodiment, the present application provides another method for constructing a digital twin model of a converter station, specifically comprising:

[0131] ① High-precision modeling of converter stations based on 3D laser scanning

[0132] The main structure of a 3D laser scanner is a high-precision laser rangefinder and a reflective prism that can guide the laser and scan at a uniform angle. The laser rangefinder actively emits lasers and receives signals reflected from the surface of natural objects to measure distances. For each scanning point, the slant distance from the measuring station to the scanning point can be measured. Combined with the horizontal and vertical angles of the scan, the spatial relative coordinates of each scanning point and the measuring point can be obtained. If the spatial coordinates of the measuring station are known, the 3D coordinates of each scanning point can be obtained.

[0133] The specific operation is that the 3D laser scanner transmitter sends out a laser pulse signal, which is diffusely reflected by the surface of the object and then transmitted back to the receiver along almost the same path. The distance between the target point P and the scanner can be calculated, and the encoder is controlled to synchronously measure the scanning angle observation value α and the longitudinal scanning angle observation value β of each laser pulse, so that the spatial coordinates of the target point P can be obtained. The 3D laser scanning measurement is generally a custom indicator system of the instrument, with the X-axis in the horizontal scanning plane, the Y-axis perpendicular to the X-axis in the horizontal scanning plane, and the Z-axis perpendicular to the horizontal scanning plane. Its spatial structure is as follows: Figure 5 As shown. The whole system consists of a ground 3D laser scanner, a digital camera, post-processing software, a power supply and auxiliary equipment. It uses a non-contact high-speed laser measurement method to obtain geometric data and image data of terrain or complex objects. The post-processing software then processes the collected point cloud data and image data and converts them into spatial position coordinates or models in an absolute coordinate system. Finally, it is output in a variety of different formats to meet the data source of the spatial database and the needs of different applications.

[0134] The data collected by the 3D laser scanning system is point cloud data. Point cloud data processing generally includes the following steps: noise removal, multi-view alignment, data simplification, and surface reconstruction.

[0135] 1) Noise removal refers to removing data other than the scanned object from the point cloud data. During the scanning process, due to the influence of certain environmental factors, such as surrounding people being captured by the scanner, these data need to be deleted in the post-processing stage.

[0136] 2) Multi-view alignment means that because the object under test is too large or has a complex shape, it is often impossible to scan all the data at once, and multiple scans are required from different positions and multiple perspectives. These point clouds need to be aligned and spliced, which is called multi-view alignment. Point cloud alignment and splicing can be achieved by placing control points with the same name on the surface of the object.

[0137] 3) Data simplification means that since point cloud data is massive, it is necessary to simplify the data without affecting the surface reconstruction and maintaining a certain accuracy. For example, 3D model simplification based on quadric error metrics (QEM) is used.

[0138] 4) Surface reconstruction refers to the use of accurate surfaces to represent scanned data. After surface reconstruction, 3D modeling can be performed to restore the original appearance of the scanned object.

[0139] ② Intelligent fusion of multi-dimensional data of converter station digital twin

[0140] At present, power equipment status monitoring projects are often based on visible light imaging, infrared thermal imaging temperature measurement, ultraviolet discharge, and audio intelligent sensor monitoring. Among them, visible light imaging detection has the characteristics of high resolution and rich texture details, while other sensors have the characteristics of detecting other band status information and displaying it in two-dimensional form. Based on the above characteristics analysis, the following is proposed: Figure 6 The process of multi-source data spatial synthesis and three-dimensional display method of converter station is shown. The process includes three steps: equipment grid model reconstruction, data registration and transformation, and data three-dimensional display.

[0141] 1) Equipment mesh model reconstruction: Most current monitoring devices are equipped with monocular visible light digital cameras. Therefore, this framework uses monocular visible light 3D reconstruction technology to automatically match images from visible light image sequences, calculate the camera shooting posture and equipment spatial position, and finally reconstruct a detailed 3D equipment mesh model. This model is used as the model basis for subsequent 3D display of equipment status data. The process is as follows: Figure 7 As shown. Figure 7 In the implementation process shown, the Structure from Motion (SfM) technology part is based on the theoretical basis of epipolar geometry and triangulation principles. It performs local or global feature matching on the input ordered or unordered image sequence, then solves the camera pose and performs incremental or global bundle adjustment optimization, and finally restores the precise shooting pose of the detection camera and the sparse point cloud model of the device.

[0142] Next, the Multi-View Stereo (MVS) technology uses the camera pose output by the SfM technology as input, further traverses and matches the input image, obtains more image matching points, and then restores the dense point cloud, surface mesh, and texture model of the scene. The SfM technology based on the camera epipolar geometry theory can solve the monocular camera pose parameters corresponding to the image sequence. The camera epipolar geometry model is as follows: Figure 8 shown. Figure 8 In the model shown, I 1 ,I 2 are the numbers of single images taken by the monocular camera at different positions; 1 , O 2 are the camera optical center positions corresponding to the two images respectively; P is the object point, p 1 、p 2 They are respectively1 ,I 2 The projection point in 1 and O 2 The intersection point e of the line connecting the two 1 、e 2 is the extreme point; p 1 e 1 、p 2 e 2 is the epipolar line; R and t are the camera’s 2 Position relative to O 1 The rotation and translation transformation matrix of the position.

[0143] After the camera's optimal pose is solved, the MVS technology is further used to construct a 3D mesh model of the device. The camera pose parameters and images output by SfM are used as input, and the block matching method is used to obtain Figure 7 The dense point cloud model of the device is shown. Compared with the sparse point cloud, the dense point cloud model has more device structure details. Then the weak texture support area classification method is used to classify all the point clouds in the dense point cloud according to their spatial positions. Based on the classification, the point cloud meshing weights in the weak texture support area are improved, so that the area with fewer point clouds can also construct a complete surface mesh, and finally the device mesh model and visible light texture mapping relationship are obtained.

[0144] 2) Data registration and transformation: Based on the visible light image, other multi-source image registration technologies are used to traverse and register other heterogeneous data with it in two-dimensional space to obtain the spatial transformation relationship between the visible light image and other band images, and then the data decision method is used to obtain the one-to-one matching relationship between the heterogeneous data. In addition, the best matching image in each other multi-source image needs to be screened out.

[0145] 3) Data 3D display: 3D reconstruction technology restores the fine mesh model of the equipment and the texture mapping relationship between the visible light image and the mesh model. At the same time, the image registration technology establishes the spatial matching relationship between heterogeneous images. At this time, UV coordinate texture mapping is used to perform multi-source texture mapping on the mesh model to obtain a 3D visualized multi-source texture model. The schematic diagram of multi-source texture mapping of the converter station model is shown in the figure. Fig. 9 As shown. Fig. 9 It can be seen that the visible light fine grid model gives texture to multi-source heterogeneous data, which not only ensures the consistency of multi-source heterogeneous data during spatial data mapping, but also enables low-resolution data to have a high-resolution three-dimensional stereoscopic display effect.

[0146] In this embodiment, the point cloud data of the converter station is first obtained by three-dimensional laser scanning, and a preliminary equipment mesh model is generated after noise removal, data simplification, surface reconstruction and other processing. Then, the dense point cloud and surface mesh of the equipment are restored from the visible light image using motion recovery structure (SfM) and multi-view stereo (MVS) technology to ensure the fineness and structural accuracy of the model. Subsequently, through data registration and transformation technology, multi-source images (such as infrared, ultraviolet images, etc.) are combined for spatial alignment to generate a multi-source texture mapping relationship, which is mapped to the mesh model to form a complete digital twin model. This model can not only accurately display the geometric structure of the equipment, but also integrate multi-dimensional physical property information, providing comprehensive technical support for equipment status monitoring, fault diagnosis and predictive maintenance, and significantly improving the digital and intelligent management capabilities of the converter station.

[0147] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0148] Based on the same inventive concept, the embodiment of the present application also provides a digital twin model construction device for a converter station for implementing the digital twin model construction method of the converter station involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the digital twin model construction device for one or more converter stations provided below can refer to the limitations of the digital twin model construction method for the converter station above, and will not be repeated here.

[0149] In an exemplary embodiment, Fig.10 As shown, a digital twin model construction device for a converter station is provided, comprising: a dense point cloud acquisition module 1001, a point cloud data fusion module 1002, a grid model construction module 1003, a texture mapping acquisition module 1004 and a digital twin construction module 1005, wherein:

[0150] A dense point cloud acquisition module 1001 is used to process the visible light image of the converter station using a motion recovery structure model and a multi-view stereo model to obtain dense point cloud data of the converter station;

[0151] The point cloud data fusion module 1002 is used to fuse the dense point cloud data and the laser scanning point cloud data of the converter station to obtain the geometric point cloud data of the converter station;

[0152] A grid model building module 1003 is used to build a grid model of the converter station according to the geometric point cloud data;

[0153] The texture mapping acquisition module 1004 is used to obtain a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and to perform registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to other multi-source images;

[0154] The digital twin construction module 1005 is used to obtain a multi-source texture digital twin model of the converter station based on the grid model, visible light texture mapping relationship and other texture mapping relationships.

[0155] In one of the embodiments, the above-mentioned dense point cloud acquisition module 1001 is also used to use a motion recovery structure model to perform feature matching processing on the visible light image to obtain the shooting posture information corresponding to the visible light image and the sparse point cloud data of the converter station; the visible light image, shooting posture information and sparse point cloud data are input into the multi-view stereo model to obtain the dense point cloud data of the converter station.

[0156] In one of the embodiments, the mesh model construction module 1003 is also used to perform surface reconstruction on the geometric point cloud data to obtain an initial mesh model of the converter station; and use a quadratic error metric simplification model to simplify the vertices in the initial mesh model to obtain a simplified mesh model as the mesh model of the converter station.

[0157] In one of the embodiments, the mesh model construction module 1003 is further used to use the initial mesh model as the current iterative mesh model, calculate the quadratic error metric value of each edge of the current iterative mesh model; determine the target edge of the quadratic error metric value, and merge the two vertices corresponding to the target edge into one vertex to obtain a new current iterative mesh model; return to execute the step of calculating the quadratic error metric value of each edge of the current iterative mesh model until the new current iterative mesh model reaches a preset complexity target, and use the new current iterative mesh model as the simplified mesh model.

[0158] In one of the embodiments, the texture mapping acquisition module 1004 is also used to extract image feature points from visible light images and other multi-source images, and use a scale-invariant feature transformation model to perform feature point matching processing on the image feature points to obtain matching feature points; based on the matching feature points, calculate the spatial transformation matrix between the other multi-source images and the visible light image; based on the spatial transformation matrix, map the textures in the other multi-source images to the grid model to obtain other texture mapping relationships corresponding to the other multi-source images.

[0159] In one embodiment, the other multi-source images include a plurality of other multi-source images, and each other multi-source image includes a plurality of candidate images; the texture mapping acquisition module 1004 is further used to determine, for any other multi-source image, a degree of matching between each candidate image and the visible light image according to a spatial transformation matrix of each candidate image in any other multi-source image; according to the degree of matching, determine a target matching image of any other multi-source image from the candidate images of any other multi-source image; according to the spatial transformation matrix of the target matching image, map the texture in the target matching image to the grid model to obtain other texture mapping relationships corresponding to any other multi-source image.

[0160] Each module in the digital twin model construction device of the converter station can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0161] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.11As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for constructing a digital twin model of a converter station is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0162] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0163] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0165] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0167] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0168] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0169] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a digital twin model of a converter station, characterized in that: The method comprises: Using a structure-from-motion model and a multi-view stereo model, the visible light image of the converter station is processed to obtain dense point cloud data of the converter station; fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station; Constructing a grid model of the converter station according to the geometric point cloud data; Obtaining a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and performing registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images; According to the grid model, the visible light texture mapping relationship and the other texture mapping relationships, a multi-source texture digital twin model of the converter station is obtained.

2. The method according to claim 1, characterized in that The method uses the motion recovery structure model and the multi-view stereo model to process the visible light image of the converter station to obtain dense point cloud data of the converter station, including: Using the motion recovery structure model, perform feature matching processing on the visible light image to obtain shooting posture information corresponding to the visible light image and sparse point cloud data of the converter station; The visible light image, the shooting posture information and the sparse point cloud data are input into the multi-view stereo model to obtain dense point cloud data of the converter station.

3. The method according to claim 1, characterized in that The step of constructing a grid model of the converter station according to the geometric point cloud data comprises: Performing surface reconstruction processing on the geometric point cloud data to obtain an initial grid model of the converter station; The quadratic error metric simplified model is adopted to simplify the vertices in the initial mesh model to obtain a simplified mesh model as the mesh model of the converter station.

4. The method according to claim 3, characterized in that The method of simplifying the model by using quadratic error metric to simplify the vertices in the initial mesh model to obtain a simplified mesh model includes: Taking the initial grid model as the current iterative grid model, calculating the quadratic error metric value of each edge of the current iterative grid model; Determine a target edge of the quadratic error metric value, and merge two vertices corresponding to the target edge into one vertex to obtain a new current iterative mesh model; Return to the step of calculating the quadratic error metric value of each edge of the current iterative mesh model until the new current iterative mesh model reaches a preset complexity target, and use the new current iterative mesh model as the simplified mesh model.

5. The method according to claim 1, characterized in that The registering process is performed on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images, including: Extracting image feature points from the visible light image and the other multi-source images, and performing feature point matching processing on the image feature points using a scale-invariant feature transformation model to obtain matching feature points; Calculating a spatial transformation matrix between the other multi-source images and the visible light image according to the matching feature points; According to the spatial transformation matrix, the textures in the other multi-source images are mapped onto the grid model to obtain other texture mapping relationships corresponding to the other multi-source images.

6. The method according to claim 5, characterized in that The other multi-source images include a plurality of other multi-source images, each of which includes a plurality of candidate images; Mapping the textures in the other multi-source images onto the grid model according to the spatial transformation matrix to obtain other texture mapping relationships corresponding to the other multi-source images includes: For any other multi-source image, determining the matching degree between each candidate image and the visible light image according to the spatial transformation matrix of each candidate image in the any other multi-source image; Determining a target matching image of any other multi-source image from candidate images of any other multi-source image according to the matching degree; According to the spatial transformation matrix of the target matching image, the texture in the target matching image is mapped onto the grid model to obtain other texture mapping relationships corresponding to any other multi-source image.

7. A digital twin model construction device for a converter station, characterized in that: The device comprises: A dense point cloud acquisition module is used to process the visible light image of the converter station by using a motion recovery structure model and a multi-view stereo model to obtain dense point cloud data of the converter station; A point cloud data fusion module, used for fusing the dense point cloud data with the laser scanning point cloud data of the converter station to obtain geometric point cloud data of the converter station; A grid model building module, used to build a grid model of the converter station according to the geometric point cloud data; A texture mapping acquisition module, configured to obtain a visible light texture mapping relationship corresponding to the visible light image according to the visible light image and the grid model, and to perform registration processing on other multi-source images of the converter station according to the visible light image to obtain other texture mapping relationships corresponding to the other multi-source images; A digital twin construction module is used to obtain a multi-source texture digital twin model of the converter station based on the grid model, the visible light texture mapping relationship and the other texture mapping relationships.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images