Real-time 3D modeling method and device for converter station transformer

Abstract

The present application discloses a real-time three-dimensional modeling method for a converter station transformer, which is applied to the field of three-dimensional modeling technology. The method provided by the present application includes: obtaining a first set of images of a preset first number of target transformers according to a preset first resolution; inputting the first set of images into a preset image feature matching algorithm to obtain a second set of second images of a second number; converting the resolution of the images in the second set of images into a preset second resolution to obtain a third set of images; inputting the third set of images into a preset NeRF model to obtain a three-dimensional model component of the target transformer; embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer; repeating the above steps to obtain a real-time updated three-dimensional model of the target transformer.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional modeling technology, and in particular to a method, device, computer equipment and storage medium for real-time three-dimensional modeling of a converter station transformer. Background Art

[0002] With the construction of digital intelligent converter stations, the intelligent operation and maintenance platform in the digital intelligent converter stations provides important data support for equipment fault diagnosis with the help of digital twin online monitoring technology, and the three-dimensional modeling of power equipment in the digital twin system is the basis for digital twin visualization. Compared with the two-dimensional model, the three-dimensional model provides more intuitive visual spatial information, providing technical support for the interactive application and unattended and automated operation and maintenance of the digital twin of the intelligent converter station.

[0003] Existing 3D modeling technologies are mainly divided into active time-of-flight method, laser triangulation method, structured light method, and passive binocular vision method, monocular vision method, image reconstruction method of convolutional neural network, deep learning feature method, BIM modeling and other rendering methods. However, active 3D modeling methods are easily affected by ambient light and are rarely used in 3D modeling of outdoor scenes. Passive 3D modeling methods have defects such as complex algorithms, cumbersome data extraction, poor accuracy, and weak rendering capabilities. Finally, existing 3D modeling technologies all have the problem of long model update cycles, which makes it impossible to observe the latest operating conditions of converter station transformer equipment in real time. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, computer equipment and storage medium for real-time three-dimensional modeling of a converter station transformer to solve the problem of long update cycle of the existing three-dimensional model of the converter station transformer.

[0005] In a first aspect of the present application, a real-time three-dimensional modeling method for a converter station transformer is provided, comprising:

[0006] Acquire a first set of images of a first quantity preset for the target transformer according to a preset first resolution;

[0007] Inputting the first image set into a preset image feature matching algorithm to obtain a second number of second image sets;

[0008] Converting the resolution of the images in the second image set to a preset second resolution to obtain a third image set;

[0009] Inputting the third image set into a preset NeRF model to obtain a three-dimensional model component of the target transformer;

[0010] Embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer;

[0011] The first image set is updated using the received image of the target transformer, and the aforementioned steps are repeated to update the three-dimensional model.

[0012] A second aspect of the present application provides a real-time three-dimensional modeling device for a converter station transformer, comprising:

[0013] A first image set module, used to obtain a first set of first images of a target transformer according to a preset first resolution;

[0014] A second image set module, used for inputting the first image set into a preset image feature matching algorithm to obtain a second number of second image sets;

[0015] A third image set module, configured to convert the resolution of the images in the second image set into a preset second resolution to obtain a third image set;

[0016] A three-dimensional model component module, used for inputting the third image set into a preset NeRF model to obtain a three-dimensional model component of the target transformer;

[0017] A three-dimensional model module, used for embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer;

[0018] A step repeating module is used to update the first image set using the received image of the target transformer, and repeat the above steps to update the three-dimensional model.

[0019] According to a third aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for real-time three-dimensional modeling of a converter station transformer when executing the computer program.

[0020] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for real-time three-dimensional modeling of a converter station transformer are implemented.

[0021] The above-mentioned converter station transformer real-time three-dimensional modeling method, device, computer equipment and storage medium obtain a first set of first images of a target transformer according to a preset first resolution; input the first set of images into a preset image feature matching algorithm to obtain a second set of second images of a second number; convert the resolution of the images in the second set of images into a preset second resolution to obtain a third set of images; input the third set of images into a preset NeRF model to obtain a three-dimensional model component of the target transformer; embed the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer; repeat the above steps to obtain a three-dimensional model of the target transformer updated in real time. Not only does it realize the real-time update of the three-dimensional model of the converter station transformer, but it also achieves the beneficial effect of updating and monitoring the three-dimensional model of the converter station transformer at the component level. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 This is a schematic diagram of an application environment of a real-time three-dimensional modeling method for a converter station transformer in an embodiment of the present application;

[0024] Figure 2 is a flow chart of a real-time three-dimensional modeling method for a converter station transformer in one embodiment of the present application;

[0025] Figure 3 It is a structural schematic diagram of a real-time three-dimensional modeling device for a converter station transformer in one embodiment of the present application;

[0026] Figure 4 It is a schematic diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The real-time three-dimensional modeling method of converter station transformer provided in this application can be applied to Figure 1In the application environment, the computer device can be but not limited to various personal computers and laptops. The computer device can also be a server. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. It is understandable that Figure 1 The number of computer devices in the figure is only for reference and can be expanded to any number according to actual needs.

[0029] In one embodiment, if Figure 2 As shown, a real-time three-dimensional modeling method for converter station transformer is provided. Figure 1 The computer device in the example is used to illustrate, and the method includes the following steps S101 to S106:

[0030] S101. Acquire a first set of images of a target transformer in a preset first quantity according to a preset first resolution.

[0031] Among them, in the converter station of the current power system, there has been an application scenario of unmanned and automated operation and maintenance of the converter station, and part of the monitoring of the unmanned and automated operation and maintenance of the converter station depends on monitoring cameras and robots. At the same time, the real-time three-dimensional modeling method of the converter station transformer of this embodiment is also based on the inspection images taken by the monitoring cameras and robots of the converter station. Among them, the first number also changes with the actual effect of the three-dimensional modeling result of the target converter station transformer obtained by the real-time three-dimensional modeling method of the converter station transformer of this embodiment. If the first number is small, the three-dimensional features of the target converter station transformer obtained by the image in the first image set are small, which will further affect the poor visual effect of the three-dimensional modeling of the target converter station transformer. If the first number is large, the number of images that need to be processed by the real-time three-dimensional modeling method of the converter station transformer of this embodiment becomes large, resulting in a decrease in the operating efficiency of the real-time three-dimensional modeling method of the converter station transformer of this embodiment, and then resulting in a decrease in the timeliness of the three-dimensional model of the target converter station transformer. For example, at present, the resolution of the inspection images taken by the monitoring cameras and robots of the converter station is basically 1080P, and the number of the inspection images must reach at least 200 images.

[0032] S102: Input the first image set into a preset image feature matching algorithm to obtain a second number of second image sets.

[0033] Among them, the function of the image feature matching algorithm is to streamline the images with high similarity in the first image set, further reduce the number of images in the first image set, and obtain a second image set of a second number, wherein the second number is smaller than the first number, but the effect of the images in the second image set in representing the three-dimensional features of the converter station transformer is not reduced.

[0034] Specifically, first, the images in the first image set are sorted according to the acquisition time, and the first image is selected from the first image set as the reference image. Because the images of the converter station transformer are acquired in batches according to the shooting time, the similarity of the images corresponding to adjacent acquisition times is high, and the feature changes of the images per unit time are small. Then, the feature point change rate of each image after the reference image is calculated in turn, wherein the feature point change rate is the rate of change of the number of feature points of each image relative to the previous image, and the number of feature points is calculated according to the image feature matching algorithm. In a more specific embodiment, the number of feature points is calculated using the SIFT feature point matching algorithm. At the same time, when the feature point change rate is within the preset change rate range, the image corresponding to the feature point change rate is used as the key image, and other images between the key image and the reference image are removed. For example, if the feature point change rate of an image corresponding to a certain acquisition time during the traversal process exceeds the preset change rate range relative to the reference image, the image corresponding to the acquisition time is used as the key image. Finally, the key image is selected as the reference image, and the steps of calculating the feature point change rate of each image after the reference image in sequence until the feature point change rate is within a preset change rate range are repeated until all images in the first image set are traversed to obtain the second image set.

[0035] S103: Convert the resolution of the images in the second image set to a preset second resolution to obtain a third image set.

[0036] The image of the converter station transformer is used for the training and application of the machine learning model and the three-dimensional modeling process, and the machine learning model for image processing has image parameter settings for the input data, i.e., image data. For example, some machine learning models for image processing do not accept images with extremely small or extremely large image resolutions or images with extremely small or extremely large storage space. Therefore, it is necessary to convert the resolution of the images in the second image set into the second resolution that can be processed by the machine learning model and required by the three-dimensional modeling process.

[0037] S104: Input the third image set into a preset NeRF model to obtain a three-dimensional model component of the target transformer.

[0038] Among them, the NeRF (Neural Radiance Fields) model is mainly used for three-dimensional modeling of scenes, and has a fast modeling speed and a small sample requirement. However, the NeRF model cannot achieve modeling and programming at the transformer component level. Therefore, in this embodiment, the traditional BIM modeling method is combined with the improved NeRF model to obtain the real-time three-dimensional modeling result of the converter station transformer.

[0039] Furthermore, the conversion of the resolution of the images in the second image set to the preset second resolution includes: first, obtaining the preset second resolution set by the NeRF model for the input image. Then, comparing the second resolution with the first resolution. If the second resolution is greater than the first resolution, the resolution of the images in the second image set is converted to the preset second resolution using a preset super-resolution algorithm. If the second resolution is less than the first resolution, the resolution of the images in the second image set is converted to the preset second resolution using a preset downsampling algorithm. For example, the resolution of the images taken by the surveillance camera and the robot is 1080P, and the super-resolution algorithm is used to convert the image with a resolution of 1080P to an image with a resolution of 4K, and the downsampling algorithm is used to convert the image with a resolution of 1080P to an image with a resolution of 360P or 480P.

[0040] Furthermore, before inputting the third image set into the preset NeRF model, the method further includes: representing the three-dimensional continuous scene of the target transformer as a 5D vector function, wherein the input of the 5D vector function is a 3D position and a 2D viewing direction, and the output of the 5D vector function is a color and a volume density. For example, the 3D position is represented as x(x, y, z), the 2D viewing direction is represented as d(θ, φ), the color is represented as RGB color c(r, g, b), and the volume density is represented as σ, then the 5D vector function expression is (σ, c) = g θ (x,d).

[0041] Furthermore, before representing the three-dimensional continuous scene of the target transformer as a 5D vector function, it also includes: using a preset encoding function to process the coordinate value in each direction of the 3D position and the 2D viewing direction, and the preset encoding function is as follows:

[0042] γ(a)=(sin(2 0 πa),cos(2 0 πa),...,sin(2 L-1 πa),cos(2 L-1 πa))

[0043] Among them, represents the coordinate value in each direction of the 3D position or the 2D viewing direction, and represents the periodic order of the trigonometric function.

[0044] Furthermore, the training process of the NeRF model is divided into a simple sampling method and a fine sampling method, and the number of images collected by the simple sampling method is less than the number of images collected by the fine sampling method. However, the color of the target transformer predicted under different sampling methods has a certain error, because it is necessary to ensure that the total square error between the color of the target transformer predicted under different sampling methods and the real color of the target transformer is minimized to quickly obtain a three-dimensional complex image of the target transformer. Among them, the first predicted color under the simple sampling method and the second predicted color under the fine sampling method are output using volume rendering technology, and the real color of the target transformer is obtained using the ground truth camera posture, intrinsic parameters and the boundary of synthetic data.

[0045] Furthermore, the error function of the NeRF model during training is:

[0046]

[0047] Where, represents the set of sampled light rays, C(r) represents the true color of the target transformer, represents the first predicted color obtained by sampling the target transformer in a first sampling manner, represents the second predicted color obtained by sampling the target transformer by the second sampling method. Furthermore, the second predicted color under the fine sampling method can also be used for the final rendering. Therefore, after using the NeRF form of scene representation, the color of any light passing through the scene can be solved by using classic volume rendering, and a new three-dimensional image can be quickly rendered and synthesized to achieve the beneficial effect of quickly or in real time updating the three-dimensional scene model of the converter station transformer.

[0048] S105. Embed the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer.

[0049] Among them, the pre-modeling result of the target transformer is obtained according to the BIM modeling method. The pre-modeling result is obtained using the architectural data of the converter station transformer. The BIM modeling method makes up for the defect that the NeRF model cannot achieve component-level modeling and programming, and the NeRF model makes up for the defect that the BIM modeling method cannot quickly update the model. However, the BIM modeling method and the improved NeRF model realize the component-level modeling and real-time update of the converter station transformer.

[0050] S106: Use the received image of the target transformer to update the first image set, and repeat the above steps to update the three-dimensional model.

[0051] Among them, the existing power system converter station mainly collects real-time data through robot inspection and camera. The robot collects the data of the converter station (including video data and image data) according to the preset inspection cycle, and regularly uploads the inspection records and inspection contents.

[0052] Furthermore, the method of updating the first image set using the received image of the target transformer and repeating the aforementioned steps to update the three-dimensional model further includes: first, checking whether the three-dimensional model component has changed before and after the three-dimensional model is updated. If the three-dimensional model component has changed, it is determined whether the target transformer corresponding to the three-dimensional model component has failed, wherein the degree of change of the three-dimensional model component compared with the preset standard transformer three-dimensional model component is used as a condition for determining whether a failure has occurred. Finally, if the target transformer has failed, the information of the target transformer corresponding to the three-dimensional model component is used to generate an alarm prompt message.

[0053] The real-time three-dimensional modeling method of the converter station transformer provided in this embodiment is as follows: obtaining a first set of first images of a preset first resolution of the target transformer; inputting the first set of images into a preset image feature matching algorithm to obtain a second set of second images of a second number; converting the resolution of the images in the second set of images into a preset second resolution to obtain a third set of images; inputting the third set of images into a preset NeRF model to obtain a three-dimensional model component of the target transformer; embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer; repeating the above steps to obtain a three-dimensional model of the target transformer updated in real time. Not only does it realize the real-time update of the three-dimensional model of the converter station transformer, but it also achieves the beneficial effect of updating and monitoring the three-dimensional model of the converter station transformer at the component level.

[0054] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0055] In one embodiment, a converter station transformer real-time 3D modeling device 100 is provided, and the converter station transformer real-time 3D modeling device 100 corresponds to the converter station transformer real-time 3D modeling method in the above embodiment. Figure 3As shown, the converter station transformer real-time 3D modeling device 100 includes a first image collection module 11, a second image collection module 12, a third image collection module 13, a 3D model component module 14, a 3D model module 15 and a step repeating module 16. The functional modules are described in detail as follows:

[0056] A first image set module 11 is used to obtain a first set of images of a preset first quantity of a target transformer according to a preset first resolution;

[0057] A second image set module 12, configured to input the first image set into a preset image feature matching algorithm to obtain a second number of second image sets;

[0058] A third image set module 13, configured to convert the resolution of the images in the second image set into a preset second resolution to obtain a third image set;

[0059] A three-dimensional model component module 14 is used to input the third image set into a preset NeRF model to obtain a three-dimensional model component of the target transformer;

[0060] A three-dimensional model module 15, used for embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer;

[0061] The step repeating module 16 is used to update the first image set using the received image of the target transformer, and repeat the above steps to update the three-dimensional model.

[0062] Furthermore, the second image collection module 12 further includes:

[0063] A first image sorting submodule, used to sort the images in the first image set according to acquisition time;

[0064] A reference image submodule, configured to select a first image from the first image set as a reference image;

[0065] A feature point change rate submodule, used to sequentially calculate the feature point change rate of each image after the reference image, wherein the feature point change rate is the rate of change of the number of feature points of each image relative to the previous image, and the number of feature points is calculated according to the image feature matching algorithm;

[0066] A key image submodule, used to use the image corresponding to the feature point change rate as a key image when the feature point change rate is within a preset change rate range, and remove other images between the key image and the reference image;

[0067] The first image set loop submodule is used to select the key image as the reference image, and repeatedly execute the steps of sequentially calculating the feature point change rate of each image after the reference image until the feature point change rate is within a preset change rate range, until all images in the first image set are traversed to obtain the second image set.

[0068] Furthermore, the third image collection module 13 further includes:

[0069] A second resolution submodule, used to obtain the preset second resolution set by the NeRF model for the input image;

[0070] A resolution comparison submodule, used for comparing the second resolution with the first resolution;

[0071] a super-resolution processing submodule, configured to convert the resolution of the images in the second image set into the preset second resolution using a preset super-resolution algorithm if the second resolution is greater than the first resolution;

[0072] The downsampling processing submodule is used to convert the resolution of the images in the second image set to the preset second resolution using a preset downsampling algorithm if the second resolution is smaller than the first resolution.

[0073] Furthermore, the three-dimensional model component module 14 also includes:

[0074] The three-dimensional continuous scene submodule is used to represent the three-dimensional continuous scene of the target transformer as a 5D vector function, wherein the input of the 5D vector function is a 3D position and a 2D viewing direction, and the output of the 5D vector function is a color and a volume density.

[0075] Furthermore, the three-dimensional continuous scene submodule also includes:

[0076] The coding function processing subunit is used to process the coordinate value in each direction of the 3D position and the 2D viewing direction using a preset coding function, wherein the preset coding function is as follows:

[0077] γ(a)=(sin(2 0 πa),cos(2 0 πa),...,sin(2 L-1 πa),cos(2 L-1 πa))

[0078] Wherein, a represents the coordinate value in each direction of the 3D position or the 2D viewing direction, and L represents the periodic order of the trigonometric function.

[0079] Furthermore, the three-dimensional model component module 14 also includes:

[0080] The error function submodule, the error function used in the NeRF model during training is:

[0081]

[0082] Where r represents the set of sampled rays, C(r) represents the true color of the target transformer, represents the first predicted color obtained by sampling the target transformer in a first sampling manner, Represents a second predicted color obtained by sampling the target transformer through a second sampling method.

[0083] Furthermore, the step repetition module 16 further includes:

[0084] An update checking subunit, used to check whether the three-dimensional model components are changed before and after the three-dimensional model is updated;

[0085] A fault judgment subunit is used to judge whether the target transformer corresponding to the three-dimensional model component has a fault if the three-dimensional model component changes, wherein the degree of change between the three-dimensional model component and the preset standard transformer three-dimensional model component is used as a judgment condition for whether a fault has occurred.

[0086] The alarm information subunit generates alarm prompt information using the information of the target transformer corresponding to the three-dimensional model component if a fault occurs.

[0087] The meaning of "first" and "second" in the above modules / units is only to distinguish different modules / units, and is not used to define which module / unit has a higher priority or other limiting meanings. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules in this application is only a logical division, and there may be other division methods when implemented in actual applications.

[0088] For the specific limitations of the real-time three-dimensional modeling device for converter station transformers, please refer to the limitations of the real-time three-dimensional modeling method for converter station transformers mentioned above, which will not be repeated here. Each module in the above-mentioned real-time three-dimensional modeling device for converter station transformers can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0089] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the real-time three-dimensional modeling method of the converter station transformer. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a real-time three-dimensional modeling method of the converter station transformer is implemented.

[0090] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for real-time three-dimensional modeling of a transformer in a converter station in the above embodiment are implemented, such as Figure 2 Steps S101 to S106 and other extensions of the method and extensions of related steps are shown. Alternatively, when the processor executes the computer program, the functions of each module / unit of the real-time three-dimensional modeling device for converter station transformers in the above embodiment are realized, for example Figure 3 The functions of modules 11 to 16 are shown in Figure 1. To avoid repetition, they will not be described here.

[0091] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and various interfaces and lines are used to connect various parts of the entire computer device.

[0092] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.), etc.

[0093] The memory may be integrated into the processor or may be arranged separately from the processor.

[0094] In one embodiment, a computer-readable storage medium is provided on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for real-time three-dimensional modeling of a converter station transformer in the above embodiment are implemented, for example Figure 2 Steps S101 to S106 and other extensions of the method and related steps are shown. Alternatively, when the computer program is executed by the processor, the functions of each module / unit of the real-time three-dimensional modeling device for converter station transformers in the above embodiment are realized, for example Figure 3 The functions of modules 11 to 16 are shown in Figure 1. To avoid repetition, they will not be described here.

[0095] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0096] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0097] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A real-time three-dimensional modeling method for converter station transformer, characterized in that: include: Acquire a first set of images of a first quantity preset for the target transformer according to a preset first resolution; Inputting the first image set into a preset image feature matching algorithm to obtain a second number of second image sets; Converting the resolution of the images in the second image set to a preset second resolution to obtain a third image set; Inputting the third image set into a preset NeRF model to obtain a three-dimensional model component of the target transformer; Embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer; Using the received image of the target transformer to update the first image set, repeating the above steps to update the three-dimensional model; Among them, the step of inputting the first image set into a preset image feature matching algorithm to obtain a second number of second image sets includes: sorting the images in the first image set according to the acquisition time; selecting the first image from the first image set as the reference image; sequentially calculating the feature point change rate of each image after the reference image, wherein the feature point change rate is the rate of change of the number of feature points of each image relative to the previous image, and the number of feature points is calculated according to the image feature matching algorithm; when the feature point change rate is within a preset change rate range, taking the image corresponding to the feature point change rate as a key image, and removing other images between the key image and the reference image; selecting the key image as the reference image, and repeating the steps of sequentially calculating the feature point change rate of each image after the reference image to the step of when the feature point change rate is within the preset change rate range, until all the images in the first image set are traversed to obtain the second image set.

2. The real-time three-dimensional modeling method for converter station transformer according to claim 1 is characterized in that: The converting the resolution of the images in the second image set into a preset second resolution comprises: Obtaining the preset second resolution set by the NeRF model for the input image; comparing the second resolution with the first resolution; If the second resolution is greater than the first resolution, converting the resolution of the images in the second image set to the preset second resolution using a preset super-resolution algorithm; If the second resolution is smaller than the first resolution, a preset downsampling algorithm is used to convert the resolutions of the images in the second image set to the preset second resolution.

3. The real-time three-dimensional modeling method for converter station transformer according to claim 1 is characterized in that: Before inputting the third image set into a preset NeRF model, the method further includes: The three-dimensional continuous scene of the target transformer is represented as a 5D vector function, wherein the input of the 5D vector function is a 3D position and a 2D viewing direction, and the output of the 5D vector function is a color and a volume density.

4. The real-time three-dimensional modeling method for converter station transformer according to claim 3 is characterized in that: The step of representing the three-dimensional continuous scene of the target transformer as a 5D vector function also includes: A preset encoding function is used to process the coordinate value in each direction of the 3D position and the 2D viewing direction, and the preset encoding function is as follows: γ(a)=(sin(2 0 πa),cos(2 0 πa),...,sin(2 L-1 πa),cos(2 L-1 (a))), Wherein, a represents the coordinate value in each direction of the 3D position or the 2D viewing direction, and L represents the periodic order of the trigonometric function.

5. The real-time three-dimensional modeling method for converter station transformer according to claim 1, characterized in that: The error function of the NeRF model during training is: ， Where r represents the set of sampled rays, C(r) represents the true color of the target transformer, represents the first predicted color obtained by sampling the target transformer in a first sampling manner, Represents a second predicted color obtained by sampling the target transformer using a second sampling method.

6. The real-time three-dimensional modeling method for converter station transformer according to claim 1, characterized in that: The step of updating the first image set using the received image of the target transformer and repeating the aforementioned steps to update the three-dimensional model further comprises: Checking whether the three-dimensional model components are changed before and after the three-dimensional model is updated; If yes, then determine whether the target transformer corresponding to the three-dimensional model component is faulty, wherein the degree of change between the three-dimensional model component and a preset standard transformer three-dimensional model component is used as a condition for determining whether a fault occurs; If a fault occurs, the information of the target transformer corresponding to the three-dimensional model component is used to generate an alarm prompt message.

7. A real-time three-dimensional modeling device for a converter station transformer, characterized in that: include: A first image set module, used to obtain a first set of first images of a target transformer according to a preset first resolution; A second image set module, used for inputting the first image set into a preset image feature matching algorithm to obtain a second number of second image sets; A third image set module, configured to convert the resolution of the images in the second image set into a preset second resolution to obtain a third image set; A three-dimensional model component module, used for inputting the third image set into a preset NeRF model to obtain a three-dimensional model component of the target transformer; A three-dimensional model module, used for embedding the three-dimensional model component into the pre-modeling result of the target transformer to obtain an updated three-dimensional model of the target transformer; A step repeating module, configured to update the first image set using the received image of the target transformer, and repeat the above steps to update the three-dimensional model; Among them, the second image set module 12 also includes: a first image sorting submodule, which is used to sort the images in the first image set according to the acquisition time; a reference image submodule, which is used to select the first image from the first image set as the reference image; a feature point change rate submodule, which is used to sequentially calculate the feature point change rate of each image after the reference image, wherein the feature point change rate is the rate of change of the number of feature points of each image relative to the previous image, and the number of feature points is calculated according to the image feature matching algorithm; a key image submodule, which is used to take the image corresponding to the feature point change rate as the key image when the feature point change rate is within a preset change rate range, and remove other images between the key image and the reference image; a first image set loop submodule, which is used to select the key image as the reference image, and repeatedly execute the steps of sequentially calculating the feature point change rate of each image after the reference image to the step of when the feature point change rate is within the preset change rate range, until all the images in the first image set are traversed to obtain the second image set.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the real-time three-dimensional modeling method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the real-time three-dimensional modeling method according to any one of claims 1 to 6 are implemented.

Images