A power operation scene and virtual display method

Abstract

The present application relates to the field of three-dimensional reconstruction technology, and in particular to a method for displaying an electric power operation scene and virtualization. It includes: obtaining an image of each preset image acquisition position of a target electric power operation scene on each preset point cloud acquisition track to obtain a scene image list set; determining a target point cloud acquisition track corresponding to the target electric power operation scene according to the accuracy coefficient corresponding to the scene image list set and each preset point cloud acquisition track; obtaining a number of point cloud data containing a target operation area according to the target point cloud acquisition track; constructing a three-dimensional model of the target electric power operation scene according to the point cloud data containing the target operation area, and displaying it; the present application not only ensures the modeling accuracy of key areas, but also saves computing resources. A complete and relatively accurate three-dimensional model can be constructed.

Description

Background Art

[0002] In order to ensure the safety of power operation scenes, modeling of power operation scenes is the main method to obtain power operation scene information. Among the related technologies, the three-dimensional scene modeling method mainly relies on binocular vision technology and lidar technology. The former uses the left and right cameras to capture images, and uses parallax to calculate the depth information of the target to generate three-dimensional information of the scene; the latter is to emit a laser beam to the target, and receive the returned signal for ranging, and construct the corresponding point cloud for three-dimensional reconstruction. However, binocular vision technology is difficult to reconstruct areas with unclear features; and the ranging range of lidar technology is limited by cost, the reconstruction accuracy of multiple scans is poor, and the objects in the power operation scene are complex and diverse, so it is difficult to directly construct a complete and accurate three-dimensional image for the scene panorama. Summary of the invention

[0003] The technical problem to be solved by this application is: how to provide an electric power operation scene and a virtual display method to solve some problems existing in the prior art.

[0004] In view of the above technical problems, according to the first aspect of the present application, a power operation scene and a virtual display method are provided, comprising the following steps:

[0005] S100, acquiring an image of each preset image acquisition position of the target power operation scene on each preset point cloud acquisition track to obtain a scene image list set; the preset point cloud acquisition track is a circle with the geometric center of the minimum bounding box where the target operation area of ​​the target power operation scene is located as the center;

[0006] S200, determining a target point cloud acquisition track corresponding to a target power operation scene according to the scene image list set and the accuracy coefficient corresponding to each preset point cloud acquisition track;

[0007] S300, acquiring a track according to the target point cloud, and acquiring a number of point cloud data including a target operation area;

[0008] S400, constructing a three-dimensional model of a target power operation scene based on a number of point cloud data including a target operation area, and displaying the model; wherein the target power operation scene includes a target operation area and a non-target operation area; the three-dimensional model of the target power operation scene is constructed based on the point cloud data and two-dimensional image data of the target operation area and the two-dimensional image data of the non-target operation area.

[0009] This application has at least the following beneficial effects:

[0010] The present application provides an electric power operation scene and a virtual display method, in which a target electric power operation scene includes a target operation area and a non-target operation area. Since the target operation area in the electric power operation scene is an electric power target on a pole, including lightning arresters, insulators, etc., which is at a relatively high height, if it is in a jungle or a city, when constructing a three-dimensional model, there will be occlusion when collecting point cloud data, which will cause inaccurate collected data. In order to improve the accuracy of point cloud data acquisition as much as possible and avoid occlusion, the present application first obtains the corresponding image according to the preset image acquisition position set on several preset point cloud acquisition tracks, and then selects the one with the least occlusion or no occlusion on several preset point cloud acquisition tracks as the target point cloud acquisition track. However, due to the different parameters and configurations of different point cloud acquisition devices, the quality, acquisition speed, difficulty, etc. of the point cloud data acquired at different distances from the target are also different. Therefore, each preset point cloud acquisition track has a corresponding accuracy coefficient, which indicates the accuracy (including the quality, acquisition speed, acquisition difficulty, etc. of the point cloud data) when the point cloud acquisition device collects the point cloud data of the target operation area on the corresponding preset point cloud acquisition track; that is, the higher the accuracy coefficient, the more balanced the quality, difficulty and accuracy of the point cloud data collected on the corresponding preset point cloud acquisition track. Thus, the comprehensive score of each preset point cloud acquisition track is obtained. Here, since some preset point cloud acquisition tracks have a suitable distance, but may have more occlusions, the comprehensive score of each preset point cloud acquisition track is comprehensively determined according to the scene image list set and the accuracy coefficient corresponding to each preset point cloud acquisition track. The higher the comprehensive score, the more accurate the point cloud data collected in the corresponding preset point cloud acquisition track.

[0011] Furthermore, after determining the target preset point cloud acquisition track, point cloud data of the target operation area is collected in all directions on the track. Due to the large range of the non-target operation area and the limited ranging unit of the point cloud acquisition equipment, the corresponding two-dimensional image data of the non-target operation area is obtained, and finally the three-dimensional model corresponding to the target power operation area is obtained.

[0012] This application uses different modeling methods to perform three-dimensional modeling on the target operating area and the target non-operating area of ​​the target power operating area, solving the modeling difficulty problem of directly constructing a complete three-dimensional image due to factors such as complex environment. And obtain the three-dimensional data of the target operating area, and the two-dimensional image data of the non-target operating area, that is, for the target operating area with high accuracy requirements and small area, the laser radar technology with high accuracy but small corresponding range is used, and for the non-target operating area with low accuracy requirements and large area, two-dimensional image data is used, which saves resources, and on the basis of ensuring the modeling accuracy of the key area (target operating area), it also saves computing resources. A complete and relatively accurate three-dimensional model can be constructed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 A flowchart of an electric power operation scenario and a virtual display method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0016] like Figure 1 As shown, an embodiment of the present application provides a power operation scene and a virtual display method, including:

[0017] S100, acquiring an image of each preset image acquisition position of the target power operation scene on each preset point cloud acquisition track to obtain a scene image list set; the preset point cloud acquisition track is a circle with the geometric center of the minimum bounding box where the target operation area of ​​the target power operation scene is located as the center.

[0018] Specifically, the target power scene is a scene containing power targets, wherein the power targets include power poles, etc. The target power scene includes a target operation area and a non-target operation area. Among them, the target operation area includes power targets on the power poles, etc., specifically including lightning arresters, insulators, etc., which are at a high height. The non-target operation area may be trees, houses, fences, etc. Further, step S100 also includes:

[0019] S110, determining a point cloud acquisition distance range of the target electric power operation scene according to a target operation area of ​​the target electric power operation scene.

[0020] Here, the point cloud acquisition device has different point cloud acquisition distance ranges for work areas of different sizes, that is, within the point cloud acquisition distance range, the point cloud acquisition device can obtain point cloud data containing the complete work area, but the workload, difficulty and quality of acquiring point cloud data at different distances are different. When scanning at close range, the density of laser point clouds is usually higher, and more detailed information can be captured. However, a too close scanning distance may also increase the difficulty and cost of scanning because the scanning device needs to be moved more frequently or multiple scanning sites need to be set up. Long-distance scanning can cover a wider area, reduce the number of scanning sites and mobile scanning devices, and improve scanning efficiency, but as the scanning distance increases, the density of the laser point cloud will decrease, and it may not be possible to capture all the detailed information of the object. Therefore, for each work area, there is a corresponding optimal point cloud acquisition distance, and the comprehensive workload, difficulty and quality of collecting point cloud data at this optimal point cloud acquisition distance are the most balanced. In this embodiment, since the target power operation scene may contain high obstacles that block the target operation area, in this embodiment, the optimal point cloud acquisition distance and the closest distance that can acquire point cloud data containing the complete operation area are used as the two end values ​​of the point cloud acquisition distance range, respectively, to prevent the presence of many obstacles in the preset point cloud acquisition track corresponding to the optimal point cloud acquisition distance.

[0021] S120, according to the point cloud acquisition distance range of the target power operation scene, obtain a preset point cloud acquisition track list G=(G1, G2, ..., G i , …, G n );i=1,2,…,n;where n is the number of preset point cloud acquisition tracks; G i is the identifier of the i-th preset point cloud acquisition track; each preset point cloud acquisition track has a corresponding accuracy coefficient; each preset point cloud acquisition track has a corresponding preset image acquisition position; the distance between any two adjacent preset point cloud acquisition tracks is equal; the horizontal distance between any preset point cloud acquisition track and the edge of the target operating area belongs to the point cloud acquisition distance range; any two preset point cloud acquisition tracks are on the same plane.

[0022] Here, within the above point cloud acquisition distance range, a preset point acquisition track is set for each preset distance, wherein the preset point cloud acquisition track is a circle with the geometric center of the minimum bounding box where the target operation area of ​​the target power operation scene is located as the center. In order to improve the accuracy of the target operation area data acquisition and the obtained model, the target operation area is surrounded by data acquisition, and therefore, a preset point cloud acquisition track is set to surround the collection of point cloud data.

[0023] Furthermore, due to the differences in the quality, acquisition speed and difficulty of point cloud data acquired at different distances, each point cloud acquisition track has a corresponding accuracy coefficient, among which the preset point cloud acquisition track corresponding to the optimal point cloud acquisition distance (the maximum value of the point cloud acquisition distance range) has the largest accuracy coefficient.

[0024] S130, according to G, acquiring a corresponding scene image at each preset image acquisition position corresponding to each preset point cloud acquisition track, so as to obtain a scene image list set C=(C1, C2, ..., C i , …, C n );C i Get the scene image list corresponding to the track for the i-th preset point cloud; C i =(CT i，1 , CT i，2 ,…,CT i，a ,…,CT i，f（i） ); a = 1, 2, ..., f (i); f (i) is the number of preset image acquisition positions corresponding to the i-th preset point cloud acquisition track; CT i，a A scene image corresponding to the ath preset image acquisition position of the i-th preset point cloud acquisition track; each scene image has a corresponding viewing angle range; the edges corresponding to the viewing angle ranges of any two adjacent scene images on the same preset point cloud acquisition track overlap.

[0025] Here, a corresponding preset image acquisition position is set on each preset point cloud acquisition track, and each scene image has a corresponding viewing angle range; the corresponding edges of the viewing angle ranges of any two adjacent scene images on the same preset point cloud acquisition track coincide with each other, that is, according to each corresponding preset image acquisition position, all images on the preset point cloud acquisition track are acquired (every two adjacent images can be seamlessly spliced).

[0026] S200, determining a target point cloud acquisition track corresponding to a target power operation scene according to the scene image list set and the accuracy coefficient corresponding to each preset point cloud acquisition track.

[0027] Specifically, the step S200 includes:

[0028] S210, according to the scene image list set C, obtain the scene image scoring list set CP = (CP1, CP2, ..., CP i , …, CP n );CP i Get the scene image score list corresponding to the track for the i-th preset point cloud; CP i =(CPT i，1 , CPT i，2 , …, CPT i，a , …, CPT i，f（i）); CPT i，a The occlusion score of the scene image corresponding to the a-th preset image acquisition position of the acquired i-th preset point cloud acquisition track; CPT i，a =Z i，a / F i，a ; Z i，a The area of ​​the obstacle in the scene image corresponding to the a-th preset image acquisition position of the acquired i-th preset point cloud acquisition track; F i，a The total area of ​​the scene image corresponding to the a-th preset image acquisition position of the acquired i-th preset point cloud acquisition track.

[0029] Here, after acquiring each scene image, each scene image is scored, wherein the score is scored according to the area of ​​obstacles contained in each scene image, that is, the larger the area of ​​obstacles in the scene image, the higher the corresponding occlusion score. The occlusion score indicates the corresponding obstacle occlusion of the preset point cloud acquisition track at this viewing angle.

[0030] S220, according to CP and the accuracy coefficient corresponding to each preset point cloud to obtain the track, a preset point cloud to obtain the track comprehensive score list GP = (GP1, GP2, ..., GP i ,…,GP n );GP i G i The corresponding preset point cloud obtains the track comprehensive score; GP i =1 / (avg(CP i ))*D i ;avg() is the preset average value determination function;D i G i The corresponding precision coefficient.

[0031] Here, the comprehensive score of the preset point cloud acquisition track represents the occlusion situation of the preset point cloud acquisition track at each viewing angle and the accuracy coefficient corresponding to the position of the preset point cloud acquisition track itself. The more occlusions (the more obstacles there are), the more difficult it is for the preset point cloud acquisition track to collect point cloud information, and the collected point cloud data will also be biased. Therefore, the comprehensive score of the preset point cloud acquisition track is inversely proportional to the corresponding occlusion score and directly proportional to the corresponding accuracy coefficient.

[0032] S230, determining the preset point cloud acquisition track corresponding to MAX(GP) as the target point cloud acquisition track; wherein MAX() is a preset maximum value determination function.

[0033] Here, the preset point cloud acquisition track with the largest comprehensive score in the GP is determined as the target point cloud acquisition track. That is, the target point cloud acquisition track is the best point cloud acquisition track obtained by comprehensively considering the occlusion when collecting point cloud data at the corresponding position, as well as the collection difficulty, collection speed, data accuracy and other dimensions at the corresponding position.

[0034] S300, obtaining a track according to the target point cloud, and obtaining a number of point cloud data including a target operation area.

[0035] Specifically, the target point cloud acquisition track is provided with a plurality of point cloud acquisition positions, where the corresponding point cloud data can be acquired by an airborne point cloud acquisition device.

[0036] Before step S400, the method further includes:

[0037] S001, jointly calibrate the point cloud acquisition device and the binocular camera according to a preset joint calibration method.

[0038] Specifically, first, the intrinsic and extrinsic parameters of the binocular camera are obtained through the checkerboard calibration method; secondly, a rectangular calibration plate with labels on the four corners is constructed; then, the calibration plate is collected by the binocular camera to obtain the center coordinates of the label; after that, the calibration plate is scanned by a point cloud acquisition device (lidar, etc.) to obtain the center coordinates of the label in the point cloud; the centers of the four labels on the calibration plate are used as feature points, and constraints are established for the center coordinates of the labels in the lidar coordinate system and the binocular camera coordinate system, and the rotation and translation matrices of the lidar coordinate system and the binocular camera coordinate system are calculated to complete the joint calibration of the binocular camera and the point cloud acquisition device.

[0039] S400, constructing a three-dimensional model of a target power operation scene based on a number of point cloud data including a target operation area, and displaying the model; wherein the target power operation scene includes a target operation area and a non-target operation area; the three-dimensional model of the target power operation scene is constructed based on the point cloud data and two-dimensional image data of the target operation area and the two-dimensional image data of the non-target operation area.

[0040] Specifically, the target point cloud acquisition track is provided with a plurality of point cloud acquisition positions; the step S400 comprises:

[0041] S410, acquiring point cloud data and two-dimensional image data of a corresponding viewing angle according to each point cloud acquisition position, so as to construct a three-dimensional model of the target operation area.

[0042] Wherein, the step S410 includes:

[0043] S411, collecting point cloud data of a corresponding viewing angle according to each point cloud collection position.

[0044] Here, the point cloud data is dedistorted.

[0045] S412, acquiring corresponding two-dimensional image data from each point cloud acquisition position according to the binocular camera.

[0046] S413, mapping the point cloud data corresponding to each point cloud acquisition position with the corresponding two-dimensional image data to obtain target point cloud data corresponding to each point cloud acquisition position; the target point cloud data is the point cloud data corresponding to the target operation area.

[0047] S414, stitching the target point cloud data corresponding to each point cloud acquisition position to construct a three-dimensional model of the target operation area.

[0048] In this embodiment, for the target working area, point cloud recognition is relied upon to perform three-dimensional reconstruction of the working target, and a three-dimensional model proportional to the size of the target working area can be constructed.

[0049] S420, collecting two-dimensional image data of the non-target operation area according to the binocular camera to construct a three-dimensional model of the non-target operation area.

[0050] Wherein, the step S420 includes:

[0051] S421, collecting two-dimensional image data of the non-target operation area according to the binocular camera.

[0052] Wherein, after step S421, the method further includes:

[0053] S002: pre-process the two-dimensional image data.

[0054] Here, the preprocessing includes denoising, image enhancement, light intensity adjustment, etc.

[0055] S422, performing target recognition on the two-dimensional image data of the non-target operation area according to a preset deep learning model to obtain a first recognition target and a second recognition target; wherein the first recognition target is a recognizable target; and the second recognition target is a non-recognizable target.

[0056] As an example: the first recognizable target may be a tree, a house, a fence, etc., and the second recognizable target may be a road, etc.

[0057] S423: Replace each corresponding first recognition target according to a preset model in the model library.

[0058] Here, for each first recognition target, it is directly replaced with a preset model in the model library to reduce the complexity and calculation amount of the modeling algorithm.

[0059] S424, identifying each second recognition target according to the point cloud contour.

[0060] Here, radar point cloud contours are used to represent unrecognizable targets, and the specific implementation algorithm is the Marching cubes point cloud contour generation algorithm.

[0061] Furthermore, the Marching cubes point cloud contour generation algorithm reconstructs the surface by fitting the isosurface in the three-dimensional discrete point cloud data field, thereby generating the three-dimensional contour of the object, wherein the voxel is a cube grid composed of eight adjacent voxel points in the point cloud data. According to the three-dimensional point cloud model, the voxel is divided by the minimum bounding box method; the voxel points containing the volume data content are real points, the background voxel points are virtual points, and the voxel containing both real points and virtual points is the boundary voxel. Traverse all the voxels and extract the boundary voxels; use the triangle piece to fit the isosurface between the real points and virtual points in the boundary voxel, and determine the triangle piece according to the known triangle table; finally, combine all the triangle pieces to construct the point cloud contour reconstruction model.

[0062] S430: aligning the target operation area three-dimensional model with the non-target operation area three-dimensional model to obtain a three-dimensional model of a target power operation scene.

[0063] Here, the three-dimensional model of the target operating area is registered with the three-dimensional model of the non-target operating area by an iterative closest point method to obtain optimized posture parameters to update the global map.

[0064] This embodiment further divides the non-target operation area into recognizable targets and non-recognizable targets through image recognition algorithms. This method directly replaces recognizable targets with preset models, reducing the need for annotation training of traditional data. Non-recognizable targets are represented by point cloud contours, and the target operation area is constructed proportionally using three-dimensional point clouds. The actual size of the target operation area can be measured, which reduces the size of the three-dimensional model while meeting the needs of actual operations and ensuring operation accuracy, taking into account the calculation amount and effect of the model.

[0065] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0066] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0067] In an exemplary embodiment of the present application, an electronic device capable of implementing the above method is also provided.

[0068] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0069] The electronic device according to this embodiment of the present application is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0070] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0071] The storage device stores program codes, which can be executed by the processor, so that the processor executes the steps described in the “exemplary method” section of the specification according to various exemplary embodiments of the present application.

[0072] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0073] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0074] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0075] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface. In addition, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0076] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0077] In an exemplary embodiment of the present application, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product, which includes a program code. When the above program product is run on a terminal device, the above program code is used to enable the above terminal device to execute the steps according to various exemplary implementations of the present application described in the above "Exemplary Method" section of the present specification.

[0078] The above-mentioned program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0079] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0080] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0081] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0082] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0083] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0084] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A power operation scene and virtual display method, characterized in that: The method comprises: S100, acquiring an image of each preset image acquisition position of the target power operation scene on each preset point cloud acquisition track to obtain a scene image list set; the preset point cloud acquisition track is a circle with the geometric center of the minimum bounding box where the target operation area of ​​the target power operation scene is located as the center; S200, determining a target point cloud acquisition track corresponding to a target power operation scene according to the scene image list set and the accuracy coefficient corresponding to each preset point cloud acquisition track; S300, acquiring a track according to the target point cloud, and acquiring a number of point cloud data including a target operation area; S400, constructing a three-dimensional model of a target power operation scene based on a plurality of point cloud data including a target operation area, and displaying the model; wherein the target power operation scene includes a target operation area and a non-target operation area; the three-dimensional model of the target power operation scene is constructed based on the point cloud data and two-dimensional image data of the target operation area and the two-dimensional image data of the non-target operation area; The step S200 includes: S210, according to the scene image list set C, obtain the scene image scoring list set CP = (CP1, CP2, ..., CP i , …, CP n );CP i Get the scene image score list corresponding to the track for the i-th preset point cloud; CP i =(CPT i，1 , CPT i，2 ,…,CPT i，a ,…,CPT i，f（i） ); CPT i，a The occlusion score of the scene image corresponding to the a-th preset image acquisition position of the acquired i-th preset point cloud acquisition track; CPT i，a =Z i，a / F i，a ; Z i，a The area of ​​the obstacle in the scene image corresponding to the a-th preset image acquisition position of the acquired i-th preset point cloud acquisition track; F i，a The total area of ​​the scene image corresponding to the ath preset image acquisition position of the acquired i-th preset point cloud acquisition track; S220, according to CP and the accuracy coefficient corresponding to each preset point cloud to obtain the track, a preset point cloud to obtain the track comprehensive score list GP = (GP1, GP2, ..., GP i ,…,GP n );GP i G i The corresponding preset point cloud obtains the track comprehensive score; GP i =1 / (avg(CP i ))*D i ;avg() is the preset average value determination function;D i G i The corresponding coefficient of precision; S230, determining the preset point cloud acquisition track corresponding to MAX(GP) as the target point cloud acquisition track; wherein MAX() is a preset maximum value determination function.

2. The method for displaying power operation scenes and virtualization according to claim 1, characterized in that: The step S100 includes: S110, determining a point cloud acquisition distance range of the target power operation scene according to a target operation area of ​​the target power operation scene; S120, according to the point cloud acquisition distance range of the target power operation scene, obtain a preset point cloud acquisition track list G=(G1, G2, ..., G i , …, G n );i=1,2,…,n;where n is the number of preset point cloud acquisition tracks; G i is the identifier of the i-th preset point cloud acquisition track; each preset point cloud acquisition track has a corresponding accuracy coefficient; each preset point cloud acquisition track has a corresponding preset image acquisition position; the distance between any two adjacent preset point cloud acquisition tracks is equal; the horizontal distance between any preset point cloud acquisition track and the edge of the target operation area belongs to the point cloud acquisition distance range; any two preset point cloud acquisition tracks are on the same plane; S130, according to G, acquiring a corresponding scene image at each preset image acquisition position corresponding to each preset point cloud acquisition track, so as to obtain a scene image list set C=(C1, C2, ..., C i , …, C n );C i Get the scene image list corresponding to the track for the i-th preset point cloud; C i =(CT i，1 , CT i，2 ,…,CT i，a ,…,CT i，f（i） ); a = 1, 2, ..., f (i); f (i) is the number of preset image acquisition positions corresponding to the i-th preset point cloud acquisition track; CT i，a A scene image corresponding to the ath preset image acquisition position of the i-th preset point cloud acquisition track; each scene image has a corresponding viewing angle range; the edges corresponding to the viewing angle ranges of any two adjacent scene images on the same preset point cloud acquisition track overlap.

3. The method for displaying power operation scenes and virtualization according to claim 2, characterized in that: Before step S400, the method further includes: S001, jointly calibrate the point cloud acquisition device and the binocular camera according to a preset joint calibration method.

4. The method for displaying power operation scenes and virtualization according to claim 3 is characterized in that: The target point cloud acquisition track is provided with a plurality of point cloud acquisition positions; The step S400 includes: S410, acquiring point cloud data and two-dimensional image data of a corresponding viewing angle according to each point cloud acquisition position, so as to construct a three-dimensional model of a target operation area; S420, collecting two-dimensional image data of the non-target operation area according to the binocular camera to construct a three-dimensional model of the non-target operation area; S430: aligning the target operation area three-dimensional model with the non-target operation area three-dimensional model to obtain a three-dimensional model of a target power operation scene.

5. The method for displaying power operation scenes and virtualization according to claim 4, characterized in that: The step S410 includes: S411, collecting point cloud data of a corresponding viewing angle according to each point cloud collection position; S412, acquiring corresponding two-dimensional image data from each point cloud acquisition position according to the binocular camera; S413, mapping the point cloud data corresponding to each point cloud acquisition position with the corresponding two-dimensional image data to obtain target point cloud data corresponding to each point cloud acquisition position; the target point cloud data is the point cloud data corresponding to the target operation area; S414, stitching the target point cloud data corresponding to each point cloud acquisition position to construct a three-dimensional model of the target operation area.

6. The method for displaying power operation scenes and virtualization according to claim 4, characterized in that: The step S420 includes: S421, collecting two-dimensional image data of the non-target operation area using a binocular camera; S422, performing target recognition on the two-dimensional image data of the non-target operation area according to a preset deep learning model to obtain a first recognition target and a second recognition target; wherein the first recognition target is a recognizable target; and the second recognition target is a non-recognizable target; S423, replacing each corresponding first recognition target according to a preset model in the model library; S424, identifying each second recognition target according to the point cloud contour.

7. The method for displaying power operation scenes and virtualization according to claim 6, characterized in that: After step S421, the method further includes: S002: pre-process the two-dimensional image data.

Images