A non-inductive personnel positioning method and system based on a digital twin model

Abstract

The application discloses a kind of personnel positioning methods and systems based on digital twin model, belong to personnel positioning technical field.The method of the present application, including: laser scanning is carried out to transformer substation, to obtain laser scanning data, based on the laser scanning data, the real scene three-dimensional digital twin model of transformer substation is established, and virtual video image acquisition equipment is constructed to the real scene three-dimensional digital twin model;Video image data collected by video acquisition image equipment in transformer substation is obtained, based on the three-dimensional digital twin model, the change area of video image in video image data is determined, personnel in change area are identified, and the spatial relationship of personnel and constructed virtual video image acquisition equipment is determined;Based on the spatial relationship, personnel in the transformer substation is positioned in three dimensions.The personnel in transformer substation is positioned by digital twin model in the present application, greatly reduces the cost of personnel positioning, and provides safety guarantee for personnel in transformer substation.

Description

Technical Field

[0001] The present invention relates to the field of personnel positioning technology, and more specifically, to a non-sensing personnel positioning method and system based on a digital twin model. Background Art

[0002] The number of newly added smart substations exceeds 7,000; in addition, conventional old substations also need to be digitalized and intelligently transformed. Therefore, construction personnel, operation and maintenance personnel, equipment factory personnel, and auxiliary control device factory personnel in the substations need to enter the site for construction, installation and maintenance. The safe operation of on-site personnel has become a major hidden danger to the safety of power grid operation. First, substation operators typically wear near-field sensor badges, and a large number of electronic fences are deployed on-site to ensure safe operation of on-site operators. However, electronic badges and electronic fences are difficult to deploy, and if an outsider is not wearing an electronic badge, the person cannot be located. Second, the spatial location information of personnel can be measured using methods such as lidar, millimeter-wave radar, and binocular cameras, but the cost is at least 100% higher than that of conventional monocular cameras, making large-scale promotion and use impossible, especially since a large number of monocular cameras are already deployed in substations, and the cost of upgrading is huge. Third, personnel positioning based on monocular cameras generally lacks depth information and can only simply identify changing areas in the camera image, but cannot accurately locate personnel. Fourth, all current personnel positioning methods cannot identify personnel globally and can only use audio and video alarms to remind monitoring personnel to review the monitoring images in the background. However, it is impossible to locate multiple locations in the substation on the same map. Summary of the Invention

[0003] To address the above issues, the present invention proposes a method for locating insensitive personnel based on a digital twin model, comprising:

[0004] Performing laser scanning on the substation to obtain laser scanning data, establishing a real-life three-dimensional digital twin model of the substation based on the laser scanning data, and constructing a virtual video image acquisition device for the real-life three-dimensional digital twin model;

[0005] Obtaining video image data collected by a video image acquisition device within the substation, determining a changed area of ​​the video image in the video image data based on the three-dimensional digital twin model, identifying personnel within the changed area, and determining a spatial relationship between the personnel and the constructed virtual video image acquisition device;

[0006] Based on the spatial relationship, personnel in the substation are positioned in three dimensions.

[0007] Optional video capture equipment includes: cameras, drones, and robots within the substation.

[0008] Optionally, a virtual video image acquisition device is constructed for the real-scene three-dimensional digital twin model, including: constructing a virtual camera, a virtual drone and a virtual robot.

[0009] Optionally, constructing a virtual camera includes: calibrating the camera in the substation to determine the internal and external parameters and distortion parameters of the camera in the substation, and constructing the virtual camera based on the internal and external parameters and distortion parameters of the camera.

[0010] Optionally, building a virtual drone includes: obtaining positioning data of the drone in the substation, and aligning the positioning data with the real-scene three-dimensional digital twin model to build a virtual robot.

[0011] Optionally, building a virtual robot includes: obtaining laser point cloud data scanned by the robot in the substation, aligning the laser point cloud data with the real-scene three-dimensional digital twin model to locate the position of the robot, and building a virtual robot based on the robot's position positioning.

[0012] Optionally, the video image data includes: video image data collected by a camera, video image data collected by a drone, and video image data collected by a robot.

[0013] Optionally, the video image data collected by the camera includes: the video image of the substation taken by the camera and the angle information and posture information of the camera.

[0014] Optionally, the video image data collected by the drone includes: video images of the substation taken by the drone and the drone's position information, attitude information and angle information.

[0015] Optionally, the video image data collected by the robot includes: video images of the substation taken by the robot and point cloud information and angle information of the robot.

[0016] On the other hand, the present invention also provides a non-sensitized personnel positioning system based on a digital twin model, comprising:

[0017] A model building module is used to perform laser scanning on the substation to obtain laser scanning data, establish a real-life three-dimensional digital twin model of the substation based on the laser scanning data, and construct a virtual video image acquisition device for the real-life three-dimensional digital twin model;

[0018] A computing module is configured to obtain video image data collected by a video image acquisition device within the substation, determine a changed area of ​​the video image in the video image data based on the three-dimensional digital twin model, identify personnel within the changed area, and determine a spatial relationship between the personnel and the constructed virtual video image acquisition device;

[0019] A positioning module is used to perform three-dimensional positioning of personnel in the substation based on the spatial relationship.

[0020] Optional video capture equipment includes: cameras, drones, and robots within the substation.

[0021] Optionally, a model construction module constructs a virtual video image acquisition device for the real-scene three-dimensional digital twin model, including: constructing a virtual camera, a virtual drone and a virtual robot.

[0022] Optionally, the model construction module constructs a virtual camera, including: calibrating the camera in the substation to determine the internal and external parameters and distortion parameters of the camera in the substation, and constructing the virtual camera based on the internal and external parameters and distortion parameters of the camera.

[0023] Optionally, the model building module builds a virtual drone, including: obtaining positioning data of the drone in the substation, and aligning the positioning data with the real-scene three-dimensional digital twin model to build a virtual robot.

[0024] Optionally, the model building module builds a virtual robot, including: obtaining laser point cloud data scanned by the robot in the substation, aligning the laser point cloud data with the real-scene three-dimensional digital twin model to locate the position of the robot, and building the virtual robot based on the position of the robot.

[0025] Optionally, the video image data includes: video image data collected by a camera, video image data collected by a drone, and video image data collected by a robot.

[0026] Optionally, the video image data collected by the camera includes: the video image of the substation taken by the camera and the angle information and posture information of the camera.

[0027] Optionally, the video image data collected by the drone includes: video images of the substation taken by the drone and the drone's position information, attitude information and angle information.

[0028] Optionally, the video image data collected by the robot includes: video images of the substation taken by the robot and point cloud information and angle information of the robot.

[0029] In yet another aspect, the present invention further provides a computing device comprising: one or more processors;

[0030] a processor for executing one or more programs;

[0031] When the one or more programs are executed by the one or more processors, the above-described method is implemented.

[0032] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, the method described above is implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention provides a method for non-sensing personnel positioning based on a digital twin model, comprising: performing a laser scan on a substation to obtain laser scanning data, establishing a real-life three-dimensional digital twin model of the substation based on the laser scanning data, and constructing a virtual video image acquisition device for the real-life three-dimensional digital twin model; obtaining video image data captured by the video image acquisition device within the substation, determining a change area of ​​the video image in the video image data based on the three-dimensional digital twin model, identifying personnel within the change area, and determining a spatial relationship between the personnel and the constructed virtual video image acquisition device; and performing three-dimensional positioning of personnel within the substation based on the spatial relationship. The present invention locates substation personnel through a digital twin model, greatly reducing the cost of personnel positioning and providing safety protection for personnel within the substation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of the process of the present invention;

[0036] Figure 2 Schematic diagram of a process for implementing the present invention;

[0037] Figure 3 Schematic diagram of the association mapping of the spatial position of the virtual camera in the top view of the substation digital twin model according to an embodiment of the present invention;

[0038] FIG4( a ) is a schematic diagram of the association mapping between virtual camera data and a substation digital twin model according to an embodiment of the present invention;

[0039] FIG4( b ) is a schematic diagram of a process for constructing a virtual camera according to an embodiment of the present invention;

[0040] FIG5( a ) is a schematic diagram illustrating the association mapping of the spatial position of a virtual robot in a top view of a digital twin model of a substation according to an embodiment of the present invention;

[0041] FIG5( b ) is a schematic diagram of precise positioning of a virtual robot according to an embodiment of the present invention;

[0042] FIG5( c ) is a schematic diagram of the association mapping between the virtual robot camera data and the substation digital twin model according to an embodiment of the present invention;

[0043] FIG6( a ) is a schematic diagram illustrating the association mapping of the spatial position of a virtual drone in a top view of a digital twin model of a substation according to an embodiment of the present invention;

[0044] FIG6( b ) is a schematic diagram of the association mapping between the virtual drone camera data and the substation digital twin model according to an embodiment of the present invention;

[0045] FIG7( a ) is a schematic diagram of a person entering a picture changing area according to an embodiment of the present invention;

[0046] FIG7( b ) is a schematic diagram showing the position change of people in the image change area according to an embodiment of the present invention;

[0047] Figure 8 Schematic diagram of the process of mapping the spatial relationship between cameras and personnel according to an embodiment of the present invention;

[0048] Figure 9 Schematic diagram of the structure of the system of the present invention. DETAILED DESCRIPTION

[0049] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to provide a thorough and complete disclosure of the present invention and to fully convey the scope of the present invention to those skilled in the art. The terminology used in the exemplary embodiments shown in the accompanying drawings is not intended to limit the present invention. In the accompanying drawings, identical elements are denoted by the same reference numerals.

[0050] Unless otherwise specified, the terms used herein (including technical terms) have the meanings commonly understood by those skilled in the art. In addition, it is understood that terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0051] Example 1:

[0052] This paper proposes a method for locating non-sensitized personnel based on digital twin models, such as Figure 1 Shown, including:

[0053] Step 1: Perform laser scanning on the substation to obtain laser scanning data, establish a real-life three-dimensional digital twin model of the substation based on the laser scanning data, and construct a virtual video image acquisition device for the real-life three-dimensional digital twin model;

[0054] Step 2: Obtain video image data collected by the video image acquisition device in the substation, determine the changed area of ​​the video image in the video image data based on the three-dimensional digital twin model, identify the personnel in the changed area, and determine the spatial relationship between the personnel and the constructed virtual video image acquisition device;

[0055] Step 3: Based on the spatial relationship, three-dimensionally locate the personnel in the substation.

[0056] Among them, video acquisition image equipment includes: cameras, drones and robots in substations.

[0057] Among them, constructing a virtual video image acquisition device for the real-scene three-dimensional digital twin model includes: constructing a virtual camera, a virtual drone and a virtual robot.

[0058] Among them, constructing a virtual camera includes: calibrating the camera in the substation to determine the internal and external parameters and distortion parameters of the camera in the substation, and constructing the virtual camera based on the internal and external parameters and distortion parameters of the camera.

[0059] Among them, building a virtual drone includes: obtaining the positioning data of the drone in the substation, aligning the positioning data with the real-scene three-dimensional digital twin model to build a virtual robot.

[0060] Among them, building a virtual robot includes: obtaining laser point cloud data scanned by the robot in the substation, aligning the laser point cloud data with the real-scene three-dimensional digital twin model to locate the position of the robot, and building a virtual robot based on the position of the robot.

[0061] Among them, video image data includes: video image data collected by cameras, video image data collected by drones, and video image data collected by robots.

[0062] The video image data collected by the camera includes: the video image of the substation taken by the camera and the angle information and posture information of the camera.

[0063] The video image data collected by the drone includes: video images of the substation taken by the drone and the drone's position information, attitude information and angle information.

[0064] The video image data collected by the robot includes: the video image of the substation taken by the robot and the point cloud information and angle information of the robot.

[0065] The present invention will be further described below in conjunction with embodiments:

[0066] Implementation process, such as Figure 2 Shown, including:

[0067] Step 101: construct a real-life 3D digital twin model of the entire substation by performing a LiDAR scan on the substation.

[0068] Specifically, substation lidar scanning refers to multi-station laser point cloud scanning, where the number of sites is set according to the substation scale, area, equipment density, and equipment occlusion conditions, and lidars are set up at different sites to scan the laser point cloud of the substation.

[0069] Specifically, the spatial coordinates of the lidar are recorded at each lidar scanning site, the laser point cloud is roughly aligned using the spatial coordinates, and precise positioning is achieved using the feature points of the laser point cloud to construct a complete real-life 3D digital twin model of the substation, with the point set P{p1, p2, p3, ..., pn}.

[0070] Specifically, each area and equipment in the substation includes at least lidar scans at different angles to ensure a complete real-life 3D model of the area and equipment.

[0071] Specifically, if Figure 3 As shown, the hardware includes: cameras, robots and drones deployed in substations, calibration tools, secure encryption gateways, and substation digital twin data processing modules.

[0072] Among them, the visible light camera is used to collect electromagnetic wave imaging with a wavelength of 400nm to 780nm, reflecting color, texture and edge information. The camera pixel is not less than 2 million and the resolution is not less than 1920×1080; the lidar is used to transmit and receive electromagnetic waves with wavelengths such as 850nm, 905nm, and 1550nm to reflect the appearance space shape and spatial coordinates; the security encryption gateway is used for data security encryption to realize the secure encryption upload of data from visible light cameras, robots and drones, and can be connected via wired or wireless methods.

[0073] Among them, the system services are deployed on the substation digital twin data processing module, including: 1) digital twin 3D visualization service, which is used to display the substation digital twin model in 3D, and to display the registration and fusion of perception data and inspection data in the digital twin model; 2) substation inspection business service, which is used for task formulation, plan query, task status viewing, result confirmation, result analysis, real-time monitoring, report export, etc. for cameras, robots and drones; 4) digital twin data fusion service, which is used for real-time registration and fusion of camera data in the digital twin model, and time-series-based 3D data query; 5) non-sensing personnel 3D positioning service, which is used for non-sensing 3D positioning based on the digital twin model monocular camera.

[0074] Step 102 (a) calibrates the cameras in the substation, determines the internal and external parameters and distortion parameters of all cameras in the substation, and constructs virtual cameras in the substation digital twin model;

[0075] Specifically, as shown in Figure 4(a), the spatial position (Xc, Yc, Zc) of the camera is calibrated in the digital twin model of the substation, and the camera's measurement direction, effective measurement range and other parameters are set. The camera's internal and external parameters and distortion parameters are obtained through the calibration tool.

[0076] Specifically, by deploying a checkerboard calibration plate near the camera, we obtain the camera's intrinsic parameters: 1 / dx, 1 / dy, r, u0, v0, f, the camera's extrinsic parameters: ω, δ, θ, Tx, Ty, Tz, and the even parameters: k1, k2, k3, p1, p2.

[0077] Specifically, the position and posture of the actually deployed sensing device are transmitted back through a secure encryption gateway, and the position and posture of the virtual sensor are adjusted through the digital twin data processing module to ensure that the data collection orientation of the virtual sensing device is consistent with that of the actually deployed sensing device.

[0078] Specifically, the sensing device is mapped to the substation digital twin model, as shown in Figure 4(b). The camera's coordinate system is Oc-XcYcZc, and the substation digital twin model's coordinate system is Ow-XwYwZw. The coordinates of Oc-XcYcZc are converted to the Ow-XwYwZw coordinate system through the PC transformation.

[0079] In step 102 (b), the robot can align the laser point cloud scanned by its installed laser radar with the substation real-life 3D digital twin model to achieve precise positioning of the robot and build a virtual robot in the substation digital twin model;

[0080] Specifically, as shown in Figure 5(a), the robot's trajectory and the spatial coordinates of the track points (Xr, Yr, Zr) are planned in the substation digital twin model, and the camera's measurement direction, effective measurement range and other parameters are set. The internal and external parameters and distortion parameters of the robot camera are obtained through the calibration tool.

[0081] Specifically, the position and posture of the robot are combined with the substation digital twin model for positioning, which is divided into coarse positioning and fine positioning.

[0082] Specifically, the robot's spatial position coarse positioning is collected through a positioning module, including but not limited to Beidou positioning system, GPS, RTK and other positioning methods.

[0083] Specifically, the positioning module carried by the robot can provide meter-level positioning accuracy. The digital twin model of the substation has absolute spatial coordinates. The meter-level spatial positioning of the positioning module can realize the coarse positioning of the robot in the digital twin model of the substation.

[0084] Specifically, the precise positioning of the inspection equipment in the digital twin model of the substation is shown in Figure 5(b). 1) Feature points are extracted from the coarsely positioned local digital twin model; 2) Feature points are extracted from the point cloud data collected by the robot's lidar; 3) The feature points of the point cloud collected by the robot's lidar are matched with the feature points of the local digital twin model under coarse positioning, and the matching accuracy can reach the millimeter level; 4) The spatial coordinate position and posture of the robot are inferred through the point cloud collected by the robot's lidar; 5) The spatial coordinate position of the robot is mapped to the digital twin model of the substation, as shown in Figure 5(a), realizing the precise positioning of the inspection equipment in the digital twin model of the substation and constructing virtual inspection equipment in the digital twin model of the substation.

[0085] Specifically, the robot camera data is registered and fused with the substation digital twin model, as shown in Figure 5(c). The robot camera's coordinate system is Or-XrYrZr, and the substation digital twin model's coordinate system is Ow-XwYwZw. The Or-XrYrZr coordinates are transformed into the Ow-XwYwZw coordinate system through the Pr transformation.

[0086] In step 102 (c), the UAV can be accurately positioned by using its own RTK, lidar and other precise positioning devices, by registering with the substation real-life 3D digital twin model, and constructing a virtual UAV in the substation digital twin model;

[0087] Specifically, as shown in Figure 6(a), the UAV’s trajectory and the spatial coordinates of the track points (Xu, Yu, Zu) are planned in the substation digital twin model, and the camera’s measurement direction, effective measurement range and other parameters are set. The internal and external parameters and distortion parameters of the UAV camera are obtained through the calibration tool.

[0088] Specifically, the spatial position of the drone is collected through the positioning module installed on it, including but not limited to Beidou positioning system, GPS, RTK and lidar positioning methods.

[0089] Specifically, the registration and fusion of drone camera data and the substation digital twin model is shown in Figure 6(b). The drone camera's coordinate system is Ou-XuYuZu, and the substation digital twin model's coordinate system is Ow-XwYwZw. The Ou-XuYuZu coordinates are converted to the Ow-XwYwZw coordinate system through Pu transformation.

[0090] Step 103 (a): The camera with a pan-tilt system transmits the pan-tilt angle information while transmitting the video image; the camera with a fixed posture only transmits the video image;

[0091] Specifically, the camera pan / tilt angle information and video image data are transmitted back to the substation digital twin data processing module through a secure encrypted gateway to update the posture of the virtual camera in the digital twin model, and perform data registration and fusion and personnel positioning analysis.

[0092] Step 103 (b), when the robot transmits the video image, it also transmits the point cloud and the angle information of the pan / tilt platform;

[0093] Specifically, the robot's laser point cloud, camera pan-tilt angle information, and video image data are transmitted back to the substation digital twin data processing module through a secure encrypted gateway to update the virtual camera's posture in the digital twin model, and perform data registration and fusion and personnel positioning analysis.

[0094] Step 103 (c), when the drone transmits the video image, it also transmits the drone's position, attitude, and gimbal angle information;

[0095] Specifically, the drone's spatial position information, drone posture, camera gimbal angle information, and video image data are transmitted back to the substation digital twin data processing module through a secure encrypted gateway for alignment, fusion, and personnel positioning analysis.

[0096] Step 104, based on the change of the camera image, extract the image change area;

[0097] Specifically, the camera refers to a fixed-position camera, a gimbal-mounted camera, a robot-mounted camera, and a drone-mounted camera.

[0098] Specifically, the spatial position of the device in the image captured by the camera is fixed, such as Figure 7(a) and 7(b) As shown in the figure, when a moving object enters the picture, a changing area will be formed in the adjacent frames. Through the grayscale change of the picture, the digital twin data processing module can quickly extract the changing area of ​​the picture for further analysis.

[0099] Step 105: Identify people in the area where the image changes, give the pixels where the people are located, and transmit the camera information of the people entering back to the substation digital twin model;

[0100] Specifically, as shown in Figure 6(b), for the extracted image change area, based on the trained deep learning personnel recognition algorithm, it can effectively determine whether the image change area is a person intrusion. In the case of a person intrusion, the camera information of the person intrusion is transmitted back to the substation digital twin model;

[0101] Step 106: Based on the position and posture of the camera, internal and external parameters, and the video image captured by the camera, the spatial relationship of the person relative to the camera is determined by measuring the monocular image;

[0102] Specifically, the camera refers to a fixed-position camera, a gimbal-mounted camera, a robot-mounted camera, and a drone-mounted camera.

[0103] Specifically, the spatial position, posture, internal and external parameters, and distortion parameters of the camera are used for monocular measurement, which can determine the spatial coordinates of pixels in the coordinate system of the substation digital twin model.

[0104] Step 107: Map the spatial relationship between the camera and the personnel to the virtual camera / virtual inspection equipment and the substation digital twin model to achieve seamless three-dimensional positioning of all personnel in the station.

[0105] Specifically, the camera refers to a fixed-position camera, a gimbal-mounted camera, a robot-mounted camera, and a drone-mounted camera.

[0106] Specifically, a virtual camera refers to a camera with a fixed position or a camera mounted on a gimbal.

[0107] Specifically, virtual inspection equipment refers to virtual robots and virtual drones.

[0108] Specifically, if Figure 8 As shown in the figure, the spatial coordinates corresponding to the pixels of the personnel are mapped to the digital twin model of the substation, which can realize the three-dimensional positioning of the personnel on the digital twin model of the substation.

[0109] This invention aims to address 1) the challenges of non-contact positioning; 2) the inability of monocular cameras to acquire spatial information; and 3) the difficulty of locating personnel at different locations within the same map. It enables three-dimensional, non-contact positioning of personnel using standard monocular cameras, robot-mounted monocular cameras, and drone-mounted monocular cameras at substations. Compared to methods such as electronic ID badges and lidar, this significantly reduces the cost of safely locating personnel in substations, while enabling precise positioning of personnel on a three-dimensional model and improving the efficiency of remote substation operation and maintenance.

[0110] Example 2:

[0111] The present invention also provides a non-sensitized personnel positioning system 200 based on a digital twin model, such as Figure 9 Shown, including:

[0112] A model building module 201 is configured to perform laser scanning on the substation to obtain laser scanning data, establish a real-life 3D digital twin model of the substation based on the laser scanning data, and construct a virtual video image acquisition device for the real-life 3D digital twin model;

[0113] A calculation module 202 is configured to obtain video image data collected by a video image acquisition device within the substation, determine a changed area of ​​the video image in the video image data based on the three-dimensional digital twin model, identify personnel within the changed area, and determine a spatial relationship between the personnel and the constructed virtual video image acquisition device;

[0114] The positioning module 203 is configured to perform three-dimensional positioning of personnel in the substation based on the spatial relationship.

[0115] Among them, video acquisition image equipment includes: cameras, drones and robots in substations.

[0116] Among them, the model construction module constructs a virtual video image acquisition device for the real-scene three-dimensional digital twin model, including: constructing a virtual camera, a virtual drone and a virtual robot.

[0117] Among them, the model construction module constructs a virtual camera, including: calibrating the camera in the substation to determine the internal and external parameters and distortion parameters of the camera in the substation, and constructing the virtual camera based on the internal and external parameters and distortion parameters of the camera.

[0118] Among them, the model construction module constructs a virtual drone, including: obtaining the positioning data of the drone in the substation, aligning the positioning data with the real-scene three-dimensional digital twin model to build a virtual robot.

[0119] Among them, the model construction module constructs a virtual robot, including: obtaining laser point cloud data scanned by the robot in the substation, aligning the laser point cloud data with the real-scene three-dimensional digital twin model to locate the position of the robot, and building a virtual robot based on the position of the robot.

[0120] Among them, video image data includes: video image data collected by cameras, video image data collected by drones, and video image data collected by robots.

[0121] The video image data collected by the camera includes: the video image of the substation taken by the camera and the angle information and posture information of the camera.

[0122] The video image data collected by the drone includes: video images of the substation taken by the drone and the drone's position information, attitude information and angle information.

[0123] The video image data collected by the robot includes: the video image of the substation taken by the robot and the point cloud information and angle information of the robot.

[0124] The present invention locates substation personnel through a digital twin model, greatly reducing the cost of personnel positioning and providing safety protection for personnel in the substation.

[0125] Example 3:

[0126] Based on the same inventive concept, the present invention also provides a computer device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of the method in the above embodiment.

[0127] Example 4:

[0128] Based on the same inventive concept, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiment.

[0129] It will be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0133] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0134] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for locating non-sensitized personnel based on a digital twin model, characterized in that: The method comprises: Performing laser scanning on the substation to obtain laser scanning data, establishing a real-life three-dimensional digital twin model of the substation based on the laser scanning data, and constructing a virtual video image acquisition device for the real-life three-dimensional digital twin model; Obtaining video image data collected by a video image acquisition device within the substation, determining a changed area of ​​the video image in the video image data based on the three-dimensional digital twin model, identifying personnel within the changed area, and determining a spatial relationship between the personnel and the constructed virtual video image acquisition device; Based on the spatial relationship, three-dimensionally locate the personnel in the substation; The step of constructing a virtual video image acquisition device for the real-scene three-dimensional digital twin model includes: constructing a virtual camera, a virtual drone, and a virtual robot; The construction of the virtual camera includes: calibrating the camera in the substation to determine the internal and external parameters and distortion parameters of the camera in the substation, and constructing the virtual camera based on the internal and external parameters and distortion parameters of the camera; transmitting the position and posture of the actually deployed sensing device through a secure encryption gateway, and adjusting the position and posture of the virtual sensor through a digital twin data processing module to ensure that the data collection orientation of the virtual sensing device is consistent with that of the actually deployed sensing device; The construction of the virtual drone includes: obtaining the positioning data of the drone in the substation, aligning the positioning data with the real-scene three-dimensional digital twin model to construct the virtual drone; extracting feature points from the coarsely positioned local digital twin model; extracting feature points from the point cloud data collected by the robot's laser radar; matching the feature points of the point cloud collected by the robot's laser radar with the feature points of the local digital twin model under coarse positioning, with the matching accuracy reaching the millimeter level; inferring the robot's spatial coordinate position and posture through the point cloud collected by the robot's laser radar; mapping the robot's spatial coordinate position to the substation digital twin model, realizing the precise positioning of the inspection equipment in the substation digital twin model, and constructing the virtual inspection equipment in the substation digital twin model; The construction of the virtual robot includes: obtaining laser point cloud data scanned by the robot in the substation, aligning the laser point cloud data with the real-scene three-dimensional digital twin model to locate the position of the robot, and constructing the virtual robot based on the robot's position positioning; planning the drone's track and the spatial coordinates of the track points in the substation digital twin model, setting the camera's measurement direction and effective measurement range, and obtaining the drone's camera's internal and external parameters and distortion parameters through a calibration tool.

2. The method according to claim 1, characterized in that The video image acquisition equipment includes: cameras, drones and robots in the substation.

3. The method according to claim 1, characterized in that The video image data includes: video image data collected by a camera, video image data collected by a drone, and video image data collected by a robot.

4. The method according to claim 3, characterized in that The video image data collected by the camera includes: the video image of the substation taken by the camera and the angle information and posture information of the camera.

5. The method according to claim 3, characterized in that The video image data collected by the drone includes: the video image of the substation taken by the drone and the position information, attitude information and angle information of the drone.

6. The method according to claim 3, characterized in that The video image data collected by the robot includes: the video image of the substation taken by the robot and the point cloud information and angle information of the robot.

7. A non-sensitized personnel positioning system based on digital twin model, characterized in that: The system comprises: A model building module is used to perform laser scanning on the substation to obtain laser scanning data, establish a real-life three-dimensional digital twin model of the substation based on the laser scanning data, and construct a virtual video image acquisition device for the real-life three-dimensional digital twin model; A computing module is configured to obtain video image data collected by a video image acquisition device within the substation, determine a changed area of ​​the video image in the video image data based on the three-dimensional digital twin model, identify personnel within the changed area, and determine a spatial relationship between the personnel and the constructed virtual video image acquisition device; a positioning module, configured to perform three-dimensional positioning of personnel in the substation based on the spatial relationship; The model construction module constructs a virtual video image acquisition device for the real-scene three-dimensional digital twin model, including: constructing a virtual camera, a virtual drone and a virtual robot; The model construction module constructs a virtual camera, including: calibrating the camera in the substation to determine the internal and external parameters and distortion parameters of the camera in the substation, and constructing the virtual camera based on the internal and external parameters and distortion parameters of the camera; transmitting the position and posture of the actually deployed sensing device through a secure encryption gateway, and adjusting the position and posture of the virtual sensor through the digital twin data processing module to ensure that the data collection orientation of the virtual sensing device is consistent with that of the actually deployed sensing device; The model construction module constructs a virtual drone, including: obtaining the positioning data of the drone in the substation, aligning the positioning data with the real-scene three-dimensional digital twin model to construct the virtual drone; extracting feature points from the coarsely positioned local digital twin model; extracting feature points from the point cloud data collected by the robot's lidar; matching the feature points of the point cloud collected by the robot's lidar with the feature points of the local digital twin model under coarse positioning, with the matching accuracy reaching the millimeter level; inferring the robot's spatial coordinate position and posture through the point cloud collected by the robot's lidar; mapping the robot's spatial coordinate position to the substation digital twin model, realizing the precise positioning of the inspection equipment in the substation digital twin model, and constructing the virtual inspection equipment in the substation digital twin model; The model construction module constructs a virtual robot, including: obtaining laser point cloud data scanned by the robot in the substation, aligning the laser point cloud data with the real-scene three-dimensional digital twin model to locate the robot's position, and constructing the virtual robot based on the robot's position positioning; planning the drone's track and the spatial coordinates of the track points in the substation digital twin model, setting the camera's measurement direction and effective measurement range, and obtaining the drone's camera's internal and external parameters and distortion parameters through calibration tools.

8. The system according to claim 7, characterized in that The video image acquisition equipment includes: cameras, drones and robots in the substation.

9. The system according to claim 7, wherein: The video image data includes: video image data collected by a camera, video image data collected by a drone, and video image data collected by a robot.

10. The system according to claim 9, characterized in that The video image data collected by the camera includes: the video image of the substation taken by the camera and the angle information and posture information of the camera.

11. The system according to claim 9, wherein: The video image data collected by the drone includes: the video image of the substation taken by the drone and the position information, attitude information and angle information of the drone.

12. The system according to claim 9, wherein: The video image data collected by the robot includes: the video image of the substation taken by the robot and the point cloud information and angle information of the robot.

13. A computer device, characterized in that: include: one or more processors; a processor for executing one or more programs; When the one or more programs are executed by the one or more processors, the method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

Images