Substation safety control method and system based on three-dimensional reconstruction model

Abstract

The invention provides a substation safety management and control method and system based on a three-dimensional reconstruction model, and relates to the technical field of safety management and control, and the method comprises the steps: obtaining real-time point cloud data inputted by a laser point cloud sensor and a real-time image collected by a camera; comparing the real-time point cloud data with a background point cloud model, and extracting a changed dynamic region by using a background difference algorithm; the dynamic reconstruction module is used for identifying a first position of a first dynamic target in the dynamic region based on the real-time point cloud data; calculating a second position of a second dynamic target contained in the real-time image; if the first position is successfully matched with the second position and the sparse structure of the real-time point cloud data is successfully matched with dense pixels of the real-time image, determining that the first dynamic target and the second dynamic target are both dynamic flight targets; and determining the position of the dynamic flight target according to the first position and the second position, and marking in the three-dimensional model. The safety management and control precision of the transformer substation can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of safety management and control, and specifically to a substation safety management and control method and system based on a three-dimensional reconstruction model. Background Art

[0002] In the power system, substations are the core hubs for power transmission and conversion, and their safety and stability are directly related to the normal operation of the entire power grid. However, traditional substation monitoring and early warning methods mainly rely on manual inspections and a small number of fixed monitoring equipment, which not only limits the efficiency and coverage of monitoring, but also leads to high operating costs. More importantly, these traditional methods are relatively slow in responding to emergencies, and it is difficult to meet the stringent requirements of modern power systems for substations to achieve real-time monitoring and efficient safety early warning. Therefore, there is an urgent need for an efficient and intelligent monitoring system to improve the safety and responsiveness of substations and ensure the stable operation of the power grid.

[0003] At present, with the development of 3D laser scanning technology and computer vision technology, the application of 3D remodeling technology in power systems has gradually received attention. Using 3D point cloud data for monitoring and early warning of substation equipment can obtain the 3D spatial information of the equipment more comprehensively and accurately. When dealing with diverse scenarios in complex environments, the current abnormal behavior recognition algorithm is prone to false alarms and missed alarms. For example, for drones or small target objects, the accuracy of recognition is affected by the size and speed of the object, and missed alarms often occur. Summary of the invention

[0004] The present application provides a substation safety management and control method and system based on a three-dimensional reconstruction model, which can improve the safety management and control accuracy of the substation.

[0005] In a first aspect of the present application, a substation safety management and control method based on a three-dimensional reconstruction model is provided, the method comprising: Obtain real-time point cloud data input by the laser point cloud sensor and real-time images collected by the camera; Comparing the real-time point cloud data with the background point cloud model point by point, and extracting the changed dynamic area using a background difference algorithm; A dynamic reconstruction module based on the real-time point cloud data identifies a first position of a first dynamic target in the dynamic area; Calculating a second position of a second dynamic target included in the real-time image; If the first position is successfully matched with the second position, and the sparse structure of the real-time point cloud data is successfully matched with the dense pixels of the real-time image, it is determined that the first dynamic target and the second dynamic target are both dynamic flying targets; The position of the dynamic flying target is determined according to the first position and the second position, and is marked in the three-dimensional model.

[0006] Optionally, comparing the real-time point cloud data with the background point cloud model point by point and extracting the changed dynamic area using a background difference algorithm specifically includes: Generate a real-time point cloud model according to the real-time point cloud data; Clustering and segmenting the real-time point cloud model to obtain a plurality of real-time voxel units, and clustering and segmenting the background point cloud model to obtain a plurality of background voxel units; Compare the plurality of real-time voxel units with the plurality of background voxel units one by one, and determine a first voxel unit corresponding to the background voxel unit among the plurality of real-time voxel units; A first voxel unit among the plurality of real-time voxel units is removed to obtain a second voxel unit, where the second voxel unit is the dynamic area.

[0007] Optionally, the dynamic reconstruction module based on the real-time point cloud data identifies a first position of a first dynamic target in the dynamic area, specifically comprising: Determining a relative position of the second voxel unit and the first voxel unit according to the real-time point cloud model; Aligning the real-time point cloud model with the background point cloud model through a point cloud registration algorithm; Obtaining the actual coordinate position of the background point cloud model; The first position of the first dynamic target is determined according to the relative position and the actual coordinate position.

[0008] Optionally, before successfully matching the sparse structure of the real-time point cloud data with the dense pixels of the real-time image, the method further includes: Spatially align the laser point cloud sensor and the camera to establish a unified coordinate system; Calculating a feature descriptor for a first key point in the real-time image to obtain a first descriptor, and calculating a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor; comparing the first descriptor with the second descriptor, and matching the first descriptor with the second descriptor whose similarity exceeds a preset threshold; If it is determined that the matching rate of the plurality of first descriptors reaches a preset condition, or the matching rate of the plurality of second descriptors reaches a preset condition, the coordinates of the first key point are mapped to the dynamic reconstruction model to calculate the position of the dynamic flying target.

[0009] Optionally, comparing the real-time point cloud data with the background point cloud model point by point, and extracting the changed dynamic area using a background difference algorithm, specifically includes: According to a predefined side length size, subdividing the real-time point cloud model into a plurality of first voxel points and subdividing the background point cloud model into a plurality of second voxel points; Calculate the attribute characteristics of each real-time voxel unit according to the geometric data of the first voxel point contained in the real-time voxel unit, wherein the attribute characteristics include point density, average coordinates and geometric center; Calculate the attribute characteristics of each background voxel unit according to the geometric data of the second voxel point contained in the background voxel unit, wherein the attribute characteristics include point density, average coordinates and geometric center; The real-time voxel units are compared with the background voxel units one by one, and dynamic voxel units whose attribute feature change values ​​exceed a preset standard among the multiple real-time voxel units are determined, and the dynamic voxel units are determined to be the dynamic areas.

[0010] Optionally, calculating a feature descriptor for a first key point in the real-time image to obtain a first descriptor specifically includes: The real-time image is downsampled multiple times to generate a series of image hierarchical structures with different resolutions. For each image hierarchical structure with different resolutions, Gaussian filtering is applied to the image to smooth the image and generate Gaussian blurred images of different scales. The Gaussian blurred images are represented as follows:

[0011] Wherein, I(x,y) is the Gaussian blurred image, L(x,y,σ) is the Gaussian blurred image, G(x,y,σ) is the Gaussian kernel, Represents the convolution operation; The difference between Gaussian blurred images of adjacent scales is calculated to generate a differential Gaussian image, which is represented as follows:

[0012] Wherein, D(x, y, σ) is a differential Gaussian image, L(x, y, σ) is the Gaussian blurred image, L(x, y, kσ) is a Gaussian blurred image adjacent to the Gaussian blurred image, and k is a constant multiple; Compare any pixel point in the differential Gaussian image with multiple adjacent pixel points. If the spatial value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, and the scale value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, then the any pixel point is marked as a key point, and the first key point is obtained. The amplitude and direction of the pixel points in the neighborhood of the first key point are calculated, specifically by the following formula:

[0013]

[0014] Wherein, m(x, y) is the amplitude of the pixel points in the neighborhood of the first key point, θ is the direction of the pixel points in the neighborhood of the first key point, L(x+1, y) and L(x-1, y) respectively represent the pixel values ​​of the left and right adjacent pixels of the first key point in the horizontal direction of the differential Gaussian image, and L(x, y+1) and L(x, y-1) respectively represent the pixel values ​​of the upper and lower adjacent pixels of the first key point in the vertical direction of the differential Gaussian image; Divide the area of ​​the first key point into a plurality of sub-areas of the same size, and calculate a histogram of each of the sub-areas according to the amplitude and direction of the pixel points in the neighborhood of the key point; The multiple histograms are combined to form a multi-dimensional vector to obtain the first descriptor.

[0015] Optionally, calculating a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor specifically includes: Calculate the point curvature of any point cloud point corresponding to the real-time point cloud data, and the point curvature is calculated by the following formula:

[0016] Where C is the point curvature, n i is the normal vector of the neighboring points of any one of the point cloud points, n is the normal vector of any one of the point cloud points, and k is the number of neighboring points of any one of the point cloud points; If it is determined that the curvature of any one of the point cloud points is greater than a preset curvature, the any one of the point cloud points is determined as a key point, and the second key point is obtained; Taking the second key point as the center, dividing the neighborhood into a plurality of spherical cells according to the direction of the normal vector of the second key point; A normal vector direction histogram is calculated in each of the spherical cells, and features of the spherical cells are accumulated to generate an overall direction histogram, wherein data of the direction histogram is the second descriptor.

[0017] In a second aspect of the present application, a substation safety management and control system based on a three-dimensional reconstruction model is provided, the system comprising an acquisition module, an extraction module, an identification module, a processing module and a judgment module, wherein: The acquisition module is used to acquire real-time point cloud data input by the laser point cloud sensor and real-time images collected by the camera; The extraction module is used to compare the real-time point cloud data with the background point cloud model point by point, and extract the changed dynamic area using the background difference algorithm; The recognition module is used to identify the first position of the first dynamic target in the dynamic area based on the dynamic reconstruction module of the real-time point cloud data; The processing module is used to calculate a second position of a second dynamic target included in the real-time image; The judgment module is used to determine that the first dynamic target and the second dynamic target are both dynamic flying targets if the first position is successfully matched with the second position and the sparse structure of the real-time point cloud data is successfully matched with the dense pixels of the real-time image; The processing module is used to determine the position of the dynamic flying target according to the first position and the second position, and mark it in the three-dimensional model.

[0018] Optionally, the processing module is used to generate a real-time point cloud model according to the real-time point cloud data; The processing module is used to cluster and segment the real-time point cloud model to obtain a plurality of real-time voxel units, and cluster and segment the background point cloud model to obtain a plurality of background voxel units; The judgment module is used to compare the multiple real-time voxel units with the multiple background voxel units one by one, and determine the first voxel unit corresponding to the background voxel unit among the multiple real-time voxel units; The processing module is used to remove a first voxel unit from the plurality of real-time voxel units to obtain a second voxel unit, where the second voxel unit is the dynamic area.

[0019] Optionally, the processing module is used to determine the relative position of the second voxel unit and the first voxel unit according to the real-time point cloud model; The processing module is used to align the real-time point cloud model with the background point cloud model through a point cloud registration algorithm; The acquisition module is used to acquire the actual coordinate position of the background point cloud model; The judgment module is used to determine the first position of the first dynamic target according to the relative position and the actual coordinate position.

[0020] Optionally, the processing module is used to spatially align the laser point cloud sensor and the camera to establish a unified coordinate system; The processing module is used to calculate a feature descriptor for a first key point in the real-time image to obtain a first descriptor, and calculate a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor; The judgment module is used to compare the first descriptor with the second descriptor, and match the first descriptor and the second descriptor whose similarity exceeds a preset threshold; The identification module is used to map the coordinates of the first key point to the dynamic reconstruction model to calculate the position of the dynamic flying target if it is determined that the matching rate of multiple first descriptors reaches a preset condition, or the matching rate of multiple second descriptors reaches a preset condition.

[0021] Optionally, the processing module is used to subdivide the real-time point cloud model into a plurality of first voxel points and subdivide the background point cloud model into a plurality of second voxel points according to a predefined side length size; The processing module is used to calculate the attribute characteristics of each real-time voxel unit according to the geometric data of the first voxel point contained in the real-time voxel unit, and the attribute characteristics include point density, average coordinates and geometric center; The processing module is used to calculate the attribute characteristics of each background voxel unit according to the geometric data of the second voxel point contained in the background voxel unit, and the attribute characteristics include point density, average coordinates and geometric center; The judgment module is used to compare the real-time voxel units with the background voxel units one by one, determine the dynamic voxel units whose attribute feature change values ​​exceed the preset standard among the multiple real-time voxel units, and determine the dynamic voxel units as the dynamic areas.

[0022] Optionally, the processing module is used to downsample the real-time image multiple times to generate a series of image hierarchical structures with different resolutions, and for each image hierarchical structure with different resolutions, apply Gaussian filtering to the image to smooth the image and generate Gaussian blurred images of different scales, wherein the Gaussian blurred image is represented as follows:

[0023] Wherein, I(x,y) is the Gaussian blurred image, L(x,y,σ) is the Gaussian blurred image, G(x,y,σ) is the Gaussian kernel, Represents the convolution operation; The processing module is used to calculate the difference between Gaussian blurred images of adjacent scales to generate a differential Gaussian image, and the differential Gaussian image is represented as follows:

[0024] Wherein, D(x, y, σ) is a differential Gaussian image, L(x, y, σ) is the Gaussian blurred image, L(x, y, kσ) is a Gaussian blurred image adjacent to the Gaussian blurred image, and k is a constant multiple; The processing module is used to compare any pixel point in the differential Gaussian image with multiple adjacent pixel points. If the spatial value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, and the scale value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, then the any pixel point is marked as a key point, and the first key point is obtained; The processing module is used to calculate the amplitude and direction of the pixel points in the neighborhood of the first key point, specifically by the following formula:

[0025]

[0026] Wherein, m(x, y) is the amplitude of the pixel points in the neighborhood of the first key point, θ is the direction of the pixel points in the neighborhood of the first key point, L(x+1, y) and L(x-1, y) respectively represent the pixel values ​​of the left and right adjacent pixels of the first key point in the horizontal direction of the differential Gaussian image, and L(x, y+1) and L(x, y-1) respectively represent the pixel values ​​of the upper and lower adjacent pixels of the first key point in the vertical direction of the differential Gaussian image; The processing module is used to divide the area of ​​the first key point into a plurality of sub-areas of the same size, and calculate the histogram of each sub-area according to the amplitude and direction of the pixel points in the neighborhood of the key point; The processing module is used to merge the multiple histograms to form a multi-dimensional vector to obtain the first descriptor.

[0027] Optionally, the processing module is used to calculate the point curvature of any point cloud point corresponding to the real-time point cloud data, and the point curvature is calculated by the following formula:

[0028] Where C is the point curvature, n i is the normal vector of the neighboring points of any one of the point cloud points, n is the normal vector of any one of the point cloud points, and k is the number of neighboring points of any one of the point cloud points; The processing module is used for determining the any one point cloud point as a key point if it is determined that the curvature of the any one point cloud point is greater than a preset curvature, and obtaining the second key point; The processing module is used to divide the neighborhood into a plurality of spherical cells based on the direction of the normal vector of the second key point with the second key point as the center; The processing module is used to calculate the normal vector direction histogram in each of the spherical cells, accumulate the features of the spherical cells, and generate an overall direction histogram, and the data of the direction histogram is the second descriptor.

[0029] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0030] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0031] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. This application collects real-time data through laser point cloud sensors and cameras, uses background difference algorithms to detect dynamic areas, and combines the matching results of real-time point cloud data and image data to achieve accurate recognition and positioning of dynamic flying targets. The three-dimensional spatial information of the point cloud and the dense visual details of the image are combined. After the dynamic target is identified, its type and position are confirmed through multi-data source matching, which significantly reduces the false alarm and missed alarm rates and ensures efficient monitoring and positioning of small and fast targets such as drones. Finally, by marking the position of dynamic targets in real time in the three-dimensional model, comprehensive, accurate and efficient safety control of the substation environment is achieved, thereby greatly improving the safety control accuracy of the substation.

[0032] 2. Dynamic areas are extracted by comparing the real-time point cloud data with the background point cloud model point by point, and combining voxel segmentation and background difference algorithms. Specifically, by subdividing the point cloud model into voxel units, the geometric properties of the real-time voxel units and the background voxel units are compared one by one, and the dynamic voxel areas with obvious changes are identified. This process can effectively filter out the static background and only retain the dynamically changing areas, so that the system can accurately identify abnormal activities around the equipment. The probability of false alarms and missed alarms is significantly reduced in complex environments, thereby greatly improving the accuracy and response efficiency of substation safety monitoring.

[0033] 3. Perform multi-scale processing on real-time images, apply Gaussian blur and differential Gaussian calculations, accurately extract key points and generate their feature descriptors. This process effectively improves the detection capability of feature points at different scales and ensures the robustness of feature points to image rotation and illumination changes. By calculating the amplitude and direction of pixels in the neighborhood and constructing a directional histogram, the first descriptor finally generated not only fully describes the local features of the key points, but also effectively distinguishes different objects in the image. The efficient extraction and accurate description of key points in a dynamic environment provides a solid foundation for subsequent target recognition and matching, thereby significantly improving the overall performance and reliability of the substation safety monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a substation safety management and control method based on a three-dimensional reconstruction model disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of a substation safety management and control system based on a three-dimensional reconstruction model disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0035] Explanation of the reference numerals: 201, acquisition module; 202, extraction module; 203, identification module; 204, processing module; 205, judgment module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0036] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0037] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0038] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0039] As the core hub of the power system, the traditional monitoring method of substations relies on manual inspections and fixed equipment. Not only is the monitoring efficiency low and the operating cost high, but the response is slow in emergencies, making it difficult to meet the real-time monitoring needs. With the development of 3D laser scanning and computer vision technology, monitoring methods based on 3D point clouds are gradually applied to substations, which can accurately obtain the spatial information of equipment and improve safety. However, the current abnormal behavior recognition algorithm is prone to false alarms and missed alarms in complex scenarios, and the detection effect of small and fast targets (such as drones) is poor. Therefore, more efficient safety control methods are urgently needed to ensure the stable operation of the power grid.

[0040] This embodiment discloses a substation safety management and control method based on a three-dimensional reconstruction model. Figure 1 , including the following steps S110-S160: S110, acquiring real-time point cloud data input by the laser point cloud sensor and real-time images captured by the camera.

[0041] The embodiment of the present application discloses a substation safety management and control method based on a three-dimensional reconstruction model, which is applied to a server. The server includes but is not limited to electronic devices such as mobile phones, tablet computers, wearable devices, and PCs (Personal Computers), and can also be a background server that runs a substation safety management and control method based on a three-dimensional reconstruction model. The server can be implemented as an independent server or a server cluster composed of multiple servers.

[0042] Deploy laser point cloud sensors in substations to collect 3D point cloud data in the substation area. Point cloud data can accurately describe the shape and position of equipment in space and generate basic data required for 3D reconstruction models. Install cameras in the monitoring area to collect real-time image data to capture 2D information of equipment, personnel and other moving objects. These image data can provide texture and color information to make up for the shortcomings of point cloud data.

[0043] The data collected by the laser point cloud sensor and the camera are transmitted to the server through the network. To ensure that the point cloud and image data are collected in the same time period, the timestamps of the laser sensor and the camera need to be synchronized for accurate alignment in subsequent processing.

[0044] The collected point cloud data is processed by denoising, downsampling, filtering, etc. to remove redundant data and noise. When there is no dynamic interference (such as drones) manually determined, the static background model of the substation is constructed using multi-frame point cloud data to represent the static layout and spatial position of the equipment, providing a benchmark for subsequent dynamic area detection. The background point cloud model is used on the server to generate a high-precision 3D reconstruction model of the substation, which is used for display and monitoring.

[0045] S120, comparing the real-time point cloud data with the background point cloud model point by point, and extracting the changed dynamic area using a background difference algorithm.

[0046] In a possible implementation, the real-time point cloud data is compared point by point with the background point cloud model, and the changed dynamic area is extracted using a background difference algorithm, specifically including: generating a real-time point cloud model based on the real-time point cloud data; clustering and segmenting the real-time point cloud model to obtain multiple real-time voxel units, and clustering and segmenting the background point cloud model to obtain multiple background voxel units; comparing the multiple real-time voxel units with the multiple background voxel units one by one to determine the first voxel unit among the multiple real-time voxel units that corresponds to the background voxel unit; removing the first voxel unit among the multiple real-time voxel units to obtain a second voxel unit, and the second voxel unit is the dynamic area.

[0047] In a possible implementation, the real-time point cloud data is compared point by point with the background point cloud model, and the changed dynamic area is extracted using a background difference algorithm, which specifically includes: according to a predefined side length size, the real-time point cloud model is subdivided into a plurality of first voxel points, and the background point cloud model is subdivided into a plurality of second voxel points; according to the geometric data of the first voxel points contained in the real-time voxel unit, the attribute characteristics of each real-time voxel unit are calculated, and the attribute characteristics include point density, average coordinates and geometric center; according to the geometric data of the second voxel points contained in the background voxel unit, the attribute characteristics of each background voxel unit are calculated, and the attribute characteristics include point density, average coordinates and geometric center; the real-time voxel unit is compared with the background voxel unit one by one, and the dynamic voxel unit whose attribute characteristic change value exceeds the preset standard among the multiple real-time voxel units is determined, and the dynamic voxel unit is determined as a dynamic area.

[0048] Specifically, based on the point cloud data of the current substation collected in real time from the laser point cloud sensor, a 3D point cloud model containing the real-time status of the substation is generated. An appropriate voxel grid size (e.g., 0.1m or 0.2m side length) is determined to divide the spatial range of the point cloud model. The voxel size depends on the size of the target object and the detection accuracy requirements.

[0049] Perform voxel division on the point data of the real-time point cloud model, and refine the point cloud points within the spatial range into multiple "first voxel points". Perform the same voxel division on the point data in the background point cloud model, and divide the background point cloud data into multiple "second voxel points". For each voxel point, calculate the voxel index (V x ,V y ,V z ), assign the point cloud points to the corresponding voxel units, and the voxel index is calculated by the following formula:

[0050] Among them, (x, y, z) are the spatial coordinates of the voxel point, (V x ,V y ,V z ) is the voxel index of the voxel point, and d is the side length of the voxel unit.

[0051] Voxel indexing divides the three-dimensional space into voxel units (i.e., small cubes). The index can be used to quickly find the voxel unit to which each point belongs, making it easier to divide the entire space into multiple small areas. This can simplify the management and processing of large-scale point cloud data, making the data inside each voxel unit easier to access and process. For example, when you need to find the neighborhood points of a point in the point cloud, you can directly find the voxel unit where the point is located, as well as the adjacent voxel units, through the voxel index, thereby speeding up the query of neighborhood points.

[0052] According to the geometric data of the first voxel point contained in the real-time voxel unit, the attribute characteristics of each real-time voxel unit are calculated, and the attribute characteristics include point density, average coordinates and geometric center. According to the geometric data of the second voxel point contained in the background voxel unit, the attribute characteristics of each background voxel unit are calculated, and the attribute characteristics include point density, average coordinates and geometric center.

[0053] Specifically, the total number of points in the voxel unit is calculated, that is, the point density. The point density can reflect the density of the voxel unit and is often used to detect whether a new object appears. The average coordinates of all points in the voxel unit are calculated to represent the average position of the point cloud in space. The calculation formula for the average coordinates is:

[0054] Among them, (X, Y, Z) is the average coordinate of the voxel unit, N is the number of voxel points contained in the voxel unit, (x i ,y i ,z i ) is the coordinate of the ith voxel point, that is, the geometric data of each voxel point. The average coordinate of the voxel unit is used as the geometric center to represent the center position of the voxel unit in three-dimensional space.

[0055] The attribute characteristics of the real-time voxel unit and the calculation method of the background voxel unit are both calculated in the above manner, but these characteristic values ​​of the background voxel unit are fixed in most cases, reflecting the static structural information of the substation.

[0056] Pair the real-time voxel units with the background voxel units one by one and compare whether there are significant differences in their attribute characteristics. For each voxel unit, compare the attribute characteristics (point density, average coordinates and geometric center) of the real-time voxel unit and the corresponding background voxel unit. Use the following comparison method: Calculate the difference in point density between the real-time voxel unit and the background voxel unit. If the difference exceeds the preset threshold, it indicates that a new object or dynamic change may have appeared in the area. Calculate the average coordinate difference between the real-time voxel unit and the background voxel unit. If the difference exceeds the threshold, it means that the position of the area has changed. The change in the center position of the voxel unit is judged by the difference in the geometric center. If the change is large, it may indicate that the object has moved or a new target has been added.

[0057] A predefined change threshold is set, and the voxel units whose attribute feature differences exceed the threshold are judged as "dynamic voxel units", that is, areas with significant changes. All voxels judged as dynamic voxel units are marked as "second voxel units", that is, areas with significant changes. The dynamic voxel unit represents a possible abnormal dynamic area in the substation, such as the entry of drones or personnel. Cluster analysis is performed on the marked dynamic voxel units, and spatially adjacent voxel units are merged to form a complete dynamic area. DBSCAN or connected component labeling algorithm can be used to aggregate dynamic voxel units with a close distance into a coherent dynamic area.

[0058] Alternatively, static voxel units can be identified and removed to obtain dynamic areas. Specifically, the voxel units whose comparison results conform to the static area are marked as "first voxel units", that is, static areas. All first voxel units (static voxel units) are removed from the voxel unit set of the real-time point cloud model, so that the remaining "second voxel units" are obtained, that is, the voxel units containing dynamic changes. The extracted second voxel units are clustered to merge the spatially adjacent dynamic voxel units. A distance-based clustering algorithm (such as DBSCAN) can be used to aggregate voxel units with a close distance into a complete dynamic area. The clustered dynamic area is the area that has changed, which may indicate the activity of a drone or other object. The voxel units in the dynamic area are marked in the three-dimensional model to facilitate subsequent monitoring and early warning.

[0059] S130, a dynamic reconstruction module based on real-time point cloud data identifies a first position of a first dynamic target in a dynamic area.

[0060] In a possible implementation, a dynamic reconstruction module based on real-time point cloud data identifies the first position of the first dynamic target in the dynamic area, specifically including: determining the relative position of the second voxel unit and the first voxel unit according to the real-time point cloud model; aligning the real-time point cloud model with the background point cloud model through a point cloud registration algorithm; obtaining the actual coordinate position of the background point cloud model; and determining the first position of the first dynamic target according to the relative position and the actual coordinate position.

[0061] Specifically, based on the previously extracted dynamic voxel unit (second voxel unit) and static voxel unit (first voxel unit), their three-dimensional positions and voxel indexes are recorded respectively. Based on the voxel indexes of the second voxel unit and the first voxel unit, their relative position difference is calculated. For example: Δx=V x2 -V x1 , Δy=V y2 -V y1 , Δz=V z2 -V z1 , where (V x1 ,V y1 ,V z1 ) is the index position of the first voxel unit (static unit), (V x2 ,V y2 ,V z2 ) is the index position of the second voxel unit (dynamic unit).

[0062] Select an appropriate point cloud registration algorithm (such as the ICP algorithm) to align the real-time point cloud model with the background point cloud model. The ICP (Iterative Closest Point) algorithm is often used for rigid body registration and can align the model by minimizing the Euclidean distance between the two point clouds. Specifically, select an initial alignment method to estimate the initial poses of the real-time point cloud model and the background point cloud model. For the points in the real-time point cloud model, find the closest points in the background point cloud model as the corresponding points. Iteratively update the rotation and translation parameters to minimize the square error between the matching point pairs until convergence. Get a transformation matrix, which is used to align the real-time point cloud data with the background point cloud data in the same coordinate system.

[0063] According to the preset reference of the background point cloud model, the actual coordinate position of each background voxel unit is obtained. For example, the background point cloud data can contain the three-dimensional spatial coordinates (X, Y, Z) of each static voxel unit for positioning. In the unified coordinate system after registration, the coordinate position (X, Y, Z) of each background voxel unit (i.e., the first voxel unit) can be used as a reference reference to provide the actual spatial position.

[0064] Using the calculated relative position difference (Δx, Δy, Δz), the actual coordinate position of the background model is combined with the relative position transformation to determine the actual position of the dynamic voxel unit. Specifically, the actual position (X b ,Y b ,Z b ) as the reference position. According to the relative position difference, the position of the dynamic voxel unit is mapped to the actual coordinate system: X d =X b +Δx·d,Y d =Y b +Δy·d,Z d =Z b +Δz·d where d is the side length of the voxel unit, (X d ,Y d ,Z d ) is the actual spatial coordinate of the dynamic target. If the dynamic region contains multiple voxel units, the coordinates of all dynamic voxel units can be averaged to obtain the center position of the first dynamic target.

[0065] S140, calculating a second position of a second dynamic target included in the real-time image.

[0066] The key feature points in the real-time image captured by the camera are detected using a feature point detection algorithm (such as SIFT, ORB or FAST). The SIFT algorithm is used to detect feature points and calculate descriptors. Through the selected feature points, important locations that may represent dynamic targets (such as drones) can be found in the image. The specific real-time method will be described in the subsequent steps.

[0067] Use target detection algorithms (such as YOLO, Faster R-CNN, etc.) to identify dynamic targets in real-time images. Extract the bounding box position of the dynamic target from the target detection result, which usually includes the coordinates of the upper left corner and the lower right corner (or the center coordinates), and then calculate the second dynamic position of the second dynamic target in the real-time image based on the bounding box position. In this application, calculating the second dynamic position of the second dynamic target in the real-time image is only a conventional technical means in the relevant technical field, and will not be further described here.

[0068] S150: If the first position is successfully matched with the second position, and the sparse structure of the real-time point cloud data is successfully matched with the dense pixels of the real-time image, it is determined that the first dynamic target and the second dynamic target are both dynamic flying targets.

[0069] Ensure that the first position and the second position are in the same coordinate system. If one is in the image coordinate system and the other is in the point cloud coordinate system, they need to be converted to a unified three-dimensional coordinate system through corresponding transformations (such as external parameter matrices). Extract the descriptors of feature points near the first position and the second position (such as SIFT, ORB, FPFH, etc.) to obtain the local feature information of the target. For the converted first position (X 1 ,Y 1 ,Z 1 ) and the second position (X 2 ,Y 2 ,Z 2 ), calculate the Euclidean distance between them, and set a suitable distance threshold (such as 0.1 meters) to determine whether the two locations can be considered to be the same target. If the distance d is less than the threshold, the match is considered successful.

[0070] In a possible implementation, before successfully matching the sparse structure of real-time point cloud data with the dense pixels of the real-time image, the method further includes: spatially aligning the laser point cloud sensor and the camera to establish a unified coordinate system; calculating a feature descriptor for a first key point in the real-time image to obtain a first descriptor, and calculating a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor; comparing the first descriptor with the second descriptor, and matching the first descriptor and the second descriptor whose similarity exceeds a preset threshold; if it is determined that the matching rate of multiple first descriptors meets the preset condition, or the matching rate of multiple second descriptors meets the preset condition, mapping the coordinates of the first key point to the dynamic reconstruction model to calculate the position of the dynamic flying target.

[0071] Specifically, the laser point cloud sensor and the camera are calibrated to determine their relative position and orientation in three-dimensional space. Usually, some known calibration plates or specific geometric shapes are used to record the collected data at different angles and positions. The internal parameters (focal length, principal point, distortion parameters, etc.) and external parameters (position and posture relative to the camera) of the laser point cloud sensor are obtained to establish the spatial relationship between the two. Using the calibration results, the image data captured by the camera and the laser point cloud data are mapped to a common three-dimensional coordinate system. This ensures that subsequent data processing (such as feature point matching) is performed in the same coordinate system, which is convenient for mutual comparison. In the real-time image, the first key points are extracted using feature point detection algorithms (such as SIFT, ORB, etc.), and the positions of these key points in the image are recorded. For each extracted first key point, its feature descriptor is calculated. Taking SIFT as an example, a 128-dimensional descriptor vector is generated.

[0072] In a possible implementation manner, calculating a feature descriptor for a first key point in a real-time image to obtain a first descriptor specifically includes: The real-time image is downsampled multiple times, and the original image is downsampled multiple times (for example, halved each time) to generate a series of image hierarchical structures with different resolutions. These hierarchical structures can capture features at different scales. For each resolution image hierarchy, Gaussian filtering is applied to the image to smooth the image and generate Gaussian blurred images of different scales. The Gaussian blurred images are represented as follows:

[0073] Among them, I(x,y) is the Gaussian blurred image, L(x,y,σ) is the Gaussian blurred image, G(x,y,σ) is the Gaussian kernel, Represents the convolution operation, and the Gaussian kernel is defined as:

[0074] Calculate the difference between Gaussian blurred images of adjacent scales to generate a differential Gaussian image. The differential Gaussian image is represented as follows:

[0075] Among them, D(x,y,σ) is the difference Gaussian image, L(x,y,σ) is the Gaussian blurred image, L(x,y,kσ) is the Gaussian blurred image adjacent to the Gaussian blurred image, and k is a constant multiple. By calculating the difference between Gaussian blurred images of adjacent scales, the difference Gaussian image (DoG) is obtained. This processing method can effectively highlight the edges and significant features in the image. The difference Gaussian image can help the algorithm identify the parts with obvious changes in the image, so as to be more accurate in feature point detection.

[0076] For each pixel in the difference Gaussian image, check its local extreme value in space and scale. The specific method is to compare a pixel with the 26 adjacent points. If the point is a local extreme value (that is, the value of the point is larger or smaller than the surrounding values), then the point is marked as a key point candidate.

[0077] Compare any pixel in the differential Gaussian image with multiple adjacent pixels (including four directions of up, down, left, right, and eight directions of diagonal). If the value of any pixel in space is greater than or less than the value of multiple adjacent pixels in space, and the value of any pixel in scale is greater than or less than the value of multiple adjacent pixels in space, then any pixel is marked as a key point and the first key point is obtained. By comparing the value of the current pixel with the adjacent pixel, the local extreme point (key point) can be identified. The local extreme point corresponds to the significant structure or edge in the image and can represent the important features in the image. This step ensures that only important key points are extracted, thereby improving the efficiency and accuracy of subsequent processing.

[0078] Calculate the amplitude and direction of the pixels in the neighborhood of the first key point, specifically using the following formula:

[0079]

[0080] Among them, m(x,y) is the amplitude of the pixel points in the neighborhood of the first key point, and the magnitude of the gradient amplitude reflects the degree of grayscale change of the image at this point, that is, the edge strength. Generally speaking, points with larger gradient amplitudes appear at the edges of the image or in areas with significant texture. θ is the direction of the pixel points in the neighborhood of the first key point. The value range of the gradient direction is [-π,π], which represents the direction angle of the grayscale change. The direction is usually used to construct a direction histogram in the generation of feature descriptors, thereby giving the key point a main direction and making the SIFT feature rotationally invariant. L(x+1,y) and L(x-1,y) represent the pixel values ​​of the left and right adjacent pixels of the first key point in the horizontal direction of the differential Gaussian image, respectively, and L(x,y+1) and L(x,y-1) represent the pixel values ​​of the upper and lower adjacent pixels of the first key point in the vertical direction of the differential Gaussian image, respectively; The area of ​​the first key point is divided into multiple sub-regions of the same size. The histogram of each sub-region is calculated based on the amplitude and direction of the pixels in the neighborhood of the key point. Usually, the 16×16 neighborhood is divided into 4×4 sub-regions (that is, each sub-region contains 4×4 pixels, a total of 16 sub-regions) for local calculation and construction of the direction histogram. The direction histogram is calculated for each 4×4 sub-region, which can refine the gradient information around the key point and enhance the discrimination of the descriptor.

[0081] Each sub-region generates an 8-direction histogram (it can also be other numbers of directions, but 8 directions are a common choice) to represent the main directional distribution of the gradient. The gradient direction of each pixel in the neighborhood is classified into the corresponding direction interval (usually divided into 8 directions, each direction corresponds to a 45° angle interval), and the gradient amplitude m(x,y) is added as a weight to the histogram count of the direction. The direction is divided into 8 equally divided intervals, each interval represents a 45° direction range (the range from 0° to 360° is evenly divided into 8 intervals). Multiple histograms are merged to form a multidimensional vector to obtain the first descriptor.

[0082] In a possible implementation, calculating a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor specifically includes: Calculate the point curvature of any point cloud point corresponding to the real-time point cloud data. Point curvature is used to measure the surface curvature of each point in the point cloud. The larger the point curvature, the more significant the geometric structure of the area where the point is located. Therefore, point curvature can be used to detect key points. Point curvature is calculated using the following formula:

[0083] Where C is the point curvature, n i is the normal vector of the neighboring points of any point cloud point, n is the normal vector of any point cloud point, k is the number of neighboring points of any point cloud point (defining a radius or a fixed number of neighboring points), for any point cloud point, calculate its normal vector n. The normal vector is usually estimated by the neighborhood points of the point, for example, PCA (principal component analysis) is used to find the normal vector of the local plane.

[0084] The point curvature of each point is compared. If the point curvature value of the point is greater than the preset curvature threshold, the point is marked as a key point to obtain the second key point. Selecting an appropriate curvature threshold can effectively reduce the redundant information in the point cloud and only retain points with higher curvature and richer information as key points.

[0085] All points that meet the curvature condition are gathered together to generate a set containing the second key point. These key points are usually located at the edge of the object or in areas with significant geometric changes, and can better describe the geometric features of the point cloud data. With the second key point as the center, its neighborhood is divided into multiple spherical cells along the direction of its normal vector. The spherical cell is a series of spherical shells (layers) centered on the key point. The thickness of each spherical shell is equal. The center of the sphere is the key point, and the direction of the normal vector is used to determine the direction of the sub-area division. The spherical cell is further divided in the direction of the normal vector. The neighborhood of the key point is usually divided into several sub-areas (such as eight directions) to ensure that the descriptor is rotationally invariant and improve the regional resolution.

[0086] In each spherical cell, the direction of the normal vector of the points therein is calculated, and the distribution of these directions is counted to generate a direction histogram. Each direction histogram represents the direction distribution of the normal vector of the points in the spherical cell. The direction histogram is usually constructed by dividing the normal vector direction into several direction intervals, and each interval represents a fixed direction range. For example, when divided into 8 intervals, each interval corresponds to an angle range of 45°. In each spherical cell, the normal vector directions of all points are accumulated to generate a histogram representing the direction distribution of the spherical cell. This histogram can effectively characterize the local geometry of the area. The direction histograms of all spherical cells are accumulated to form an overall direction histogram. This overall direction histogram contains the characteristics of each spherical cell, forming a high-dimensional vector, namely the second descriptor.

[0087] In order to reduce the amplitude difference of the data, the generated directional histogram is usually normalized so that the value of the descriptor is in a relatively consistent range. The integrated and normalized directional histogram forms the feature descriptor of the key point (i.e., the second descriptor). This descriptor not only contains the spatial position and direction information of the key point, but also reflects the local geometric structure characteristics of the point, making it highly robust in the matching process.

[0088] Furthermore, the first descriptor of the image is compared with the second descriptor of the point cloud, and the Euclidean distance or the Hamming distance is used as a similarity metric. The distance between the first descriptor and the second descriptor is calculated, and if the distance is less than a preset similarity threshold, the pair of descriptors is considered to be matched successfully.

[0089] In the first descriptor set and the second descriptor set, the number of matching pairs that meet the similarity condition (i.e., the distance is less than the threshold) is counted. The matching success rate between the two descriptor sets is calculated. The matching rate refers to the ratio of the number of successfully matched feature point pairs to the total number of feature points, which is used to judge the reliability of the match. If the matching rate of the first descriptor or the matching rate of the second descriptor reaches the preset condition (e.g., 80%), it is judged that the image feature points and point cloud feature points in the area are matched successfully. At this time, it can be determined that the first dynamic target in the dynamic reconstruction model and the second dynamic target in the real-time image are both dynamic flying targets (e.g., drones).

[0090] S160: Determine the position of the dynamic flying target according to the first position and the second position, and mark it in the three-dimensional model.

[0091] The camera and laser point cloud sensor are jointly calibrated to obtain the relative position relationship between the two in space, that is, the external parameters (rotation matrix R and translation vector T). During the calibration process, it is necessary to complete multiple shots with a known three-dimensional calibration plate or feature marker to accurately calculate the external parameter parameters. The calibration parameters are used to align the image coordinate system of the camera and the three-dimensional coordinate system of the laser point cloud to ensure that the first position (in the image) and the second position (in the point cloud) can be mapped to the same coordinate system for position matching.

[0092] Set the first position coordinate in the image to (x img ,y img ). Using the camera's intrinsic and extrinsic parameters, the image plane coordinates are mapped to a three-dimensional coordinate system to form a three-dimensional position in the image (X img ,Y img ,Z img ). The conversion formula is as follows (assuming the depth Z img Can be estimated from point cloud data or known models):

[0093] Among them, R and T are rotation matrices and translation vectors, which are used to transform the image coordinate system into the three-dimensional point cloud coordinate system. The second position (X pc ,Y pc ,Z pc ) is already in the point cloud, so it is already in a 3D coordinate system. img ,Y img ,Z img ) and the second position (X pc ,Y pc ,Z pc ) to reduce the uncertainty of a single data source. The following weighting formula can be used: X target =αX img +(1-α)X pc Y target =αY img +(1-α)Y pc Z target =αZ img +(1-α)Z pc Among them, α is the weight coefficient, which can be set according to the accuracy of image data and point cloud data. It can usually be taken as 0.5 (average fusion) or adjusted according to the reliability of the actual data source.

[0094] In the 3D reconstruction model, the calculated dynamic flight target position is used as a marking point and displayed on the 3D monitoring interface for real-time monitoring by maintenance personnel. When marking, special symbols (such as dots, arrows, or drone icons) can be used to highlight the dynamic flight target in the 3D model for easy identification and tracking. If the target moves within the monitoring area, its marked position is updated in real time so that the 3D model dynamically reflects the current position of the target.

[0095] This embodiment also discloses a substation safety management and control system based on a three-dimensional reconstruction model. Figure 2 The system includes an acquisition module 201, an extraction module 202, an identification module 203, a processing module 204 and a judgment module 205, wherein: The acquisition module 201 is used to acquire real-time point cloud data input by the laser point cloud sensor and real-time images collected by the camera.

[0096] The extraction module 202 is used to compare the real-time point cloud data with the background point cloud model point by point, and extract the dynamic area that has changed using the background difference algorithm.

[0097] The identification module 203 is a dynamic reconstruction module for identifying a first position of a first dynamic target in a dynamic area based on real-time point cloud data.

[0098] The processing module 204 is used to calculate a second position of a second dynamic target included in the real-time image.

[0099] The judgment module 205 is used to determine that the first dynamic target and the second dynamic target are both dynamic flying targets if the first position is matched with the second position successfully and the sparse structure of the real-time point cloud data is matched with the dense pixels of the real-time image successfully.

[0100] The processing module 204 is used to determine the position of the dynamic flying target according to the first position and the second position, and mark it in the three-dimensional model.

[0101] In a possible implementation, the processing module 204 is configured to generate a real-time point cloud model according to the real-time point cloud data.

[0102] The processing module 204 is used to cluster and segment the real-time point cloud model to obtain a plurality of real-time voxel units, and cluster and segment the background point cloud model to obtain a plurality of background voxel units.

[0103] The judgment module 205 is used to compare the multiple real-time voxel units with the multiple background voxel units one by one, and determine the first voxel unit corresponding to the background voxel unit among the multiple real-time voxel units.

[0104] The processing module 204 is used to remove a first voxel unit from the multiple real-time voxel units to obtain a second voxel unit, where the second voxel unit is a dynamic area.

[0105] In a possible implementation, the processing module 204 is configured to determine the relative position of the second voxel unit with respect to the first voxel unit according to the real-time point cloud model.

[0106] The processing module 204 is used to align the real-time point cloud model with the background point cloud model through a point cloud registration algorithm.

[0107] The acquisition module 201 is used to acquire the actual coordinate position of the background point cloud model.

[0108] The determination module 205 is used to determine the first position of the first dynamic target according to the relative position and the actual coordinate position.

[0109] In a possible implementation, the processing module 204 is used to spatially align the laser point cloud sensor and the camera to establish a unified coordinate system.

[0110] The processing module 204 is used to calculate a feature descriptor for a first key point in the real-time image to obtain a first descriptor, and calculate a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor.

[0111] The judgment module 205 is used to compare the first descriptor with the second descriptor, and match the first descriptor and the second descriptor whose similarity exceeds a preset threshold.

[0112] The identification module 203 is used to map the coordinates of the first key point to the dynamic reconstruction model to calculate the position of the dynamic flying target if it is determined that the matching rate of the plurality of first descriptors reaches a preset condition, or the matching rate of the plurality of second descriptors reaches a preset condition.

[0113] In a possible implementation, the processing module 204 is configured to subdivide the real-time point cloud model into a plurality of first voxel points and subdivide the background point cloud model into a plurality of second voxel points according to a predefined side length size.

[0114] The processing module 204 is used to calculate the attribute characteristics of each real-time voxel unit according to the geometric data of the first voxel point contained in the real-time voxel unit, and the attribute characteristics include point density, average coordinates and geometric center.

[0115] The processing module 204 is used to calculate the attribute characteristics of each background voxel unit according to the geometric data of the second voxel point contained in the background voxel unit, and the attribute characteristics include point density, average coordinates and geometric center.

[0116] The judgment module 205 is used to compare the real-time voxel units with the background voxel units one by one, determine the dynamic voxel units whose attribute feature change values ​​exceed the preset standard among the multiple real-time voxel units, and determine the dynamic voxel units as dynamic areas.

[0117] In a possible implementation, the processing module 204 is used to perform multiple downsampling on the real-time image to generate a series of image hierarchical structures with different resolutions. For each image hierarchical structure with different resolutions, Gaussian filtering is applied to the image to smooth the image and generate Gaussian blurred images of different scales. The Gaussian blurred images are represented as follows:

[0118] Among them, I(x,y) is the Gaussian blurred image, L(x,y,σ) is the Gaussian blurred image, G(x,y,σ) is the Gaussian kernel, Represents a convolution operation.

[0119] The processing module 204 is used to calculate the difference between Gaussian blurred images of adjacent scales to generate a differential Gaussian image. The differential Gaussian image is represented as follows:

[0120] Among them, D(x, y, σ) is the difference Gaussian image, L(x, y, σ) is the Gaussian blurred image, L(x, y, kσ) is the Gaussian blurred image adjacent to the Gaussian blurred image, and k is a constant multiple.

[0121] The processing module 204 is used to compare any pixel point in the differential Gaussian image with multiple adjacent pixel points. If the spatial value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, and the scale value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, then any pixel point is marked as a key point to obtain the first key point.

[0122] The processing module 204 is used to calculate the amplitude and direction of the pixel points in the neighborhood of the first key point, specifically by the following formula:

[0123]

[0124] Wherein, m(x,y) is the amplitude of the pixel in the neighborhood of the first key point, θ is the direction of the pixel in the neighborhood of the first key point, L(x+1,y) and L(x-1,y) represent the pixel values ​​of the left and right adjacent pixels of the first key point in the horizontal direction of the differential Gaussian image, and L(x,y+1) and L(x,y-1) represent the pixel values ​​of the upper and lower adjacent pixels of the first key point in the vertical direction of the differential Gaussian image.

[0125] The processing module 204 is used to divide the area of ​​the first key point into a plurality of sub-areas of the same size, and calculate the histogram of each sub-area according to the amplitude and direction of the pixel points in the neighborhood of the key point.

[0126] The processing module 204 is used to merge multiple histograms to form a multi-dimensional vector to obtain a first descriptor.

[0127] In a possible implementation, the processing module 204 is used to calculate the point curvature of any point cloud point corresponding to the real-time point cloud data. The point curvature is calculated by the following formula:

[0128] Where C is the point curvature, n i is the normal vector of the adjacent points of any point cloud point, n is the normal vector of any point cloud point, and k is the number of adjacent points of any point cloud point.

[0129] The processing module 204 is used to determine any point cloud point as a key point if it is determined that the curvature of any point cloud point is greater than a preset curvature, and obtain a second key point.

[0130] The processing module 204 is used to divide the neighborhood into a plurality of spherical cells based on the direction of the normal vector of the second key point and taking the second key point as the center.

[0131] The processing module 204 is used to calculate the normal vector direction histogram in each spherical cell, accumulate the features of the spherical cell, and generate an overall direction histogram. The data of the direction histogram is the second descriptor.

[0132] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0133] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0134] The communication bus 302 is used to realize the connection and communication between these components.

[0135] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0136] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0137] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0138] Among them, the memory 305 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be at least one storage device located away from the aforementioned processor 301. The memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface 303 module, and an application program for a substation safety management and control method based on a three-dimensional reconstruction model.

[0139] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call an application program stored in the memory 305 for a substation safety management and control method based on a three-dimensional reconstruction model. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0140] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0141] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory 305 and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0146] The present application also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores instructions. When executed by one or more processors 301, the electronic device executes one or more of the methods described in the above embodiments.

[0147] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A substation safety management and control method based on a three-dimensional reconstruction model, characterized in that: The method comprises: Obtain real-time point cloud data input by the laser point cloud sensor and real-time images collected by the camera; Comparing the real-time point cloud data with the background point cloud model point by point, and extracting the changed dynamic area using a background difference algorithm; A dynamic reconstruction module based on the real-time point cloud data identifies a first position of a first dynamic target in the dynamic area; Calculating a second position of a second dynamic target included in the real-time image; If the first position is successfully matched with the second position, and the sparse structure of the real-time point cloud data is successfully matched with the dense pixels of the real-time image, it is determined that the first dynamic target and the second dynamic target are both dynamic flying targets; The position of the dynamic flying target is determined according to the first position and the second position, and is marked in the three-dimensional model.

2. A substation safety management and control method based on a three-dimensional reconstruction model according to claim 1, characterized in that: The real-time point cloud data is compared point by point with the background point cloud model, and the changed dynamic area is extracted using a background difference algorithm, specifically including: Generate a real-time point cloud model according to the real-time point cloud data; Clustering and segmenting the real-time point cloud model to obtain a plurality of real-time voxel units, and clustering and segmenting the background point cloud model to obtain a plurality of background voxel units; Compare the plurality of real-time voxel units with the plurality of background voxel units one by one, and determine a first voxel unit corresponding to the background voxel unit among the plurality of real-time voxel units; A first voxel unit among the plurality of real-time voxel units is removed to obtain a second voxel unit, where the second voxel unit is the dynamic area.

3. A substation safety management and control method based on a three-dimensional reconstruction model according to claim 2, characterized in that: The dynamic reconstruction module based on the real-time point cloud data identifies a first position of a first dynamic target in the dynamic area, specifically comprising: Determining a relative position of the second voxel unit and the first voxel unit according to the real-time point cloud model; Aligning the real-time point cloud model with the background point cloud model through a point cloud registration algorithm; Obtaining the actual coordinate position of the background point cloud model; The first position of the first dynamic target is determined according to the relative position and the actual coordinate position.

4. A substation safety management and control method based on a three-dimensional reconstruction model according to claim 1, characterized in that: Before successfully matching the sparse structure of the real-time point cloud data with the dense pixels of the real-time image, the method further includes: Spatially align the laser point cloud sensor and the camera to establish a unified coordinate system; Calculating a feature descriptor for a first key point in the real-time image to obtain a first descriptor, and calculating a feature descriptor for a second key point in the real-time point cloud data to obtain a second descriptor; comparing the first descriptor with the second descriptor, and matching the first descriptor with the second descriptor whose similarity exceeds a preset threshold; If it is determined that the matching rate of the plurality of first descriptors reaches a preset condition, or the matching rate of the plurality of second descriptors reaches a preset condition, the coordinates of the first key point are mapped to the dynamic reconstruction model to calculate the position of the dynamic flying target.

5. A substation safety management and control method based on a three-dimensional reconstruction model according to claim 2, characterized in that: The step of comparing the real-time point cloud data with the background point cloud model point by point and extracting the changed dynamic area using a background difference algorithm specifically includes: According to a predefined side length size, subdividing the real-time point cloud model into a plurality of first voxel points and subdividing the background point cloud model into a plurality of second voxel points; Calculate the attribute characteristics of each real-time voxel unit according to the geometric data of the first voxel point contained in the real-time voxel unit, wherein the attribute characteristics include point density, average coordinates and geometric center; Calculate the attribute characteristics of each background voxel unit according to the geometric data of the second voxel point contained in the background voxel unit, wherein the attribute characteristics include point density, average coordinates and geometric center; The real-time voxel units are compared with the background voxel units one by one, and dynamic voxel units whose attribute feature change values ​​exceed a preset standard among the multiple real-time voxel units are determined, and the dynamic voxel units are determined to be the dynamic areas.

6. A substation safety management and control method based on a three-dimensional reconstruction model according to claim 4, characterized in that: Calculating a feature descriptor for a first key point in the real-time image to obtain a first descriptor specifically includes: The real-time image is downsampled multiple times to generate a series of image hierarchical structures with different resolutions. For each image hierarchical structure with different resolutions, Gaussian filtering is applied to the image to smooth the image and generate Gaussian blurred images of different scales. The Gaussian blurred images are represented as follows: ； Wherein, I(x,y) is the Gaussian blurred image, L(x,y,σ) is the Gaussian blurred image, G(x,y,σ) is the Gaussian kernel, Represents the convolution operation; The difference between Gaussian blurred images of adjacent scales is calculated to generate a differential Gaussian image, which is represented as follows: ； Wherein, D(x, y, σ) is a differential Gaussian image, L(x, y, σ) is the Gaussian blurred image, L(x, y, kσ) is a Gaussian blurred image adjacent to the Gaussian blurred image, and k is a constant multiple; Compare any pixel point in the differential Gaussian image with multiple adjacent pixel points. If the spatial value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, and the scale value of any pixel point is greater than or less than the spatial values ​​of multiple adjacent pixel points, then the any pixel point is marked as a key point, and the first key point is obtained. The amplitude and direction of the pixel points in the neighborhood of the first key point are calculated, specifically by the following formula: ； ； Wherein, m(x, y) is the amplitude of the pixel points in the neighborhood of the first key point, θ is the direction of the pixel points in the neighborhood of the first key point, L(x+1, y) and L(x-1, y) respectively represent the pixel values ​​of the left and right adjacent pixels of the first key point in the horizontal direction of the differential Gaussian image, and L(x, y+1) and L(x, y-1) respectively represent the pixel values ​​of the upper and lower adjacent pixels of the first key point in the vertical direction of the differential Gaussian image; Divide the area of ​​the first key point into a plurality of sub-areas of the same size, and calculate a histogram of each of the sub-areas according to the amplitude and direction of the pixel points in the neighborhood of the key point; The multiple histograms are combined to form a multi-dimensional vector to obtain the first descriptor.

7. A substation safety management and control method based on a three-dimensional reconstruction model according to claim 4, characterized in that: The calculating a feature descriptor for the second key point in the real-time point cloud data to obtain a second descriptor specifically includes: Calculate the point curvature of any point cloud point corresponding to the real-time point cloud data, and the point curvature is calculated by the following formula: ； Where C is the curvature of the point, n i is the normal vector of the neighboring points of any one of the point cloud points, n is the normal vector of any one of the point cloud points, and k is the number of neighboring points of any one of the point cloud points; If it is determined that the curvature of any one of the point cloud points is greater than a preset curvature, the any one of the point cloud points is determined as a key point, and the second key point is obtained; Taking the second key point as the center, dividing the neighborhood into a plurality of spherical cells according to the direction of the normal vector of the second key point; A normal vector direction histogram is calculated in each of the spherical cells, and features of the spherical cells are accumulated to generate an overall direction histogram, wherein data of the direction histogram is the second descriptor.

8. A substation safety management and control system based on a three-dimensional reconstruction model, characterized in that: The system comprises an acquisition module (201), an extraction module (202), an identification module (203), a processing module (204) and a judgment module (205), wherein: The acquisition module (201) is used to acquire real-time point cloud data input by the laser point cloud sensor and real-time images collected by the camera; The extraction module (202) is used to compare the real-time point cloud data with the background point cloud model point by point, and extract the dynamic area that has changed using a background difference algorithm; The identification module (203) is used for identifying the first position of the first dynamic target in the dynamic area based on the dynamic reconstruction module of the real-time point cloud data; The processing module (204) is used to calculate a second position of a second dynamic target included in the real-time image; The judgment module (205) is used to determine that the first dynamic target and the second dynamic target are both dynamic flying targets if the first position is successfully matched with the second position and the sparse structure of the real-time point cloud data is successfully matched with the dense pixels of the real-time image; The processing module is used to determine the position of the dynamic flying target according to the first position and the second position, and mark it in the three-dimensional model.

9. An electronic device, characterized in that: The electronic device comprises a processor (301), a communication bus (302), a memory (305), a user interface (303), a network interface (304) and a memory (305), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are both used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images