A large mechanical movement monitoring and early warning method and system

Abstract

The application discloses a large-scale mechanical movement monitoring and early warning method and system. The application generates target substation point cloud data by carrying out point cloud data processing on initial substation point cloud data corresponding to substation information; generates a target large-scale mechanical model by using three-dimensional position information and a preset large-scale mechanical simulation model; searches for the nearest points of each live equipment point cloud in the target substation point cloud data and target points of the target large-scale mechanical model; calculates the shortest distances between the nearest points and a model surface of the target large-scale mechanical model; and generates and outputs early warning information when the shortest distances are less than a warning distance threshold. The application solves the technical problem of low accuracy of existing technologies and easy occurrence of accidents. The application uses a proximity search algorithm to find the nearest live equipment point to the current position of the construction machinery in a very short time and calculates the distance between the two. The application provides strong safety protection for large-scale construction machinery operation of substations.

Description

Technical Field

[0001] This invention relates to the field of motion monitoring technology for large machinery, and in particular to a method and system for motion monitoring and early warning of large machinery. Background Technology

[0002] With the continuous development of the economy and the continuous improvement of urban infrastructure, it is necessary to rely on large machinery such as cranes to lift objects. However, due to the complexity of the operation of cranes in substations, drivers and supervisors cannot accurately judge the actual distance between the crane and the live equipment, resulting in frequent collision accidents. Once an accident occurs, it often causes significant economic losses.

[0003] Therefore, sensors such as infrared cameras are usually installed on cranes to detect and sense live equipment in substations. However, single infrared sensing detection has a large working error and low accuracy, and is prone to hitting live equipment in substations, leading to accidents. Summary of the Invention

[0004] This invention provides a method and system for monitoring and early warning of the movement of large machinery, which solves the technical problem that existing methods of installing infrared cameras and other sensors on cranes to detect and sense live equipment in substations have large working errors, low accuracy, and are prone to hitting live equipment in substations, leading to accidents.

[0005] The first aspect of this invention provides a method for monitoring and early warning of the motion of large machinery, comprising:

[0006] In response to a received motion request from a large machine, the substation information and the three-dimensional position information of the large machine corresponding to the motion request are obtained.

[0007] The initial substation cloud data corresponding to the substation information is processed into point cloud data to generate target substation cloud data.

[0008] Using the aforementioned three-dimensional position information and a preset large-scale mechanical simulation model, a target large-scale mechanical model is generated;

[0009] Search for the nearest neighbor points of each live equipment point cloud in the target substation site cloud data and the target points of the target large mechanical model;

[0010] Calculate the shortest distance between the nearest point and the surface of the target large mechanical model;

[0011] When the shortest distance is less than the warning distance threshold, a warning message is generated and output.

[0012] Optionally, the step of responding to the received large machinery movement request and obtaining the substation information and three-dimensional position information of the large machinery corresponding to the large machinery movement request includes:

[0013] In response to a received request for movement of large machinery, obtain the substation information corresponding to the request for movement of large machinery;

[0014] The 3D position information of large machinery is collected by an RTK locator and transmitted through a remote communication module.

[0015] Optionally, the point cloud data processing includes global offset of point cloud coordinates, point cloud coloring, and point cloud clipping; the step of processing the initial substation site cloud data corresponding to the substation information to generate target substation site cloud data includes:

[0016] Calculate the center coordinates of the original coordinates of the initial substation cloud data corresponding to the substation information;

[0017] Using the center coordinates as the offset vector, the coordinates of each point cloud in the initial substation site cloud data are offset according to the offset vector to generate the first updated substation site cloud data;

[0018] The first updated substation cloud data is colored according to the preset point cloud coloring effect to generate the second updated substation cloud data.

[0019] The point cloud picking tool is used to select the point clouds located within the substation area in the second updated substation site cloud data, and each unselected point cloud is deleted to generate and store the target substation site cloud data.

[0020] Optionally, the preset point cloud coloring effect includes elevation coloring display, category coloring display, and original RGB information coloring display; the step of coloring the first updated substation site cloud data according to the preset point cloud coloring effect to generate the second updated substation site cloud data includes:

[0021] When the point cloud corresponding to the first updated substation station cloud data needs to be displayed according to elevation color, the maximum and minimum current point cloud elevation corresponding to the first updated substation station cloud data are calculated.

[0022] The maximum and minimum elevation values ​​of the current point cloud are set as the two ends of the preset color band, respectively.

[0023] The current maximum point cloud elevation is used as the position parameter on the color band to color each point cloud, generating the second updated substation point cloud data with elevation coloring;

[0024] When the point cloud corresponding to the first updated substation site cloud data needs to be displayed in color according to category, the corresponding category color is set according to the preset category of live equipment.

[0025] The point cloud of each energized device is colored according to the categories described above, and the second updated substation point cloud data is generated by category coloring.

[0026] When the point cloud corresponding to the first updated substation cloud data needs to be displayed according to the original RGB information, the point cloud is colored according to the RGB channel values ​​in the point cloud data corresponding to the first updated substation cloud data, and RGB colored second updated substation cloud data is generated.

[0027] Optionally, the point cloud data processing further includes point cloud ground filtering and category editing; the step of using a point cloud picking tool to select point clouds located within the substation area in the second updated substation site cloud data, and deleting the unselected point clouds to generate and store the target substation site cloud data includes:

[0028] The point cloud picking tool is used to select the point clouds within the substation area in the second updated substation site cloud data, and each unselected point cloud is deleted to generate the third updated substation site cloud data.

[0029] Select the local lowest point from the third updated substation cloud data as a seed point, set all the seed points as ground points and construct a triangular network;

[0030] Calculate each non-ground point according to the preset ground point judgment method;

[0031] Determine whether the distance and maximum angle from the non-ground point to the triangulation meet the preset ground threshold condition;

[0032] If so, mark the non-ground point as a ground point and record the current total number of ground points;

[0033] Traverse the preset grid cells, reset the remaining ground points in the preset grid cells except for the lowest ground point to non-ground points, and determine whether the current proportion of the total number of ground points is greater than the proportion threshold.

[0034] If so, then proceed to the step of selecting the local lowest point from the third updated substation cloud data as a seed point, setting all the seed points as ground points and constructing a triangular network;

[0035] If not, then all the aforementioned non-ground points are determined as ground points, and the fourth updated substation cloud data is generated;

[0036] The point clouds in the fourth updated substation site cloud data are classified according to the preset categories of energized equipment to generate multiple energized equipment point clouds.

[0037] The target substation site cloud data is constructed using the point cloud of all the described energized equipment.

[0038] Optionally, the step of generating the target large-scale machinery model using the three-dimensional position information and a preset large-scale machinery simulation model includes:

[0039] Construct a simulation model of the large machinery based on the tracks, fuselage, and mechanical boom corresponding to the large machinery;

[0040] The position and orientation of the large mechanical simulation model in the point cloud scene are determined using the three-dimensional position information, and the large mechanical model is then generated and updated.

[0041] The latest three-dimensional position information is obtained at preset intervals, and updated three-dimensional position information is generated.

[0042] The position and orientation of the large mechanical simulation model in the point cloud scene are updated according to the updated 3D position information to generate the target large mechanical model.

[0043] Optionally, the step of searching for the nearest neighbor points of each energized equipment point cloud in the target substation site cloud data and the target point of the target large machinery model includes:

[0044] Calculate the variance of the dimensional data of each energized equipment point cloud in the target substation site cloud data.

[0045] Calculate the average value of all the point cloud data of the powered equipment in the first dimension corresponding to the maximum variance value, and set the median data corresponding to the average value as the intermediate node;

[0046] Determine whether the point cloud data of the electrical equipment corresponding to the first dimension is less than the median data;

[0047] If so, the point cloud data of the powered equipment is divided into the left subtree node;

[0048] If not, the point cloud data of the powered equipment is divided into the right subtree node;

[0049] Jump to execute the step of calculating the average value of all the point cloud data of the powered equipment in the first dimension corresponding to the maximum variance value, and setting the median data corresponding to the average value as the intermediate node, and construct a kd tree;

[0050] The nearest neighbor search method is used to search for the nearest neighbor point between each subtree node of the kd-tree and the target point of the target large mechanical model.

[0051] Optionally, the step of searching for the nearest neighbor point between each subtree node of the kd-tree and the target point of the target large mechanical model using the nearest neighbor search method includes:

[0052] Input the root node of the kd-tree into the priority queue;

[0053] Extract the point cloud data of the powered equipment corresponding to the optimal node in the priority queue;

[0054] Calculate the first target distance between the point cloud data of the electrical equipment and the target point of the target large mechanical model, and set the first target distance as the target point proximity value;

[0055] Extract the point cloud data of the electrical equipment corresponding to each node in the priority queue, and calculate the second target distance between the point cloud data of the electrical equipment corresponding to the current node and the target point of the target large mechanical model;

[0056] Update the neighbor value of the target point based on the second target distance;

[0057] The first subtree node in the dimension of the optimal node is searched according to a preset threshold; wherein, when the subtree node of the optimal node is a left subtree node, the first subtree node is a right subtree node.

[0058] Input the first subtree node into the priority queue, and jump to execute the step of extracting the point cloud data of the electrical equipment corresponding to the optimal node in the priority queue until the number of searches exceeds the preset backtracking number, and generate multiple target point proximity values;

[0059] Select the nearest neighbor value of the target point that is less than the preset nearest neighbor threshold from all the nearest neighbor values ​​of the target point as the nearest neighbor point of each subtree node to the target point of the target large mechanical model.

[0060] Optionally, it also includes:

[0061] Create a coordinate system with three mutually perpendicular unit axes and the camera's position as the origin;

[0062] A visual following matrix is ​​constructed using the three unit axes and translation vectors; the expression for the visual following matrix is ​​as follows:

[0063]

[0064] Where R is the unit axis, representing the right vector; U is the unit axis, representing the up vector; D is the unit axis, representing the direction vector; and P is the camera position vector.

[0065] The 3D position information of the target large mechanical model in the point cloud scene is updated in the opposite direction to the coordinate position information of the visual following matrix.

[0066] A second aspect of the present invention provides a large-scale mechanical motion monitoring and early warning system, comprising:

[0067] The three-dimensional position information module is used to respond to the received motion request of large machinery and obtain the substation information and the three-dimensional position information of the large machinery corresponding to the motion request.

[0068] The target substation cloud data module is used to process the initial substation cloud data corresponding to the substation information into point cloud data to generate target substation cloud data.

[0069] The target large machinery model module is used to generate a target large machinery model using the three-dimensional position information and a preset large machinery simulation model;

[0070] The nearest neighbor module is used to search for the nearest neighbor point between each live equipment point cloud in the target substation site cloud data and the target point of the target large machinery model.

[0071] The shortest distance module is used to calculate the shortest distance between the nearest point and the surface of the target large mechanical model;

[0072] The early warning information module is used to generate and output early warning information when the shortest distance is less than the early warning distance threshold.

[0073] As can be seen from the above technical solutions, the present invention has the following advantages:

[0074] This invention responds to a received motion request from a large machine, acquiring the corresponding substation information and the three-dimensional position information of the large machine. It then processes the initial substation site cloud data corresponding to the substation information into point cloud data to generate target substation site cloud data. Using the three-dimensional position information and a preset large machine simulation model, it generates a target large machine model. It searches for the nearest neighbor points between the point clouds of each energized device in the target substation site cloud data and the target point of the target large machine model. It calculates the shortest distance between the nearest neighbor point and the surface of the target large machine model. When the shortest distance is less than a warning distance threshold, it generates and outputs a warning message. This invention solves the technical problem of existing methods that use infrared cameras and other sensors mounted on cranes to detect energized equipment in substations. However, single infrared sensing detection has large operating errors, low accuracy, and is prone to hitting energized equipment, leading to accidents. This invention has the following innovations and advantages:

[0075] (1) Explore and realize the combination of point cloud scene visualization technology and 3D modeling technology to achieve intuitive display of large construction machinery 3D visualization models in high-density, high-precision 3D point cloud scenes. Furthermore, it realizes the interaction between the construction machinery model and the point cloud scene;

[0076] (2) Explore and implement a construction data positioning data acquisition method that combines hardware and software. Install several portable RTKs at the corners and boom nodes of construction machinery. Use real-time remote communication technologies such as TCP / IP to transmit the field survey data monitored by the RTK hardware to the software for parsing and processing. This replaces the real-time early warning device that relies on lidar, thereby greatly reducing the hardware procurement cost and ensuring the reliability of the construction machinery positioning simulation.

[0077] (3) Explore and implement a method for rapid search of collision risk points and rapid calculation of early warning distances during construction machinery operation. A KNN nearest neighbor search algorithm based on KD-TREE is used to find the nearest live equipment location to the current position of the construction machinery in a very short time and calculate the distance between them. This provides strong safety assurance for the operation of large construction machinery in substations. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a flowchart illustrating the steps of a method for monitoring and early warning of motion in large machinery according to Embodiment 1 of the present invention.

[0080] Figure 2 This is a flowchart illustrating the steps of a method for monitoring and warning the motion of large machinery according to Embodiment 2 of the present invention.

[0081] Figure 3 This is a schematic diagram of a method for determining whether a point is a ground point according to Embodiment 2 of the present invention;

[0082] Figure 4 This is a flowchart illustrating a triangular network encryption strategy for high-density point cloud data provided in Embodiment 2 of the present invention.

[0083] Figure 5 This is a flowchart illustrating the dynamic simulation of the operation process of a target large-scale mechanical model according to Embodiment 2 of the present invention.

[0084] Figure 6 This is a schematic diagram of the early warning distance calculation process between the nearest point and the target large mechanical model provided in Embodiment 2 of the present invention;

[0085] Figure 7 This is a schematic diagram of a spatial segmentation structure for a node provided in Embodiment 2 of the present invention;

[0086] Figure 8 This is a schematic diagram of the kd-tree structure corresponding to the spatial partitioning of a node, provided in Embodiment 2 of the present invention.

[0087] Figure 9 This is a flowchart illustrating a KNN nearest neighbor search algorithm based on KD-TREE provided in Embodiment 2 of the present invention;

[0088] Figure 10 This is a schematic diagram of a camera and observation space provided in Embodiment 2 of the present invention;

[0089] Figure 11 This is a structural block diagram of a large-scale mechanical motion monitoring and early warning system provided in Embodiment 3 of the present invention. Detailed Implementation

[0090] This invention provides a method and system for monitoring and early warning of the movement of large machinery, which addresses the technical problem that existing methods, such as installing infrared cameras and other sensors on cranes to detect and sense live equipment in substations, suffer from large errors and low accuracy due to the reliance on single infrared sensing, which can easily lead to accidents caused by contact with live equipment in the substation.

[0091] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0092] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a method for monitoring and early warning of the movement of large machinery, as provided in Embodiment 1 of the present invention.

[0093] This invention provides a method for monitoring and early warning of the movement of large machinery, comprising the following steps:

[0094] Step 101: Respond to the received motion request of the large machinery, and obtain the substation information and the three-dimensional position information of the large machinery corresponding to the motion request.

[0095] It should be noted that substation information refers to the point cloud information of each point within the substation;

[0096] The three-dimensional position information of large machinery refers to the three-dimensional pose parameter information of large machinery.

[0097] In this embodiment of the invention, the point cloud information within the substation and the three-dimensional pose parameter information of the large machinery are obtained by acquiring the motion request of the large machinery.

[0098] Step 102: Process the initial substation site cloud data corresponding to the substation information into point cloud data to generate target substation site cloud data.

[0099] It should be noted that the initial substation point cloud data refers to the raw point cloud data that has not undergone point cloud data processing.

[0100] Point cloud data processing includes algorithmic data processing such as global offset of point cloud coordinates, point cloud coloring, point cloud clipping, point cloud ground filtering, and category editing.

[0101] The target substation site cloud data refers to the point cloud data obtained after preprocessing the initial substation site cloud data through algorithms such as global offset of point cloud coordinates, point cloud coloring, point cloud clipping, point cloud ground filtering, and category editing.

[0102] In a specific embodiment, the initial substation site cloud data corresponding to the substation information is sequentially processed by algorithms such as global offset of point cloud coordinates, point cloud coloring, point cloud clipping, point cloud ground filtering, and category editing to obtain target substation site cloud data with point clouds of each energized device.

[0103] Step 103: Using three-dimensional position information and a preset large-scale mechanical simulation model, generate the target large-scale mechanical model.

[0104] It should be noted that the preset large-scale machinery simulation model refers to a simulation model in which the large construction machinery is abstracted into three parts: tracks, fuselage and control room, and boom, and represented in a point cloud scene. Specifically, the boom is divided into two-section booms and three-section booms. Based on this, the construction machinery model is constructed in three parts. The tracks and fuselage are close to regular cubes in shape, so they are simulated using a cube bounding box. The boom model is simulated using multiple cylindrical sections. The two-section boom is simulated using two connected cylinders, and the three-section boom is simulated using three connected cylinders.

[0105] The target large-scale mechanical model refers to a large-scale mechanical simulation model in a point cloud scene that combines its positioning data and can be updated with new 3D position information to achieve the effect of dynamic simulation.

[0106] In this embodiment of the invention, the corresponding three-dimensional position information is input into a preset large-scale mechanical simulation model, and the position and posture of the large-scale mechanical simulation model are updated through three-dimensional pose parameter information, thereby obtaining a target large-scale mechanical model with dynamic simulation effect.

[0107] Step 104: Search for the nearest neighbor points of each live equipment point cloud in the target substation site cloud data and the target point of the target large mechanical model.

[0108] It should be noted that the point cloud of each electrical device refers to the point cloud identified according to the type of each electrical device.

[0109] The nearest point is the point with the shortest Euclidean distance.

[0110] The target point refers to a target point selected from the target large mechanical model.

[0111] In a specific embodiment, the nearest neighbor search method is used to search for the nearest neighbor point between each live equipment point cloud in the target substation site cloud data and the target point of the target large mechanical model.

[0112] Step 105: Calculate the shortest distance between the nearest point and the surface of the target large mechanical model.

[0113] It should be noted that the shortest distance refers to the shortest distance calculated using Euclidean distance.

[0114] In this embodiment of the invention, the nearest point in the target substation cloud data and the shortest distance on the model surface are used.

[0115] Step 106: When the shortest distance is less than the warning distance threshold, generate and output the warning information.

[0116] It should be noted that the warning distance threshold refers to the distance at which getting within 1 meter may cause damage to the live equipment by large machinery, but the specific warning distance is set by the management personnel and is not limited here.

[0117] In a specific embodiment, when the shortest distance is less than the warning distance threshold, it may cause damage to the electrical equipment by large machinery. It is necessary to generate and output warning information in a timely manner to warn the operators of the large machinery or notify the relevant management personnel to stop it in time.

[0118] This invention responds to a received motion request from a large machine, acquiring the corresponding substation information and the three-dimensional position information of the large machine. It then processes the initial substation site cloud data corresponding to the substation information into point cloud data to generate target substation site cloud data. Using the three-dimensional position information and a preset large machine simulation model, it generates a target large machine model. It searches for the nearest neighbor points between the point clouds of each energized device in the target substation site cloud data and the target point of the target large machine model. It calculates the shortest distance between the nearest neighbor point and the surface of the target large machine model. When the shortest distance is less than a warning distance threshold, it generates and outputs a warning message. This invention solves the technical problem of existing methods that use infrared cameras and other sensors mounted on cranes to detect energized equipment in substations. However, single infrared sensing detection has large operating errors, low accuracy, and is prone to hitting energized equipment, leading to accidents. This invention has the following innovations and advantages:

[0119] (1) Explore and realize the combination of point cloud scene visualization technology and 3D modeling technology to achieve intuitive display of large construction machinery 3D visualization models in high-density, high-precision 3D point cloud scenes. Furthermore, it realizes the interaction between the construction machinery model and the point cloud scene;

[0120] (2) Explore and implement a construction data positioning data acquisition method that combines hardware and software. Install several portable RTKs at the corners and boom nodes of construction machinery. Use real-time remote communication technologies such as TCP / IP to transmit the field survey data monitored by the RTK hardware to the software for parsing and processing. This replaces the real-time early warning device that relies on lidar, thereby greatly reducing the hardware procurement cost and ensuring the reliability of the construction machinery positioning simulation.

[0121] (3) Explore and implement a method for rapid search of collision risk points and rapid calculation of early warning distances during construction machinery operation. A KNN nearest neighbor search algorithm based on KD-TREE is used to find the nearest live equipment location to the current position of the construction machinery in a very short time and calculate the distance between them. This provides strong safety assurance for the operation of large construction machinery in substations.

[0122] Please see Figure 2-6 , Figure 2 This is a flowchart illustrating the steps of a method for monitoring and early warning of the movement of large machinery, as provided in Embodiment 2 of the present invention.

[0123] This invention provides a method for monitoring and early warning of the movement of large machinery, comprising the following steps:

[0124] Step 201: Respond to the received large machinery movement request and obtain the substation information corresponding to the large machinery movement request.

[0125] In this embodiment of the invention, the specific implementation process of step 201 is similar to that of step 101, and will not be repeated here.

[0126] Step 202: Collect the three-dimensional position information of the large machinery through the RTK locator and transmit the three-dimensional position information through the remote communication module.

[0127] It should be noted that RTK positioners are used to measure the positioning data of large machinery in real time at construction sites. TCP / IP refers to a protocol suite that enables information transmission between multiple different networks. It has the advantages of data transmission stability and confidentiality, and network addresses are uniformly assigned, with each device and terminal in the network having a unique address. Therefore, it is quite suitable for use in the remote transmission of RTK positioning information.

[0128] In a specific embodiment, the three-dimensional position information obtained by the RTK locator is used to refresh the current position and attitude of the large machinery in the point cloud scene in real time.

[0129] Specifically, the communication process based on the TCP / IP protocol is as follows: 1. Server communication process: (1) Create a listening socket object and a communication socket object; (2) The listening socket listens on the specified IP and port; (3) If there is a new connection request, the listening socket obtains the client information and initializes the communication socket; (4) Use the communication socket to read and write data. 2. Client communication process: (1) Create a communication socket object; (2) The communication socket connects to the specified IP and port; (3) After successfully connecting to the server, use the communication socket to read and write data.

[0130] The remote communication module uses a header-body separation approach for sending and receiving files, commonly employing a header-plus-body format. The header format is generally fixed, defined as the packet path plus the packet size. On the server side, after selecting the file to send, it first sends the header information, then uses a timer to delay for a few milliseconds before sending the packet body data. A fixed-size buffer is defined to continuously read data from the file. In each loop, the socket sends the data read from the buffer to the client, accumulating the size of the currently read data. When the size of the read data equals the actual packet body size recorded in the header, the file is considered sent successfully. On the client side, the socket first parses the header information, reads the packet body size, and then accumulates the size of the currently received data after each data packet is received. When this size equals the packet body size, the current file is considered received successfully. This solves the problem of packet merging, an unavoidable issue encountered when using the TCP / IP protocol for file sending and receiving, where the end information of one file is combined with the header information of the next. This phenomenon causes data confusion, making it impossible to receive the actual data.

[0131] Specifically, the remote communication module uses Socket heartbeat detection. To ensure the safety of construction, the receiving end of the 3D position information file collected by the RTK locator needs to maintain a connection with the sending end for a long time. During this process, there is a chance of server crash or network instability causing disconnection. At this time, it is necessary to use the Socket heartbeat detection mechanism to monitor the connection status in real time and try to reconnect after disconnection.

[0132] The principle of heartbeat detection is that the client periodically sends heartbeat packets to the server, and the server immediately replies to the client upon receiving the heartbeat. During this process, the client checks whether the heartbeat was sent successfully and detects heartbeat timeouts by recording the number of heartbeats awaiting a reply. A timer is defined and started when the client and server successfully connect. Every 30ms, the client sends a heartbeat message to the server and accumulates the number of heartbeats awaiting a reply. Once the server receives a heartbeat message from the client, it replies to it. After receiving the server's reply, the client resets its count of heartbeats awaiting a reply to zero. When the number of heartbeats awaiting a reply exceeds three, the client is considered to have lost connection with the server, a warning is issued, and a reconnection attempt is made.

[0133] Step 203: Process the initial substation site cloud data corresponding to the substation information into point cloud data to generate target substation site cloud data.

[0134] Optionally, point cloud data processing includes global offset of point cloud coordinates, point cloud coloring, and point cloud clipping; step 203 includes the following steps S11-S14:

[0135] S11. Calculate the center coordinates of the original coordinates of the initial substation cloud data corresponding to the substation information;

[0136] S12. Using the center coordinates as the offset vector, offset the coordinates of each point cloud in the initial substation site cloud data according to the offset vector to generate the first updated substation site cloud data.

[0137] S13. Color the first updated substation site cloud data according to the preset point cloud coloring effect to generate the second updated substation site cloud data.

[0138] S14. Use a point cloud picking tool to select the point clouds within the substation area in the second updated substation site cloud data, and delete the unselected point clouds to generate and store the target substation site cloud data.

[0139] It should be noted that the x and y coordinate values ​​of each point in the initial substation site cloud data are usually quite large, reaching tens or even millions. Directly visualizing this data would cause the point cloud scene to be far from the origin of the view coordinate system, significantly reducing the effectiveness and efficiency of point cloud visualization. Therefore, a global coordinate offset is applied to the entire initial substation site cloud data, translating it to the vicinity of the view origin before visualization.

[0140] The preset point cloud shading effects include elevation shading display, category shading display, and original RGB information shading display.

[0141] In a specific embodiment, the center coordinates of the original coordinates of the initial substation cloud data are first calculated. Then, the center coordinates are used as the offset vector to translate the center of the initial substation cloud data to the original position in the view. Finally, the coordinates of each point cloud in the initial substation cloud data are offset to obtain the first updated substation cloud data that is near the origin of the view after global offset.

[0142] Specifically, in order to enrich the point cloud rendering effect, it supports coloring and displaying the first updated substation site cloud data according to the point cloud elevation, category, and original RGB information, and generating the second updated substation site cloud data.

[0143] Optionally, in addition to the initial substation site cloud data concentrated in the middle, the collected raw point cloud data will also scan and record some terrain features and clutter around the substation. For the convenience and efficiency of subsequent data processing, it is necessary to crop and remove points outside the substation. The specific processing method is to use a point cloud picking tool to select the point cloud of the substation area, and then allocate two parts of memory according to the index of the selected point cloud. The raw point cloud is divided into two parts: selected points and unselected points, which are stored separately. Finally, the part of the point cloud memory that does not belong to the substation is deleted as needed to generate the target substation site cloud data.

[0144] Optionally, the preset point cloud coloring effect includes elevation coloring display, category coloring display, and original RGB information coloring display; step S13 includes the following steps S131-S136:

[0145] S131. When the point cloud corresponding to the first updated substation station cloud data needs to be displayed according to elevation color, the maximum value and minimum value of the current point cloud elevation corresponding to the first updated substation station cloud data are calculated.

[0146] S132. Set the current maximum point cloud elevation and the current minimum point cloud elevation to the two ends of the preset color band, respectively.

[0147] S133. Use the position parameter of the current maximum point cloud elevation on the color band to color each point cloud, and generate the second updated substation point cloud data with elevation coloring.

[0148] S134. When the point cloud corresponding to the first updated substation site cloud data needs to be displayed in color according to category, set the corresponding category color according to the preset category of live equipment.

[0149] S135. Assign colors to the point clouds of each live equipment according to the color of each category, and generate the second updated substation point cloud data with category coloring.

[0150] S136. When the point cloud corresponding to the first updated substation cloud data needs to be colored according to the original RGB information, each point cloud is colored according to the RGB channel values ​​in the point cloud data corresponding to the first updated substation cloud data, and RGB colored second updated substation cloud data is generated.

[0151] It should be noted that the second update of the elevation coloring for substations refers to the substation cloud data generated according to the point cloud coloring effect displayed by the elevation coloring.

[0152] Category-based coloring for the second update of substation cloud data refers to substation cloud data generated by displaying point cloud coloring effects according to category coloring.

[0153] The second update of RGB shading substation cloud data refers to the substation cloud data generated by shading the point cloud according to the original RGB information.

[0154] In a specific embodiment, when displaying by elevation coloring, the maximum and minimum elevation values ​​of the current point cloud are first calculated. Based on a pre-set color band, the maximum and minimum elevation values ​​of the current point cloud are set as the colors at both ends of the color band. During this period, the position of the point cloud on the color band is calculated according to the maximum elevation value of the current point cloud and then colored accordingly, generating the second updated substation cloud data with elevation coloring. When displaying by category coloring, different colors are assigned to different categories of live equipment, generating the second updated substation cloud data with category coloring. When displaying by original RGB information coloring, the RGB channel values ​​recorded in each point when scanning and acquiring the point cloud data are used for coloring, generating the second updated substation cloud data with original RGB coloring.

[0155] Optionally, point cloud data processing also includes point cloud ground filtering and category editing; step S14 includes the following steps S141-S1410:

[0156] S141. Use the point cloud picking tool to select the point clouds in the second updated substation site cloud data that are located within the substation area, and delete the unselected point clouds to generate the third updated substation site cloud data.

[0157] S142. Select the local lowest point from the cloud data of the third updated substation site as the seed point, set all seed points as ground points and construct a triangular network;

[0158] S143. Calculate each non-ground point according to the preset ground point judgment method;

[0159] S144. Determine whether the distance and maximum angle from non-ground points to the triangulation network meet the preset ground threshold conditions.

[0160] S145. If so, mark the non-ground points as ground points and record the current total number of ground points;

[0161] S146. Traverse the preset grid cells, reset the remaining ground points in the preset grid cells (excluding the lowest ground point) to non-ground points, and determine whether the current percentage of the total number of ground points is greater than the percentage threshold.

[0162] S147. If so, then proceed to the steps of selecting the local lowest point as the seed point from the cloud data of the third updated substation, setting all seed points as ground points and constructing a triangular network.

[0163] S148. If not, then determine all non-ground points as ground points and generate the fourth updated substation cloud data.

[0164] S149. Classify the point clouds in the fourth updated substation site cloud data according to the preset categories of live equipment, and generate multiple live equipment point clouds.

[0165] S1410. Construct target substation site cloud data using point clouds of all energized equipment.

[0166] It should be noted that the point cloud ground filtering employs an improved progressively encrypted triangular mesh filtering algorithm, mainly consisting of two parts: point cloud mesh construction and point cloud filtering. Its primary purpose is to separate ground points from non-ground points in the point cloud data. The preset ground threshold conditions can be set according to actual conditions and are not limited here.

[0167] In a specific embodiment, the Bowyer-Watson algorithm is used to construct a Delaunay triangulation of the point cloud to be processed. The specific steps are as follows:

[0168] 1) Construct a super triangle containing all scattered points and put it into a triangle linked list.

[0169] 2) Insert the scattered points in the discrete point cloud one by one. Find the triangle whose circumcircle contains the insertion point in the triangle linked list (called the influence triangle of the point). Delete the common edge of the influence triangle. Connect the insertion point with all the vertices of the influence triangle to complete the insertion of a point in the Delaunay triangle linked list.

[0170] 3) Optimize the newly formed triangles locally according to the optimization criteria. Add the newly formed triangles to the Delaunay triangle linked list.

[0171] 4) Repeat step 2) above until all scattered points have been inserted.

[0172] Point cloud filtering employs the classic Adaptive Triangulated Network (ATIN) algorithm to filter the point cloud data in the fourth updated substation site cloud data. The main processing steps can be divided into the following three parts:

[0173] (1) Select the local lowest point as the seed point. The range of the fourth updated substation site cloud data is divided into windows of the same size according to the plane coordinates, and then the point with the lowest elevation in each window is selected to complete the selection of the seed point. Since the existence of buildings needs to be considered, and at least one ground point needs to be ensured in each window, the window size is generally determined according to the size of the largest building in the processing area.

[0174] (2) Construct a triangulation based on seed points to form an initial terrain surface. Mark all seed points as ground points and construct a Delonix triangulation for them. This triangulation can be regarded as a rough terrain surface. Methods for constructing the triangulation include the segmentation and merging method, the point-by-point insertion method, and the triangulation growth method.

[0175] (3) Find new ground points to gradually encrypt the triangulation network. Traverse all non-ground points, find points that meet the threshold conditions, mark them as new ground points, and encrypt them into the current triangulation network. Iterate until all points that meet the requirements have been added, and generate the fifth updated substation cloud data.

[0176] The pre-defined ground point determination method is a method for determining whether a point is a ground point, as follows: Figure 3 As shown: Calculate the perpendicular distance d from the point to the triangle in TIN (provided that the foot of the perpendicular from the point to the triangle is inside the triangle). Simultaneously, connect the point to the three vertices of the triangle to form sides, and calculate the angles α, β, and γ between these three sides and the plane containing the triangle. These distances and angles must all satisfy the following:

[0177]

[0178] Where, d max The maximum allowable distance is θ, and the maximum allowable included angle is θ.

[0179] Specifically, considering the efficiency issue of triangular mesh encryption for high-density point cloud data, a new triangular mesh encryption strategy for high-density point cloud data is proposed, based on the classic progressive encryption triangular mesh, such as... Figure 4 As shown, the process is as follows:

[0180] 1) Mark all seed points as ground points.

[0181] 2) Construct a triangulation network based on ground points.

[0182] 3) Traverse all non-ground points, calculate the distance and angle between each point and its corresponding triangle. If a threshold condition is met, the point is marked as a ground point. After traversing all non-ground points, record the current total number of ground points. Where i represents the number of iterations.

[0183] 4) Create a grid with a cell size of m, and traverse all grid cells. If there are ground points, keep only the lowest ground point, reclassify all other ground points as non-ground points, mark them, and skip these points in the traversal in step 3).

[0184] 5) Repeat steps 2) to 4). In step 3), the proportion of the increased points to all points is:

[0185]

[0186] Where, N total This represents the total number of point cloud data.

[0187] The loop ends when p is less than the proportional threshold. In this invention, the proportional threshold is set to 0.001.

[0188] 6) Reclassify the marked non-ground points as ground points to obtain the final ground point results.

[0189] In this strategy, only the lowest point in the grid cell is used to construct the TIN during the encryption process. Therefore, the number of grid points is controlled, memory overhead is limited, and algorithm runtime is optimized.

[0190] Specifically, compared to surrounding vegetation, buildings, and other features, live equipment has a larger spatial volume and a relatively concentrated distribution, making it easy to visually identify and less prone to misjudgment. Furthermore, each point in the LAS format 3D point cloud data has an attribute field that can be used to record the category to which that point belongs. Considering the above, this invention uses a manual selection method to identify live equipment within the substation. The specific implementation method is as follows: A field or number representing the category of live equipment is defined in advance. This invention defines six common types of live equipment for classification: main transformers, switches, disconnectors, busbars, cross-line devices, and current transformers. Then, based on ground filtering, non-ground points are edited. Using a point cloud selection tool, live equipment points are manually selected visually from the non-ground points, and their category fields are assigned values, thereby constructing the target substation point cloud data.

[0191] Step 204: Using three-dimensional position information and a preset large-scale mechanical simulation model, generate the target large-scale mechanical model.

[0192] Optionally, step 204 includes the following steps S21-S24:

[0193] S21. Construct a simulation model of large machinery based on the tracks, fuselage, and mechanical boom of the large machinery.

[0194] S22. Use 3D position information to determine the position and orientation of the large mechanical simulation model in the point cloud scene, and generate and update the large mechanical model.

[0195] S23. Obtain the latest three-dimensional position information according to the preset interval time, and generate updated three-dimensional position information;

[0196] S24. Update the position and orientation of the large mechanical simulation model in the point cloud scene according to the updated 3D position information to generate the target large mechanical model.

[0197] In a specific embodiment, a simulation model is constructed in a point cloud scene according to the tracks, fuselage and mechanical boom of the large machinery. The position and attitude of the target large machinery model in the point cloud scene are completely determined by the three-dimensional pose parameter data transmitted by the RTK locator. Whenever new RTK positioning information is transmitted, the position and attitude of the target large machinery model in the substation point cloud scene are also updated, thereby achieving the effect of dynamic simulation.

[0198] The core idea of ​​dynamic simulation of the operation process of a large-scale target mechanical model is: Figure 5 As shown, a rectangular bounding box is used to approximate the surface of the construction machinery. A timer is defined to refresh the screen view of the point cloud scene every 20ms. If the system receives new RTK positioning information during the two refresh attempts, the three-dimensional pose parameters of the target large machinery model in the screen domain are updated according to the RTK positioning information, and its surface bounding box is re-rendered to dynamically display the latest position of the large machinery in the substation.

[0199] Step 205: Search for the nearest points of each live equipment point cloud in the target substation site cloud data and the target point of the target large mechanical model.

[0200] Optionally, step 205 includes the following steps S31-S37:

[0201] S31. Calculate the variance of the dimension data of each live equipment point cloud in the target substation site cloud data.

[0202] S32. Calculate the average value of all point cloud data of powered devices in the first dimension corresponding to the maximum variance value, and set the median data corresponding to the average value as the intermediate node.

[0203] S33. Determine whether the point cloud data of the corresponding electrical equipment in the first dimension is less than the median data.

[0204] S34. If so, divide the point cloud data of the powered equipment into the left subtree node;

[0205] S35. If not, then divide the point cloud data of the powered equipment into the right subtree node;

[0206] S36. Jump to the step of calculating the average value of all point cloud data of powered devices in the first dimension corresponding to the maximum variance value, and setting the median data corresponding to the average value as the intermediate node, and constructing a kd tree;

[0207] S37. Search for the nearest neighbor point between each subtree node in the kd-tree and the target point of the target large mechanical model using the nearest neighbor search method.

[0208] It's important to note that a kd-tree, or k-dimensional tree for short, is a spatially partitioned data structure. It's commonly used for searches in high-dimensional spaces, such as range searches and nearest neighbor searches. Since the number of points in a 3D point cloud is generally large, using a kd-tree for retrieval can significantly reduce time consumption, ensuring that the finding and registration of related points in the point cloud is real-time. For example... Figure 6 As shown, the core of this early warning distance calculation is to construct a three-dimensional kd-tree from the point cloud of the identified energized equipment, and then search for its nearest neighbor in the kd-tree using the three-dimensional position information of the target large mechanical model.

[0209] In this embodiment of the invention, a kd-tree can be viewed as a k-dimensional binary tree, where each node is k-dimensional data, and the node should contain the following fields:

[0210]

[0211] Specifically, the construction of a kd-tree involves arranging the unordered point clouds of energized equipment in a specific order from the target substation site cloud data, facilitating quick and efficient retrieval. For example... Figure 7-8 As shown, the biggest problem in building a tree lies in the choice of the pivot point, and two rules must be followed:

[0212] (1) The tree should be as balanced as possible. The more balanced the tree, the more evenly it is divided, and the less time is required for the search.

[0213] (2) Maximize the pruning opportunities of neighborhood search.

[0214] The construction algorithm is as follows:

[0215] Input: Unordered point cloud of charged equipment, dimension k

[0216] Output: kd-tree corresponding to the point cloud of the charged device

[0217] Algorithm:

[0218] (1) Initialize the splitting axis: Calculate the variance of the data in each dimension, take the dimension with the largest variance as the splitting axis, and label it as r;

[0219] (2) Determine the node: Search the current data according to the split axis dimension, find the median data (middle node), and put it into the current node;

[0220] (3) Divide into two branches:

[0221] Split into the left subtree: On the current split axis, all values ​​less than the median are split into the left subtree;

[0222] Partition to the right subtree: On the current partition axis, all values ​​greater than or equal to the median are partitioned into the right subtree;

[0223] (4) Update the partition axis: r = (r+1)%k;

[0224] (5) Determine child nodes:

[0225] Determine the left subtree node: Perform step 2 in the data of the left subtree;

[0226] Determine the right subtree node: Perform step 2 in the data of the right subtree.

[0227] The nearest neighbor search method is used to search for the nearest neighbor point between each subtree node in the kd-tree and the target point of the target large mechanical model.

[0228] Optionally, step S37 includes the following steps S371-S378:

[0229] S371. Input the root node of the kd-tree into the priority queue;

[0230] S372. Extract the point cloud data of the powered equipment corresponding to the best node in the priority queue;

[0231] S373. Calculate the first target distance between the point cloud data of the electrical equipment and the target point of the target large mechanical model, and set the first target distance as the target point proximity value;

[0232] S374. Extract the point cloud data of the electrical equipment corresponding to each node in the priority queue, and calculate the second target distance between the point cloud data of the electrical equipment corresponding to the current node and the target point of the target large mechanical model.

[0233] S375. Update the proximity value of the target point based on the distance to the second target;

[0234] S376. Search for the first subtree node in the dimension where the optimal node is located according to the preset threshold; wherein, when the subtree node where the optimal node is located is the left subtree node, the first subtree node is the right subtree node.

[0235] S377. Input the first subtree node into the priority queue, and jump to execute the step of extracting the point cloud data of the powered equipment corresponding to the optimal node in the priority queue until the number of searches exceeds the preset backtracking number, and generate multiple target point proximity values.

[0236] S378. Select the nearest neighbor value of the target point that is less than the preset nearest neighbor threshold from all the nearest neighbor values ​​of the target point as the nearest neighbor point between each subtree node and the target point of the target large mechanical model.

[0237] It should be noted that the kd-tree algorithm performs two parts in its nearest neighbor search: binary search and backtracking search. The number of backtracking steps directly determines the algorithm's efficiency. The KNN algorithm, based on kd-tree search, requires backtracking to the root node for each search, resulting in many irrelevant backtracking steps and unnecessary time consumption. To improve the efficiency of the k-nearest neighbor search, the BBF algorithm is proposed. This algorithm provides a priority queue to store nodes missed during binary search, sorted in ascending order of distance from the hyperplane containing each node to the target point A of the target large mechanical model. The preset nearest neighbor threshold refers to the threshold determined by all the k-nearest neighbor values ​​of the target point in ascending order. The maximum number of backtracking steps H depends on the actual situation and is not limited here.

[0238] In a specific embodiment, such as Figure 9 As shown, for a given target point A, its k-nearest neighbor search algorithm based on BBF is as follows:

[0239] Input: kd-tree, specifying the search point A, and the maximum number of backtracking steps H.

[0240] Output: Nk(A)

[0241] 1:t-node←prioritylist optimal dequeue node

[0242] 2: c-node ← t-node

[0243] 3: if dist(A, c-node) <max(Nk(A))

[0244] 4: Update Nk(A)

[0245] 5: if A[split] > t-node[split]

[0246] 6: Add the l-tree of t-node to prioritylist, and update the r-tree of t-node to c-node.

[0247] 7:else

[0248] 8: Add the r-tree of t-node to the priority list, and update the l-tree of t-node to c-node.

[0249] 9: Determine if a leaf node was found. If no, proceed to step 3; if yes, proceed to the next step.

[0250] 10: If prioritylist is empty or exceeds H, no: proceed to step 1; yes: proceed to the next step.

[0251] 11: Return Nk(A)

[0252] Input the root node of the kd-tree into a priority queue. Take the point cloud data of the powered equipment corresponding to the best node in the priority queue. Compare the distance between the current node and the target point A. Generate or update the k-nearest neighbor value of the target point. Search the subtree according to the preset threshold. Input the other subtree (the first subtree) of the current node into the priority queue. Specifically, when the node in the subtree where the current node is located is the left subtree node, then the node in the first subtree (the other subtree) is the right subtree node. Determine whether a leaf node has been found. If no leaf node has been found, return to the step of comparing the distance between the current node and the target point A and generating or updating the k-nearest neighbor value of the target point. If a leaf node has been found, proceed to the next step. Determine whether the priority queue is empty or whether the number of backtracking times exceeds the maximum number of backtracking times. When the number of backtracking times exceeds the maximum number of backtracking times, compare all the k-nearest neighbor values ​​of the target point and sort them from smallest to largest. Select the M target point k-nearest neighbor values ​​that are less than the preset proximity threshold as the nearest neighbor points.

[0253] Step 206: Calculate the shortest distance between the nearest point and the surface of the target large mechanical model.

[0254] In embodiments of the present invention, such as Figure 6 As shown, the shortest distance from the nearest point to the surface (bounding box surface) of the target large mechanical model is the required warning distance.

[0255] kd-tree input data: various types of electrical equipment points;

[0256] Query point: Current location of the target large mechanical model RTK and UWB points;

[0257] Output: The nearest electrical equipment points to each RTK and UWB point of the target large mechanical model and the Euclidean distance between them.

[0258] Step 207: When the shortest distance is less than the warning distance threshold, generate and output the warning information.

[0259] In this embodiment of the invention, the specific implementation process of step 207 is similar to that of step 106, and will not be repeated here.

[0260] Optionally, this method further includes the following steps S41-S43:

[0261] S41. Create a coordinate system with three unit axes perpendicular to each other and the camera position as the origin;

[0262] S42. Construct a visual following matrix using three unit axes and translation vectors; the expression for the visual following matrix is ​​as follows:

[0263]

[0264] Where R is the unit axis, representing the right vector; U is the unit axis, representing the up vector; D is the unit axis, representing the direction vector; and P is the camera position vector.

[0265] S43. Update the 3D position information of the target large mechanical model in the point cloud scene in the opposite direction of the coordinate position information of the visual following matrix.

[0266] It should be noted that since the position and orientation of the target large machinery model in the point cloud scene are updated each time the 3D pose parameter data of the RTK positioning information is transmitted, in order to capture the dynamic update process of the construction machinery in a timely manner, this invention adopts the view following function. When carrying out construction control, the field of view in the point cloud view is always focused on the target large machinery model and follows the movement and changes of the target large machinery model.

[0267] Viewfollowing is primarily implemented using the LookAt module in OpenGL. The look matrix transforms all world coordinates into look coordinates relative to the camera's position and orientation. When defining a camera, you need its position in world space, the direction of view, a vector pointing to its right, and a vector pointing upwards. For example... Figure 10 As shown, create a coordinate system with three mutually perpendicular unit axes and the camera's position as the origin.

[0268] In a specific embodiment, such as Figure 10As shown, Position represents the camera position, and Direction represents the camera direction. The camera direction is the direction in which the camera points towards the scene origin (0,0,0). The camera direction is obtained by subtracting the scene origin vector from the camera position vector, which is the direction of Z in the diagram. Right is the right vector, representing the positive direction of the x-axis in camera space. To obtain the right vector, an up vector must first be defined. Then, the up vector and the direction vector obtained in the second step are cross-producted. The result of the cross product of the two vectors will be perpendicular to both vectors, thus we obtain the vector pointing in the positive x-axis direction (if the order of the cross product is reversed, we obtain the opposite vector pointing in the negative x-axis direction). Finally, UP represents the upper axis.

[0269] The visual following matrix is ​​constructed from these three unit axes and translation vectors, as shown in the following expression:

[0270]

[0271] Specifically, R is the right vector, U is the up vector, D is the direction vector, and P is the camera position vector. The position vectors are inversely related because the world coordinate system is ultimately translated in the opposite direction to the movement of the monitoring personnel. Therefore, the 3D position information of the target large mechanical model in the point cloud scene is updated according to the opposite direction of the coordinate position information of the visual follow matrix. Using the visual follow matrix as the observation matrix allows for efficient transformation of all world coordinates to the newly defined observation space.

[0272] This invention responds to a received motion request from a large machine, acquiring the corresponding substation information and the three-dimensional position information of the large machine. It then processes the initial substation site cloud data corresponding to the substation information into point cloud data to generate target substation site cloud data. Using the three-dimensional position information and a preset large machine simulation model, it generates a target large machine model. It searches for the nearest neighbor points between the point clouds of each energized device in the target substation site cloud data and the target point of the target large machine model. It calculates the shortest distance between the nearest neighbor point and the surface of the target large machine model. When the shortest distance is less than a warning distance threshold, it generates and outputs a warning message. This invention solves the technical problem of existing methods that use infrared cameras and other sensors mounted on cranes to detect energized equipment in substations. However, single infrared sensing detection has large operating errors, low accuracy, and is prone to hitting energized equipment, leading to accidents. This invention has the following innovations and advantages:

[0273] (1) Explore and realize the combination of point cloud scene visualization technology and 3D modeling technology to achieve intuitive display of large construction machinery 3D visualization models in high-density, high-precision 3D point cloud scenes. Furthermore, it realizes the interaction between the construction machinery model and the point cloud scene;

[0274] (2) Explore and implement a construction data positioning data acquisition method that combines hardware and software. Install several portable RTKs at the corners and boom nodes of construction machinery. Use real-time remote communication technologies such as TCP / IP to transmit the field survey data monitored by the RTK hardware to the software for parsing and processing. This replaces the real-time early warning device that relies on lidar, thereby greatly reducing the hardware procurement cost and ensuring the reliability of the construction machinery positioning simulation.

[0275] (3) Explore and implement a method for rapid search of collision risk points and rapid calculation of early warning distances during construction machinery operation. A KNN nearest neighbor search algorithm based on KD-TREE is used to find the nearest live equipment location to the current position of the construction machinery in a very short time and calculate the distance between them. This provides strong safety assurance for the operation of large construction machinery in substations.

[0276] Please see Figure 11 , Figure 11 This is a structural block diagram of a large-scale mechanical motion monitoring and early warning system provided in Embodiment 3 of the present invention.

[0277] This invention provides a large-scale machinery motion monitoring and early warning system, comprising:

[0278] The three-dimensional position information module 1101 is used to respond to the received motion request of large machinery and obtain the substation information and the three-dimensional position information of the large machinery corresponding to the motion request.

[0279] The target substation cloud data module 1102 is used to process the initial substation cloud data corresponding to the substation information into point cloud data to generate target substation cloud data.

[0280] The target large machinery model module 1103 is used to generate a target large machinery model using three-dimensional position information and a preset large machinery simulation model;

[0281] The nearest neighbor module 1104 is used to search for the nearest neighbor of each live equipment point cloud in the target substation site cloud data and the target point of the target large machinery model.

[0282] The shortest distance module 1105 is used to calculate the shortest distance between the nearest point and the surface of the target large mechanical model;

[0283] The early warning information module 1106 is used to generate and output early warning information when the shortest distance is less than the early warning distance threshold.

[0284] Optionally, the three-dimensional location information module 1101 includes:

[0285] The substation information submodule is used to respond to received large machinery movement requests and obtain the substation information corresponding to the large machinery movement requests.

[0286] The information acquisition submodule is used to acquire the three-dimensional position information of large machinery through the RTK positioner and transmit the three-dimensional position information through the remote communication module.

[0287] Optionally, point cloud data processing includes global offset of point cloud coordinates, point cloud coloring, and point cloud clipping; the target substation point cloud data module 1102 includes:

[0288] The coordinate calculation submodule is used to calculate the center coordinates of the original coordinates of the initial substation cloud data corresponding to the substation information.

[0289] The offset vector submodule is used to offset the coordinates of each point cloud in the initial substation site cloud data according to the offset vector, using the center coordinates as the offset vector, to generate the first updated substation site cloud data.

[0290] The second point cloud data submodule is used to color the first updated substation point cloud data according to the preset point cloud coloring effect, and generate the second updated substation point cloud data.

[0291] The delete point cloud submodule is used to select point clouds within the substation area in the second updated substation site cloud data using a point cloud picking tool, and delete the unselected point clouds to generate and store the target substation site cloud data.

[0292] Optionally, the preset point cloud coloring effects include elevation coloring display, category coloring display, and original RGB information coloring display; the second point cloud data submodule includes:

[0293] The elevation coloring submodule is used to calculate the maximum and minimum elevation values ​​of the current point cloud corresponding to the first updated substation station cloud data when the point cloud corresponding to the first updated substation station cloud data needs to be displayed according to elevation coloring.

[0294] The preset color band submodule is used to set the current maximum point cloud elevation and the current minimum point cloud elevation as the two ends of the color band according to the preset color band;

[0295] The color band submodule is used to assign color to each point cloud using the position parameter of the current maximum point cloud elevation on the color band, and generate the second updated substation point cloud data with elevation coloring.

[0296] The category coloring submodule is used to set the corresponding category color according to the preset category of live equipment when the point cloud corresponding to the first updated substation site cloud data needs to be displayed according to category coloring.

[0297] The category color submodule is used to assign colors to the point clouds of each energized device according to each category color, and generate category-colored second update substation point cloud data;

[0298] The RGB information submodule is used to assign color to each point cloud according to the RGB channel values ​​in the point cloud data corresponding to the first updated substation cloud data when the point cloud data corresponding to the first updated substation cloud data needs to be colored according to the original RGB information, so as to generate RGB colored second updated substation cloud data.

[0299] Optionally, point cloud data processing also includes point cloud ground filtering and category editing; the point cloud deletion submodule includes:

[0300] The third update substation cloud data submodule is used to select the point clouds in the second update substation cloud data that are located within the substation area using a point cloud picking tool, and delete the unselected point clouds to generate the third update substation cloud data.

[0301] The triangulation submodule is used to select the local lowest point as the seed point from the cloud data of the third updated substation site, set all seed points as ground points and construct the triangulation.

[0302] The non-ground point calculation submodule is used to calculate each non-ground point according to a preset ground point judgment method.

[0303] The ground threshold condition submodule is used to determine whether the distance and maximum angle from non-ground points to the triangulation network meet the preset ground threshold conditions.

[0304] The current total number of ground points submodule is used to mark non-ground points as ground points and record the current total number of ground points if the condition is met.

[0305] The grid cell submodule is used to traverse the preset grid cells, reset the remaining ground points in the preset grid cells except for the lowest ground point to non-ground points, and determine whether the current proportion of the total number of ground points is greater than the proportion threshold.

[0306] The jump to the execution submodule is used to select the local lowest point as the seed point from the cloud data of the third updated substation site, set all seed points as ground points and construct the triangular network if the condition is met.

[0307] The fourth update substation cloud data submodule is used to determine all non-ground points as ground points and generate the fourth update substation cloud data if no.

[0308] The live equipment point cloud submodule is used to classify each point cloud in the fourth updated substation site cloud data according to the preset live equipment category, and generate multiple live equipment point clouds.

[0309] The target substation cloud data submodule is used to construct target substation cloud data using the point cloud of all energized equipment.

[0310] Optionally, the target large mechanical model module 1103 includes:

[0311] The large machinery simulation model submodule is used to construct large machinery simulation models according to the tracks, fuselage, and mechanical boom of the corresponding large machinery.

[0312] The large machinery model update submodule is used to determine the position and orientation of the large machinery simulation model in the point cloud scene using three-dimensional position information, and to generate an updated large machinery model.

[0313] The 3D position information update submodule is used to obtain the latest 3D position information at preset intervals and generate updated 3D position information.

[0314] The target large-scale mechanical model generation submodule is used to update the position and orientation of the large-scale mechanical simulation model in the point cloud scene according to the updated 3D position information, and generate the target large-scale mechanical model.

[0315] Optionally, the nearest neighbor module 1104 includes:

[0316] The variance value submodule is used to calculate the variance value of the dimension data of each live equipment point cloud in the target substation site cloud data;

[0317] The intermediate node submodule is used to calculate the average value of all point cloud data of powered devices in the first dimension corresponding to the maximum variance value, and set the median data corresponding to the average value as the intermediate node.

[0318] The judgment submodule is used to determine whether the point cloud data of the corresponding electrical equipment in the first dimension is less than the median data.

[0319] The left subtree node submodule is used to divide the point cloud data of the powered equipment into the left subtree node if the condition is met.

[0320] The right subtree node submodule is used to divide the point cloud data of the powered equipment into the right subtree node if no;

[0321] Construct a kd-tree submodule to jump to the step of calculating the average value of all point cloud data of powered devices in the first dimension corresponding to the maximum variance value, and setting the median data corresponding to the average value as the intermediate node, and constructing the kd-tree;

[0322] The nearest neighbor submodule is used to search for the nearest neighbor point between each subtree node in the kd-tree and the target point of the target large mechanical model using the nearest neighbor search method.

[0323] Optionally, the nearest neighbor submodule includes:

[0324] The priority queue submodule is used to input the root node of the kd-tree into the priority queue;

[0325] The optimal node submodule is used to extract the point cloud data of the powered equipment corresponding to the optimal node in the priority queue;

[0326] The target point proximity value submodule is used to calculate the first target distance between the point cloud data of the electrical equipment and the target point of the target large mechanical model, and set the first target distance as the target point proximity value.

[0327] The point cloud data extraction submodule is used to extract the point cloud data of the electrical equipment corresponding to each node in the priority queue, and calculate the second target distance between the point cloud data of the electrical equipment corresponding to the current node and the target point of the target large mechanical model.

[0328] The "Update Target Point Proximity Values" submodule is used to update the proximity values ​​of the target point based on the distance to the second target.

[0329] The preset threshold submodule is used to search for the first subtree node in the dimension where the optimal node is located according to the preset threshold; wherein, when the subtree node where the optimal node is located is the left subtree node, the first subtree node is the right subtree node.

[0330] The target point proximity value submodule is used to input the first subtree node into the priority queue and jump to execute the step of extracting the point cloud data of the electrical equipment corresponding to the best node in the priority queue until the number of searches exceeds the preset backtracking number, and generate multiple target point proximity values.

[0331] The target point nearest neighbor submodule is used to select the target point nearest neighbor value less than a preset nearest neighbor threshold from all target point nearest neighbor values ​​as the nearest neighbor point between each subtree node and the target large mechanical model.

[0332] Optionally, this system also includes:

[0333] The coordinate system submodule is used to create a coordinate system with three unit axes perpendicular to each other and the camera's position as the origin.

[0334] The visual following matrix submodule is used to construct the visual following matrix using three unit axes and translation vectors; the expression for the visual following matrix is ​​as follows:

[0335]

[0336] Where R is the unit axis, representing the right vector; U is the unit axis, representing the up vector; D is the unit axis, representing the direction vector; and P is the camera position vector.

[0337] The 3D position information submodule is used to update the 3D position information of the target large mechanical model in the point cloud scene in the opposite direction of the coordinate position information of the visual following matrix.

[0338] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0339] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0340] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0341] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0342] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0343] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring and early warning of the movement of large machinery, characterized in that, include: In response to a received motion request from a large machine, the substation information and the three-dimensional position information of the large machine corresponding to the motion request are obtained. The initial substation cloud data corresponding to the substation information is processed into point cloud data to generate target substation cloud data. Using the aforementioned three-dimensional position information and a preset large-scale mechanical simulation model, a target large-scale mechanical model is generated; Search for the nearest neighbor points of each live equipment point cloud in the target substation site cloud data and the target points of the target large mechanical model; Calculate the shortest distance between the nearest point and the surface of the target large mechanical model; When the shortest distance is less than the warning distance threshold, a warning message is generated and output. The point cloud data processing includes global offset of point cloud coordinates, point cloud coloring, and point cloud clipping. The step of processing the initial substation site cloud data corresponding to the substation information into point cloud data to generate target substation site cloud data includes: Calculate the center coordinates of the original coordinates of the initial substation cloud data corresponding to the substation information; Using the center coordinates as the offset vector, the coordinates of each point cloud in the initial substation site cloud data are offset according to the offset vector to generate the first updated substation site cloud data; The first updated substation cloud data is colored according to the preset point cloud coloring effect to generate the second updated substation cloud data. The point cloud picking tool is used to select the point clouds within the substation area in the second updated substation site cloud data, and each unselected point cloud is deleted to generate and store the target substation site cloud data. The step of generating the target large-scale machinery model using the three-dimensional position information and the preset large-scale machinery simulation model includes: Construct a simulation model of the large machinery based on the tracks, fuselage, and mechanical boom corresponding to the large machinery; The position and orientation of the large mechanical simulation model in the point cloud scene are determined using the three-dimensional position information, and the large mechanical model is then generated and updated. The latest three-dimensional position information is obtained at preset intervals, and updated three-dimensional position information is generated. The position and orientation of the large mechanical simulation model in the point cloud scene are updated according to the updated three-dimensional position information to generate the target large mechanical model; The step of searching for the nearest neighbor points of each live equipment point cloud in the target substation site cloud data and the target point of the target large mechanical model includes: Calculate the variance of the dimensional data of each energized equipment point cloud in the target substation site cloud data. Calculate the average value of all the point cloud data of the powered equipment in the first dimension corresponding to the maximum variance value, and set the median data corresponding to the average value as the intermediate node; Determine whether the point cloud data of the electrical equipment corresponding to the first dimension is less than the median data; If so, the point cloud data of the powered equipment is divided into the left subtree node; If not, the point cloud data of the powered equipment is divided into the right subtree node; Jump to execute the step of calculating the average value of all the point cloud data of the powered equipment in the first dimension corresponding to the maximum variance value, and setting the median data corresponding to the average value as the intermediate node, and construct a kd tree; The nearest neighbor search method is used to search for the nearest neighbor point between each subtree node of the kd-tree and the target point of the target large mechanical model.

2. The method for monitoring and early warning of large-scale machinery motion according to claim 1, characterized in that, The steps of responding to a received large machinery movement request and obtaining the substation information and three-dimensional position information of the large machinery corresponding to the large machinery movement request include: In response to a received request for movement of large machinery, obtain the substation information corresponding to the request for movement of large machinery; The 3D position information of large machinery is collected by an RTK locator and transmitted through a remote communication module.

3. The method for monitoring and early warning of large-scale machinery motion according to claim 1, characterized in that, The preset point cloud coloring effect includes elevation coloring display, category coloring display, and original RGB information coloring display; the step of assigning color to the first updated substation site cloud data according to the preset point cloud coloring effect to generate the second updated substation site cloud data includes: When the point cloud corresponding to the first updated substation station cloud data needs to be displayed according to elevation color, the maximum and minimum current point cloud elevation corresponding to the first updated substation station cloud data are calculated. The maximum and minimum elevation values ​​of the current point cloud are set as the two ends of the preset color band, respectively. The current maximum point cloud elevation is used as the position parameter on the color band to color each point cloud, generating the second updated substation point cloud data with elevation coloring; When the point cloud corresponding to the first updated substation site cloud data needs to be displayed in color according to category, the corresponding category color is set according to the preset category of live equipment. The point cloud of each energized device is colored according to the categories described above, and the second updated substation point cloud data is generated by category coloring. When the point cloud corresponding to the first updated substation cloud data needs to be displayed according to the original RGB information, the point cloud is colored according to the RGB channel values ​​in the point cloud data corresponding to the first updated substation cloud data, and RGB colored second updated substation cloud data is generated.

4. The method for monitoring and early warning of large-scale machinery motion according to claim 1, characterized in that, The point cloud data processing also includes point cloud ground filtering and category editing; the step of using a point cloud picking tool to select point clouds within the substation area in the second updated substation site cloud data, and deleting the unselected point clouds to generate and store the target substation site cloud data includes: The point cloud picking tool is used to select the point clouds within the substation area in the second updated substation site cloud data, and each unselected point cloud is deleted to generate the third updated substation site cloud data. Select the local lowest point from the third updated substation cloud data as a seed point, set all the seed points as ground points and construct a triangular network; Calculate each non-ground point according to the preset ground point judgment method; Determine whether the distance and maximum angle from the non-ground point to the triangulation meet the preset ground threshold condition; If so, mark the non-ground point as a ground point and record the current total number of ground points; Traverse the preset grid cells, reset the remaining ground points in the preset grid cells except for the lowest ground point to non-ground points, and determine whether the current proportion of the total number of ground points is greater than the proportion threshold. If so, then proceed to the step of selecting the local lowest point from the third updated substation cloud data as a seed point, setting all the seed points as ground points and constructing a triangular network; If not, then all the aforementioned non-ground points are determined as ground points, and the fourth updated substation cloud data is generated; The point clouds in the fourth updated substation site cloud data are classified according to the preset categories of energized equipment to generate multiple energized equipment point clouds. The target substation site cloud data is constructed using the point cloud of all the described energized equipment.

5. The method for monitoring and early warning of large-scale machinery motion according to claim 1, characterized in that, The step of searching for the nearest neighbor point between each subtree node of the kd-tree and the target point of the target large mechanical model using the nearest neighbor search method includes: Input the root node of the kd-tree into the priority queue; Extract the point cloud data of the powered equipment corresponding to the optimal node in the priority queue; Calculate the first target distance between the point cloud data of the electrical equipment and the target point of the target large mechanical model, and set the first target distance as the target point proximity value; Extract the point cloud data of the electrical equipment corresponding to each node in the priority queue, and calculate the second target distance between the point cloud data of the electrical equipment corresponding to the current node and the target point of the target large mechanical model; Update the neighbor value of the target point based on the second target distance; The first subtree node in the dimension of the optimal node is searched according to a preset threshold; wherein, when the subtree node of the optimal node is a left subtree node, the first subtree node is a right subtree node. Input the first subtree node into the priority queue, and jump to execute the step of extracting the point cloud data of the powered equipment corresponding to the optimal node in the priority queue until the number of searches exceeds the preset backtracking number, and generate multiple target point proximity values; Select the nearest neighbor value of the target point that is less than the preset nearest neighbor threshold from all the nearest neighbor values ​​of the target point as the nearest neighbor point of each subtree node to the target point of the target large mechanical model.

6. The method for monitoring and early warning of large machinery motion according to claim 1, characterized in that, Also includes: Create a coordinate system with three mutually perpendicular unit axes and the camera's position as the origin; A visual following matrix is ​​constructed using the three unit axes and translation vectors; the expression for the visual following matrix is ​​as follows: ； Where R is the unit axis, representing the right vector; U is the unit axis, representing the up vector; D is the unit axis, representing the direction vector; and P is the camera position vector. The 3D position information of the target large mechanical model in the point cloud scene is updated in the opposite direction to the coordinate position information of the visual following matrix.

7. A large-scale mechanical motion monitoring and early warning system, characterized in that, include: The three-dimensional position information module is used to respond to the received motion request of large machinery and obtain the substation information and the three-dimensional position information of the large machinery corresponding to the motion request. The target substation cloud data module is used to process the initial substation cloud data corresponding to the substation information into point cloud data to generate target substation cloud data. The target large machinery model module is used to generate a target large machinery model using the three-dimensional position information and a preset large machinery simulation model; The nearest neighbor module is used to search for the nearest neighbor point between each live equipment point cloud in the target substation site cloud data and the target point of the target large machinery model. The shortest distance module is used to calculate the shortest distance between the nearest point and the surface of the target large mechanical model; The early warning information module is used to generate and output early warning information when the shortest distance is less than the early warning distance threshold; The point cloud data processing includes global offset of point cloud coordinates, point cloud coloring, and point cloud clipping. The target substation cloud data module includes: The coordinate calculation submodule is used to calculate the center coordinates of the original coordinates of the initial substation cloud data corresponding to the substation information; The offset vector submodule is used to offset the coordinates of each point cloud in the initial substation cloud data according to the offset vector, using the center coordinates as the offset vector, to generate the first updated substation cloud data. The second point cloud data submodule is used to color the first updated substation site cloud data according to the preset point cloud coloring effect, and generate the second updated substation site cloud data. The delete point cloud submodule is used to select the point clouds within the substation area in the second updated substation site cloud data using a point cloud picking tool, and delete each unselected point cloud, generate the target substation site cloud data and store it. The target large mechanical model module includes: The large machinery simulation model submodule is used to construct a large machinery simulation model according to the tracks, fuselage and mechanical boom of the large machinery. The large machinery model update submodule is used to determine the position and orientation of the large machinery simulation model in the point cloud scene using the three-dimensional position information, and to generate an updated large machinery model. The three-dimensional position information update submodule is used to obtain the latest three-dimensional position information at preset intervals and generate updated three-dimensional position information. A target large machinery model generation submodule is used to update the position and orientation of the large machinery simulation model in the point cloud scene according to the updated three-dimensional position information, and generate the target large machinery model; The nearest neighbor module includes: The variance value submodule is used to calculate the variance value of the dimension data of each live equipment point cloud in the target substation site cloud data. The intermediate node submodule is used to calculate the average value of all the point cloud data of the powered equipment in the first dimension corresponding to the maximum variance value, and set the median data corresponding to the average value as the intermediate node. The judgment submodule is used to determine whether the point cloud data of the electrical equipment corresponding to the first dimension is less than the median data. The left subtree node submodule is used to divide the point cloud data of the powered equipment into the left subtree node if the condition is met. The right subtree node submodule is used to divide the point cloud data of the powered equipment into the right subtree node if no; A kd-tree submodule is constructed to jump to and execute the step of averaging all the point cloud data of the powered equipment in the first dimension corresponding to the maximum variance value, and setting the median data corresponding to the average value as the intermediate node, thereby constructing a kd-tree. The nearest neighbor submodule is used to search for the nearest neighbor point between each subtree node on the kd-tree and the target point of the target large mechanical model according to the nearest neighbor search method.

Images