A hidden danger target ranging method and device based on a three-dimensional image

Abstract

The application provides a hidden danger target ranging method and device based on a three-dimensional image, and belongs to the technical field of power transmission channel hidden danger identification. In the case that an image acquisition unit in a monitoring and shooting device collects an image including a preset marker, a server acquires to-be-fused environment images through a laser radar and the image acquisition unit in the monitoring and shooting device, so as to determine an environment three-dimensional image according to the to-be-fused environment images. The preset marker is a hidden danger target with pixel attributes meeting preset requirements, and the preset requirements are obtained by comparing the attributes of pixel points in sample data. Pixel region information of the preset marker in the environment three-dimensional image is determined. The pixel region information includes a preset marker position and a preset marker volume. According to the pixel region information, the distance between the preset marker and a power transmission area in the environment three-dimensional image is determined.

Description

Technical Field

[0001] This application relates to the field of power transmission channel hazard identification technology, and in particular to a hazard target ranging method and device based on three-dimensional images. Background Technology

[0002] With the development of technology, the distribution of power transmission lines is becoming increasingly complex. As the medium for power transmission, the safety and integrity of power transmission lines require constant attention from power transmission departments. If power transmission lines are damaged, it will not only cause inconvenience in electricity use, but may also threaten the lives of people near the power transmission lines.

[0003] Currently, power transmission regulatory authorities rely on drone inspections to monitor potential hazards around transmission lines, such as construction vehicles, damaged protective soil membranes, and large floating objects. While drone inspections can identify potential hazards, they cannot provide immediate alerts when such hazards are detected, thus hindering hazard identification efforts along transmission lines. Summary of the Invention

[0004] This application provides a method and device for ranging potential hazards based on three-dimensional images, which is used to solve the problem that the hazard investigation of power transmission channels is not real-time and affects the efficiency of hazard investigation.

[0005] On the one hand, this application provides a method for ranging potential hazards based on three-dimensional images, the method comprising: When the image acquisition unit in the monitoring equipment acquires an image including a preset marker, the server uses the lidar and image acquisition unit in the monitoring equipment to acquire images of the environment to be fused, and determines a 3D environmental image based on each image. The preset marker is a potential hazard target whose pixel attributes meet preset requirements, which are obtained by comparing the attributes of pixels in sample data. The pixel region information of the preset marker in the 3D environmental image is determined. This pixel region information includes the position and volume of the preset marker. Based on the pixel region information, the distance between the preset marker and the power transmission area in the 3D environmental image is determined.

[0006] In one implementation of this application, when the image acquisition unit acquires an image including preset markers, the image acquisition unit determines the corresponding LiDAR. The image acquisition unit is connected to the LiDAR via the control unit of the monitoring device. The LiDAR acquires point cloud data to be fused within the corresponding field of view of the image acquisition unit. Furthermore, when the LiDAR acquires the point cloud data to be fused, the image acquisition unit acquires the corresponding two-dimensional image to be fused. The environment image to be fused includes the point cloud data to be fused and the two-dimensional image to be fused. Based on the image fusion processing of the point cloud data to be fused and the two-dimensional image to be fused, the server generates a three-dimensional environment image.

[0007] In one implementation of this application, when the image acquisition unit acquires an image including preset markers, the image acquisition unit determines a corresponding preset device group. The preset device group includes several monitoring devices set at multiple angles. The corresponding monitoring devices of the image acquisition unit send synchronous acquisition commands to each monitoring device in the preset device group via a wireless connection unit connected to the control unit. Based on the synchronous acquisition commands, each image acquisition unit in the preset device group acquires the two-dimensional image to be fused, and each LiDAR acquires the point cloud data to be fused, so that the server performs fusion and stitching processing on each two-dimensional image to be fused and the corresponding point cloud data to be fused, thereby determining a three-dimensional environmental image.

[0008] In one implementation of this application, a first monitoring device in a preset device group generates a device synchronization signal with a preset delay based on a synchronization acquisition command. The preset delay is determined according to the number of monitoring devices in the preset device group. The first monitoring device in the preset device group sequentially transmits the device synchronization signal to second monitoring devices via a wireless connection unit. The second monitoring devices can be any monitoring device other than the first monitoring device in the preset device group, and each second monitoring device communicates with the first monitoring device and / or the server.

[0009] In one implementation of this application, when the image acquisition unit fails to acquire an image including a preset marker, the wireless connection unit of each monitoring device in the preset device group starts operating according to preset work cycle data. When the image acquisition unit acquires an image including the preset marker, the wireless connection unit starts operating according to preset work cycle data after sending the 3D environmental image to the server.

[0010] In one implementation of this application, when the distance between a preset marker and the power transmission area in the 3D environmental image is less than a preset threshold, the server generates an alarm message. The alarm message includes one or more of the following: text, sound, and image. The server determines the corresponding management terminal with the acquisition location of the 3D environmental image as the center and the preset distance as the radius. The server sends the alarm message to the management terminal.

[0011] In one implementation of this application, when the preset marker is a moving object, the server determines the moving object's trajectory using a Kalman filter model. The server determines the shortest distance between the boundary line of the 3D environmental image and the moving object as the escape distance. If the escape distance is less than a preset value, the server determines whether a power transmission area exists in the direction of the moving trajectory. If a power transmission area is determined to exist in the direction of the moving trajectory, the server generates a monitoring command to activate and monitor the corresponding monitoring device in the direction of the moving trajectory.

[0012] In one implementation of this application, the server divides the power transmission area into several sub-regions. The server determines the center pixel of each sub-region and the geometric center pixel of a preset marker in the 3D image of the environment. The server determines the center distance between each center pixel and the geometric center pixel, and uses the center distances that satisfy a predetermined condition as the first distance. The predetermined condition is the minimum value among the center distances. The server determines the second distance between the sub-region corresponding to the first distance and several pixel blocks corresponding to the preset marker. The pixel blocks are obtained by dividing the preset marker according to a predetermined rule, which is determined based on the volume of the preset marker. The server uses the minimum value among the second distances as the distance between the preset marker and the power transmission area in the 3D image of the environment.

[0013] In one implementation of this application, the monitoring device sends a heartbeat packet to the server via a wireless connection unit. If the server does not receive the heartbeat packet, it determines the device serial number of the monitoring device that is missing the heartbeat packet. Based on the device serial number, the server generates device detection information and sends the device detection information to the corresponding maintenance terminal.

[0014] On the other hand, this application also provides a hazard target ranging device based on three-dimensional images, the device comprising: At least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: When the image acquisition unit in the monitoring equipment acquires an image including a preset marker, the server uses the lidar and image acquisition unit in the monitoring equipment to acquire images of the environment to be fused, and determines a 3D environmental image based on each image. The preset marker is a potential hazard target whose pixel attributes meet preset requirements, which are obtained by comparing the attributes of pixels in sample data. The pixel region information of the preset marker in the 3D environmental image is determined. This pixel region information includes the position and volume of the preset marker. Based on the pixel region information, the distance between the preset marker and the power transmission area in the 3D environmental image is determined.

[0015] By using the above-described method, fixed monitoring equipment on transmission towers can obtain three-dimensional environmental images of potential hazards. These images allow for accurate detection of the distance between the hazard and the transmission line, thus enabling hazard distance measurement. The fixed monitoring equipment in this application can provide immediate alarms for potential hazards and simultaneously monitor the distance between the hazard and the transmission line, improving the efficiency of identifying potential transmission line hazards. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic flowchart of a method for ranging potential hazards based on three-dimensional images, as described in an embodiment of this application. Figure 2 This is a schematic diagram of the monitoring device in a method for ranging potential hazards based on three-dimensional images, as described in an embodiment of this application. Figure 3 This is a schematic diagram of another structure of the monitoring device in a method for ranging potential hazards based on three-dimensional images, as described in an embodiment of this application. Figure 4 This is another structural schematic diagram of the monitoring device in a method for ranging potential hazards based on three-dimensional images, as described in an embodiment of this application. Figure 5 This is a schematic diagram of a monitoring device in a method for ranging potential hazards based on three-dimensional images, as described in an embodiment of this application. Figure 6 This is a schematic diagram of the structure of a hazard target ranging device based on three-dimensional images in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] While visual monitoring of power transmission channels can, to some extent, avoid the need for inspectors to conduct field inspections, the lack of depth information in the images can lead to non-hazardous targets being misidentified as hazardous targets. This undoubtedly increases the workload for image inspectors in removing misidentified images.

[0019] Drones equipped with lidar can be used to inspect power transmission channels and obtain images of potential hazards with depth information. However, drone inspections are conducted infrequently and cannot monitor the occurrence of potential hazards near power transmission channels in real time.

[0020] Based on this, this application provides a method and device for ranging potential hazards based on three-dimensional images, which can be used to monitor potential hazards in power transmission channels in real time.

[0021] The various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] This application provides a method for ranging potential hazards based on three-dimensional images, such as... Figure 1 As shown, the method may include steps S101-S106: S101, when the server acquires an image including a preset marker in the image acquisition unit of the monitoring device, it acquires the environment image to be fused by the lidar in the monitoring device and the image acquisition unit, so as to determine the three-dimensional image of the environment based on each of the environment images to be fused.

[0023] Among them, the preset markers are potential hazards whose pixel attributes meet preset requirements, which are obtained by comparing the attributes of pixels in the sample data.

[0024] In this embodiment, the image acquisition unit and the lidar are included in the hazard target monitoring device, and can acquire images of the same field of view. The lidar can acquire point cloud data in the field of view, and the image acquisition unit acquires two-dimensional images containing color. In this embodiment, the image acquisition unit can be a camera, webcam, or other similar device, and the lidar can be a lidar with a wavelength of 905 nanometers.

[0025] The range of a 905-nanometer laser radar is relatively short, while the cost of a 1550-nanometer laser radar is relatively high. This application embodiment collects the potential hazards of power transmission towers by installing monitoring equipment at multiple angles, so as to achieve the effect of both real-time acquisition of potential hazards and cost savings.

[0026] In this embodiment, multiple or a single transmission tower has several monitoring devices, which are installed in the following three ways: 1. The monitoring devices are installed facing each other, that is, the monitoring devices are set on the transmission towers within a span and installed in opposite directions; 2. The monitoring devices are installed around the transmission tower to collect images around the transmission tower; 3. Monitoring devices are set on multiple transmission towers so that the collection areas of each monitoring device are stitched together to form a ring area.

[0027] In the first installation method, the system structure can be as follows: Figure 2 As shown, the monitoring devices communicate wirelessly; the system structure for the second installation method can be as follows: Figure 3 As shown, the main monitoring device and the server can communicate via 4G or 5G mobile communication technology; the system structure of the third installation method can be as follows: Figure 4 As shown, the main monitoring device and each slave monitoring device are connected wirelessly, and the main monitoring device and each slave monitoring device communicate with the server via 4G or 5G. Wireless communication can be via Bluetooth, a 433MHz wireless module, or ZigBee Long Range Radio (LoRA), etc. By employing the three installation methods described above, the field of view for acquiring potential hazards can be improved, compensating for the narrow field of view limitation of low-wavelength lidar. It also allows for real-time acquisition of potential hazards around power transmission lines. Furthermore, this application utilizes a combination of lidar and an image acquisition unit to acquire potential hazards, resulting in color-coded 3D images with richer information and more accurate distance calculations from the 3D images.

[0028] In this embodiment, the image acquisition unit can acquire external images in real time or after a set time, such as 4 seconds. The image acquisition unit can then send the external images to the control unit, i.e., the ARM core board, via the Mobile Industry Processor Interface (MIPI). The core board can identify whether the external images contain preset markers. Alternatively, the core board can send the external images to a server for the server to identify the preset markers. A structural diagram of the device installed on the transmission tower is shown below. Figure 5 As shown, the core board connects to the wireless connection unit via a serial general purpose input / output (GPIO) port and to the LiDAR via a network port. The core board also connects to a rechargeable battery unit and is connected to a charging management unit via an MCU control unit. The charging management unit is connected to the rechargeable battery unit to provide power to the hazard monitoring equipment.

[0029] When the external image contains preset markers, the core board can control the LiDAR and image acquisition unit to simultaneously acquire images. The core board can then fuse the images acquired by the LiDAR and image acquisition unit to obtain a 3D environmental image, which is then sent to the server. Alternatively, the core board can directly send the images acquired by the LiDAR and image acquisition unit to the server. The server then uses SLAM point cloud matching to complete the image stitching and fusion operation to obtain a 3D environmental image.

[0030] The 3D environmental image can be obtained by stitching together images captured by multiple monitoring devices, with overlapping areas in the fields of view of the multiple monitoring devices. In this embodiment, the preset marker can be obtained by extracting the pixel attributes of potential hazards from several sample data. The pixel attributes include at least: color, pixel shape, etc. The core board or server of the monitoring device can be pre-set with a trained image recognition model to identify the preset marker. The image recognition model identifies whether the image contains the preset marker. The image recognition model includes, but is not limited to, models based on CNN.

[0031] In step S101, the server acquires images of the environment to be fused using the lidar and the image acquisition unit, respectively, to determine a three-dimensional environmental image based on each of the images, specifically including: When the image acquisition unit acquires an image including a preset marker, the image acquisition unit determines the corresponding lidar.

[0032] The image acquisition unit is connected to the lidar through the control unit of the monitoring equipment.

[0033] The lidar acquires point cloud data to be fused within the corresponding field of view of the image acquisition unit, and when the lidar acquires the point cloud data to be fused, the image acquisition unit acquires the corresponding two-dimensional image to be fused.

[0034] The environmental images to be fused include point cloud data to be fused and two-dimensional images to be fused.

[0035] Based on the image fusion processing of the point cloud data to be fused and the two-dimensional image to be fused, the server generates the three-dimensional image of the environment.

[0036] Through the above scheme, the image acquisition unit and the lidar can communicate directly with the server through the wireless connection unit, so as to send the point cloud data to be fused to the server.

[0037] Because radar ranging targets are limited, the images of potential hazards collected by a single monitoring device are also limited. If the number and area of ​​potential hazards are large, relying solely on a single monitoring device cannot obtain complete images of the potential hazards. Therefore, step S101 also specifically includes the following methods: First, when the image acquisition unit acquires an image including a preset marker, the image acquisition unit determines the corresponding preset device group.

[0038] The preset equipment group includes several monitoring devices set up at multiple angles.

[0039] The pre-installed equipment group can use any one or more of the three installation methods mentioned above, depending on the actual installation environment.

[0040] Then, the corresponding monitoring device of the image acquisition unit sends a synchronous acquisition command to each monitoring device in the preset device group through the wireless connection unit connected to the control unit.

[0041] In this embodiment, the first monitoring device in the preset device group generates a device synchronization signal delayed by a preset time based on a synchronization acquisition command. The preset time is determined according to the number of monitoring devices in the preset device group.

[0042] During installation, the server can designate one monitoring device from a pre-set device group as the master device and the remaining monitoring devices as slave devices, with the master device being the first monitoring device. The preset delay time can be set as follows: if there are n monitoring devices, where n is a natural number greater than 1, then based on the communication time t between the monitoring devices, the delay time of the first monitoring device is nt. After nt, the first monitoring device acquires an image. The delay of the monitoring device that first sends a signal is (n-1)t, and the delays of each monitoring device are (n-2)t, (n-3)t, ..., t.

[0043] The first monitoring device in the preset device group transmits the device synchronization signal sequentially to the second monitoring device via a wireless connection unit. The second monitoring device can be any monitoring device other than the first monitoring device in the preset device group, and each second monitoring device communicates with the first monitoring device and / or the server.

[0044] The above scheme enables each monitoring device to simultaneously acquire images of preset markers, laying the foundation for the subsequent stitching process.

[0045] Finally, based on the synchronous acquisition command, each image acquisition unit in the preset device group acquires the two-dimensional image to be fused and each lidar acquires the point cloud data to be fused, so that the server can perform fusion and stitching processing on each two-dimensional image to be fused and the corresponding point cloud data to be fused to determine the three-dimensional image of the environment.

[0046] In this application embodiment, real-time operation of the monitoring equipment would consume unnecessary energy. Therefore, this application provides the following solution: If the image acquisition unit fails to acquire an image including the preset marker, the wireless connection unit of each monitoring device in the preset device group will operate according to the preset working cycle data.

[0047] Work cycle data is as follows: from 0:00 to 5:00, wake up once every 10 seconds; from 6:00 to 18:00, wake up once every 4 seconds; from 19:00 to 23:00, wake up once every 8 seconds. Specific work cycle data can be set in actual use. This is only an example of work cycle data, and this application does not impose any specific limitations on it.

[0048] When the image acquisition unit acquires an image including preset markers, the wireless connection unit sends the 3D environmental image to the server and then starts running according to the preset work cycle data.

[0049] The above solution can save energy consumption of the wireless connection unit of the surveillance equipment, reduce power consumption and avoid resource waste in places with low communication frequency and low energy consumption.

[0050] S102, the server determines the pixel region information of the preset marker in the three-dimensional image of the environment.

[0051] The pixel region information includes the preset marker position and the preset marker volume.

[0052] After obtaining the stitched or unstitched 3D environmental image, the server determines preset markers, such as cranes, their pixel locations within the 3D environmental image, and the volume formed by those pixels. Using this pixel area information, the server can further monitor for potential hazards. In this embodiment, the server can also use the volume and location of preset markers to raise awareness of certain preset markers. For example, if the volume of a preset marker exceeds a preset value, the server can focus on the location of that marker.

[0053] S103, the server determines the distance between the preset marker and the power transmission area in the 3D image of the environment based on the pixel area information.

[0054] In this embodiment of the application, step S103 can be implemented by performing the following method, specifically: First, the server divides the power transmission area into several sub-regions.

[0055] For example, a power transmission area includes transmission towers and transmission lines. The server can define the locations of the transmission towers and transmission lines as the power transmission area. Based on actual usage, the power transmission area can be divided into several sub-areas.

[0056] Secondly, the server determines the center pixel of each sub-region and the geometric center pixel of the preset marker in the 3D image of the environment.

[0057] The preset marker in the 3D image of the environment can be a cube, cuboid, trihedron, or an irregular shape. The server can obtain the center pixel of the set of pixels of the preset marker through the pixels of the graphic.

[0058] The server determines the center distance between each center pixel and the geometric center pixel, and uses the center distances that satisfy a predetermined condition as the first distance. The predetermined condition is the minimum value among all center distances.

[0059] The predetermined condition could be that after the server calculates several center distances, it compares the numerical values ​​of each center distance and selects the center distance with the smallest numerical value as the center distance that satisfies the predetermined condition.

[0060] Next, the server determines the second distance between the sub-region corresponding to the first distance and a number of pixel blocks corresponding to the preset marker.

[0061] The second distance is the distance between each pixel in the sub-region and the corresponding pixel blocks of the preset marker.

[0062] Finally, the server uses the minimum value among the second distances as the distance between the preset marker and the power transmission area in the 3D image of the environment.

[0063] The above method can be used to obtain the distance between potential hazards and power transmission channels and towers through three-dimensional environmental images, thereby timely identifying potential power transmission hazards and improving the efficiency of hazard identification.

[0064] S104, if the distance between the preset marker and the power transmission area in the 3D image of the environment is less than a preset threshold, the server generates an alarm message.

[0065] Alarm information includes one or more of the following: text, sound, and image.

[0066] Warning messages include, for example, the potential hazard of a crane entering the danger zone of a power transmission channel in area A, or the potential hazard of a flock of birds entering within 10 meters of a power transmission tower in area B.

[0067] S105, the server uses the location where the 3D environmental image was acquired as the center and a preset distance as the radius to determine the corresponding management terminal.

[0068] In this embodiment of the application, the server can use GPS to obtain the location of the management terminal within the range of the acquisition location of the three-dimensional environmental image, and filter the management terminals with the acquisition location of the three-dimensional environmental image as the center and a preset distance as the radius.

[0069] S106, the server sends the alarm information to the management terminal.

[0070] Since potential hazards may be constantly moving, if the monitoring area of ​​the corresponding preset equipment group of the surveillance equipment is limited, the movement of the hazard may cause it to escape the current monitoring area. In this embodiment, the following method can be used to monitor the hazard in real time. Specifically: First, with the preset marker being a moving object, the server determines the moving object's trajectory using a Kalman filter model.

[0071] The server can determine whether a preset marker is moving using a preset image recognition model, and estimate the trajectory of the moving object using a Kalman filter model.

[0072] Then, the server determines the shortest distance between the boundary line of the 3D image of the environment and the moving object, as the escape distance.

[0073] As the moving object continues to move, the server can calculate the distance between the boundary line and the moving object in real time. This boundary line corresponds to the maximum area boundary that the preset equipment group of the monitoring device can collect.

[0074] Then, if the escape distance is less than a preset value, the server determines whether there is a power transmission area in the direction of the movement trajectory.

[0075] Following the direction of the movement trajectory, the server can determine the area that the moving potential hazard is about to enter. If the potential hazard is about to enter the monitoring area of ​​another preset equipment group, meaning there is a power transmission area in the direction of the movement trajectory, the server can pre-activate the monitoring function of the preset equipment group.

[0076] Finally, the server determines that there is a power transmission area in the direction of the movement trajectory, so that the corresponding monitoring equipment in the direction of the movement trajectory is activated and begins monitoring.

[0077] The server can set the distance between the preset marker and the boundary line of the 3D environment image to be less than a preset value, such as 10 meters or 5 meters. Alternatively, it can calculate the distance based on the device group's startup time and the preset marker's movement speed. For example, if the movement speed is v, the startup time is t1, and the escape distance is x, the preset value could be: v (x / v-t1)+1, where x / v is greater than t1.

[0078] Since power transmission lines typically pass through open areas, the maintenance of surveillance equipment requires on-site inspection by maintenance personnel, resulting in a waste of human resources. Therefore, this application provides the following technical solution: The monitoring device sends heartbeat packets to the server via a wireless connection unit.

[0079] A heartbeat packet is a user-defined command message sent periodically between the client and server to notify each other of the client's status. It's sent at regular intervals, similar to a heartbeat, hence the name. Heartbeat packets can be used to monitor the operational status of surveillance equipment.

[0080] If the server does not receive a heartbeat packet, determine the device serial number of the monitoring device that is missing a heartbeat packet.

[0081] The device serial number can be the production serial number of the image acquisition unit or LiDAR in the monitoring device, or it can be the physical address of the network card of the monitoring device. The device serial number only needs to be able to identify the monitoring device. As for the specific type of device serial number, such as SIM card number or production serial number, this application does not make specific restrictions.

[0082] Based on the device serial number, the server generates device detection information and sends the device detection information to the corresponding maintenance terminal.

[0083] The above solution can monitor the operating status of the corresponding monitoring equipment in the power transmission channel, preventing the problem of the monitoring equipment malfunctioning and thus failing to identify potential hazards.

[0084] Figure 6 A schematic diagram of a hazard target ranging device based on three-dimensional images is provided as an embodiment of this application, such as... Figure 6 As shown, the device includes: At least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: When the image acquisition unit in the monitoring equipment acquires an image including a preset marker, the server uses the lidar and image acquisition unit in the monitoring equipment to acquire images of the environment to be fused, and determines a 3D environmental image based on each image. The preset marker is a potential hazard target whose pixel attributes meet preset requirements, which are obtained by comparing the attributes of pixels in sample data. The pixel region information of the preset marker in the 3D environmental image is determined. This pixel region information includes the position and volume of the preset marker. Based on the pixel region information, the distance between the preset marker and the power transmission area in the 3D environmental image is determined.

[0085] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0086] The computer device and method provided in this application are one-to-one correspondences. Therefore, the computer device also has similar beneficial technical effects as its corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the computer device will not be repeated here.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0088] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for ranging potential hazards based on three-dimensional images, characterized in that, The method includes: When the image acquisition unit in the monitoring device acquires an image including preset markers, the server uses the lidar in the monitoring device and the image acquisition unit to acquire images of the environment to be fused, respectively, to determine a three-dimensional image of the environment based on each image of the environment to be fused. Specifically, this includes: the image acquisition unit determining a corresponding preset device group; the preset device group including several monitoring devices set at multiple angles; the monitoring devices corresponding to the image acquisition unit sending a synchronization acquisition command to each monitoring device in the preset device group through a wireless connection unit connected to the control unit; and based on the synchronization acquisition command, each image in the preset device group... The acquisition unit acquires two-dimensional images to be fused, and each lidar acquires point cloud data to be fused, so that the server performs fusion and stitching processing on each of the two-dimensional images to be fused and the corresponding point cloud data to be fused to determine the three-dimensional environmental image; wherein, the preset marker is a potential hazard target whose pixel attributes meet preset requirements, and the preset requirements are obtained by comparing the attributes of pixels in sample data; the environmental image to be fused includes point cloud data to be fused and two-dimensional images to be fused; based on the image fusion processing of the point cloud data to be fused and the two-dimensional images to be fused, the server generates the three-dimensional environmental image; the lidar used is a lidar with a wavelength of 905 nanometers; Multiple or a single transmission tower may be equipped with several monitoring devices, installed in the following ways:

1. Monitoring devices are installed facing each other, meaning they are placed on transmission towers within a span and installed in opposite directions; 2. Monitoring devices are installed around the transmission tower to collect images of the surrounding area; 3. Monitoring devices are installed on multiple transmission towers, so that the areas collected by each device are stitched together to form a ring-shaped area. In the first installation method, the monitoring devices communicate wirelessly with each other; in the second installation method, the main monitoring device communicates with the server via 4G or 5G; in the third installation method, the main monitoring device and each slave monitoring device are connected wirelessly, and the main monitoring device and each slave monitoring device communicate with the server via 4G or 5G. The server determines the pixel region information of the preset marker in the 3D image of the environment; wherein, the pixel region information includes the position and volume of the preset marker; The server determines the distance between the preset marker and the power transmission area in the 3D environmental image based on the pixel region information; specifically including: The server divides the power transmission area into several sub-areas; The server determines the center pixel of each sub-region and the geometric center pixel of the preset marker in the 3D image of the environment; The server determines the center distance between each of the center pixels and the geometric center pixel, and takes each of the center distances that satisfies a predetermined condition as the first distance; the predetermined condition is the minimum value among the center distances. The server determines a second distance between the sub-region corresponding to the first distance and a plurality of pixel blocks corresponding to the preset marker; wherein the pixel blocks are obtained by dividing the preset marker according to a preset rule, and the preset rule is determined based on the volume of the preset marker; The server uses the minimum value of each of the second distances as the distance between the preset marker and the power transmission area in the three-dimensional image of the environment.

2. The method according to claim 1, characterized in that, The laser radar and the image acquisition unit acquire images of the environment to be fused, specifically including: When the image acquisition unit acquires an image including the preset marker, the image acquisition unit determines the corresponding lidar; wherein, the image acquisition unit is connected to the lidar through the control unit of the monitoring device; The lidar acquires point cloud data to be fused from the corresponding field of view of the image acquisition unit; and When the lidar acquires the point cloud data to be fused, the image acquisition unit acquires the corresponding two-dimensional image to be fused from the point cloud data.

3. The method according to claim 1, characterized in that, Before each image acquisition unit in the preset device group acquires the two-dimensional image to be fused and each lidar acquires the point cloud data to be fused, the method further includes: The first monitoring device in the preset device group generates a device synchronization signal with a preset delay based on the synchronous acquisition command; wherein, the preset delay is determined according to the number of monitoring devices in the preset device group; The first monitoring device in the preset device group sends the device synchronization signal to the second monitoring device sequentially through the wireless connection unit; the second monitoring device is any monitoring device other than the first monitoring device in the preset device group, and each second monitoring device communicates with the first monitoring device and / or the server.

4. The method according to claim 1, characterized in that, The method further includes: If the image acquisition unit fails to acquire an image including the preset marker, the wireless connection unit of each monitoring device in the preset device group starts operation according to the preset working cycle data. When the image acquisition unit acquires an image including preset markers, the wireless connection unit sends the three-dimensional environmental image to the server and then starts running according to the preset work cycle data.

5. The method according to claim 1, characterized in that, After determining the distance between the preset marker and the power transmission area in the three-dimensional environmental image, the method further includes: If the distance between the preset marker and the power transmission area in the 3D environmental image is less than a preset threshold, the server generates an alarm message; the alarm message includes one or more of the following: text, sound, and image; The server determines the corresponding management terminal with the acquisition location of the three-dimensional environmental image as the center and a preset distance as the radius. The server sends the alarm information to the management terminal.

6. The method according to claim 1, characterized in that, After determining the distance between the preset marker and the power transmission area in the three-dimensional environmental image based on the pixel region information, the method further includes: When the preset marker is a moving object, the server determines the moving trajectory of the moving object using a Kalman filter model. The server determines the shortest distance between the boundary line of the 3D environmental image and the moving object, as the escape distance; If the escape distance is less than a preset value, the server determines whether there is a power transmission area in the direction of the movement trajectory; If present, the server generates a monitoring command to activate and monitor the corresponding monitoring device along the movement trajectory.

7. The method according to claim 1, characterized in that, The method further includes: The monitoring device sends heartbeat packets to the server via a wireless connection unit; If the server does not receive the heartbeat packet, determine the device serial number of the monitoring device that is missing the heartbeat packet; Based on the device serial number, the server generates device detection information and sends the device detection information to the corresponding maintenance terminal.

8. A hazard target ranging device based on three-dimensional images, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method for ranging potential targets based on three-dimensional images as described in any one of claims 1-7.

Images