Construction equipment threatening power grid line safety detection method and device

Abstract

The present application provides a method and device for detecting the safety of power grid lines threatened by construction machinery, wherein the method comprises: based on a rotating target detection algorithm, identifying the mechanical arm segmentation target and the base target contained in the collected scene graph, and locating the transmission line; identifying the type of the mechanical arm segmentation target, and merging the mechanical arm segmentation targets of the same type that meet the preset merging rules to obtain a complete mechanical arm target; matching the complete mechanical arm target with the base target, determining the complete construction machinery target, and determining the construction status information of the complete construction machinery target and the transmission line; determining the shortest distance between the complete construction machinery target and the transmission line, and the threshold distance between the complete construction machinery target and the transmission line according to the construction status information; comparing the shortest distance with the threshold distance, and pushing safety warning information according to the comparison result. Realize the monitoring of the safety of power grid lines threatened by construction machinery, and reduce the situation of construction machinery touching the transmission line and destroying the safety of electricity from the source.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and more particularly to a method for detecting the safety of a power grid line threatened by a construction tool. The present application also relates to a device for detecting the safety of a power grid line threatened by a construction tool. Background Art

[0002] In power transmission scenarios, construction equipment or other targets often appear near power transmission lines, which poses certain risks to the safety of power transmission lines. Specifically, when construction equipment such as cranes, cement pumps, and pile drivers are operating near power transmission channels, the long mechanical arms of the construction equipment are prone to collision and damage to the surrounding power transmission lines, causing power accidents, threatening power safety, causing inconvenience to the production and life of enterprises and residents along the line, and also causing huge economic losses. Therefore, how to identify potential danger targets in power transmission scenarios is of great significance and can greatly improve the safety of power transmission lines.

[0003] The existing line safety detection uses straight line detection combined with traditional machine learning algorithms, which is prone to false detection when encountering objects such as decorative lines on buildings and straight paths. Specifically, in the existing line safety detection, the projections of the two leftmost and rightmost wires in the wire area and the bottom line of the tower together constitute the wire danger warning area, and then the distance between the bottom of the target device and the warning area is solved to represent the distance between the target device and the wire.

[0004] However, this type of algorithm is an indirect distance algorithm with low accuracy. As a result, even if the low construction equipment is located in the danger zone below the conductor, it will not pose a threat to the conductor due to the huge height difference and long distance between it and the conductor. However, the indirect distance algorithm cannot reflect this distance difference.

[0005] In addition, in the existing line safety detection, after calculating the distance between the equipment and the wire, the distance threshold is determined only according to the type of the target equipment, without considering that the same target in different postures, such as a vertical crane and a horizontal crane, poses different threats to the wire. If this is not distinguished, a horizontal crane on the road will also cause false alarms; and for construction equipment, different models of the same target will have different lengths, and the lengths of the mechanical arms of construction equipment of different lengths will be different, so the dangerous distance to the wire should be different; finally, for construction equipment, when its relative position to the wire is different, such as the wire is on the left side of the crane, and the crane arm is tilted to the right or to the left, there will be inconsistent danger levels. The existing line safety detection cannot effectively solve the above problems. Summary of the invention

[0006] In view of this, the embodiment of the present application provides a method for detecting the safety of power grid lines threatened by construction equipment to solve the technical defects existing in the prior art. The embodiment of the present application also provides a device for detecting the safety of power grid lines threatened by construction equipment, a computing device, and a computer-readable storage medium.

[0007] According to a first aspect of an embodiment of the present application, a method for detecting a construction device threatening the safety of a power grid line is provided, comprising:

[0008] Based on the rotating target detection algorithm, the robot arm segment targets and base targets contained in the collected scene graph are identified, and the transmission line is located;

[0009] Identify the type of the robotic arm segmented target, and merge the robotic arm segmented targets of the same type that meet the preset merging rules to obtain a complete robotic arm target;

[0010] Matching the complete robot arm target with the base target, determining a complete construction machine target, and determining the construction status information of the complete construction machine target and the transmission line;

[0011] Determine, based on the construction status information, the shortest distance between the complete construction machine target and the transmission line, and a threshold distance between the complete construction machine target and the transmission line;

[0012] The shortest distance is compared with the threshold distance, and safety warning information is pushed according to the comparison result.

[0013] Optionally, merging the same type of the robotic arm segment targets that meet a preset merging rule to obtain a complete robotic arm target includes:

[0014] Based on the robotic arm segmentation targets of the same type, construct a robotic arm segmentation target set;

[0015] Selecting one of the robotic arm segmentation targets in the robotic arm segmentation target set as a candidate robotic arm segmentation target;

[0016] Determine the left and right endpoints of all the robot arm segment targets in the robot arm segment target set, wherein the left and right endpoints include a left endpoint and a right endpoint;

[0017] sequentially calculating a first endpoint distance between one of the left and right endpoints of the candidate robotic arm segment and the other of the left and right endpoints of the other robotic arm segment targets in the robotic arm segment target set;

[0018] When the first endpoint distance is less than a preset first endpoint distance threshold, recording the robotic arm segment target associated with the first endpoint distance into a complete robotic arm target set;

[0019] The step of selecting one of the robotic arm segmentation targets in the robotic arm segmentation target set as a candidate robotic arm segmentation target is executed until all the robotic arm segmentation targets in the robotic arm segmentation target set are selected, and the complete robotic arm target is determined based on the complete robotic arm target set.

[0020] Optionally, matching the complete mechanical arm target with the base target to determine the complete construction machine target includes:

[0021] Calculating a second endpoint distance between the endpoint position of the complete robot arm target and the endpoint position of the base target;

[0022] When the second endpoint distance is less than a preset second endpoint distance threshold, the complete robotic arm target and the base target associated with the second endpoint distance are input into a pre-trained instrument authenticity recognition model, and confidence data is output;

[0023] The complete robot arm target associated with the highest confidence data is matched with the base target to determine the complete construction machine target.

[0024] Optionally, the determining the construction status information of the complete construction equipment target and the transmission line includes:

[0025] Based on a preset magnification factor, the base target included in the complete construction machine target is magnified to obtain a base reference target;

[0026] Calculate the intersection-and-union ratio between the complete mechanical arm target included in the complete construction machine target and the base reference target, and determine whether the intersection-and-union ratio calculation result is greater than a preset intersection-and-union ratio threshold value,

[0027] If not, determining that the construction status information is the first construction status;

[0028] If so, the construction status information is determined based on the complete robot arm target, the base target and the transmission line position of the transmission line.

[0029] Optionally, determining the construction status information according to the complete robot arm target, the base target and the transmission line position of the transmission line includes:

[0030] Identifying a positional relationship between the complete robot arm target, the base target, and the transmission line position;

[0031] When the base target is between the complete robot arm target and the transmission line position, determining that the construction state information is a second construction state;

[0032] When the base target is not between the complete robot arm target and the transmission line position, the construction status information is determined to be the third construction status.

[0033] Optionally, determining the shortest distance between the complete construction machine target and the transmission line, and a threshold distance between the complete construction machine target and the transmission line according to the construction status information includes:

[0034] When the construction status information is the first construction status, setting the shortest distance to be greater than the threshold distance;

[0035] When the construction state information is the second construction state, determine the minimum value of the distance between the complete mechanical arm target contained in the complete construction machine target and the transmission line as the shortest distance, and N times the diagonal of the circumscribed rectangle of the complete construction machine target as the threshold distance;

[0036] When the construction status information is the third construction status, determine the connection point between the complete robotic arm target and the base target contained in the complete construction equipment target, take the distance between the connection point and the transmission line as the shortest distance, and take M times the robotic arm length of the complete robotic arm target as the threshold distance.

[0037] Optionally, the determining the minimum value of the distance between the complete mechanical arm target included in the complete construction machine target and the power transmission line as the shortest distance comprises:

[0038] Based on a preset sampling distance, performing equidistant sampling on the robotic arm segmented target included in the complete robotic arm target;

[0039] The vertical distances between all sampling points and the transmission line are calculated, and the minimum value among the calculated results is taken as the shortest distance.

[0040] Optionally, taking M times the length of the robotic arm of the complete robotic arm target as the threshold distance comprises:

[0041] Determining the robot arm segment length of the robot arm segment object contained in the complete robot arm object;

[0042] The segment lengths of the robotic arm are summed to obtain the robotic arm length, and M times the robotic arm length is used as the threshold distance.

[0043] Optionally, comparing the shortest distance with the threshold distance, and pushing the safety warning information according to the comparison result, includes:

[0044] Determine whether the shortest distance is greater than the threshold distance,

[0045] If yes, no security risk information will be reported;

[0046] If not, report the security risk information and issue a security alert through the preset security risk alert device.

[0047] According to a second aspect of an embodiment of the present application, a device for detecting the safety of a power grid line threatened by a construction tool is provided, comprising:

[0048] The recognition module is configured to recognize the segmented target of the robot arm and the base target contained in the collected scene graph based on the rotating target detection algorithm, and locate the transmission line;

[0049] A merging module is configured to identify the type of the robotic arm segmented target and merge the robotic arm segmented targets of the same type that meet the preset merging rules to obtain a complete robotic arm target;

[0050] a state confirmation module configured to match the complete mechanical arm target with the base target, determine a complete construction machine target, and determine construction state information of the complete construction machine target and the transmission line;

[0051] a calculation module configured to determine, based on the construction status information, a shortest distance between the complete construction machine target and the transmission line, and a threshold distance between the complete construction machine target and the transmission line;

[0052] The alarm module is configured to compare the shortest distance with the threshold distance and push safety alarm information according to the comparison result.

[0053] According to a third aspect of an embodiment of the present application, a computing device is provided, including:

[0054] Memory and processor;

[0055] The memory is used to store computer-executable instructions, and the processor implements the steps of the method for detecting the safety of power grid lines threatened by construction equipment when executing the computer-executable instructions.

[0056] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the method for detecting the safety of power grid lines threatening to construction equipment are implemented.

[0057] According to a fifth aspect of the embodiments of the present application, a chip is provided, which stores a computer program. When the computer program is executed by the chip, the steps of the method for detecting the safety of power grid lines threatening by construction equipment are implemented.

[0058] The construction equipment threat to power grid line safety detection method provided by the present application is based on the rotating target detection algorithm, identifies the mechanical arm segmentation target and the base target contained in the collected scene graph, and locates the transmission line; identifies the type of the mechanical arm segmentation target, and merges the mechanical arm segmentation targets of the same type that meet the preset merging rules to obtain the complete mechanical arm target; matches the complete mechanical arm target with the base target, determines the complete construction equipment target, and determines the construction status information of the complete construction equipment target and the transmission line; according to the construction status information, determines the shortest distance between the complete construction equipment target and the transmission line, and the threshold distance between the complete construction equipment target and the transmission line; compares the shortest distance with the threshold distance, and pushes the safety warning information according to the comparison result. It realizes real-time detection of the direct distance between the construction equipment and each transmission line, and can distinguish construction equipment in different positions and different postures, and accurately judges whether the current posture of the construction equipment may threaten the transmission line according to the situation. In the case where the construction equipment may threaten the transmission channel, the power supervisor is warned to send people to intervene in advance, so as to reduce the collision of the construction equipment with the transmission line and damage the power safety from the source. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 This is a flow chart of a method for detecting the safety of power grid lines threatened by construction equipment provided in one embodiment of the present application;

[0061] Figure 2 This is a complete mechanical arm target schematic diagram of a method for detecting the safety of a power grid line threatened by a construction machine provided in an embodiment of the present application;

[0062] Figure 3 It is a schematic diagram of calculating the vertical distance between a sampling point and a transmission line of a method for detecting the safety of a power grid line threatened by a construction machine provided in an embodiment of the present application;

[0063] Figure 4 This is a schematic diagram of the structure of a device for detecting the safety of a power grid line threatened by a construction tool provided in an embodiment of the present application;

[0064] Figure 5 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0065] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0066] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0067] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0068] In the present application, a method for detecting the safety of a power grid line threatened by a construction machine is provided. The present application also relates to a device for detecting the safety of a power grid line threatened by a construction machine, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0069] Figure 1 A flowchart of a method for detecting the safety of a power grid line threatened by a construction tool provided in accordance with an embodiment of the present application is shown, which specifically includes the following steps:

[0070] Step S102: Based on the rotating target detection algorithm, identify the robot arm segmentation target and the base target contained in the collected scene graph, and locate the transmission line;

[0071] Step S104: Identify the type of the robotic arm segmented target, and merge the robotic arm segmented targets of the same type that meet the preset merging rules to obtain a complete robotic arm target;

[0072] Step S106: matching the complete robot arm target with the base target, determining a complete construction equipment target, and determining the construction status information of the complete construction equipment target and the transmission line;

[0073] Step S108: determining the shortest distance between the complete construction machine target and the transmission line, and the threshold distance between the complete construction machine target and the transmission line according to the construction status information;

[0074] Step S110: Compare the shortest distance with the threshold distance, and push safety warning information according to the comparison result.

[0075] Among them, the scene graph is an image collected in a construction scene, which contains any number of robot arm segment targets and base targets. The specific number is determined by the actual usage scenario and is not limited in this embodiment; the type of the robot arm segment target represents the type of construction equipment to which the robot arm segment belongs, such as tower cranes, forklifts, bulldozers, excavators, dump trucks, cranes, cement pump trucks, ship cranes, pile drivers and long-arm excavators, etc. The specific type of construction equipment and its corresponding type of robot arm segment target are determined by the actual usage scenario and are not limited in this embodiment; the complete robot arm target represents the robot arm corresponding to the same construction equipment; the construction status information represents the positional relationship between the complete construction equipment target and the transmission line.

[0076] Based on this, based on the rotating target detection algorithm, target recognition is performed on the collected scene graph. The selected recognition algorithm can be YOLO, ‌R2CNN algorithm, etc. The specific selected recognition algorithm is determined by the actual usage scenario, and this embodiment does not limit it. After identifying the robotic arm segmentation target, base target and transmission line in the scene graph, the type of robotic arm segmentation target is identified, and the robotic arm segmentation targets of the same type that meet the merging rules are merged to obtain a complete robotic arm target. The merging rules characterize the distance between each robotic arm segmentation target, and meet the conditions that can be identified as robotic arm segments on the same robotic arm.

[0077] Afterwards, after determining the complete robotic arm target, match the complete mechanical target and the base target belonging to the same construction equipment to obtain the complete construction equipment target, and determine the construction status information through the positional relationship between the complete construction equipment target and the transmission line; according to the construction conditions of different construction equipment corresponding to different construction status information, determine the shortest distance between the complete construction equipment target and the transmission line, as well as the threshold distance between the complete construction equipment target and the transmission line according to different standards; finally, by comparing the shortest distance with the threshold distance, determine whether the construction equipment in the current scene diagram threatens the power grid line according to the comparison result, and if so, issue a safety alarm to prompt relevant personnel to intervene, thereby reducing the collision of construction equipment with transmission lines and damaging power safety from the source.

[0078] Furthermore, in step S104, the same type of robot arm segment targets that meet the preset merging rule are merged to obtain a complete robot arm target. In this embodiment, the specific implementation method is as follows:

[0079] Based on the same type of the robotic arm segmentation targets, a robotic arm segmentation target set is constructed; one of the robotic arm segmentation targets in the robotic arm segmentation target set is selected as a candidate robotic arm segmentation target; the left and right endpoints of all the robotic arm segmentation targets in the robotic arm segmentation target set are determined, wherein the left and right endpoints include a left endpoint and a right endpoint; the first endpoint distance between one of the left and right endpoints of the candidate robotic arm segment and the other of the left and right endpoints of the other robotic arm segmentation targets in the robotic arm segmentation target set is calculated in sequence; when the first endpoint distance is less than a preset first endpoint distance threshold, the robotic arm segmentation target associated with the first endpoint distance is recorded in the complete robotic arm target set; the step of selecting one of the robotic arm segmentation targets in the robotic arm segmentation target set as a candidate robotic arm segmentation target is executed until all the robotic arm segmentation targets in the robotic arm segmentation target set are selected, and the complete robotic arm target is determined based on the complete robotic arm target set.

[0080] Among them, there is a situation where the same scene graph contains multiple construction machines, then the scene graph also contains robotic arms corresponding to the multiple construction machines. In order to distinguish the robotic arms corresponding to each robotic arm segment target, the robotic arm segment targets belonging to the same complete robotic arm target need to be merged.

[0081] Based on this, the same type of robotic arm segmentation targets are placed in the same robotic arm segmentation target set. For any robotic arm segmentation target set, one of the robotic arm segmentation targets is selected as the candidate robotic arm segmentation target, the left and right endpoints of all the robotic arm segmentation targets in the robotic arm segmentation target set are determined, and the left and right endpoint positions between the candidate robotic arm segmentation target and other robotic arm segmentation targets are compared. Specifically, when the left endpoint of the candidate robotic arm segmentation target is the comparison object, the distance between the right endpoint of other robotic arm segmentation targets and the left endpoint of the candidate robotic arm segmentation target is compared, which is recorded as the first endpoint distance; conversely, the right endpoint of the candidate robotic arm segmentation target can also be selected as the comparison object, and the distance between the left endpoint of other robotic arm segmentation targets and the right endpoint of the candidate robotic arm segmentation target is compared, which is recorded as the first endpoint distance.

[0082] When the first endpoint distance is less than the first endpoint distance threshold, the robotic arm segment target associated with the first endpoint distance is recorded in the complete robotic arm target set, wherein the selection of the first endpoint distance threshold is determined by the type of robotic arm segment target. For example, the robotic arms corresponding to the tower crane and the long-arm excavator have different spacings between each robotic arm segment, and the corresponding first endpoint distance threshold is also selected differently. The specific numerical selection of the first endpoint distance threshold is determined by the actual usage scenario, and this embodiment does not limit it.

[0083] After the first endpoint distances between the current candidate robotic arm segmentation target and other robotic arm segmentation targets in the same robotic arm segmentation target set are all judged, the candidate robotic arm segmentation targets are reselected in the robotic arm segmentation target set until all the robotic arm segmentation targets are selected to obtain the final complete robotic arm target set. All the robotic arm segmentation targets in the complete robotic arm target set together constitute the complete robotic arm target.

[0084] For example, Figure 2 As shown in the schematic diagram of a complete robotic arm target of a construction equipment threat to power grid line safety detection method provided, there are robotic arm segment targets 1, 2, and 3 in the robotic arm segment target set, and one of the robotic arm segment targets is selected in turn, and the distance between the left and right endpoints is determined with the other robotic arm segment targets, and compared with the preset first endpoint distance threshold, it is determined that the robotic arm segment targets 1, 2, and 3 are robotic arm segment targets on the same robotic arm, and a complete robotic arm target including the robotic arm segment targets 1, 2, and 3 is obtained.

[0085] Furthermore, in step S106, the process of matching the complete mechanical arm target with the base target to determine the complete construction equipment target is specifically implemented as follows in this embodiment:

[0086] Calculate the endpoint position of the complete robotic arm target and the second endpoint distance between it and the endpoint position of the base target; when the second endpoint distance is less than a preset second endpoint distance threshold, input the complete robotic arm target and the base target associated with the second endpoint distance into a pre-trained equipment authenticity recognition model, and output confidence data; match the complete robotic arm target and the base target associated with the highest confidence data to determine the complete construction equipment target.

[0087] If a scene graph contains multiple construction machines, then there are multiple complete robot arm targets and multiple base targets corresponding to the multiple construction machines. In order to clearly identify the complete robot arm target and the base target belonging to the same construction machine, it is necessary to match the complete robot arm target with the base target.

[0088] Based on this, we first determine the distance between each complete robotic arm target and each base target, that is, the second endpoint distance, and the judgment process is completed through the endpoint position of the complete robotic arm target and the endpoint position of the base target. Specifically, the complete robotic arm target is regarded as a line segment, and the two edgemost endpoints of the line segment are used as the endpoint positions of the complete robotic arm target. For the endpoint position selection of the base target, the four corners of the circumscribed rectangle of the base target, or the geometric center of the circumscribed rectangle, or through the target recognition algorithm, the robotic arm connection point of the base target is identified as the endpoint position of the base target.

[0089] It should be noted that the selection of the second endpoint distance threshold is correlated with the endpoint position selection standard of the base target. For example, if one of the four corners of the circumscribed rectangle of the base target is selected as the endpoint position of the base target, then the corresponding second endpoint distance threshold will be greater than the second endpoint distance threshold corresponding to selecting the robotic arm connection point of the base target as the endpoint position of the base target.

[0090] After determining the second endpoint distance between all complete robotic arm targets and each base target, the second endpoint distance is compared with the second endpoint distance threshold. When the second endpoint distance is less than the second endpoint distance threshold, the complete robotic arm target and the base target corresponding to the second endpoint distance are input into the equipment authenticity recognition model; the equipment authenticity recognition model will determine the probability that the input robotic arm target and the base target belong to the same construction equipment, and output the recognition result in the form of confidence data.

[0091] A group of complete robotic arm targets and base targets with the highest confidence data are selected as complete construction machinery targets. After removing this group of complete robotic arm targets and base targets, for other complete robotic arm targets and base targets whose second endpoint distance is less than the second endpoint distance threshold, repeat the matching through the equipment authenticity recognition model to obtain new complete construction machinery targets. The corresponding complete robotic arm targets and base targets are removed again, and the above-mentioned cyclic matching process is repeated until all complete robotic arm targets and base targets whose second endpoint distance is less than the second endpoint distance threshold are matched.

[0092] Furthermore, in step S106, the process of determining the complete construction equipment target and the construction status information of the transmission line is specifically implemented as follows in this embodiment:

[0093] Based on a preset magnification factor, the base target included in the complete construction equipment target is magnified to obtain a base reference target; the intersection and union ratio between the complete robotic arm target included in the complete construction equipment target and the base reference target is calculated to determine whether the intersection and union ratio calculation result is greater than a preset intersection and union ratio threshold; if not, the construction status information is determined to be the first construction status; if so, the construction status information is determined based on the complete robotic arm target, the base target and the transmission line position of the transmission line.

[0094] Among them, for any complete construction machine target, the base target included therein is enlarged to obtain the base reference target. The selection of the magnification factor is determined by the actual use scenario, and this embodiment does not limit it. The selection range of the magnification factor in the actual measurement is generally 5%-30%. The intersection-and-union ratio between the base reference target and the complete robotic arm target in the same complete construction machine target is calculated. The calculation process of the intersection-and-union ratio can be achieved by the area intersection-and-union ratio between the circumscribed rectangle of the base reference target and the circumscribed rectangle of the complete robotic arm target, or by the perimeter intersection-and-union ratio between the circumscribed rectangle of the base reference target and the circumscribed rectangle of the complete robotic arm target.

[0095] Based on this, for any complete construction equipment target, the relationship between the corresponding intersection and union ratio calculation result and the intersection and union ratio threshold is judged. When the intersection and union ratio calculation result is less than or equal to the intersection and union ratio threshold, the construction status information is determined to be the first construction status; when the intersection and union ratio calculation result is greater than the intersection and union ratio threshold, it is necessary to perform a secondary judgment on the construction status information based on the relationship between the complete mechanical arm target, the base target, and the transmission line position of the transmission line of the complete construction equipment target.

[0096] It should be noted that the selection of the intersection-and-merge ratio threshold is determined by the type of the robotic arm segment target in the complete construction equipment target. For example, the tower crane and the long-arm excavator have different corresponding intersection-and-merge ratio thresholds. The specific numerical selection of the intersection-and-merge ratio threshold is determined by the actual usage scenario, and this embodiment does not limit it. The first construction state indicates that the corresponding complete equipment is in a folded and flat shutdown state, rather than in a startup working state.

[0097] Furthermore, when the above intersection-and-merge ratio calculation result is greater than a preset intersection-and-merge ratio threshold, the process of determining the construction status information according to the complete robot arm target, the base target and the transmission line position of the transmission line is specifically implemented as follows in this embodiment:

[0098] Identify the positional relationship between the complete robotic arm target, the base target and the transmission line position; when the base target is between the complete robotic arm target and the transmission line position, determine that the construction status information is the second construction status; when the base target is not between the complete robotic arm target and the transmission line position, determine that the construction status information is the third construction status.

[0099] Among them, the second construction state represents that the construction machinery is in a working state with its back to the transmission line during the construction process; the third construction state has two situations, one is that the transmission line is between the complete robotic arm target and the base target, representing that the construction machinery is below the transmission line during the construction process; the other is that the complete robotic arm target is between the base target and the transmission line, representing that the construction machinery is in a working state facing the transmission line during the construction process.

[0100] Based on this, for any complete construction equipment target that is not in the first working state, the positional relationship between the complete robotic arm target, the base target, and the transmission line it contains determines that its corresponding construction status information is the second construction state or the third construction state.

[0101] Furthermore, in step S108, the shortest distance between the complete construction equipment target and the transmission line, and the threshold distance between the complete construction equipment target and the transmission line are determined according to the construction status information. In this embodiment, the specific implementation method is as follows:

[0102] When the construction status information is the first construction status, the shortest distance is set to be greater than the threshold distance; when the construction status information is the second construction status, the minimum value of the distance between the complete robotic arm target contained in the complete construction equipment target and the transmission line is determined as the shortest distance, and N times the diagonal of the circumscribed rectangle of the complete construction equipment target is used as the threshold distance; when the construction status information is the third construction status, the connection point between the complete robotic arm target contained in the complete construction equipment target and the base target is determined, and the distance between the connection point and the transmission line is used as the shortest distance, and M times the length of the robotic arm of the complete robotic arm target is used as the threshold distance.

[0103] Among them, for the complete construction machinery target whose construction status is the first construction status, since it is in a non-working shutdown state, it can be determined at this time that it will not pose a threat to the safety of the power grid line, so the shortest distance is set to be greater than the threshold distance; regarding the selection of the values ​​of N and M, it is determined by the type of the robotic arm segment target in the complete construction machinery target. For example, the N and M values ​​corresponding to the tower crane and the long-arm excavator are different. The specific selection of the values ​​of N and M is determined by the actual usage scenario, and this embodiment does not limit it.

[0104] Based on this, for the complete construction machinery target in the second construction state, the minimum distance between the complete robotic arm target and the transmission line is taken as the shortest distance, and the diagonal of the circumscribed rectangle of the complete construction machinery target is scaled N times as the threshold distance; for the complete construction machinery target in the third construction state, the connection point between the complete robotic arm target and the base target contained therein is determined, and the distance between the connection point and the transmission line is taken as the shortest distance, and the robotic arm length of the complete robotic arm target is magnified M times as the threshold distance, wherein a vertical line segment is made through the connection point to all the transmission lines, and the length of the shortest vertical line segment is the distance between the connection point and the transmission line.

[0105] Furthermore, the above process of determining the minimum value of the distance between the complete mechanical arm target included in the complete construction equipment target and the transmission line as the shortest distance is specifically implemented as follows in this embodiment:

[0106] Based on a preset sampling distance, equidistant sampling is performed on the robotic arm segmented target included in the complete robotic arm target; the vertical distances between all sampling points and the transmission line are calculated, and the minimum value in the calculation results is taken as the shortest distance.

[0107] Among them, Figure 3 A schematic diagram of calculating the vertical distance between the sampling point and the transmission line of a construction equipment threat to the power grid line safety detection method is provided. For any mechanical arm segmented target contained in a complete mechanical arm target, the mechanical arm segmented target is sampled at equal distances. The interval distance of each sampling point is a preset sampling distance. The selection of the sampling distance is determined by the actual usage scenario and is not limited in this embodiment. The vertical distance between each sampling point and each transmission line is calculated in turn, and the minimum value h of the vertical distance is taken as min as the shortest distance.

[0108] Furthermore, in the above process of taking M times the length of the robot arm of the complete robot arm target as the threshold distance, in this embodiment, the specific implementation method is as follows:

[0109] Determine the robotic arm segment length of the robotic arm segment target contained in the complete robotic arm target; sum the robotic arm segment lengths to obtain the robotic arm length, and use M times the robotic arm length as the threshold distance.

[0110] Among them, for any complete robotic arm target, the corresponding robotic arm length is the robotic arm segment length box of all robotic arm segment targets contained in the complete robotic arm target.

[0111] Furthermore, in step S110, the shortest distance is compared with the threshold distance, and the safety warning information is pushed according to the comparison result. In this embodiment, the specific implementation method is as follows:

[0112] Determine whether the shortest distance is greater than the threshold distance. If so, do not report the security risk information; if not, report the security risk information and issue a security alert through a preset security risk alert device.

[0113] Among them, for any complete construction equipment target, if it is in the first construction state, then the associated shortest distance is greater than the threshold distance, there is no safety risk, and no safety risk information is reported; if it is in the second construction state and the third construction state, and the associated shortest distance is greater than the threshold distance, there is no safety risk, and no safety risk information is reported; if it is in the second construction state and the third construction state, and the associated shortest distance is less than or equal to the threshold distance, there is a safety risk, and the safety risk information needs to be reported. The safety risk information includes the physical position corresponding to the scene graph, the position information of the complete construction equipment target with safety risks, the position information of the base target and the complete mechanical arm target included in the complete construction equipment target, and the threat posed by the complete construction equipment to the safety of the power grid line, and the corresponding transmission line position information. The safety risk warning device is composed of one or more of warning lights, speakers, displays and other devices. The specific composition is determined by the actual use scenario, and this embodiment does not limit it.

[0114] Corresponding to the above method embodiment, the present application also provides an embodiment of a device for detecting the safety of a power grid line threatened by a construction machine, Figure 4 The schematic diagram of the structure of a device for detecting the safety of power grid lines threatened by construction equipment provided in one embodiment of the present application is shown. Figure 4 As shown, the device comprises:

[0115] The identification module 402 is configured to identify the robot arm segmentation target and the base target contained in the collected scene graph based on the rotating target detection algorithm, and locate the transmission line;

[0116] A merging module 404 is configured to identify the type of the robotic arm segmented target and merge the robotic arm segmented targets of the same type that meet a preset merging rule to obtain a complete robotic arm target;

[0117] The state confirmation module 406 is configured to match the complete mechanical arm target with the base target, determine the complete construction machine target, and determine the construction state information of the complete construction machine target and the transmission line;

[0118] A calculation module 408 is configured to determine the shortest distance between the complete construction machine target and the transmission line and a threshold distance between the complete construction machine target and the transmission line according to the construction status information;

[0119] The warning module 410 is configured to compare the shortest distance with the threshold distance, and push safety warning information according to the comparison result.

[0120] In an optional embodiment, the merging module 404 is further configured to:

[0121] Based on the same type of the robotic arm segmentation targets, a robotic arm segmentation target set is constructed; one of the robotic arm segmentation targets in the robotic arm segmentation target set is selected as a candidate robotic arm segmentation target; the left and right endpoints of all the robotic arm segmentation targets in the robotic arm segmentation target set are determined, wherein the left and right endpoints include a left endpoint and a right endpoint; the first endpoint distance between one of the left and right endpoints of the candidate robotic arm segment and the other of the left and right endpoints of the other robotic arm segmentation targets in the robotic arm segmentation target set is calculated in sequence; when the first endpoint distance is less than a preset first endpoint distance threshold, the robotic arm segmentation target associated with the first endpoint distance is recorded in the complete robotic arm target set; the step of selecting one of the robotic arm segmentation targets in the robotic arm segmentation target set as a candidate robotic arm segmentation target is executed until all the robotic arm segmentation targets in the robotic arm segmentation target set are selected, and the complete robotic arm target is determined based on the complete robotic arm target set.

[0122] In an optional embodiment, the status confirmation module 406 is further configured to:

[0123] Calculate the endpoint position of the complete robotic arm target and the second endpoint distance between it and the endpoint position of the base target; when the second endpoint distance is less than a preset second endpoint distance threshold, input the complete robotic arm target and the base target associated with the second endpoint distance into a pre-trained equipment authenticity recognition model, and output confidence data; match the complete robotic arm target and the base target associated with the highest confidence data to determine the complete construction equipment target.

[0124] In an optional embodiment, the status confirmation module 406 is further configured to:

[0125] Based on a preset magnification factor, the base target included in the complete construction equipment target is magnified to obtain a base reference target; the intersection and union ratio between the complete robotic arm target included in the complete construction equipment target and the base reference target is calculated to determine whether the intersection and union ratio calculation result is greater than a preset intersection and union ratio threshold; if not, the construction status information is determined to be the first construction status; if so, the construction status information is determined based on the complete robotic arm target, the base target and the transmission line position of the transmission line.

[0126] In an optional embodiment, the status confirmation module 406 is further configured to:

[0127] Identify the positional relationship between the complete robotic arm target, the base target and the transmission line position; when the base target is between the complete robotic arm target and the transmission line position, determine that the construction status information is the second construction status; when the base target is not between the complete robotic arm target and the transmission line position, determine that the construction status information is the third construction status.

[0128] In an optional embodiment, the calculation module 408 is further configured to:

[0129] When the construction status information is the first construction status, the shortest distance is set to be greater than the threshold distance; when the construction status information is the second construction status, the minimum value of the distance between the complete robotic arm target contained in the complete construction equipment target and the transmission line is determined as the shortest distance, and N times the diagonal of the circumscribed rectangle of the complete construction equipment target is used as the threshold distance; when the construction status information is the third construction status, the connection point between the complete robotic arm target contained in the complete construction equipment target and the base target is determined, and the distance between the connection point and the transmission line is used as the shortest distance, and M times the length of the robotic arm of the complete robotic arm target is used as the threshold distance.

[0130] In an optional embodiment, the calculation module 408 is further configured to:

[0131] Based on a preset sampling distance, equidistant sampling is performed on the robotic arm segmented target included in the complete robotic arm target; the vertical distances between all sampling points and the transmission line are calculated, and the minimum value in the calculation results is taken as the shortest distance.

[0132] In an optional embodiment, the calculation module 408 is further configured to:

[0133] Determine the robotic arm segment length of the robotic arm segment target contained in the complete robotic arm target; sum the robotic arm segment lengths to obtain the robotic arm length, and use M times the robotic arm length as the threshold distance.

[0134] In an optional embodiment, the alarm module 410 is further configured to:

[0135] Determine whether the shortest distance is greater than the threshold distance. If so, do not report the security risk information; if not, report the security risk information and issue a security alert through a preset security risk alert device.

[0136] The construction equipment threat to power grid line safety detection device provided by the present application, based on the rotating target detection algorithm, identifies the mechanical arm segmentation target and the base target contained in the collected scene graph, and locates the transmission line; identifies the type of the mechanical arm segmentation target, and merges the mechanical arm segmentation targets of the same type that meet the preset merging rules to obtain the complete mechanical arm target; matches the complete mechanical arm target with the base target, determines the complete construction equipment target, and determines the construction status information of the complete construction equipment target and the transmission line; according to the construction status information, determines the shortest distance between the complete construction equipment target and the transmission line, and the threshold distance between the complete construction equipment target and the transmission line; compares the shortest distance with the threshold distance, and pushes the safety warning information according to the comparison result. It realizes real-time detection of the direct distance between the construction equipment and each transmission line, and can distinguish construction equipment in different positions and different postures, and accurately judges whether the current posture of the construction equipment may threaten the transmission line according to the situation. In the case where the construction equipment may threaten the transmission channel, the power supervisor is warned to send people to intervene in advance, so as to reduce the collision of the construction equipment with the transmission line and damage the power safety from the source.

[0137] The above is a schematic scheme of a construction equipment threat to power grid line safety detection device of this embodiment. It should be noted that the technical scheme of the construction equipment threat to power grid line safety detection device and the technical scheme of the above-mentioned construction equipment threat to power grid line safety detection method belong to the same concept. For details not described in detail in the technical scheme of the construction equipment threat to power grid line safety detection device, please refer to the description of the technical scheme of the above-mentioned construction equipment threat to power grid line safety detection method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0138] Figure 5 The block diagram of a computing device 500 provided according to an embodiment of the present application is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0139] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0140] In one embodiment of the present application, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0141] Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 500 may also be a mobile or stationary server.

[0142] The processor 520 is used to execute computer executable instructions for each step of the method for detecting the safety of power grid lines posed by construction equipment.

[0143] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned construction equipment threat to power grid line safety detection method are of the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the above-mentioned construction equipment threat to power grid line safety detection method.

[0144] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to execute the steps of the method for detecting the safety of power grid lines threatening to construction equipment.

[0145] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned construction equipment threat to power grid line safety detection method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned construction equipment threat to power grid line safety detection method.

[0146] An embodiment of the present application further provides a chip storing a computer program, which, when executed by the chip, implements the steps of the method for detecting the safety of power grid lines threatening to construction equipment.

[0147] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0149] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0150] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0151] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A method for detecting threats to power grid line safety from construction equipment, characterized in that, include: Based on the rotating target detection algorithm, the robot arm segment target and the base target contained in the collected scene map are identified, and the power transmission line is located. Identify the type of the robotic arm segment target, and merge the robotic arm segment targets of the same type that meet the preset merging rules to obtain the complete robotic arm target; Match the complete robotic arm target with the base target to determine the complete construction equipment target, and determine the construction status information of the complete construction equipment target and the power transmission line. Specifically, based on a preset magnification factor, magnify the base target included in the complete construction equipment target to obtain a base reference target. Calculate the intersection-over-union ratio (IoU) between the complete robotic arm target included in the complete construction equipment target and the base reference target. Determine whether the IoU calculation result is greater than a preset IoU threshold. If not, determine the construction status information as a first construction status. If yes, identify the positional relationship between the complete robotic arm target, the base target, and the power transmission line position. When the base target is between the complete robotic arm target and the power transmission line position, determine the construction status information as a second construction status. When the base target is not between the complete robotic arm target and the power transmission line position, determine the construction status information as a third construction status. Based on the construction status information, the shortest distance between the complete construction equipment target and the power transmission line, and a threshold distance between the complete construction equipment target and the power transmission line are determined. Specifically, when the construction status information is the first construction status, the shortest distance is set to be greater than the threshold distance. When the construction status information is the second construction status, the minimum distance between the complete robotic arm target included in the complete construction equipment target and the power transmission line is determined as the shortest distance, and N times the diagonal of the circumscribed rectangle of the complete construction equipment target is determined as the threshold distance. When the construction status information is the third construction status, the connection point between the complete robotic arm target included in the complete construction equipment target and the base target is determined, the distance between the connection point and the power transmission line is determined as the shortest distance, and M times the length of the robotic arm target is determined as the threshold distance. The shortest distance is compared with the threshold distance, and a security alarm is pushed based on the comparison result.

2. The method according to claim 1, characterized in that, The process of merging the same type of robotic arm segments that conform to preset merging rules to obtain a complete robotic arm target includes: Based on the same type of robotic arm segmentation targets, construct a set of robotic arm segmentation targets; Select one of the robotic arm segmentation targets in the set of robotic arm segmentation targets as a candidate robotic arm segmentation target; Determine the left and right endpoints of all the robotic arm segment targets in the set of robotic arm segment targets, wherein the left and right endpoints include the left endpoint and the right endpoint; Calculate the first endpoint distance between one of the left and right endpoints of the candidate robotic arm segment and the other of the left and right endpoints of the other robotic arm segment targets in the set of robotic arm segment targets; When the distance between the first endpoints is less than a preset threshold for the distance between the first endpoints, the robotic arm segment targets associated with the distance between the first endpoints are recorded in the complete robotic arm target set; The step of selecting one of the robotic arm segment targets in the set of robotic arm segment targets as a candidate robotic arm segment target is performed until all the robotic arm segment targets in the set of robotic arm segment targets are selected. Based on the obtained complete set of robotic arm targets, the complete robotic arm target is determined.

3. The method according to claim 1, characterized in that, The process of matching the complete robotic arm target with the base target to determine the complete construction equipment target includes: Calculate the second endpoint distance between the endpoint position of the complete robotic arm target and the endpoint position of the base target; When the distance between the second endpoints is less than a preset threshold for the distance between the second endpoints, the complete robotic arm target associated with the distance between the second endpoints and the base target are input into a pre-trained instrument authenticity recognition model, and confidence data is output. The complete robotic arm target associated with the highest confidence level data is matched with the base target to determine the complete construction equipment target.

4. The method according to claim 1, characterized in that, The step of determining the minimum distance between the complete robotic arm target (included in the complete construction equipment target) and the power transmission line as the shortest distance includes: Based on a preset sampling distance, equidistant sampling is performed on the segmented targets of the robotic arm included in the complete robotic arm target; Calculate the vertical distance between all sampling points and the transmission line, and take the minimum value in the calculation results as the shortest distance.

5. The method according to claim 1, characterized in that, The step of using M times the length of the complete robotic arm target as the threshold distance includes: Determine the length of the robotic arm segment of the robotic arm segment target included in the complete robotic arm target; The length of the robotic arm is obtained by summing the lengths of the segments, and M times the length of the robotic arm is taken as the threshold distance.

6. The method according to claim 1, characterized in that, The step of comparing the shortest distance with the threshold distance and pushing security alarm information based on the comparison result includes: Determine whether the shortest distance is greater than the threshold distance. If so, do not report security risk information; If not, report the security risk information and issue a security alert through the preset security risk alarm device.

7. A device for detecting threats to power grid line safety from construction machinery, characterized in that, include: The recognition module is configured to identify the robotic arm segment targets and base targets contained in the acquired scene map based on a rotating target detection algorithm, and to locate the power transmission line. The merging module is configured to identify the type of the robotic arm segment target and merge the robotic arm segment targets of the same type that conform to a preset merging rule to obtain a complete robotic arm target. The status confirmation module is configured to match the complete robotic arm target with the base target, determine the complete construction equipment target, and determine the construction status information of the complete construction equipment target and the power transmission line. Specifically, based on a preset magnification factor, the base target included in the complete construction equipment target is magnified to obtain a base reference target. The cross-intersection over union (CUI) ratio between the complete robotic arm target included in the complete construction equipment target and the base reference target is calculated. It is determined whether the CUI calculation result is greater than a preset CUI threshold. If not, the construction status information is determined to be a first construction status. If yes, the positional relationship between the complete robotic arm target, the base target, and the power transmission line is identified. When the base target is between the complete robotic arm target and the power transmission line, the construction status information is determined to be a second construction status. When the base target is not between the complete robotic arm target and the power transmission line, the construction status information is determined to be a third construction status. The calculation module is configured to determine, based on the construction status information, the shortest distance between the complete construction machinery target and the power transmission line, and a threshold distance between the complete construction machinery target and the power transmission line. Specifically, when the construction status information is the first construction status, the shortest distance is set to be greater than the threshold distance. When the construction status information is the second construction status, the minimum distance between the complete robotic arm target included in the complete construction machinery target and the power transmission line is determined as the shortest distance, and N times the diagonal of the circumscribed rectangle of the complete construction machinery target is determined as the threshold distance. When the construction status information is the third construction status, the connection point between the complete robotic arm target included in the complete construction machinery target and the base target is determined, the distance between the connection point and the power transmission line is determined as the shortest distance, and M times the length of the robotic arm target is determined as the threshold distance. The alarm module is configured to compare the shortest distance with the threshold distance and push security alarm information based on the comparison result.

Images