Early warning device and method for preventing external damage of line through cooperation of trinocular vision and 4D radar

Abstract

The invention provides a trinocular vision and 4D radar cooperative line external damage prevention early warning method, which belongs to the technical field of external damage early warning, and specifically comprises the following steps: according to mobile monitoring position data and in combination with the distance interval condition between different mobile monitoring positions and other mobile monitoring positions, determining the distance between the mobile monitoring positions and other mobile monitoring positions; the method comprises the following steps: determining a movement control strategy in a comparison processing process of a multi-view vision camera device and a 4D radar which are used for movably monitoring a position, and when it is determined that an early warning strategy needs to be updated based on comparison processing data, combining different camera devices in pairs to form a movement monitoring combination; according to comparison data of different mobile monitoring combinations in different angle intervals in the object moving process, a line external damage prevention early warning method with trinocular vision and 4D radar cooperation in different positions is determined, and the reliability of early warning processing is improved.

Description

Technical Field

[0001] This invention belongs to the field of external damage early warning technology, and in particular relates to a line protection external damage early warning device and method that combines tri-vision vision and 4D radar. Background Technology

[0002] During construction near power transmission lines, violations by construction machinery frequently lead to external damage. To achieve timely early warning of such risks, existing technologies often install cameras within the transmission lines to identify these risks and issue timely warnings, ensuring the operational stability of the transmission lines. However, this approach has the following drawbacks: When conducting external damage prevention early warning, the monitoring range of radar has certain limitations. Therefore, it is necessary to use camera devices within a certain distance range for external damage prevention early warning processing. This makes it an urgent technical problem to solve how to determine the comparison and identification scheme between radar and camera device movement distance, and to determine the early warning strategy based on the comparison and identification results, so as to improve the reliability of external damage early warning identification processing.

[0003] To address the aforementioned technical issues, this application provides a line protection early warning device and method that combines tri-vision vision with 4D radar. Summary of the Invention

[0004] To achieve the objectives of this invention, the following technical solution is adopted: Specifically, this application provides a method for early warning of external damage to power lines that combines trifocal vision with 4D radar, including: S1 uses the monitoring data of 4D radar in the transmission line to determine the historical recognition data of external objects of 4D radar in different angle ranges, and uses the historical recognition data in different locations to determine the movement monitoring position of the multi-view camera device in the transmission line. S2 determines the motion control strategy of the multi-view camera device and 4D radar at the motion monitoring location during the comparison and processing process, based on the motion monitoring location data and the distance interval between different motion monitoring locations and other motion monitoring locations. When S3 determines that an update to the early warning strategy is needed based on the comparison processing data, it combines different camera devices in pairs to form a motion monitoring combination. Based on the comparison data of different motion monitoring combinations in different angle ranges during the movement of the object, it determines the line protection early warning method of tri-vision and 4D radar coordination in different positions.

[0005] The beneficial effects of this invention are as follows: Based on motion monitoring location data and the distance intervals between different motion monitoring locations and other motion monitoring locations, the motion control strategy for the comparison and processing of multi-view vision cameras and 4D radar at the motion monitoring locations is determined. This takes into account both the differences in the number of motion monitoring locations, which leads to differences in the reliability of the comparison and processing of the multi-view vision cameras and 4D radar, and the differences in the recognition reliability requirements of the motion monitoring locations, which lead to differences in the distance intervals between the motion monitoring locations and other motion monitoring locations. This allows for the determination of the motion control strategy based on recognition reliability requirements and comparison and processing reliability, thereby ensuring the reliability of the comparison and recognition results in different angle ranges.

[0006] Based on the comparison data of different motion monitoring combinations in different angle ranges during the movement of an object, a method for early warning of external damage to railway lines in coordination with tri-vision and 4D radar at different locations is determined. This method not only considers the comparison deviation at the current location, but also the comparison deviation of different motion monitoring combinations in different angle ranges. It realizes the determination of an early warning strategy for external damage within the distance range where the monitoring accuracy of 4D radar does not meet the requirements, based on the comparison deviation of the location itself and the common problems of the comparison deviation of the motion monitoring combinations. This further improves the reliability of early warning processing and the efficiency of data processing.

[0007] Furthermore, the monitoring data of the 4D radar is determined based on the number of times the 4D radar identifies external objects in different locations.

[0008] Furthermore, the angle interval is divided into a preset number of angle intervals at equal intervals. In one possible embodiment, the preset number is 10.

[0009] Furthermore, the historical identification data includes the number of times the external object was identified during its movement.

[0010] Furthermore, the method for determining the motion monitoring position of the multi-view camera device is as follows: Based on the historical recognition data, the number of times the 4D radar recognizes external objects during their movement is determined within different angle ranges; Based on the number of recognitions, determine the recognition matching angle interval within the angle interval; Based on the identification and matching angle interval data in the location, determine whether the location is the motion monitoring location of the multi-view camera device.

[0011] Furthermore, the method for determining the line damage prevention and early warning method that combines trifocal vision and 4D radar is as follows: By comparing data from different motion monitoring combinations within different angle ranges during the movement of an object, the number of comparison processing steps where the movement distance of the motion monitoring combination and the 4D radar are inconsistent within different angle ranges during the movement of the object is determined, and these steps are taken as the number of comparison deviations. The probability of monitoring deviation of the mobile monitoring combination in different angle ranges is determined based on the number of comparison deviations in different angle ranges. By using different mobile monitoring combinations to determine the monitoring deviation probability in different angle ranges, a method for early warning of line damage prevention in the location using tri-vision and 4D radar in coordination is established.

[0012] Secondly, the present invention provides a line damage prevention early warning device that combines tri-vision and 4D radar, employing the aforementioned tri-vision and 4D radar combined line damage prevention early warning method, specifically including: Monitoring location identification module, movement strategy control module, and early warning module; The monitoring location identification module is responsible for determining the movement monitoring location of the multi-view camera device in the transmission line; The motion strategy control module is responsible for determining the motion control strategy during the comparison and processing of multi-view vision cameras and 4D radar at the motion monitoring location. The early warning module is responsible for determining the early warning method for line damage prevention through the collaboration of tri-vision and 4D radar in different locations.

[0013] Other features and advantages will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0014] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0015] The above and other features and advantages of the present invention will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0016] Figure 1 This is a flowchart of a line protection and early warning method that combines trifocal vision with 4D radar; Figure 2 This is a flowchart of a method for determining the position of a motion monitoring device using a multi-view camera. Figure 3 This is a flowchart of a method for determining the need for updating early warning strategies based on comparative data processing. Detailed Implementation

[0017] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore their detailed description will be omitted.

[0018] The terms “a,” “one,” “the,” and “the” are used to indicate the existence of one or more elements / components / etc.; the terms “including” and “having” are used to indicate an open-ended meaning of inclusion and that other elements / components / etc. may exist in addition to the listed elements / components / etc.

[0019] Example 1 To solve the above problems, according to one aspect of the present invention, such as Figure 1 As shown, a method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar is provided, specifically including: S1 uses the monitoring data of 4D radar in the transmission line to determine the historical recognition data of external objects of 4D radar in different angle ranges, and uses the historical recognition data in different locations to determine the movement monitoring position of the multi-view camera device in the transmission line. S2 determines the motion control strategy of the multi-view camera device and 4D radar at the motion monitoring location during the comparison and processing process, based on the motion monitoring location data and the distance interval between different motion monitoring locations and other motion monitoring locations. When S3 determines that an update to the early warning strategy is needed based on the comparison processing data, it combines different camera devices in pairs to form a motion monitoring combination. Based on the comparison data of different motion monitoring combinations in different angle ranges during the movement of the object, it determines the line protection early warning method of tri-vision and 4D radar coordination in different positions.

[0020] Furthermore, the monitoring data of the 4D radar is determined based on the number of times the 4D radar identifies external objects in different locations.

[0021] Furthermore, the angle interval is divided into a preset number of angle intervals at equal intervals. In one possible embodiment, the preset number is 10.

[0022] Furthermore, the historical identification data includes the number of times the external object was identified during its movement.

[0023] Specifically, such as Figure 2As shown, the method for determining the motion monitoring position of the multi-view camera device is as follows: In this embodiment, the locations most requiring focused monitoring by the multi-view vision system are identified. This allows the system to adjust its angle during monitoring, enabling faster identification of moving objects and improving the timeliness and efficiency of comparison processing with the 4D radar. Specifically, data collected over a long period by the 4D radar (historical identification data) is analyzed to determine the locations and directions where moving objects are most likely to appear. These "hotspot" locations and angles are then designated as key monitoring areas for the multi-view vision cameras.

[0024] S11 uses the historical recognition data to determine the number of times the 4D radar recognizes external objects during movement in different angle ranges; Specifically, the process involves counting the number of times objects are identified within each angle interval. Input: Historical recognition data (recording information about external objects identified by the 4D radar at different locations and times). Processing: Divide the 360-degree (or specific field of view) space into N equally spaced angle intervals (e.g., if N=10, each interval is 36 degrees). For each location, count the total number of times all moving objects are identified within each angle interval.

[0025] Output: A data table in the format: [position, number of recognitions for angle interval 1, number of recognitions for angle interval 2, ...].

[0026] S12 determines the matching angle interval in the angle interval based on the number of recognitions; In the above steps, the matching angle intervals are determined, and the number of recognitions for each angle interval is compared with a preset recognition threshold. All angle intervals with a recognition count exceeding the threshold are marked as "matching angle intervals". These are the "hotspot directions" where objects frequently appear.

[0027] S13 determines whether the location is a motion monitoring location of a multi-view camera device based on the identification matching angle interval data in the location.

[0028] It should be noted that the identification matching angle range is the angle range in which the number of identifications is greater than a preset identification number threshold.

[0029] It is understood that, based on the identification and matching angle interval data in the location, determining whether the location is a motion monitoring location of the multi-view camera device specifically includes: S131 determines the total number of times the 4D radar identifies external objects in different angle ranges during their movement by summing the number of times the 4D radar identifies them at the location, and determines whether the total number of identifications is greater than a preset threshold. If yes, the location is determined to be the movement monitoring location of the multi-view camera device; otherwise, proceed to the next step. In the above steps, the first level of screening - overall activity level - is performed. The calculation is as follows: the number of recognitions for all angle intervals of a location is added together to obtain the total number of recognitions.

[0030] Judgment: If the total number of identifications is greater than the preset threshold (e.g., 100 times in the past week), it indicates that this location is a very active area overall, and it is directly identified as a mobile monitoring location to quickly lock the global high-traffic area.

[0031] S132 determines whether there is a matching angle range in the position. If yes, proceed to the next step. If no, determine that the position does not belong to the motion monitoring position of the multi-view camera device. In the above steps, a second level of screening is performed - to check if there is a hotspot direction. If the overall activity level does not meet the standard, check if there is at least one identification matching angle interval at that location (i.e., the "hotspot direction" determined in S12).

[0032] If no activity is prominent in any direction, then this location may not be worth focusing on. If not, then it can be ruled out.

[0033] S133 determines whether the location is a motion monitoring location of a multi-view camera device based on the number of matching angle intervals in the location.

[0034] In the above steps, the third level of filtering is performed - the concentration of hotspot directions. If a hotspot direction exists, the number of matching angle intervals is further calculated. If the number of matching angle intervals is greater than the preset threshold for the number of angle intervals (e.g., 3), then the location is finally determined as a motion monitoring location.

[0035] This avoids misjudgment due to occasional high traffic in a single direction. Only when multiple adjacent or dispersed directions show high activity is the location considered to have strategic monitoring value. Therefore, mobile monitoring is conducted at these locations, and the location is periodically adjusted at different angles before identifying external objects, thereby improving the efficiency of external object identification.

[0036] It is understood that when the number of matching angle intervals in the location is greater than a preset threshold for the number of angle intervals, the location is determined to be the motion monitoring location of the multi-view camera device.

[0037] It should be noted that if the location is a motion monitoring location, the angles of different cameras in the tri-vision system are adjusted according to a preset time period, so as to detect moving external objects as soon as possible, laying the foundation for the reliability of further identification and early warning processing for preventing external damage.

[0038] Specifically, the method for determining the motion control strategy during the motion monitoring location comparison process is as follows: In this application, the reliability of the comparison between the camera device and the 4D radar is evaluated based on the number of mobile monitoring locations, and further evaluation is conducted based on the evaluation analysis results and the monitoring reliability requirements of different mobile monitoring locations.

[0039] S21 determines the number of mobile monitoring locations in the transmission line based on the mobile monitoring location data; S22 determines the interval distance between different mobile monitoring locations and the nearest mobile monitoring location based on the distance interval between different mobile monitoring locations and other mobile monitoring locations; S23 determines the motion control strategy of the motion monitoring location during the comparison process based on the number of motion monitoring locations and the interval distance between different motion monitoring locations and the nearest motion monitoring location.

[0040] It is understood that, based on the number of motion monitoring locations and the interval distance between different motion monitoring locations and the nearest motion monitoring location, the motion control strategy for the motion monitoring location during the comparison process is determined, specifically including: In the above steps, the system is initialized and the monitoring location is determined. Assuming a 100-kilometer-long power transmission line, 20 sets of "4D radar + tri-lens vision" integrated monitoring devices (i.e. 20 potential monitoring points) are installed along the line.

[0041] During the first week of system operation, by analyzing the historical identification data of the 4D radar, six locations were automatically identified as "movement monitoring locations" that required special attention. Locations A, B, and C: located on sections of the road where the line crosses highways, with frequent vehicle traffic and extremely high total number of identifications. Location D: located near a large construction site, with frequent crane and excavator activity. Locations E and F: located in remote mountainous areas, but radar data showed sporadic but continuous movement signals (which could be patrol personnel or illegal intrusions).

[0042] S231 obtains the number of the motion monitoring locations, determines whether the number of the motion monitoring locations is greater than the preset monitoring location number threshold, if so, determines that the motion control strategy of the motion monitoring location in the comparison process is that there is no need to adjust the angle of the three-view camera device, only to use the initial angle to determine whether the movement distance in the camera device is consistent with the movement distance of the 4D radar, if not, proceed to the next step. In the above steps, the total number of moving monitoring locations is 6, which is less than the threshold of 8. Therefore, the condition of "no adjustment is required" is not met. Since the number of moving monitoring locations is small, the monitoring reliability of the three-lens vision camera device cannot be effectively verified. Therefore, it is necessary to proceed to the next step.

[0043] S232 determines the proportion of the number of mobile monitoring locations in the power transmission line, the locations equipped with 4D radar and tri-vision based on the number of mobile monitoring locations, and uses this as the monitoring quantity proportion. It then determines whether the monitoring quantity proportion is less than a preset monitoring quantity proportion threshold. If so, it determines that the movement control strategy for the mobile monitoring location in the comparison process requires adjustment of the angle of the tri-vision camera device. At different angles, it determines whether the movement distance in the camera device is consistent with the movement distance of the 4D radar. If not, it proceeds to the next step. In the above steps, a density (quantity ratio) judgment is performed. The monitored quantity ratio = 6 / 20 = 30%, which is less than the threshold of 40%. This means that the monitoring points are sparsely distributed, meeting the condition of "needing angle adjustment". Theoretically, the system can then decide to enable a multi-angle adjustment strategy for all 6 locations, thereby adjusting the angle of the tri-vision camera device when the external object is moving, and thus determining the degree of consistency with the monitoring results of the 4D radar in different angle ranges.

[0044] S233 determines the mobile monitoring position of the transmission line tower that exists between the nearest mobile monitoring position and the different mobile monitoring positions, and treats it as a discrete monitoring position. Based on the proportion of the discrete monitoring positions in the transmission line where 4D radar and tri-vision are installed, and the proportion of the monitored positions, a monitoring matching factor is determined. It is then determined whether the monitoring matching factor is greater than a preset matching factor threshold. If so, the movement control strategy of the mobile monitoring position in the comparison process is determined to be that there is no need to adjust the angle of the tri-vision camera device. It is only necessary to use the initial angle to determine whether the movement distance in the camera device is consistent with the movement distance of the 4D radar. If not, proceed to the next step. In the above steps, a comprehensive judgment of dispersion and correlation is performed. The system analyzes the location spacing and finds that locations E and F are far away from other nearby monitoring locations (such as location D) and are separated by transmission line towers. Therefore, they are marked as "discrete monitoring locations".

[0045] Calculate the monitoring matching factor: This factor combines the "proportion of discrete monitoring locations" and the "proportion of total monitoring quantity". This factor reflects the reliability of the overall monitoring device's verification process. Generally, the higher the reliability, the less angle adjustment is required. In one possible embodiment, the factor is the sum of (1 - proportion of discrete monitoring locations) and the "proportion of total monitoring quantity". If the calculated factor value is not high (e.g., less than 1), it does not reach the "no adjustment required" threshold and does not meet the "no adjustment required" condition, proceeding to the final judgment.

[0046] S234 determines whether the motion monitoring position is a discrete monitoring position. If so, it determines that the motion control strategy of the motion monitoring position in the comparison process requires angle adjustment processing to determine whether the movement distance in the camera device is consistent with the movement distance of the 4D radar at different angles. If not, it determines that the motion control strategy of the motion monitoring position in the comparison process does not require angle adjustment processing of the tri-vision camera device, and only needs to use the initial angle to determine whether the movement distance in the camera device is consistent with the movement distance of the 4D radar.

[0047] The final case assessment is performed in the above steps, and the final screening is conducted at the six locations.

[0048] For locations E and F (discrete monitoring locations): the final strategy is "angle adjustment processing is required". Because they are isolated risk points, verification reliability must be improved through multi-angle visual confirmation, that is, adjusting the angle of the camera device during the monitoring of moving objects.

[0049] For locations A, B, C, and D: the final strategy is "no need to adjust the angle of the 3D vision camera device." Because they are close to each other, forming a continuous monitoring area, using the initial angle for rapid comparison can meet reliability requirements and save system resources.

[0050] Specifically, such as Figure 3 As shown, based on the comparison and processing data, it is determined that the early warning strategy needs to be updated, specifically including: This embodiment is the core manifestation of the system's intelligence. By analyzing historical collaborative work data, it can identify blind spots or weak links in perception and proactively adjust strategies to achieve closed-loop optimization.

[0051] S31 determines the number of comparison processing times in different angle ranges based on the comparison processing data; In the above steps, the system records each radar and vision collaborative comparison event within a statistical period (e.g., 24 hours) and classifies and counts them according to a preset angle range.

[0052] Data comparison processing: This refers to the historical record of each data comparison. Core fields include: timestamp, angle range, moving distance measured by radar, moving distance measured by vision, deviation rate, and consistency.

[0053] Angle interval: Dividing the horizontal direction (e.g., 360°) into several sector-shaped regions, for example, dividing it into 10 intervals, each 36°. This is the basic unit for the system to perform refined spatial management. Number of comparisons: The total number of times radar and visual data are compared within a specific angle interval.

[0054] This step is fundamental to data analysis and aims to understand in which directions the system operates most frequently. It maps the system's workload distribution, identifying "hotspot" and "coldspot" monitoring directions. This provides a basis for subsequent assessments of data sufficiency.

[0055] Based on the comparison processing data in different angle ranges, S32 determines the number of comparison processing times where the moving distance between the camera device and the 4D radar is inconsistent in different angle ranges, and uses this as the number of comparison deviations. In the above steps, within each angle range, all "inconsistent" comparison records are filtered out and their frequency is counted. Inconsistent movement distance: According to preset rules, when the deviation rate of the movement distance calculated by radar and vision for the same target exceeds the preset deviation rate threshold (e.g., 5%), it is judged as an "inconsistency". Number of comparison deviations: The total number of comparison events judged as "inconsistent" within a specific angle range.

[0056] The focus has shifted from "workload" statistics to "work quality" statistics. The aim is to quantify the frequency of errors in which areas the system makes, thus specifying the problems. The system no longer simply knows "where it's busy," but rather "where it's prone to errors."

[0057] S33 determines the angle range where there is a number of comparison deviations based on the number of comparison deviations in different angle ranges, and uses it as the deviation angle range. Based on the deviation angle range and the number of comparison processing in different angle ranges, it determines whether the early warning strategy needs to be updated.

[0058] It should be noted that the comparison processing data is determined based on whether the motion monitoring combination and the motion distance monitoring data of the 4D radar are consistent within different angle intervals within the distance range where the monitoring accuracy of the 4D radar meets the requirements. The consistency is determined based on whether the deviation rate between the motion distance monitoring data of the motion monitoring combination and the motion distance monitoring data of the 4D radar is less than a preset deviation rate threshold.

[0059] It is understood that, based on the aforementioned deviation angle range and the number of comparison processes within different angle ranges, it is determined whether an update to the early warning strategy is needed, specifically including: S331 determines whether there is a deviation angle range. If yes, proceed to the next step; otherwise, it is determined that no update of the early warning strategy is required. In the above steps, the initial judgment is made as to whether there is a problem area and whether there is any deviation angle range.

[0060] This is a rapid screening process. If the radar and visual data are 100% consistent across all angle ranges, it indicates excellent overall system coordination. No adjustments are needed, and the decision-making process ends early. Under ideal system conditions, early warning can be achieved simply by analyzing the movement distance of the motion monitoring combination, thus saving computing power.

[0061] S332 obtains the number of deviation angle intervals and determines whether the number of deviation angle intervals is greater than the preset deviation interval number threshold. If so, the monitoring reliability of the movement distance of multiple angle intervals may be poor, so the early warning strategy needs to be updated. That is, when the anti-external damage early warning is processed, the angle of the camera device is adjusted. If not, proceed to the next step. In the above steps, perform a breadth judgment – ​​is the problem widespread? Count the number of deviation angle intervals and see if it exceeds the preset threshold for the number of deviation intervals (e.g., 5).

[0062] If deviations occur in most angular directions, this may not be a local problem, but a systemic calibration error. Accurate identification of deviations in the reliability of position recognition using multi-view vision can capture system-level, widespread performance anomalies and trigger global calibration checks.

[0063] S333 determines the deviation weight coefficient for different deviation angle intervals based on the ratio of the number of comparison deviations to the number of comparison processing within different deviation angle intervals. It then determines whether there are deviation angle intervals where the deviation weight coefficient is greater than the preset weight coefficient threshold. If so, the monitoring reliability of the movement distance in multiple angle intervals may be poor, so the early warning strategy needs to be updated. That is, when processing the early warning of external damage, the angle of the camera device is adjusted. If not, proceed to the next step. In the above steps, perform a depth judgment to determine whether there are serious local problems, calculate the deviation weight coefficient for each deviation angle interval (deviation weight coefficient = number of comparison deviations / number of comparison processing), and check whether any coefficient exceeds the preset weight coefficient threshold (e.g., 0.5, i.e., 50%).

[0064] Even if the problem area is small, if the failure rate is extremely high in a particular direction, then that direction is a high-risk blind spot. This metric measures the severity of monitoring bias in the camera device, not just its range, enabling reliable identification of monitoring bias in the camera device's movement distance.

[0065] S334 determines whether the sum of the deviation weight coefficients of different deviation angle intervals is greater than the preset deviation weight coefficient threshold. If yes, proceed to the next step; otherwise, determine that no update processing of the early warning strategy is required. In the above steps, the overall impact judgment is to determine whether the identification deviation of the problem is serious. This involves calculating the sum of the deviation weight coefficients for all deviation angle intervals and determining whether it exceeds the preset deviation weight coefficient threshold (e.g., 2.0).

[0066] While no single interval may be particularly bad (S333 not triggered), multiple intervals exhibiting moderate deviations accumulate to increase the overall unreliability of the system. This is an indicator of the "severity of the overall problem."

[0067] S335 determines whether there are angle intervals where the number of comparison processing times is less than the preset comparison number threshold by comparing processing times within different angle intervals. If so, the monitoring reliability of the movement distance in multiple angle intervals may be poor, so the early warning strategy needs to be updated. That is, the angle of the camera device is adjusted during the anti-external damage early warning processing. If not, it is determined that the early warning strategy does not need to be updated.

[0068] In the above steps, the data sufficiency assessment—checking for monitoring blind spots—involves examining whether there are angle intervals where the number of comparisons is less than a preset comparison threshold (e.g., 50 times). If almost no data comparisons have been performed in a certain direction, it indicates that the system's true monitoring capability in that direction is unknown. This could be because there are no active targets in that direction, but it could also mean that once a target appears, the system cannot effectively verify it.

[0069] Taking "unknown risks" into account, the system is forced to adopt a more conservative strategy for areas with insufficient data, thereby covering potential monitoring blind spots.

[0070] Specifically, the method for determining the line protection early warning method that combines trifocal vision and 4D radar is as follows: It should be noted that "mobile monitoring combination" refers to a camera combination in a trinocular vision system, consisting of two or three cameras capable of stereoscopic ranging. For example, a trinocular system can have three different combinations: "left-center," "center-right," and "left-right." The reliability of each "mobile monitoring combination" in different directions is evaluated to develop an optimal visual early warning plan for the entire monitoring point in radar blind spots or unreliable areas.

[0071] S41 uses the comparison data of different motion monitoring combinations in different angle ranges during the movement of an object to determine the number of comparison processing steps where the movement distance of the motion monitoring combination is inconsistent with that of the 4D radar in different angle ranges during the movement of the object, and uses this as the number of comparison deviations. In the above steps, the number of comparison deviations is calculated. For a monitoring location, all its moving monitoring combinations (such as "left-center", "center-right", "left-right") are traversed, and the number of times each combination is inconsistent with the 4D radar data in different angle intervals is counted.

[0072] Mobile monitoring combination: A pair or group of cameras in a trinocular vision system that can form stereo vision. Number of comparison deviations: The total number of times that visual ranging and radar ranging are inconsistent within a specific angle range of a specific combination.

[0073] S42 determines the monitoring deviation probability of the mobile monitoring combination in different angle intervals based on the number of comparison deviations in different angle intervals; In the above steps, the monitoring deviation probability is calculated. For each (combination, angle interval) pair, its monitoring deviation probability is calculated as: number of comparison deviations / total number of comparisons. This transforms simple counting into probability, which is a normalization process that allows for fair comparison of the reliability between different combinations and different intervals.

[0074] Generate a "reliability heatmap". For example, we find that the "left-middle" combination has a deviation probability as high as 60% in the "interval 4" direction, which means that the combination is unreliable in this direction.

[0075] S43 determines the line protection and early warning method for external damage prevention in the location by using different motion monitoring combinations to determine the monitoring deviation probability in different angle ranges.

[0076] Furthermore, the monitoring deviation probability is determined based on the ratio of the number of comparison deviations of the motion monitoring assembly within the angular interval to the number of comparisons between the motion monitoring assembly and the 4D radar within the angular interval.

[0077] It is understandable that by using different combinations of motion monitoring with varying monitoring deviation probabilities within different angular ranges, the method for early warning of line damage prevention through the coordinated use of trifocal vision and 4D radar at the aforementioned location is determined, specifically including: S431 uses the monitoring deviation probability of the mobile monitoring combination in different angle intervals to determine whether there is an angle interval where the monitoring deviation probability is greater than a preset deviation probability threshold. If so, the method for preventing external damage to the line in the location by the coordination of tri-vision and 4D radar is determined as follows: within the distance range where the monitoring accuracy of 4D radar cannot meet the requirements, the camera devices of different mobile monitoring combinations are adjusted at angles through a second preset time period. Based on the monitoring data of the moving distance between the mobile monitoring combinations in different angle intervals, it is determined whether a warning signal needs to be issued. If not, proceed to the next step. In the above steps, the system checks for the existence of "high-risk combination-angle pairs," specifically checking whether any one of the motion monitoring combinations (motion monitoring combination, angle range) has a monitoring deviation probability greater than a preset deviation probability threshold (e.g., 50%). This is the most stringent check. If any "unreliable" sensor combination exists in any direction, the system considers there to be a localized risk overall.

[0078] If triggered, the "angle adjustment" warning mode will be activated. That is, when a target enters the area, the tri-vision system will periodically adjust the camera angle to perform stereoscopic measurements from multiple perspectives, and will only issue a warning when the measurement results from multiple perspectives are consistent and a danger is determined. This can effectively avoid false alarms or missed alarms caused by inherent defects of a fixed combination at a specific angle (such as lens dirt or calibration misalignment).

[0079] S432, based on the monitoring deviation probability, calculates the average value of the monitoring deviation probability of different mobile monitoring combinations in different angle ranges, and uses it as the comprehensive deviation factor of the mobile monitoring combination. It then determines whether there is a mobile monitoring combination whose comprehensive deviation factor is greater than a preset deviation factor threshold. If so, it determines that the line protection early warning method of the three-eye vision and 4D radar collaboration in the location is as follows: within the distance range where the monitoring accuracy of the 4D radar cannot meet the requirements, the camera devices of different mobile monitoring combinations are angled through a second preset time period. Based on the monitoring data of the movement distance between the mobile monitoring combinations in different angle ranges, it is determined whether an early warning signal needs to be issued. If not, it proceeds to the next step. In the above steps, check whether there is an "overall unreliable sensor combination", calculate the comprehensive deviation factor of each motion monitoring combination (i.e., the average deviation probability of the combination in all angle intervals), and check whether the average value of any combination is greater than the preset deviation factor threshold (e.g., 30%).

[0080] The system identifies low overall performance of the camera setup. For example, the "left-right" combination might have low overall reliability due to an excessively long baseline or other factors. If this is triggered, it indicates that other angles may also be affected by the same factors, thus activating the "angle adjustment" warning mode as well. Warnings are only issued when measurements from multiple perspectives are consistent and a danger is assessed. This effectively avoids false alarms or missed alarms caused by inherent defects in a fixed combination at a specific angle (such as lens dirt or calibration misalignment).

[0081] Based on the average value of the monitoring deviation probability of different mobile monitoring combinations in different angle intervals, S433 determines the identification deviation probability in different angle intervals, and judges whether there is an angle interval where the identification deviation probability is greater than the preset identification deviation probability threshold. If so, proceed to the next step; otherwise, determine the line anti-external damage early warning method in the location based on the identification deviation probability of different mobile monitoring combinations in different angle intervals. In the above steps, the recognition error probability (i.e., the average of the error probabilities of all motion monitoring combinations within this interval) is calculated for each angle interval. If there is an angle interval where the recognition error probability is greater than the preset recognition error probability threshold (5%), this reflects the degree to which "errors are likely to occur regardless of which combination is used to view this direction".

[0082] S434 determines whether the number of angle intervals with a recognition deviation probability greater than a preset recognition deviation probability threshold is greater than a preset angle interval number threshold. If so, the method for preventing external damage to the line in the location using a combination of tri-vision and 4D radar is determined as follows: within a distance range where the monitoring accuracy of the 4D radar cannot meet the requirements, the camera devices of different motion monitoring combinations are angled through a second preset time period. Based on the monitoring data of the movement distance between the motion monitoring combinations in different angle intervals, it is determined whether a warning signal needs to be issued. If not, the method for preventing external damage to the line in the location using a combination of tri-vision and 4D radar is determined based on the recognition deviation probability of different motion monitoring combinations in different angle intervals.

[0083] In the above steps, it is determined whether the number of these high-deviation angles is excessive (greater than a preset threshold for the number of angles in an angle range, such as two), thus identifying the location of the problem in the environment itself. For example, if the monitoring in one direction is inaccurate, camera devices in other directions may have the same risk of impact.

[0084] If there are too many high-deviation angles, the monitoring reliability of the camera device is poor, indicating that the visual sensor is facing a systemic challenge at that location, and the "angle adjustment" mode must be activated to deal with it.

[0085] If the number of high-deviation angles is small but they do exist, the system adopts a compromise yet still safe strategy: only activate the "angle adjustment" mode for these specific high-risk angle ranges. For other angle ranges with high reliability, the more efficient "no angle adjustment required" mode is used.

[0086] It should be noted that the determination of whether to issue an early warning signal is based on the monitoring data of the movement distance between the motion monitoring combinations within different angle ranges, specifically including: If the monitoring data of the movement distance between the construction machinery and the power transmission line are consistent within the angle range of any different movement monitoring combination, and it is determined from the monitoring data of the movement distance that the distance between the construction machinery and the power transmission line is less than the preset distance threshold, then a warning signal is issued.

[0087] Specifically, based on the recognition deviation probability of different motion monitoring combinations in different angle ranges at the location, a line damage prevention and early warning method combining trifocal vision and 4D radar at the location is determined, specifically including: When there is an angle range in the location where the probability of identification deviation is greater than a preset identification deviation probability threshold, the method for early warning of line damage prevention in the location using tri-vision and 4D radar in collaboration is determined as follows: within a distance range where the monitoring accuracy of 4D radar cannot meet the requirements, the camera devices of different motion monitoring combinations are angled through a second preset time period. Based on the monitoring data of the movement distance between the motion monitoring combinations in different angle ranges, it is determined whether an early warning signal needs to be issued. If not, the method for early warning of line damage prevention in the location using tri-vision and 4D radar in collaboration is determined as follows: within a distance range where the monitoring accuracy of 4D radar cannot meet the requirements, the camera devices of different motion monitoring combinations do not need to be angled. Based on the matching of the monitoring data of the movement distance between the different motion monitoring combinations, it is determined whether an early warning signal needs to be issued.

[0088] In one possible specific embodiment, the camera device detects an excavator in direction 5 of the zone and that it has entered the radar blind zone. According to a predetermined strategy, the system activates the "angle adjustment" mode of the tri-vision system. The tri-vision gimbal first uses the combined perspectives of B (center-right), A, and C to perform the first distance measurement on the target, calculating the distance to be 15 meters (safe threshold of 20 meters). Subsequently, the system slightly adjusts the gimbal angle according to the second preset time period (e.g., 2 seconds) and uses the combined perspectives of B (center-right), A, and C to perform the second distance measurement from another perspective, calculating the distance to be 18 meters. The system then adjusts the angle again to perform the third measurement.

[0089] Data Fusion and Decision Making: The system detected that the measurement results were completely consistent in each instance, but all were below the 20-meter safety threshold, and the trend was consistent. At this point, the system determined that a danger had been confirmed and immediately issued an early warning signal to prevent external damage.

[0090] It should be noted that the determination of whether to issue an early warning signal is based on the matching of movement distance monitoring data between different motion monitoring combinations, specifically including: When the monitoring data of the movement distance of any mobile monitoring group determines that the distance between the construction machinery and the power transmission line is less than a preset distance threshold, an early warning signal is issued.

[0091] Example 2 Secondly, the present invention provides a line damage prevention early warning device that combines tri-vision and 4D radar, employing the aforementioned tri-vision and 4D radar combined line damage prevention early warning method, specifically including: Monitoring location identification module, movement strategy control module, and early warning module; The monitoring location identification module is responsible for determining the movement monitoring location of the multi-view camera device in the transmission line; The motion strategy control module is responsible for determining the motion control strategy during the comparison and processing of multi-view vision cameras and 4D radar at the motion monitoring location. The early warning module is responsible for determining the early warning method for line damage prevention through the collaboration of tri-vision and 4D radar in different locations.

[0092] 4D radar, also known as imaging radar, is an advanced evolution of traditional millimeter-wave radar. In addition to the three dimensions of distance, horizontal azimuth, and velocity that traditional radar can measure, it adds the ability to accurately measure the target's elevation angle (i.e., altitude information), thus forming four-dimensional perception. In the context of protecting power transmission lines from external damage, it is a core all-weather perception system capable of high-resolution, four-dimensional point cloud imaging of objects within the monitored area.

[0093] Core Features Four-dimensional information perception: Distance: The straight-line distance between the object and the radar; Azimuth: The direction of the object on the horizontal plane; Elevation: The elevation angle of the object on the vertical plane. This is the key difference from traditional 3D radar, enabling it to determine whether the target is located on the ground, in the air, or at the same height as a guide wire; Velocity: The radial velocity of the object is perceived through the Doppler effect.

[0094] Through massive MIMO and advanced beamforming technology, 4D radar can output dense point cloud data with a resolution sufficient to outline the contours of objects and distinguish multiple adjacent targets.

[0095] Trinocular vision is an advanced optical sensing system based on the principle of stereo vision. It simulates the binocular parallax of the human eye by using three cameras arranged in a certain geometric relationship in space, and enhances measurement accuracy, reliability and field of view by adding a third camera.

[0096] In the context of preventing external damage to power transmission lines, it is a "smart eye" capable of performing high-precision 3D reconstruction, identity recognition, and behavior analysis of specific targets.

[0097] Eliminating matching ambiguity: In low-texture areas where binocular vision is difficult to distinguish, the third-view perspective can provide more constraints, greatly improving the accuracy and speed of feature point matching.

[0098] Increased redundancy and reliability: Even if one camera is temporarily blocked or malfunctions, the system can still continue to work by relying on the other two cameras to form a binocular system, expanding the effective field of view: A wider field of view can be covered by different camera combinations.

[0099] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0100] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0101] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for early warning of external damage to power lines using a combination of trifocal vision and 4D radar, characterized in that, Specifically, it includes: Using the monitoring data of 4D radar in the transmission line, the historical identification data of external objects of 4D radar in different angle ranges is determined. Using the historical identification data in different locations, the movement monitoring position of the multi-view camera device in the transmission line is determined. Based on the motion monitoring location data, and combined with the distance intervals between different motion monitoring locations and other motion monitoring locations, the motion control strategy of the multi-view camera device and 4D radar at the motion monitoring location is determined during the comparison and processing process. When it is determined that the early warning strategy needs to be updated based on the comparison processing data, different camera devices are combined in pairs to form a motion monitoring combination. Based on the comparison data of different motion monitoring combinations in different angle ranges during the movement of objects, a line protection early warning method that combines tri-vision and 4D radar in different positions is determined.

2. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 1, characterized in that, The monitoring data of the 4D radar is determined based on the number of times the 4D radar identifies external objects in different locations.

3. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 1, characterized in that, The angle intervals are divided into a preset number of angle intervals at equal intervals.

4. The method for early warning of line damage prevention by combining trifocal vision and 4D radar as described in claim 1, characterized in that, The historical identification data includes the number of times the external object was identified during its movement.

5. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 1, characterized in that, The method for determining the motion monitoring position of the multi-view camera device is as follows: Based on the historical recognition data, the number of times the 4D radar recognizes external objects during their movement is determined within different angle ranges; Based on the number of recognitions, determine the recognition matching angle interval within the angle interval; Based on the identification and matching angle interval data in the location, determine whether the location is the motion monitoring location of the multi-view camera device.

6. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 5, characterized in that, The recognition matching angle range is the angle range in which the number of recognitions is greater than a preset recognition number threshold.

7. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 1, characterized in that, If the location is a motion monitoring location, the angles of different cameras in the tri-vision system are adjusted according to a preset time period.

8. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 1, characterized in that, The method for determining the line protection early warning method that combines trifocal vision and 4D radar is as follows: By comparing data from different motion monitoring combinations within different angle ranges during the movement of an object, the number of comparison processing steps where the movement distance of the motion monitoring combination and the 4D radar are inconsistent within different angle ranges during the movement of the object is determined, and these steps are taken as the number of comparison deviations. The probability of monitoring deviation of the mobile monitoring combination in different angle ranges is determined based on the number of comparison deviations in different angle ranges. By using different mobile monitoring combinations to determine the monitoring deviation probability in different angle ranges, a method for early warning of line damage prevention in the location using tri-vision and 4D radar in coordination is established.

9. The method for early warning of external damage to power lines in conjunction with trifocal vision and 4D radar as described in claim 8, characterized in that, The monitoring deviation probability is determined based on the ratio of the number of comparison deviations of the mobile monitoring assembly within the angular range to the number of comparisons between the mobile monitoring assembly and the 4D radar within the angular range.

10. A line damage prevention and early warning device combining trinocular vision and 4D radar, employing the line damage prevention and early warning method combining trinocular vision and 4D radar as described in any one of claims 1-9, characterized in that, Specifically, it includes: Monitoring location identification module, movement strategy control module, and early warning module; The monitoring location identification module is responsible for determining the movement monitoring location of the multi-view camera device in the transmission line; The motion strategy control module is responsible for determining the motion control strategy during the comparison and processing of multi-view vision cameras and 4D radar at the motion monitoring location. The early warning module is responsible for determining the early warning method for line damage prevention through the collaboration of tri-vision and 4D radar in different locations.

Images