Non-contact distance measurement method for power equipment based on rotating target detection and infrared image

Abstract

The present invention provides a non-contact ranging method for power equipment based on rotating target detection and infrared images. The method comprises collecting infrared images of power equipment to generate an infrared image dataset containing target information of the power equipment; building a YOLOv5 target detection model optimized for rotating target detection and used to detect the rotating bounding box information of the power equipment target in the infrared image; using the trained YOLOv5 target detection model to perform target detection on the input infrared image and output a non-contact ranging result; and using a dynamic update algorithm based on historical target detection results to correct the current target position information and ranging results. The present invention can achieve high-precision identification and stable ranging of power equipment such as terminal heads and lightning arresters, and has the advantages of being sensor-independent, supporting target posture adaptation, and achieving high ranging accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and ranging of power equipment, and specifically to a non-contact ranging method for power equipment based on rotating target detection and infrared images. The method is applicable to scenarios such as power line inspection, substation equipment status monitoring, and remote identification and positioning of high-voltage facilities. It is particularly suitable for efficient identification and precise ranging of power components such as cable terminals and lightning arresters in complex environments, long distances, or areas that are difficult for humans to access. Background Art

[0002] As the level of intelligence in power systems continues to improve, the demand for automated inspection and intelligent identification of power equipment is growing. In particular, in scenarios such as transmission lines and substations, the status identification and spatial positioning of key components such as cable terminals and lightning arresters are crucial for equipment safety. Currently, the identification and ranging of power equipment often rely on manual inspections or the use of hardware devices such as lidar and ultrasonic sensors for auxiliary detection, which has the following shortcomings:

[0003] Recognition methods rely on manual or traditional algorithms, which are inefficient and highly subjective: Traditional image recognition methods have low recognition accuracy when faced with complex backgrounds, lighting changes, or device occlusion, and lack adaptive capabilities;

[0004] Distance measurement relies on dedicated hardware, resulting in complex and costly systems. Common distance measurement methods, such as laser and ultrasonic distance measurement, require the installation of dedicated sensors, increasing costs and maintenance complexity.

[0005] The object detection framework is insensitive to angle changes: Currently, most widely used object detection models use horizontal bounding boxes (HBoxes). However, when the device is tilted or the viewing angle is offset, detection accuracy decreases significantly, making it prone to missed or false detections.

[0006] Ranging accuracy is greatly affected by target detection errors and lacks compensation mechanisms: Most existing image ranging methods are based on calculations of a single dimension (such as width or height), and do not consider the imaging differences caused by changes in target angles, which easily leads to the accumulation of ranging errors.

[0007] In summary, existing technologies in the identification and ranging of power equipment still have problems such as insufficient accuracy, poor environmental adaptability, and high system costs. There is an urgent need for a more efficient, low-cost, and intelligent identification and ranging solution suitable for complex working conditions. Summary of the Invention

[0008] In response to the problems existing in existing power equipment identification and ranging technologies, such as low recognition accuracy, poor adaptability to inclined targets, dependence on external sensors for ranging, and insufficient environmental adaptability, the purpose of the present invention is to provide a non-contact ranging method for power equipment based on rotating target detection and infrared images, which is suitable for intelligent inspection and equipment positioning applications of power systems in complex environments. This method can achieve high-precision identification and stable ranging of power equipment such as terminal heads and lightning arresters, and has the advantages of being independent of sensors, supporting target posture adaptation, and having high ranging accuracy.

[0009] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0010] A non-contact distance measurement method for power equipment based on rotating target detection and infrared images includes the following steps:

[0011] Collect infrared images of power equipment and generate an infrared image dataset containing target information of the power equipment;

[0012] Build a YOLOv5 target detection model, which is optimized for rotating target detection and used to detect the rotating bounding box information of power equipment targets in infrared images; among them, the loss function of the YOLOv5 target detection model is L Including classification loss L cls , bounding box regression loss L reg and rotation angle prediction loss L θ ;

[0013] Use the trained YOLOv5 target detection model to detect the input infrared image and obtain the rotation bounding box information of the power equipment target B =( x c , y c , w , h , i ), determine the coordinates of the target's feature points on the image plane based on the rotation bounding box information, and calculate the spatial distance based on the distance relationship between the target feature points and the camera origin in the world coordinate system D , output the non-contact ranging result; where ( x c , y c ) is the center coordinate of the bounding box, w is the width of the bounding box, h is the height of the bounding box, i is the rotation angle of the bounding box;

[0014] Based on the historical target detection results, a dynamic update algorithm is used to correct the current target position information and ranging results; the dynamic update algorithm comprehensively considers the target detection results of the current frame and the previous n The historical detection results of the frame are used to smooth the target position and ranging values ​​through a weighted average algorithm.

[0015] According to the present invention, a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared images includes the following steps in the process of preparing an infrared image data set and preprocessing the data set:

[0016] Design cable terminal data collection tasks for various actual usage scenarios, including at least scenarios with different ambient temperatures, different lighting conditions, different weather conditions, and different operating states of the cable terminal;

[0017] All acquired infrared images are formatted uniformly and stored in categories according to timestamp, device number and scene number;

[0018] The targets in the infrared image are annotated using a rotating rectangular frame so that the rotated rectangular frame fits the target's outline. The annotation information includes the coordinates of the four vertices of the rotated target frame in the image coordinate system and the rotation angle θ. At the same time, each rotated target frame is annotated with its category information, which includes cable terminals and lightning arresters.

[0019] All labeled samples are split into training set, validation set and test set according to preset ratios; among them, multiple data enhancement operations are introduced on the training set, including at least angle rotation, contrast change, mirror flipping and cropping methods.

[0020] According to a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared images provided by the present invention, the method of determining the coordinates of feature points of the target on the image plane based on the rotating bounding box information includes:

[0021] For rotated bounding box B =( x c , y c , w , h , i ), select the four vertices of the rotation bounding box as the feature points of the target on the image plane, and set the rotation bounding box with the center point ( x c , y c ) is the center and the width direction is x Axis direction, height direction is yA local coordinate system is established in the axis direction. In the local coordinate system, the coordinates of the four vertices are P 1(- w / 2,- h / 2), P 2( w / 2,- h / 2), P 3( w / 2, h / 2) and P 4(- w / 2, h / 2);

[0022] According to the rotation angle i Rotate the vertex coordinates in the local coordinate system using the rotation matrix R θ Rotate the vertex coordinates in the local coordinate system, rotation matrix R θ The expression is:

[0023]

[0024] For the vertex coordinates in the local coordinate system ( x local , y local ), the rotated coordinates ( x rot , y rot ) is calculated as:

[0025]

[0026] The four vertices P 1. P 2. P 3. P The local coordinate transformation of 4 obtains the coordinates in the local coordinate system after rotation P 1rot ( x 1rot , y 1rot ), P 2rot ( x 2rot , y 2rot ), P 3rot ( x 3rot , y 3rot ), P 4rot (x 4rot , y 4rot );

[0027] Convert the vertex coordinates in the rotated local coordinate system to the global coordinate system of the image plane, and obtain the coordinates of the four feature points on the image plane ( x i , y i )( i =1,2,3,4), the conversion formula is: x i = x rot + x c 、 y i = y rot + y c ;

[0028] based on P 1rot 、 P 2rot 、 P 3rot 、 P 4rot The coordinates of the four feature points on the image plane are obtained by P 1img ( x 1, y 1) P 2img ( x 2, y 2) P 3img ( x 3, y 3) P 4img ( x 4, y 4).

[0029] According to the present invention, a non-contact distance measurement method for electric equipment based on rotating target detection and infrared image is provided, wherein the spatial distance is calculated according to the distance relationship between the target feature point and the camera origin in the world coordinate system. D， include:

[0030] Set the intrinsic parameter matrix of the infrared camera K for:

[0031]

[0032] in fx and f y The camera is x Axis and y The focal length along the axis, c x and c y The principal points of the image are x Axis and y Coordinates in the axis direction;

[0033] For the feature point coordinates on the image plane ( x i , y i ), perform normalization processing to obtain the normalized coordinates ( x inorm , y inorm ):

[0034] x inorm =( x i - c x ) / f x

[0035] y inorm =( y i - c y ) / f y

[0036] At this time, assuming the camera coordinate system Z ci =1, Z ci is the feature point in the camera coordinate system z Axis coordinates, the normalized coordinates ( x inorm , y inorm ,1) Approximately represents the coordinates in the camera coordinate system;

[0037] for n feature points, and the coordinates of the feature points in the camera coordinate system are ( X ci , Y ci , Z ci ), satisfying the following relationship:

[0038]

[0039] Then we have: x i = f x u i + c x ; y i = f y v i + c y ; u i =( x i - c x ) / f x ; v i =( y i - c y ) / f y ;

[0040] Establish a set of equations through the coordinates of multiple feature points and solve them using the least squares method Z ci , set up the i The equation of the characteristic point is:

[0041]

[0042] Will n The equations of the characteristic points are combined and written in matrix form Axe = b ,in A is the coefficient matrix, x is a vector of unknown numbers, which contains u i 、 v i and Z ci Related information, b is a constant vector, and is solved by the least squares method x ,get Z ci The value of , and then get the feature point coordinates in the camera coordinate system ( X ci , Y ci , Z ci ).

[0043] According to the present invention, a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared image is provided, and the external parameter matrix of the infrared camera is set [ R ∣ T ],in R is a 3×3 rotation matrix, T is a 3×1 translation vector;

[0044] The feature point coordinates in the camera coordinate system ( X ci , Y ci , Z ci ) and the feature point coordinates in the world coordinate system ( X wi , Y wi , Z wi ) is:

[0045]

[0046] Convert the feature point coordinates in the camera coordinate system to the feature point coordinates in the world coordinate system;

[0047] Calculate the distance between the target feature point and the camera origin in the world coordinate system D i , for the i feature points, distance D i The calculation formula is:

[0048]

[0049] The distance calculated for the four feature points D i ( i =1,2,3,4) and take the average value to get the final spatial distance D , the calculation formula is:

[0050]

[0051] Output the calculated spatial distance D As a result of contactless ranging.

[0052] According to a non-contact ranging method for electric equipment based on rotating target detection and infrared image provided by the present invention, the loss function of the YOLOv5 target detection model is L It is expressed as the following formula:

[0053] L = l1 L cls + l 2 L reg + l 3 L θ

[0054] in l 1. l 2. l 3 is the weight coefficient; during the training process, the validation set is used to evaluate the network model in training, and the network fitting is judged according to the evaluation indicators on the validation set.

[0055] According to the present invention, a non-contact ranging method for electric power equipment based on rotating target detection and infrared images is provided. The YOLOv5 target detection model includes an input layer, a backbone network, a neck network, and a detection head. The backbone network adopts the CSPDarknet architecture for extracting multi-scale image features from infrared images. An SPPF module is embedded between the backbone network and the detection head. The SPPF module is composed of multiple parallel maximum pooling layers of different scales. Pooling operations of different scales are performed on the input feature map, and the pooling results are spliced. The PANet structure is introduced in the neck network. The PANet structure fuses and transmits feature maps of different levels through a top-down and bottom-up bidirectional feature fusion path.

[0056] The detection head predicts the target bounding box based on feature maps of different scales. It is divided into three detection heads of different scales, which are suitable for the recognition scenarios of large, medium and small-sized targets respectively. Each detection head processes the input feature map through convolution operation, and the output includes at least the category probability of the target, the center coordinates of the rotated bounding box ( x c , y c ),width w ,high h and the rotation angle i information, thereby achieving target positioning and classification.

[0057] According to the present invention, a non-contact ranging method for power equipment based on rotating target detection and infrared images is provided. In each round of training, samples are randomly sampled from the training set in batches and input into the network. The samples are first pre-processed by the input layer of the network, and then pass through the backbone network, SPPF module, neck network and detection head in sequence to obtain the output results of the network encoder.

[0058] Compare the network encoder output with the corresponding true label through the designed loss function LCalculate the loss value and use the back propagation algorithm to update and optimize the network parameters according to the loss value, so that the network continues to approach the optimal solution;

[0059] After the last batch of the training set is extracted and trained, all batches in the validation set are fed into the network in sequence for validation. During the validation process, the network predicts the validation set samples and obtains the prediction results.

[0060] Compare the predicted results with the true labels of the validation set and calculate a series of evaluation metrics to assess the performance of the current model on unseen samples and determine whether the network is overfitting or underfitting.

[0061] Repeat the training and validation steps until all samples in the training set complete the iterative update of the set cycle, completing the entire model training process and obtaining a trained YOLOv5 target detection model.

[0062] According to the present invention, a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared image is provided. A target buffer is established. After each target detection is completed and the distance measurement result is calculated, the target category, position information, distance measurement result and current frame number are recorded in the target buffer.

[0063] For each target in the newly detected frame, calculate the intersection over union (IoU) value between it and the rotated bounding boxes of all targets of the same category in the target buffer;

[0064] Set an IoU matching threshold. When the IoU value of the target in the new detection frame and a target of the same category in the target buffer is greater than or equal to the matching threshold, the two targets are judged to be the same target. When the IoU value is less than the matching threshold, or the category information is inconsistent, the target in the new detection frame is judged to be a new target.

[0065] For the case where the target is determined to be the same, the weighted average algorithm is used to update the ranging result.

[0066] According to a non-contact ranging method for electric power equipment based on rotating target detection and infrared images provided by the present invention, the calculation of the intersection over union (IoU) value of the rotating bounding box includes:

[0067] First, the intersection area of ​​the two rotated bounding boxes is calculated based on their parameters; the rotated bounding boxes are converted into polygons, and then the area of ​​the intersection polygon is calculated using a polygon intersection algorithm;

[0068] Calculate the areas of the two rotated bounding boxes respectively;

[0069] The IoU value is obtained by dividing the area of ​​the intersection polygon by the sum of the areas of the two rotated bounding boxes minus the area of ​​the intersection polygon. IoU =S intersection / S box1 + S box2 - S intersection ,in S intersection is the area of ​​the intersection polygon, S box1 and S box2 are the areas of the two rotated bounding boxes respectively.

[0070] It can be seen that compared with the prior art, the method provided by the present invention has the following beneficial effects:

[0071] This invention uses infrared imaging to identify and measure the distance of power equipment. Leveraging infrared imaging's unrestricted lighting and stable imaging capabilities at night and in inclement weather, it can reliably detect targets such as terminal heads and lightning arresters in a variety of complex environments. Compared to traditional recognition systems that rely on visible light images, this invention offers greater applicability and stability in scenarios such as all-weather operations and long-distance monitoring.

[0072] 2. The present invention adopts a rotating target detection method based on YOLOv5, optimizes the rotating target detection, and can accurately detect the rotating bounding box information of the power equipment target in the infrared image. In the actual power equipment detection scenario, the posture of the power equipment in the image is often arbitrary. The traditional rectangular bounding box detection method is difficult to accurately describe the shape and position of the target, while the rotating bounding box can better fit the actual contour of the target, greatly improving the accuracy of target detection. For example, for tilted cable terminals, the rotating bounding box can more accurately frame the target, avoiding the situation where the traditional rectangular bounding box may contain too much background information or fail to fully cover the target, providing more reliable target position information for subsequent ranging calculations;

[0073] 3. The loss function L of the YOLOv5 target detection model of the present invention includes the classification loss L cls , bounding box regression loss L reg and the rotation angle prediction loss L θ This collaborative optimization of multiple loss functions enables the model to simultaneously focus on target classification, bounding box position and size regression, and rotation angle prediction. By jointly optimizing these loss functions, the model can more comprehensively learn the characteristics of power equipment targets, further improving the accuracy and robustness of target detection. In practical applications, even if power equipment is affected by factors such as lighting changes and occlusion, the model can still accurately detect the target and obtain its rotated bounding box information;

[0074] 4. The present invention utilizes the acquired rotating bounding box information of the power equipment target to determine the coordinates of the target's feature points on the image plane based on the rotating bounding box information, calculates the spatial distance based on the distance relationship between the target feature points and the camera origin in the world coordinate system, and outputs the non-contact ranging result. This ranging method based on image geometric features does not require the installation of additional sensors or markers on the power equipment, achieving true non-contact ranging and avoiding the potential safety hazards and equipment damage risks associated with traditional contact ranging methods. At the same time, this method fully utilizes the geometric information in the image, obtains the target's spatial distance through precise mathematical calculations, and has high ranging accuracy.

[0075] 5. Based on historical target detection results, the present invention uses a dynamic update algorithm to correct the current target position information and ranging results. The dynamic update algorithm comprehensively considers the target detection results of the current frame and the historical detection results of the previous n frames, and smoothes the target position and ranging values ​​through a weighted average algorithm. In practical applications, due to the influence of factors such as image noise and temporary occlusion of the target, the target detection results of a single frame may have certain errors. By comprehensively considering historical detection results, the dynamic update algorithm can effectively reduce the impact of these errors and improve the stability and accuracy of the target position and ranging results. In addition, for power equipment targets in motion, the dynamic update algorithm can track the target position and ranging changes in real time, and perform smoothing based on historical data, so that the ranging results are more consistent with the actual motion trajectory of the target. For example, during the inspection process of power equipment, the target may change its position and posture as the inspection equipment moves. The dynamic update algorithm can adjust the target position and ranging results in a timely manner, providing more accurate data support for subsequent analysis and decision-making.

[0076] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 The present invention is a flowchart of an embodiment of a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared images.

[0078] Figure 2 The present invention is a flowchart of processing an infrared image data set in an embodiment of a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared images.

[0079] Figure 3 This is a flow chart for determining the coordinates of feature points of a target on an image plane in an embodiment of a non-contact ranging method for electric power equipment based on rotating target detection and infrared images of the present invention.

[0080] Figure 4The present invention is a method for calculating spatial distance in a non-contact distance measurement method for electric equipment based on rotating target detection and infrared image. D The first flow chart.

[0081] Figure 5 The present invention is a method for calculating spatial distance in a non-contact distance measurement method for electric equipment based on rotating target detection and infrared image. D The second flow chart of FIG.

[0082] Figure 6 This is a schematic diagram of the first prediction result of the YOLOv5 target detection model in an embodiment of a non-contact ranging method for electric equipment based on rotating target detection and infrared images of the present invention.

[0083] Figure 7 This is a schematic diagram of the second prediction result of the YOLOv5 target detection model in an embodiment of a non-contact ranging method for electric equipment based on rotating target detection and infrared images of the present invention.

[0084] Figure 8 It is a flow chart of a dynamic update mechanism in an embodiment of a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared images of the present invention.

[0085] Figure 9 It is a schematic diagram of the results of multi-target ranging in an embodiment of a non-contact ranging method for electric power equipment based on rotating target detection and infrared images of the present invention. DETAILED DESCRIPTION

[0086] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0087] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0088] See also Figure 1 This embodiment provides a non-contact distance measurement method for electric power equipment based on rotating target detection and infrared images, the method comprising the following steps:

[0089] Step S1, collecting infrared images of power equipment to generate an infrared image data set containing target information of the power equipment;

[0090] Step S2: Build a YOLOv5 target detection model, which is optimized for rotating target detection and used to detect the rotating bounding box information of the power equipment target in the infrared image; wherein the loss function of the YOLOv5 target detection model is L Including classification loss L cls , bounding box regression loss L reg and rotation angle prediction loss L θ During the training process, the validation set is used to evaluate the network model in training, and the network fitting is judged based on the evaluation indicators on the validation set, such as average precision (AP) and recall rate (Recall). If underfitting or overfitting occurs, the model structure or training parameters are adjusted in time.

[0091] Step S3: Use the trained YOLOv5 target detection model to perform target detection on the input infrared image and obtain the rotation bounding box information of the power equipment target. B =( x c , y c , w , h , i ), determine the coordinates of the target's feature points on the image plane based on the rotation bounding box information, and calculate the spatial distance based on the distance relationship between the target feature points and the camera origin in the world coordinate system D , output the non-contact ranging result; where ( x c , y c ) is the center coordinate of the bounding box, w is the width of the bounding box, h is the height of the bounding box, i is the rotation angle of the bounding box;

[0092] Step S4, based on the historical target detection results, the dynamic update algorithm is used to correct the current target position information and ranging results; wherein the dynamic update algorithm comprehensively considers the target detection results of the current frame and the previous n The historical detection results of the frame are used to smooth the target position and ranging values ​​through a weighted average algorithm.

[0093] In the above step S1, the process of preparing the infrared image dataset and preprocessing the dataset includes the following steps:

[0094] OpenCV image processing technology is used to clean, preprocess and extract features of the dataset so that the dataset can be organized into a format suitable for model training. Python language is used for programming experiments, and Python code is written to implement the division of the dataset.

[0095] In practical applications, such as Figure 2 As shown, we first designed cable terminal data collection tasks for various practical scenarios, covering typical locations such as substations, ground piles, platforms, and towers. These scenarios include at least varying ambient temperatures (e.g., high, low, and normal), lighting conditions (e.g., strong light, weak light, and backlight), weather conditions (e.g., sunny, cloudy, and rainy), and cable terminal operating states (e.g., normal operation, minor faults, and major faults). During data collection, we repeatedly photographed multiple cable terminals in the same scenario, ensuring diverse images at different times, angles, distances, and lighting conditions to enhance the comprehensiveness and representativeness of the dataset.

[0096] All acquired infrared images are uniformly formatted and classified and stored according to acquisition timestamp, device number, and scene number. This facilitates rapid image retrieval and effective management, providing an orderly data foundation for subsequent data annotation, model training, and other stages. The timestamp is used as the first-level directory, and images are sorted and stored in chronological order of capture time. Within each timestamp directory, the device number is used as the second-level directory to distinguish images corresponding to different cable terminal devices. Within each device number directory, the scene number is used as the third-level directory to further classify and store images in different actual usage scenarios, facilitating subsequent data management and retrieval.

[0097] In order to improve the image quality and enhance the recognizability of the cable terminal in the image, the image enhancement algorithm is applied to the collected original infrared image.

[0098] Considering the angle and direction changes of the cable terminal in the image, the target in the infrared image is annotated using a rotating rectangular box. For example, the cable terminal or lightning arrester in the infrared image is annotated with a rotating target box using the Rolabelimg annotation tool so that the rotating rectangular box fits the target's outline. The annotation information includes the coordinates of the four vertices of the rotating target box in the image coordinate system and the rotation angle θ. At the same time, each rotated target box is annotated with its category information, which includes cable terminal and lightning arrester.

[0099] All labeled samples are split into training set, validation set and test set according to the preset ratio; the training set is used for model training, the validation set is used to judge the fitting of the network model during the training process, and the test set is used for the final evaluation of the model performance.

[0100] Furthermore, a variety of data enhancement operations are introduced on the training set, including at least angle rotation, contrast change, mirror flipping and cropping methods.

[0101] The data-augmented dataset is divided into training set, validation set and test set in a ratio of 8:1:1, resulting in a total of 42,655 training samples, 3,416 validation samples and 344 test samples.

[0102] In the above step S3, the feature point coordinates of the target on the image plane are determined based on the rotation bounding box information, such as Figure 3 Shown, including:

[0103] For rotated bounding box B =( x c , y c , w , h , i ), select the four vertices of the rotation bounding box as the feature points of the target on the image plane, and set the rotation bounding box with the center point ( x c , y c ) is the center and the width direction is x Axis direction, height direction is y A local coordinate system is established in the axis direction. In the local coordinate system, the coordinates of the four vertices are P 1(- w / 2,- h / 2), P 2( w / 2,- h / 2), P 3( w / 2, h / 2) and P 4(- w / 2, h / 2);

[0104] According to the rotation angle i Rotate the vertex coordinates in the local coordinate system using the rotation matrix R θ Rotate the vertex coordinates in the local coordinate system, rotation matrix R θ The expression is:

[0105]

[0106] For the vertex coordinates in the local coordinate system ( xlocal , y local ), the rotated coordinates ( x rot , y rot ) is calculated as:

[0107]

[0108] The four vertices P 1. P 2. P 3. P The local coordinate transformation of 4 obtains the coordinates in the local coordinate system after rotation P 1rot ( x 1rot , y 1rot ), P 2rot ( x 2rot , y 2rot ), P 3rot ( x 3rot , y 3rot ), P 4rot ( x 4rot , y 4rot );

[0109] Convert the vertex coordinates in the rotated local coordinate system to the global coordinate system of the image plane, and obtain the coordinates of the four feature points on the image plane ( x i , y i )( i =1,2,3,4), the conversion formula is: x i = x rot + x c 、 y i = y rot + y c ;

[0110] based on P 1rot 、 P 2rot 、 P3rot 、 P 4rot The coordinates of the four feature points on the image plane are obtained by P 1img ( x 1, y 1) P 2img ( x 2, y 2) P 3img ( x 3, y 3) P 4img ( x 4, y 4).

[0111] In the above step S3, if Figure 4 As shown, the spatial distance is calculated based on the distance relationship between the target feature point and the camera origin in the world coordinate system D， include:

[0112] Set the intrinsic parameter matrix of the infrared camera K for:

[0113]

[0114] in f x and f y The camera is x Axis and y The focal length along the axis, c x and c y The principal points of the image are x Axis and y Coordinates in the axis direction;

[0115] For the feature point coordinates on the image plane ( x i , y i ), perform normalization processing to obtain the normalized coordinates ( x inorm , y inorm ):

[0116] x inorm =( x i - c x ) / f x

[0117] y inorm =( y i - c y ) / f y

[0118] At this time, assuming the camera coordinate system Z ci =1, Z ci is the feature point in the camera coordinate system z Axis coordinates, the normalized coordinates ( x inorm , y inorm ,1) Approximately represents the coordinates in the camera coordinate system;

[0119] for n feature points, and the coordinates of the feature points in the camera coordinate system are ( X ci , Y ci , Z ci ), satisfying the following relationship:

[0120]

[0121] Then we have: x i = f x u i + c x ; y i = f y v i + c y ; u i =( x i - c x ) / f x ; v i =( y i - c y ) / f y ;

[0122] Establish a set of equations through the coordinates of multiple feature points and solve them using the least squares method Z ci , set up the i The equation of the characteristic point is:

[0123]

[0124] Will n The equations of the characteristic points are combined and written in matrix form Axe = b ,in A is the coefficient matrix, x is a vector of unknown numbers, which contains u i 、 v i and Z ci Related information, b is a constant vector, and is solved by the least squares method x ,get Z ci The value of , and then get the feature point coordinates in the camera coordinate system ( X ci , Y ci , Z ci ).

[0125] like Figure 5 As shown, set the external parameter matrix of the infrared camera [ R ∣ T ],in R is a 3×3 rotation matrix, T is a 3×1 translation vector;

[0126] The feature point coordinates in the camera coordinate system ( X ci , Y ci , Z ci ) and the feature point coordinates in the world coordinate system ( X wi , Y wi , Z wi ) is:

[0127]

[0128] Convert the feature point coordinates in the camera coordinate system to the feature point coordinates in the world coordinate system;

[0129] Calculate the distance between the target feature point and the camera origin in the world coordinate system (the camera origin coordinates in the world coordinate system are (0,0,0)) D i , for the i feature points, distance D i The calculation formula is:

[0130]

[0131] The distance calculated for the four feature points D i ( i =1,2,3,4) and take the average value to get the final spatial distance D , the calculation formula is:

[0132]

[0133] Output the calculated spatial distance D As a result of contactless ranging.

[0134] Therefore, this embodiment provides a complete process from obtaining rotated bounding box information to calculating spatial distance, including steps such as determining feature point coordinates, converting image coordinates to camera coordinates, converting camera coordinates to world coordinates, and calculating spatial distance. This effectively calculates the spatial distance between power equipment and the camera. This calculation fully considers geometric parameters such as the camera's intrinsic and extrinsic matrix, as well as the various parameters of the rotated bounding box (center coordinates, width, height, and rotation angle), ensuring the accuracy and reliability of the results.

[0135] During the target detection phase, an improved YOLOv5 model is used to optimize rotating target detection, enabling more accurate detection of the rotating bounding box information of power equipment targets in infrared images. This information is used to determine the coordinates of the target's feature points on the image plane. By more accurately detecting the rotating target bounding box and combining it with geometric projection principles to calculate distance, the accuracy of non-contact ranging can be improved, providing more reliable data support for monitoring and maintenance of power equipment.

[0136] In this embodiment, the loss function of the YOLOv5 target detection model is L It is expressed as the following formula:

[0137] L = l 1 L cls + l 2 L reg + l 3 L θ

[0138] in l 1. l 2. l 3 is the weight coefficient; during the training process, the validation set is used to evaluate the network model in training, and the network fitting is judged according to the evaluation indicators on the validation set.

[0139] The classification loss in this embodiment is calculated using binary cross entropy. In order to improve the sensitivity of the bounding box to geometric factors during the regression process, a positioning loss F-CIOU is designed. The formula is as follows:

[0140]

[0141] in, B and B mb are the prediction box and the target box respectively, b and b mb Represent the center points of the prediction box and the target box respectively, p (·) represents the Euclidean distance between the predicted center point and the target bounding box, c is the diagonal length of the smallest box that covers the two boxes, w mb and h mb is the width and height of the target box, w and h is the width and height of the predicted box. F-CIOU retains the advantages of CIOU loss.

[0142] In this embodiment, the PyTorch deep learning framework is used to build an improved YOLOv5 network, and the functions and tools provided by the framework are used to implement model initialization, training, parameter tuning, and performance evaluation. Programming experiments are carried out in combination with Python and Java languages. Python code is written to implement model construction and training, and Java code is written to calculate multi-target distances and dynamically update multi-target ranging results. The code is executed through the Python and Java runtime environments to complete the entire construction process of the non-contact ranging system for power equipment.

[0143] In this embodiment, the YOLOv5 target detection model includes an input layer, a backbone network, a neck network, and a detection head to achieve end-to-end target detection capabilities, ensuring that cable terminals can be accurately located and identified in multi-angle and multi-distance infrared images, providing an accurate and robust feature foundation for subsequent analysis tasks.

[0144] Specifically, the backbone network adopts the CSPDarknet architecture, which divides the feature map into two parts through the cross-stage partial connection (CSP) technology, processes them independently and then merges them, effectively reducing the amount of calculation while enhancing the feature extraction capability. It can efficiently extract multi-scale image features from infrared images and provide rich semantic information for subsequent network processing; an SPPF module is embedded between the backbone network and the detection head. The SPPF module consists of multiple parallel maximum pooling layers of different scales. By performing pooling operations of different scales on the input feature map and splicing the pooling results, the network's receptive field information for targets of different scales is significantly enhanced, enabling the network to better capture the contextual information of the target and improve the accuracy of target detection; the PANet structure is introduced in the neck network. The PANet structure fuses and transmits feature maps of different levels through top-down and bottom-up bidirectional feature fusion paths. Specifically, the top-down path transfers high-level semantic information to the lower layers, enhancing the semantic expression ability of the low-level feature maps; the bottom-up path transfers the low-level position information to the higher layers, making up for the lack of position information in the high-level feature maps, achieving full fusion of the upper and lower layer features, and improving the network's detection performance for targets of different scales.

[0145] The detection head predicts the target bounding box based on feature maps of different scales. It is divided into three detection heads of different scales, which are suitable for the recognition scenarios of large, medium and small-sized targets respectively. Each detection head processes the input feature map through convolution operation, and the output includes at least the category probability of the target, the center coordinates of the rotated bounding box ( x c , y c ),width w ,high h and the rotation angle i information, thereby achieving target positioning and classification.

[0146] Furthermore, the improved YOLOv5 network was loaded and the network parameters were set for training. The network training cycle was 300, the batch size was 8, the Adam optimizer was used, the network initial learning rate was set to 1e-3, the network termination update learning rate was set to 0.1, and the weight decay coefficient was set to 0.001.

[0147] This example uses the Adam optimizer and the cosine annealing decay method to update the learning rate. The formula is as follows:

[0148]

[0149] in, lr is the learning rate, E is the total number of training epochs, lrf is the learning rate for stopping updates.

[0150] In each round of training, samples are randomly drawn from the training set in batches and input into the network. The samples are first preprocessed by the network's input layer, and then pass through the backbone network, SPPF module, neck network, and detection head in sequence to obtain the output of the network encoder.

[0151] Compare the network encoder output with the corresponding true label through the designed loss function L Calculate the loss value and use the back propagation algorithm to update and optimize the network parameters according to the loss value, so that the network continues to approach the optimal solution;

[0152] After the last batch of the training set is extracted and training is completed, all batches in the validation set are input into the network in turn for verification to determine the network's fitting situation. During the verification process, the network predicts the validation set samples to obtain the prediction results. If the loss value of the training set passing through the network is basically the same as the loss value of the validation set passing through the network, the network performance is better. If the loss value of the training set passing through the network is much smaller than the loss value of the validation set passing through the network, the network is seriously overfitting.

[0153] Compare the predicted results with the true labels of the validation set and calculate a series of evaluation metrics to assess the performance of the current model on unseen samples and determine whether the network is overfitting or underfitting.

[0154] Repeat the training and validation steps until all samples in the training set complete the iterative update of the set cycle, complete the entire model training process, and obtain the trained YOLOv5 target detection model. The prediction results of the improved YOLOv5 target detection model are as follows: Figure 6 and 7 shown.

[0155] In step S4 above, a target buffer is established. After each target detection is completed and the ranging result is calculated, the target category, position information, ranging result, and current frame number are recorded in the target buffer. If a record with the same target number already exists in the target buffer, the position information, ranging result, and most recently updated frame number in the record are updated. If not, a unique target number is assigned to the new target and a new data record is created for storage.

[0156] In order to avoid storing too much invalid or outdated target information in the target cache, a maximum cache duration threshold is set. The difference between the most recently updated frame number and the current frame number of each target in the target cache is regularly checked. If the difference exceeds the maximum cache duration threshold, the target is considered to have left the detection area and is deleted from the target cache to free up cache space and ensure the validity of the target information stored in the cache.

[0157] For each target in the newly detected frame, calculate the intersection over union (IoU) value between it and the rotated bounding boxes of all targets of the same category in the target buffer;

[0158] Set an IoU matching threshold (for example, 0.5). When the IoU value of the target in the new detection frame and a target of the same category in the target buffer is greater than or equal to the matching threshold, the two targets are judged to be the same target; when the IoU value is less than the matching threshold, or the category information is inconsistent, the target in the new detection frame is judged to be a new target;

[0159] For the case where the target is determined to be the same, the weighted average algorithm is used to update the ranging result.

[0160] The weighted average formula is:

[0161]

[0162] in D new is the updated ranging result, D current is the ranging result of the current frame, D i-1 For the previous i -1 frame of ranging results, α is the weight coefficient of the current frame ranging result, 0< α <1; This weighted averaging algorithm eliminates ranging fluctuations caused by factors such as single-frame image noise or temporary target occlusion, thereby improving the system's ranging stability and robustness.

[0163] If a target is identified as new, a new unique target number is assigned to it. A new data record is created according to the target buffer's data record structure. The new target's category, location, ranging result, and current frame number are stored in the record and added to the target buffer. This new target will then participate in target matching and ranging result updates in subsequent detection frames.

[0164] Specifically, the calculation of the intersection over union (IoU) value of the rotated bounding box includes:

[0165] First, the intersection area of ​​the two rotated bounding boxes is calculated based on their parameters; the rotated bounding boxes are converted into polygons, and then the area of ​​the intersection polygon is calculated using a polygon intersection algorithm;

[0166] Calculate the areas of the two rotated bounding boxes respectively;

[0167] The IoU value is obtained by dividing the area of ​​the intersection polygon by the sum of the areas of the two rotated bounding boxes minus the area of ​​the intersection polygon. IoU = S intersection / S box1 + S box2 - S intersection ,in S intersection is the area of ​​the intersection polygon, S box1 and S box2 are the areas of the two rotated bounding boxes respectively.

[0168] This embodiment further proposes a dynamic update mechanism based on historical detection status to improve the stability of target ranging, such as Figure 8 When the system first detects an image, it performs a confidence assessment on all identified targets and only retains targets with a confidence score above a preset threshold as valid targets for ranging. The system then caches their pixel positions, category labels, and ranging results in system memory for use as a reference for subsequent images.

[0169] When the system enters the subsequent detection cycle, it first attempts to perform target detection. If the target detection module fails to identify any targets, or the confidence level of the detection result falls below the set threshold, the system automatically calls the previously cached target information and outputs it as the recognition result for the current image. This avoids target loss due to temporary image occlusion, thereby improving ranging continuity and output stability.

[0170] When an object is detected in a new image, the system matches the newly detected object with the cached object in the previous image. To improve matching accuracy, this embodiment adopts a matching strategy based on IoU (Intersection over Union) overlap, calculating the overlap ratio between the current object and the historical object bounding box. The specific formula is as follows:

[0171]

[0172] in, B new Represents the target bounding box detected in the current image, Bold represents the target bounding box of the previous image cache, A inter is the area of ​​the intersection, A union is the area of ​​the union region.

[0173] If the IoU between the current target and the cached target is greater than or equal to the set matching threshold and the category information is consistent, they are determined to be the same target and the system updates its position and ranging information; otherwise, it is determined to be a new target, the number is reassigned and the ranging process is entered. The results of multi-target ranging are as follows: Figure 9 shown.

[0174] Through this mechanism, the present invention achieves effective target tracking and dynamic updates, boasting exceptional fault tolerance and optimized ranging continuity. It is particularly well-suited for power inspection scenarios subject to unstable factors such as occlusion, varying lighting conditions, and jittery photography. It can spatially locate multiple power equipment, providing effective distance information support for intelligent inspections, equipment control analysis, and early warning systems in high-voltage environments, and possesses broad engineering application value.

[0175] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A non-contact distance measurement method for power equipment based on rotating target detection and infrared image, characterized in that: The following steps are involved: Collect infrared images of power equipment and generate an infrared image dataset containing target information of the power equipment; Build a YOLOv5 target detection model, which is used to detect the rotation bounding box information of power equipment targets in infrared images; among them, the loss function of the YOLOv5 target detection model is L Including classification loss L cls , bounding box regression loss L reg and rotation angle prediction loss L θ ; Use the trained YOLOv5 target detection model to detect the input infrared image and obtain the rotation bounding box information of the power equipment target B =( x c , y c , w , h , θ ), determine the coordinates of the target's feature points on the image plane based on the rotation bounding box information, and calculate the spatial distance based on the distance relationship between the target feature points and the camera origin in the world coordinate system D , output the non-contact ranging result; where ( x c , y c ) is the center coordinate of the bounding box, w is the width of the bounding box, h is the height of the bounding box, θ is the rotation angle of the bounding box; Based on the historical target detection results, a dynamic update algorithm is used to correct the current target position information and ranging results; the dynamic update algorithm comprehensively considers the target detection results of the current frame and the previous n The historical detection results of the frame are smoothed by the weighted average algorithm to calculate the target position and distance value; The spatial distance is calculated based on the distance relationship between the target feature point and the camera origin in the world coordinate system. D， include: Set the intrinsic parameter matrix of the infrared camera K for: in f x and f y The camera is x Axis and y The focal length along the axis, c x and c y The principal points of the image are x Axis and y Coordinates in the axis direction; For the feature point coordinates on the image plane ( x i , y i ), perform normalization processing to obtain the normalized coordinates ( x inorm , y inorm ): x inorm =（ x i - c x ） / f x y inorm =（ y i - c y ） / f y At this time, the camera coordinate system Z ci =1, Z ci is the feature point in the camera coordinate system z Axis coordinates, the normalized coordinates ( x inorm , y inorm ,1) Approximately represents the coordinates in the camera coordinate system; for n feature points, and the coordinates of the feature points in the camera coordinate system are ( X ci , Y ci , Z ci ), satisfying the following relationship: Then we have: x i = f x u i + c x ; y i = f y v i + c y ; u i =( x i - c x ) / f x ; v i =( y i - c y ) / f y ; Establish a set of equations through the coordinates of multiple feature points and solve them using the least squares method Z ci , set up the i The equation of the characteristic point is: Will n The equations of the characteristic points are combined and written in matrix form Ax = b ,in A is the coefficient matrix, x is a vector of unknown numbers, which contains u i 、 v i and Z ci Related information, b is a constant vector, and is solved by the least squares method x ,get Z ci The value of , and then get the feature point coordinates in the camera coordinate system ( X ci , Y ci , Z ci ).

2. The method according to claim 1, characterized in that The process of preparing the infrared image dataset and preprocessing the dataset includes the following steps: Design cable terminal data collection tasks for various actual usage scenarios, including at least scenarios with different ambient temperatures, different lighting conditions, different weather conditions, and different operating states of the cable terminal; All acquired infrared images are formatted uniformly and stored in categories according to timestamp, device number and scene number; The targets in the infrared image are annotated using a rotating rectangular frame so that the rotated rectangular frame fits the target's outline. The annotation information includes the coordinates of the four vertices of the rotated target frame in the image coordinate system and the rotation angle θ. At the same time, each rotated target frame is annotated with its category information, which includes cable terminals and lightning arresters. All labeled samples are split into training set, validation set and test set according to preset ratios; among them, multiple data enhancement operations are introduced on the training set, including at least angle rotation, contrast change, mirror flipping and cropping methods.

3. The method according to claim 1, characterized in that The determining of the coordinates of feature points of the target on the image plane based on the rotation bounding box information includes: For rotated bounding box B =( x c , y c , w , h , θ ), select the four vertices of the rotation bounding box as the feature points of the target on the image plane, and set the rotation bounding box with the center point ( x c , y c ) is the center and the width direction is x Axis direction, height direction is y A local coordinate system is established in the axis direction. In the local coordinate system, the coordinates of the four vertices are P 1(- w / 2,- h / 2), P 2( w / 2,- h / 2), P 3( w / 2, h / 2) and P 4(- w / 2, h / 2); According to the rotation angle θ Rotate the vertex coordinates in the local coordinate system using the rotation matrix R θ Rotate the vertex coordinates in the local coordinate system, rotation matrix R θ The expression is: For the vertex coordinates in the local coordinate system ( x local , y local ), the rotated coordinates ( x rot , y rot ) is calculated as: The four vertices P 1. P 2. P 3. P The local coordinate transformation of 4 obtains the coordinates in the local coordinate system after rotation P 1rot ( x 1rot , y 1rot ), P 2rot ( x 2rot , y 2rot ), P 3rot ( x 3rot , y 3rot ), P 4rot ( x 4rot , y 4rot ); Convert the vertex coordinates in the rotated local coordinate system to the global coordinate system of the image plane, and obtain the coordinates of the four feature points on the image plane ( x i , y i )( i =1,2,3,4), the conversion formula is: x i = x rot + x c 、 y i = y rot + y c ; based on P 1rot 、 P 2rot 、 P 3rot 、 P 4rot The coordinates of the four feature points on the image plane are obtained by P 1img ( x 1, y 1) P 2img ( x 2, y 2) P 3img ( x 3, y 3) P 4img ( x 4, y 4).

4. The method according to claim 1, wherein: Set the external parameter matrix of the infrared camera [ R ∣ T ],in R is a 3×3 rotation matrix, T is a 3×1 translation vector; The feature point coordinates in the camera coordinate system ( X ci , Y ci , Z ci ) and the feature point coordinates in the world coordinate system ( X wi , Y wi , Z wi ) is: Convert the feature point coordinates in the camera coordinate system to the feature point coordinates in the world coordinate system; Calculate the distance between the target feature point and the camera origin in the world coordinate system D i , for the i feature points, distance D i The calculation formula is: The distance calculated for the four feature points D i ( i =1,2,3,4) and take the average value to get the final spatial distance D , the calculation formula is: Output the calculated spatial distance D As a result of contactless ranging.

5. The method according to claim 2, wherein: The loss function of the YOLOv5 target detection model L It is expressed as the following formula: L = λ 1 L cls + λ 2 L reg + λ 3 L θ in λ 1. λ 2. λ 3 is the weight coefficient; During the training process, the validation set is used to evaluate the network model in training, and the network fitting is judged according to the evaluation indicators on the validation set.

6. The method according to any one of claims 1 to 5, characterized in that: The YOLOv5 target detection model includes an input layer, a backbone network, a neck network, and a detection head. The backbone network adopts the CSPDarknet architecture to extract multi-scale image features from infrared images. An SPPF module is embedded between the backbone network and the detection head. The SPPF module consists of multiple parallel maximum pooling layers of different scales. It performs pooling operations of different scales on the input feature map and splices the pooling results. The PANet structure is introduced into the neck network. The PANet structure fuses and transfers feature maps at different levels through a top-down and bottom-up bidirectional feature fusion path. The detection head predicts the target bounding box based on feature maps of different scales. It is divided into three detection heads of different scales, which are suitable for the recognition scenarios of large, medium and small-sized targets respectively. Each detection head processes the input feature map through convolution operation, and the output includes at least the category probability of the target, the center coordinates of the rotated bounding box ( x c , y c ),width w ,high h and the rotation angle θ information, thereby achieving the positioning and classification of the target.

7. The method according to claim 6, characterized in that: In each round of training, samples are randomly drawn from the training set in batches and input into the network. The samples are first preprocessed by the network's input layer, and then pass through the backbone network, SPPF module, neck network, and detection head in sequence to obtain the output of the network encoder. Compare the network encoder output with the corresponding true label through the designed loss function L Calculate the loss value and use the back propagation algorithm to update and optimize the network parameters according to the loss value, so that the network continues to approach the optimal solution; When the last batch of the training set is extracted and training is completed, all batches in the validation set are input into the network in sequence for validation; During the verification process, the network predicts the verification set samples and obtains the prediction results; Compare the predicted results with the true labels of the validation set and calculate a series of evaluation metrics to assess the performance of the current model on unseen samples and determine whether the network is overfitting or underfitting. Repeat the training and validation steps until all samples in the training set complete the iterative update of the set cycle, completing the entire model training process and obtaining a trained YOLOv5 target detection model.

8. The method according to any one of claims 1 to 5, characterized in that: Establish a target buffer area. After each target detection is completed and the ranging result is calculated, the target category, location information, ranging result and current frame number are recorded in the target buffer area. For each target in the newly detected frame, calculate the intersection over union (IoU) value between it and the rotated bounding boxes of all targets of the same category in the target buffer; Set an IoU matching threshold. When the IoU value of the target in the new detection frame and a target of the same category in the target buffer is greater than or equal to the matching threshold, the two targets are judged to be the same target. When the IoU value is less than the matching threshold, or the category information is inconsistent, the target in the new detection frame is judged to be a new target. For the case where the target is determined to be the same, the weighted average algorithm is used to update the ranging result.

9. The method according to claim 8, characterized in that: The calculation of the intersection over union (IoU) value of the rotated bounding box includes: First, the intersection area of ​​the two rotated bounding boxes is calculated based on their parameters; the rotated bounding boxes are converted into polygons, and then the area of ​​the intersection polygon is calculated using a polygon intersection algorithm; Calculate the areas of the two rotated bounding boxes respectively; The IoU value is obtained by dividing the area of ​​the intersection polygon by the sum of the areas of the two rotated bounding boxes minus the area of ​​the intersection polygon. IoU = S intersection / S box1 + S box2 - S intersection ,in S intersection is the area of ​​the intersection polygon, S box1 and S box2 are the areas of the two rotated bounding boxes respectively.

Images