Safe distance monitoring method and system based on deep learning and perspective transformation

Abstract

The invention provides a safety distance monitoring method and system based on deep learning and perspective transformation, and relates to the technical field of substation safety monitoring. The method comprises the following steps: shooting transformer substation workers and marking shot images, and constructing a transformer substation worker foot key point detection data set; carrying out lightweight improvement on a YOLOv8 key point detection model, respectively adding a proposed DFF module and an ACC module to a backbone and a neck, and replacing an original PAN structure of a network with a BiFPN structure; performing perspective transformation on a substation scene based on a camera imaging principle to obtain distance information in the scene; and finally, based on the detected key points and the obtained mapping relation, completing the detection of the distance between the worker and the electrified equipment. According to the invention, an effective active monitoring means is established in a substation scene, so that early warning can be provided for workers, substation managers can be helped to take measures for timely intervention, and occurrence of electric shock accidents of the workers is effectively prevented.

Description

Technical Field

[0001] The present invention relates to the technical field of substation safety monitoring, and in particular to a safety distance monitoring method and system based on deep learning and perspective transformation. Background Art

[0002] At present, substations have implemented a number of safety measures to prevent electric shock accidents, such as setting up fences to isolate live parts from relevant personnel, adding signs for reminders, specifying safety regulations and conducting training, etc., to prevent relevant personnel from contacting live equipment, especially live parts of live equipment. These preventive measures can indeed reduce the occurrence of electric shock accidents to a certain extent, but in actual operation, they are easily affected by the subjective factors of relevant personnel and may fail due to negligence, illegal operation, etc.

[0003] In the prior art, the distance measurement methods in the power scene mainly include manual measurement methods, distance measurement methods based on laser radar and distance measurement methods based on vision. The distance measurement methods based on vision include distance measurement methods based on binocular vision and distance measurement methods based on monocular vision. First, manual measurement is easily affected by the accuracy of the measuring tool and subjective factors, and its real-time and accuracy are still poor. Second, laser radar achieves distance measurement by emitting laser pulses and measuring the time when the pulses return. It has the advantages of high precision, long-distance measurement capability, and is not affected by lighting conditions. The distance between any two points in space can be calculated through the acquired point cloud data. However, laser radar is relatively expensive, the calculation cost is also high, and it is not suitable for real-time monitoring and deployment. Third, the distance measurement method based on binocular vision obtains the depth value of each pixel on the image through the principle of triangulation to achieve distance measurement. Its cost is lower than that of laser radar. However, binocular distance measurement requires matching the pixels in the two images, which requires a large amount of calculation. Lack of texture or extreme lighting conditions may lead to reduced performance.

[0004] Deep learning has developed rapidly in recent years. Even for a single image, the image depth can be predicted through monocular depth estimation. However, data-driven methods such as monocular depth estimation are highly dependent on data and may lack the ability to generalize to new and complex scenes. In the field of monocular depth estimation, most public data sets are for various applications, especially autonomous driving. There is no depth estimation data set for substation scenes. In addition, monocular depth estimation is easily affected by camera posture and shadows in the picture. The large demand for depth estimation data sets and the lack of generalization ability have become huge challenges for monocular depth estimation. Therefore, a high-precision, low-cost, and strong real-time ranging method is needed to achieve safe distance monitoring of substations. Summary of the invention

[0005] The main purpose of the present invention is to provide a safety distance monitoring method and system based on deep learning and perspective transformation, so as to solve the current technical problems of low safety distance detection accuracy, high detection cost and relatively delayed detection due to the large demand for depth estimation data sets and insufficient generalization ability.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a safety distance monitoring method based on deep learning and perspective transformation, comprising the following steps: S1: Obtain images of relevant personnel in real-time preset scenes, and after data preprocessing, input them into the constructed detection model to obtain key point data of the feet; S2: Based on the camera imaging model, a homography matrix for perspective transformation under a preset shooting scene is calculated, and a two-dimensional mapping relationship between pixel coordinates of the image and world coordinates is obtained; S3: Calculate and obtain the real-time distance according to the foot key point data and the two-dimensional mapping relationship; S4: Compare the preset safety threshold with the real-time distance to obtain a detection result; The detection model is based on the YOLOv8 key point detection model. The DFF module is used in the backbone network, and the ACC module in the neck network is spliced ​​for feature extraction. The BiFPN module is then used to fuse the features of the image, so as to output the model and obtain the key point data of the foot.

[0007] In a preferred embodiment, training the detection model comprises the following steps: Obtaining a data set of key point detection of feet of relevant persons, dividing the data set into a training set and a test set, and performing data preprocessing; Set the number of iterations for training; Inputting the training set after data preprocessing into the detection model for iterative training until the number of iterative training times is reached, and saving the optimal detection model; The iterative training includes calculating the loss function of the detection model and back-propagating to optimize the detection model.

[0008] In a preferred embodiment, training the detection model includes: The number of iterative training is set to 128, and the optimizer is set to the AdamW optimizer with momentum of 0.9 and weight decay of 0.001; the initial learning rate is 0.001, the final learning rate is 0.00001, and the warm-up strategy is used for adjustment.

[0009] In a preferred embodiment, the data preprocessing includes: Screening the images of the key point detection data set to remove blurred images; Annotate the image and convert the annotation file into a unified format; Then perform enhancement, rotation and translation to expand the image; A benchmark dataset is adopted, and images containing preset categories are randomly extracted and re-labeled to construct a foot key point detection dataset.

[0010] In the preferred solution, the DFF module is used in the backbone network, specifically: the original C2f module is replaced with the DFF module in the YOLOv8 model backbone network, the DFF module splits the Bottleneck into two CBS modules, saves the output of each CBS module, and splices the outputs of all CBS modules; The ACC module is used in the neck network. Specifically, the original C2f module is replaced with the ACC module in the neck network of the YOLOv8 model. For different inputs, the ACC module first obtains the weights of each channel through the channel attention mechanism, then sums the weights of the first half of the channel with the weights of the second half of the channel, and finally compares the sizes of the two and selects some channels with larger weights for convolution to achieve feature extraction. The BiFPN module is used, specifically: the original PAN structure is replaced by the BiFPN module. For a single input node, the BiFPN module first deletes the node with only one input edge, and then introduces jump connections for nodes at the same level; at the same time, for different inputs, the BiFPN module adopts a dynamic weight mechanism.

[0011] In the preferred embodiment, in the DFF module, multi-scale convolution is combined with depthwise separable convolution to obtain the MDConv module. Specifically, the MDConv module divides the channels into four groups and performs depthwise separable convolution of different scales on the four groups of channels.

[0012] In the preferred solution, in the ACC module, the ECA module is used to obtain the weights of the two channels. Specifically, the ECA module first performs global average pooling on the input channels, then uses one-dimensional convolution for cross-channel interaction and obtains channel weights, and finally uses the Sigmoid function to perform nonlinear transformation on the weights and weight the channels.

[0013] In a preferred embodiment, S2 comprises the following steps: S21: Use a chessboard of preset size to take multiple images, extract corner points for analysis and calculation, determine the internal and external parameters of the camera, use the distortion correction algorithm for calibration, obtain accurate images, and normalize the points on the plane with the distortion points. The formula is: ; ; In the formula, (x, y, 1) is a point on the plane, ( x dis , y dis , 1) is the distortion point, are the horizontal and vertical coordinates of the pixel point in the ideal pixel coordinate system, They are the horizontal and vertical coordinates of the pixel point in the actual pixel coordinate system due to distortion. , are two tangential distortion parameters, is the radial distance, , , They are three radial distortion parameters; S22: Select a reference rectangle of a preset size in the image, obtain pixel coordinates of the four corner points of the reference rectangle, and restore the pixel coordinates in proportion; obtain a homography matrix according to the actual pixel coordinates of the reference rectangle in the image and the restored pixel coordinates.

[0014] In a preferred embodiment, S3 comprises the following steps: S31: Using the actual side length of the reference rectangle, after perspective transformation, obtain the pixel length occupied by the actual side length in the image, and then obtain the actual distance represented by each pixel, the formula is: ; In the formula, l a is the actual side length of the reference rectangle, l p is the pixel length of the actual side length in the image; A rectangular range is used to represent the overhead view of the live equipment area, and the straight line can be obtained based on its pixel coordinates l The equation is: ; In the formula, are the real coefficients of the general equation of a straight line; The distance between the relevant personnel and the area with energized equipment d e It can be expressed as: ; In the formula, They are the projection components of the point coordinates on the horizontal and vertical vectors of the straight line respectively; S32: Preset a rectangular reference object in the actual scene and capture an image, obtain the horizontal and vertical vanishing points in the image according to the pixel coordinates of the rectangular reference object in the captured image, select multiple horizontal vanishing points to calculate the horizontal vanishing line; obtain the height scale factor according to the reference rectangle, the formula is: ; In the formula, α represents the proportional factor obtained by selecting a reference object; b p and t p Respectively represent the pixel coordinates of the bottom and top of the relevant person in the pixel coordinate system; l and v They represent the horizontal vanishing line and the vertical vanishing point in the pixel coordinate system respectively; S33: Get the distance between the relevant personnel and the charged point, the formula is: ; In the formula, d p is the distance between the relevant personnel and the live point, is the coordinate of the charged point in the world coordinate system, It is the coordinates of the substation related personnel in the world coordinate system.

[0015] In a preferred embodiment, S4 comprises the following steps: If the distance between the relevant personnel and the live point is less than the preset safety threshold, it is a safe distance; Otherwise, it is an unsafe distance and the corresponding warning operation is initiated.

[0016] A safety distance monitoring system based on deep learning and perspective transformation, comprising: The data acquisition module is used to obtain the real-time images of relevant personnel in the preset scene, and after data preprocessing, input them into the built detection model to obtain the key point data of the feet; The detection model is based on the YOLOv8 key point detection model. The DFF module is used in the backbone network, and then the ACC module in the neck network is spliced ​​for feature extraction. The BiFPN module is then used to fuse the image features, so that the model output can obtain the key point data of the foot. A two-dimensional mapping module, used to calculate a homography matrix for perspective transformation under a preset shooting scene based on a camera imaging model, and obtain a two-dimensional mapping relationship between pixel coordinates of the image and world coordinates; A distance calculation module is used to calculate and obtain the real-time distance according to the foot key point data and the two-dimensional mapping relationship; The detection and warning module is used to compare the preset safety threshold with the real-time distance to obtain the detection result.

[0017] The present invention provides a safety distance monitoring method and system based on deep learning and perspective transformation, adding DFF module and ACC module to the backbone and neck respectively, adopting BiFPN structure, and performing feature fusion; performing perspective transformation on the substation scene based on the camera imaging principle, obtaining the distance information in the scene, and obtaining the distance between the relevant personnel and the live equipment according to the detected key points and the two-dimensional mapping relationship. The present invention uses a deep learning model to accurately extract the key point data of the foot, and combines the perspective transformation technology to map the pixel coordinates to the actual world coordinates, so as to effectively calculate the real-time distance, obtain the detection results, improve the efficiency and accuracy of safety management in the scene, establish an effective active monitoring method, and judge whether there is danger according to the position information of the relevant personnel of the substation in the world coordinate system and give a timely warning, which can not only provide a warning for the personnel, but also make timely intervention, improve personal safety, and reduce the risk of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a structural comparison diagram of the DFF module of the present invention and C2f; Figure 3 ACC module structure diagram of the present invention; Figure 4 This is the structure diagram of the ECA attention mechanism of the present invention; Figure 5 Schematic diagram of the BiFPN structure of the present invention; Figure 6 This is a schematic diagram of the loss function for model training of the present invention; Figure 7 It is a schematic diagram of the belt calibration plate of the present invention; Figure 8 This is a schematic diagram of the corner point detection result of the calibration plate of the present invention; Fig. 9 This is a schematic diagram of the relative positions of the calibration plate and the camera of the present invention; Fig.10 This is a schematic diagram of the calibration reprojection error result of the present invention; Fig.11 It is a schematic diagram of the distance monitoring results between personnel and live equipment related to the present invention. DETAILED DESCRIPTION

[0019] Example 1 like Figure 1-11As shown, a safety distance monitoring method based on deep learning and perspective transformation includes the following steps: S1: Obtain real-time images of relevant personnel in a preset scene, and after data preprocessing, input them into the constructed detection model to obtain the key point data of the feet.

[0020] S2: Based on the camera imaging model, the homography matrix for perspective transformation under the preset shooting scene is calculated, and the two-dimensional mapping relationship between the pixel coordinates of the image and the world coordinates is obtained.

[0021] S3: Calculate and obtain the real-time distance based on the foot key point data and the two-dimensional mapping relationship.

[0022] S4: Compare the preset safety threshold with the real-time distance to obtain a detection result.

[0023] The detection model is based on the YOLOv8 key point detection model. The DFF module is used in the backbone network, and the ACC module in the neck network is spliced ​​for feature extraction. The BiFPN module is then used to fuse the image features, so as to output the model and obtain the key point data of the foot.

[0024] This embodiment constructs a data set for detecting key points of the feet of substation-related personnel by photographing them and annotating the photographed images. Adaptive changes can be made according to actual needs. The selection principle is: select the key point of the foot that is closest to the live equipment as the ranging point. For example, the entire human body can be regarded as a key point.

[0025] This embodiment makes lightweight improvements to the YOLOv8 key point detection model, adds the DFF module and ACC module to the backbone and neck respectively, adopts the BiFPN structure, and performs feature fusion; based on the camera imaging principle, the substation scene is perspective transformed to obtain the distance information in the scene, and the distance between the relevant personnel and the live equipment is obtained according to the detected key points and the two-dimensional mapping relationship. The present invention uses a deep learning model to accurately extract the key point data of the foot, and combines the perspective transformation technology to map the pixel coordinates to the actual world coordinates, so as to effectively calculate the real-time distance, improve the efficiency and accuracy of safety management in the scene, establish an effective active monitoring method, and judge whether there is danger according to the position information of the relevant personnel of the substation in the world coordinate system and give a timely warning, which can not only provide warnings for personnel, but also make timely interventions, improve personal safety, reduce the risk of accidents, and reduce the cost of detection.

[0026] First, in order to obtain real-time images of relevant personnel in preset scenarios, data preprocessing is performed to ensure the accuracy of image data.

[0027] In the preferred solution, data preprocessing includes: The images of the key point detection dataset are screened to remove blurred images.

[0028] Annotate the image and convert the annotation file into a unified format.

[0029] Then enhance, rotate and translate to expand the image.

[0030] A benchmark dataset is adopted, and images containing preset categories are randomly extracted and re-labeled to construct a foot key point detection dataset.

[0031] The benchmark database in this embodiment comes from the public dataset on the Internet: COCO dataset, from which images of the person class are selected and re-labeled. The dataset used in this embodiment is divided into two parts, one is the substation work pictures taken on site, and the other is the pictures of ordinary people in the COCO dataset.

[0032] In this embodiment, in different substation scenarios, images of substation-related personnel are captured by monitoring cameras or handheld cameras in the substation; the collected transmission line channel images are screened to remove blurred images; the images are annotated using the Labelme image annotation tool and the annotation files are converted into a unified format, and the YOLO format can be selected. The images of substation-related personnel are expanded through operations including enhancement, rotation, and translation; images containing preset categories, such as person categories, are randomly extracted from the benchmark data set and re-annotated to jointly complete the construction of the key point detection data set of the feet of substation-related personnel.

[0033] In this embodiment, optimization is performed based on the YOLOv8 key point detection model.

[0034] In the preferred embodiment, Figure 2 As shown in the figure, the original C2f module is replaced by the proposed DFF module in the YOLOv8 model backbone network. The DFF module splits the Bottleneck into two CBS modules, saves the output of each CBS module, and finally splices the outputs of all CBS modules. Since a denser feature map is obtained, it also means a large increase in the number of channels, thereby increasing the number of parameters of the next convolution layer. In order to reduce the number of model parameters, a multi-scale depth-wise separable convolution is proposed by combining multi-scale convolution with depth-wise separable convolution. The use of this convolution can greatly reduce the number of network parameters while also achieving better detection results.

[0035] In the neck network of the YOLOv8 model, the original C2f module is replaced with the proposed ACC module. The specific steps of building the ACC module are as follows: In order to ensure the lightweight of the network, this paper selects the ECA attention mechanism with fewer parameters to obtain the weights of the two channels; for different inputs, the ACC module first obtains the weights of each channel through the channel attention mechanism; then, the weights of the first half of the channels are summed with the weights of the second half of the channels, and the sum represents the overall weight of the two channels; finally, the sizes of the two are compared and some channels with larger weights are selected for convolution; therefore, the ACC module not only uses local convolution to ensure the lightweight structure of the network, but also can select channels carrying more information for convolution according to different inputs, thereby achieving better feature extraction.

[0036] In the YOLOv8 model, the original PAN structure is replaced with the BiFPN module. The specific steps to build the BiFPN structure are as follows: like Figure 5 As shown in the figure, BiFPN is an improvement on PAN, which mainly solves the problem of unbalanced information propagation between different scales of PAN. Since a single input node contributes less to feature fusion, BiFPN first deletes the nodes with only one input edge to reduce the number of parameters. Secondly, skip connections are introduced to nodes at the same level to better utilize the features of different input layers. At the same time, since different inputs have different resolutions, they usually contribute unequally to the output features. To solve this problem, BiFPN adjusts the importance of feature maps by introducing a dynamic weight mechanism. This dynamic weight mechanism allows the network to adaptively adjust the weights according to the input of each node to better distribute the contribution of feature maps at different resolutions.

[0037] In the preferred embodiment, in the DFF module, multi-scale convolution is combined with depthwise separable convolution to obtain the MDConv module. Specifically, the MDConv module divides the channels into four groups and performs depthwise separable convolution of different scales on the four groups of channels.

[0038] like Figure 2 As shown, in this embodiment, the depthwise separable convolutions of different scales have convolution kernel sizes of 1, 3, 5 and 7 respectively.

[0039] Furthermore, according to the research on lightweight networks ShuffleNet and MobileNet, it is shown that the use of group convolution and depthwise separable convolution can achieve good detection results while greatly reducing the number of network parameters. Based on this, this paper combines multi-scale convolution with depthwise separable convolution and proposes MDConv. The depthwise separable convolution in MDConv can effectively cope with the large increase in the number of channels, while multi-scale convolution is mainly used to detect targets of different sizes. In the substation, the model can meet the needs of detecting relatively large-sized personnel and relatively small-sized foot key points at the same time. Figure 2 As shown in the figure, MDConv divides the channels into four groups, and performs depth-wise separable convolutions of different scales with kernel sizes of 1, 3, 5, and 7 on these four groups of channels.

[0040] In the preferred embodiment, Figure 3 As shown in the figure, in the ACC module, the ECA module is used to obtain the weights of the two channels. Specifically, the ECA module first performs global average pooling on the input channels, then uses one-dimensional convolution for cross-channel interaction and obtains channel weights, and finally uses the Sigmoid function to perform nonlinear transformation on the weights and weight the channels.

[0041] like Figure 4 As shown in the figure, in order to ensure the lightweight of the network, this paper selects the ECA module to obtain the weights of the two channels. The ECA module first performs global average pooling on the input channels, and then uses one-dimensional convolution to interact across channels and obtain channel weights, thereby greatly reducing the number of parameters. At the same time, facing different numbers of channels, it can adaptively select convolution kernels to ensure its efficiency and effectiveness. Finally, the Sigmoid function is used to perform nonlinear transformation on the weights and weight the channels.

[0042] In order to ensure better detection results, the detection model is trained.

[0043] In the preferred embodiment, training the detection model includes the following steps: Obtain a data set of key point detection of the feet of relevant persons, divide the data set into a training set and a test set, and perform data preprocessing.

[0044] Set the number of training iterations.

[0045] The training set after data preprocessing is input into the detection model for iterative training until the number of iterative training times is reached and the optimal detection model is saved.

[0046] Among them, iterative training includes calculating the loss function of the detection model and back-propagating to optimize the detection model.

[0047] like Figure 6 As shown, it is a schematic diagram of the model training loss function to obtain the optimal detection model. In the model training of this embodiment, each parameter needs to be set accordingly.

[0048] In the preferred embodiment, training the detection model includes: The number of iterative training is set to 128, and the optimizer is set to the AdamW optimizer with momentum of 0.9 and weight decay of 0.001; the initial learning rate is 0.001, the final learning rate is 0.00001, and the warm-up strategy is used for adjustment.

[0049] In this embodiment, the training set and the test set can be used in a ratio of 8:2. The training set is used for training, the test set is used to verify the model, and the optimal model is retained. The optimal model is then used to monitor the safe distance.

[0050] In this embodiment, the distortion of the camera will cause the pixel position in the image to change, affecting the accuracy and precision of the image; the internal and external parameters of the camera are obtained through camera calibration to accurately measure the size and position of the object in image processing and computer vision tasks; the internal and external parameters of the camera are determined by taking multiple images using a chessboard of known size, and extracting corner points from these images for analysis and calculation; after the parameters are determined, the distortion effect is removed from the image using a corresponding distortion correction algorithm, thereby obtaining an accurate image.

[0051] In a preferred embodiment, step S2 comprises the following steps: S21: Use a chessboard of preset size to take multiple images, extract corner points for analysis and calculation, determine the internal and external parameters of the camera, use the distortion correction algorithm for calibration, obtain accurate images, and normalize the points on the plane with the distortion points. The formula is: ; ; In the formula, ( x, y, 1) is a point on the plane, ( x dis , y dis , 1) is the distortion point, are the horizontal and vertical coordinates of the pixel point in the ideal pixel coordinate system, They are the horizontal and vertical coordinates of the pixel point in the actual pixel coordinate system due to distortion. , are two tangential distortion parameters, is the radial distance, , , They are three radial distortion parameters respectively.

[0052] S22: Select a reference rectangle of a preset size in the image, obtain pixel coordinates of the four corner points of the reference rectangle, and restore the pixel coordinates in proportion; obtain a homography matrix according to the actual pixel coordinates of the reference rectangle in the image and the restored pixel coordinates.

[0053] Furthermore, this paper uses a checkerboard calibration plate made of float glass, such as Figure 7 As shown in the figure, each corner point of the chessboard on the calibration board is detected, and the detection results are as follows: Figure 8 As shown; Fig. 9 It shows the relative position of the calibration plate and the camera. Fig.10 The reprojection error of the calibration results is shown.

[0054] Furthermore, the imaging of the camera is essentially to map points in the world coordinate system to the pixel coordinate system. The calculation process is represented by a matrix, which includes a series of transformation processes such as rotation and translation. At the same time, for an image that has been taken, the image can be processed through perspective transformation to achieve image reconstruction.

[0055] In the preferred embodiment, S3 comprises the following steps: S31: Assume that the actual side length of the rectangular reference object is l a , the pixel length of the image after perspective transformation l p , then the actual distance represented by each pixel is: ; In the formula, l a is the actual side length of the reference rectangle, l p It is the actual side length in pixels of the image.

[0056] A rectangular range is used to represent the overhead view of the live equipment area, and the straight line can be obtained based on its pixel coordinates l The equation is: ; In the formula, are the real coefficients of the general equation of a straight line.

[0057] The distance between the relevant personnel and the area with energized equipment d e It can be expressed as: ; In the formula, They are the projection components of the point coordinates on the horizontal and vertical vectors of the straight line respectively.

[0058] S32: Select a reference object to obtain the shadow vanishing points in the horizontal and vertical directions of the image; select multiple shadow vanishing points in the horizontal direction to calculate the shadow vanishing line in the horizontal direction; obtain the height scale factor according to the reference object; the height calculation formula of the relevant personnel is: ; In the formula, α represents the proportional factor obtained by selecting a reference object; b p andt p Respectively represent the pixel coordinates of the bottom and top of the relevant person in the pixel coordinate system; l and v They represent the horizontal vanishing line and the vertical vanishing point in the pixel coordinate system respectively.

[0059] S33: Get the distance between the relevant personnel and the charged point, the formula is: ; In the formula, d p is the distance between the relevant personnel and the live point, ( x e , y e , z e ) is the coordinate of the charged point in the world coordinate system, ( x p , y p , z p ) are the coordinates of the substation-related personnel in the world coordinate system.

[0060] The above steps S31-S33 are used to obtain the size and distance information in the real world from the image. In this embodiment, the preset safety threshold is .

[0061] In the preferred embodiment, S4 comprises the following steps: like , then it is the safe distance; Otherwise, it is an unsafe distance and the corresponding warning operation is initiated.

[0062] like Fig.11 As shown, based on the distance measured above, it is determined whether there is danger and provides an early warning. d p , according to the safety threshold specified in the relevant standards for substations , judge the real-time distance between different voltage levels and live equipment, and promptly warn relevant personnel if the real-time distance is insufficient.

[0063] In use, this embodiment combines deep learning with camera imaging principles to solve the problems of safe distance measurement and monitoring and early warning of live equipment in substations; trains key point detection models to realize the positioning of relevant personnel in substations; calibrates the substation scene to obtain its homography matrix of perspective transformation; combines the pixel coordinates detected by the key point detection model with the homography matrix of perspective transformation to restore the position information of relevant personnel in the substation in the world coordinate system; based on the position information of relevant personnel in the substation in the world coordinate system, determines whether there is danger and issues early warning in time.

[0064] Example 2 In combination with Example 1, a safety distance monitoring system based on deep learning and perspective transformation is proposed, which adopts a safety distance monitoring method based on deep learning and perspective transformation in Example 1, including: The data acquisition module is used to obtain real-time images of relevant personnel in preset scenarios, and after data preprocessing, input them into the constructed detection model to obtain the key point data of the feet.

[0065] The detection model is based on the YOLOv8 key point detection model. The DFF module is used in the backbone network, and the ACC module in the neck network is spliced ​​for feature extraction. The BiFPN module is then used to fuse the image features, so as to output the model and obtain the key point data of the foot.

[0066] The two-dimensional mapping module is used to calculate the homography matrix for perspective transformation under the preset shooting scene based on the camera imaging model, and obtain the two-dimensional mapping relationship between the pixel coordinates of the image and the world coordinates.

[0067] The distance calculation module is used to calculate and obtain the real-time distance based on the key point data of the foot and the two-dimensional mapping relationship; The detection and warning module is used to compare the preset safety threshold with the real-time distance to obtain the detection result.

[0068] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A safety distance monitoring method based on deep learning and perspective transformation, characterized in that: The following steps are involved: S1: Obtain images of relevant personnel in real-time preset scenes, and after data preprocessing, input them into the constructed detection model to obtain key point data of the feet; S2: Based on the camera imaging model, a homography matrix for perspective transformation under a preset shooting scene is calculated, and a two-dimensional mapping relationship between pixel coordinates of the image and world coordinates is obtained; S3: Calculate and obtain the real-time distance according to the foot key point data and the two-dimensional mapping relationship; S4: Compare the preset safety threshold with the real-time distance to obtain a detection result; The detection model is based on the YOLOv8 key point detection model. The DFF module is used in the backbone network, and the ACC module in the neck network is spliced ​​for feature extraction. The BiFPN module is then used to fuse the features of the image, so as to output the model and obtain the key point data of the foot.

2. The safety distance monitoring method based on deep learning and perspective transformation according to claim 1 is characterized in that: Training the detection model comprises the following steps: Obtaining a data set of key point detection of feet of relevant persons, dividing the data set into a training set and a test set, and performing data preprocessing; Set the number of iterations for training; Inputting the training set after data preprocessing into the detection model for iterative training until the number of iterative training times is reached, and saving the optimal detection model; The iterative training includes calculating the loss function of the detection model and back-propagating to optimize the detection model.

3. The safety distance monitoring method based on deep learning and perspective transformation according to claim 2 is characterized in that: Training the detection model includes: The number of iterative training is set to 128, and the optimizer is set to the AdamW optimizer with momentum of 0.9 and weight decay of 0.001; the initial learning rate is 0.001, the final learning rate is 0.00001, and the warm-up strategy is used for adjustment.

4. The safety distance monitoring method based on deep learning and perspective transformation according to claim 2 is characterized in that: The data preprocessing includes: Screening the images of the key point detection data set to remove blurred images; Annotate the image and convert the annotation file into a unified format; Then perform enhancement, rotation and translation to expand the image; A benchmark dataset is adopted, and images containing preset categories are randomly extracted and re-labeled to construct a foot key point detection dataset.

5. The safety distance monitoring method based on deep learning and perspective transformation according to claim 1 is characterized in that: The DFF module is used in the backbone network. Specifically, the original C2f module is replaced with the DFF module in the YOLOv8 model backbone network. The DFF module splits Bottleneck into two CBS modules, saves the output of each CBS module, and concatenates the outputs of all CBS modules. The ACC module is used in the neck network. Specifically, the original C2f module is replaced with the ACC module in the neck network of the YOLOv8 model. For different inputs, the ACC module first obtains the weights of each channel through the channel attention mechanism, then sums the weights of the first half of the channel with the weights of the second half of the channel, and finally compares the sizes of the two and selects some channels with larger weights for convolution to achieve feature extraction. The BiFPN module is used, specifically: the original PAN structure is replaced by the BiFPN module. For a single input node, the BiFPN module first deletes the node with only one input edge, and then introduces jump connections for nodes at the same level; at the same time, for different inputs, the BiFPN module adopts a dynamic weight mechanism.

6. The safety distance monitoring method based on deep learning and perspective transformation according to claim 4 is characterized in that: In the DFF module, multi-scale convolution is combined with depthwise separable convolution to obtain the MDConv module. Specifically, the MDConv module divides the channels into four groups and performs depthwise separable convolution of different scales on the four groups of channels.

7. The safety distance monitoring method based on deep learning and perspective transformation according to claim 4 is characterized in that: In the ACC module, the ECA module is used to obtain the weights of the two channels. Specifically, the ECA module first performs global average pooling on the input channels, then uses one-dimensional convolution for cross-channel interaction and obtains channel weights, and finally uses the Sigmoid function to perform nonlinear transformation on the weights and weight the channels.

8. The safety distance monitoring method based on deep learning and perspective transformation according to claim 1 is characterized in that: The S2 comprises the following steps: S21: Use a chessboard of preset size to take multiple images, extract corner points for analysis and calculation, determine the internal and external parameters of the camera, use the distortion correction algorithm for calibration, obtain accurate images, and normalize the points on the plane with the distortion points. The formula is: ； ； In the formula, ( x, y, 1) is a point on the plane, ( x dis , y dis , 1) is the distortion point, are the horizontal and vertical coordinates of the pixel point in the ideal pixel coordinate system, They are the horizontal and vertical coordinates of the pixel point in the actual pixel coordinate system due to distortion. , are two tangential distortion parameters, is the radial distance, , , They are three radial distortion parameters; S22: Select a reference rectangle of a preset size in the image, obtain pixel coordinates of the four corner points of the reference rectangle, and restore the pixel coordinates in proportion; obtain a homography matrix according to the actual pixel coordinates of the reference rectangle in the image and the restored pixel coordinates.

9. The safety distance monitoring method based on deep learning and perspective transformation according to claim 8 is characterized in that: The S3 comprises the following steps: S31: Using the actual side length of the reference rectangle, after perspective transformation, obtain the pixel length occupied by the actual side length in the image, and then obtain the actual distance represented by each pixel, the formula is: ； In the formula, l a is the actual side length of the reference rectangle, l p is the pixel length of the actual side length in the image; A rectangular range is used to represent the overhead view of the live equipment area, and the straight line can be obtained based on its pixel coordinates l The equation is: ； In the formula, are the real coefficients of the general equation of a straight line; The distance between the location of relevant personnel and the area with energized equipment d e It can be expressed as: ； In the formula, They are the projection components of the point coordinates on the horizontal and vertical vectors of the straight line respectively; S32: Preset a rectangular reference object in the actual scene and capture an image, obtain the horizontal and vertical vanishing points in the image according to the pixel coordinates of the rectangular reference object in the captured image, select multiple horizontal vanishing points to calculate the horizontal vanishing line; obtain the height scale factor according to the reference rectangle, the formula is: ； In the formula, α represents the proportional factor obtained by selecting a reference object; b p and t p Respectively represent the pixel coordinates of the bottom and top of the relevant person in the pixel coordinate system; l and v They represent the horizontal vanishing line and the vertical vanishing point in the pixel coordinate system respectively; S33: Get the distance between the relevant personnel and the charged point, the formula is: ； In the formula, d p is the distance between the relevant personnel and the live point, is the coordinate of the charged point in the world coordinate system, It is the coordinates of the substation related personnel in the world coordinate system.

10. The safety distance monitoring method based on deep learning and perspective transformation according to any one of claims 1 to 9, characterized in that: The S4 comprises the following steps: If the distance between the relevant personnel and the live point is less than the preset safety threshold, it is a safe distance; Otherwise, it is an unsafe distance and the corresponding warning operation is initiated.

11. A safety distance monitoring system based on deep learning and perspective transformation, characterized in that: include: The data acquisition module is used to obtain the real-time images of relevant personnel in the preset scene, and after data preprocessing, input them into the built detection model to obtain the key point data of the feet; The detection model is based on the YOLOv8 key point detection model. The DFF module is used in the backbone network, and then the ACC module in the neck network is spliced ​​for feature extraction. The BiFPN module is then used to fuse the image features, so that the model output can obtain the key point data of the foot. A two-dimensional mapping module, used to calculate a homography matrix for perspective transformation under a preset shooting scene based on a camera imaging model, and obtain a two-dimensional mapping relationship between pixel coordinates of the image and world coordinates; A distance calculation module, used to calculate and obtain the real-time distance according to the foot key point data and the two-dimensional mapping relationship; The detection and warning module is used to compare the preset safety threshold with the real-time distance to obtain the detection result.

Images