Electric power high-altitude operation peccancy detection method
Through backlight image enhancement and cross-modal data fusion, combined with the LightenNet model and Kalman filter algorithm, the problem of detection failure caused by backlighting and occlusion in the power high-altitude operation monitoring system is solved, and the detection accuracy and reliability are improved.
Patent Information
- Application Number
- CN202510759987.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-09
- Publication Date
- 2025-09-19
AI Technical Summary
The existing power high-altitude work monitoring system has problems such as loss of visual features of protective equipment due to backlit shooting, difficulty in balancing dynamic range expansion and detail retention, insufficient feature representation of single visual modality detection, detection failure caused by occlusion, and small sample size. Traditional methods are computationally complex and rely on large-scale labeled data.
Through backlight image enhancement, cross-modal data fusion detection and occlusion compensation, the LightenNet model and Kalman filter algorithm are used to improve image quality, and the PAC-APN module is used to achieve multi-scale semantic matching and fusion, perform violation detection and compensate in the case of occlusion.
It effectively improves the detection accuracy of safety protection equipment under complex working conditions, solves the problem of detection failure caused by backlighting and occlusion, and realizes efficient violation detection.
Smart Images
Figure CN120673333A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power system safety monitoring, and in particular to a method for detecting violations of power high-altitude operations. Background Art
[0002] Existing monitoring systems for high-altitude power work are limited by the mounting angle of dome cameras, resulting in severe backlit shooting and loss of visual features of protective equipment. Traditional solutions have significant drawbacks: histogram equalization methods cause color distortion and noise amplification; multi-exposure fusion algorithms are prone to halo artifacts in areas with significant brightness differences; deep learning-based enhancement networks rely on large amounts of annotated data and are computationally complex; and existing methods generally suffer from technical bottlenecks that make it difficult to balance dynamic range expansion with detail preservation.
[0003] Since working at heights is highly dangerous, violation samples are relatively scarce, resulting in a small sample size problem. In addition, due to the complex working environment, single visual modality detection faces the problem of insufficient feature representation. Existing detection systems have two flaws: insufficient text-visual cross-modal association modeling, unable to effectively utilize the text information of the operating procedures to enhance detection features; traditional feature fusion strategies ignore multi-scale semantic matching, leading to semantic gap problems in complex scenarios. Occlusion of protective equipment caused by the limb movements of workers leads to frequent failures of horizontal protection detection. Current solutions have obvious limitations: traditional Kalman filtering suffers from severe accuracy degradation in nonlinear motion prediction and has a blind spot problem with size changes. Summary of the Invention
[0004] The purpose of the embodiment of the present invention is to provide a method for detecting violations in high-altitude power operations. The present invention effectively improves the detection accuracy of safety protection equipment under complex working conditions through backlight image enhancement, cross-modal data fusion detection and occlusion compensation.
[0005] In order to achieve the above object, the present invention provides a method for detecting violations of electrical high-altitude operations, comprising: Acquire a visual stream, and acquire an image to be detected according to the visual stream; Acquire a backlight image in the image and extract a dark area in the backlight image ; The dark area obtained Perform multi-scale transformation to enhance dark areas Dynamic range; The dark area after transformation Input the LightenNet model and generate the corresponding lighting map; The dark area Combined with the light map, an enhanced ; The non-backlit image and the enhanced The image set after is the first image; Determining whether the first image is blocked; In the case that there is no occlusion in the first image, a traffic violation detection is performed on the first image, and an alarm is issued when a traffic violation is detected.
[0006] Optionally, a backlight image in the image is obtained, and a dark area in the backlight image is extracted. ,include: Preprocessing the backlight image according to a Gaussian filtering algorithm; Obtain pixels in the backlight image, and obtain the segmentation threshold in the backlight image according to formula (1), , (1) in, is the calculated segmentation threshold, Pixel The mean value of the pixels in the local window centered at is the standard deviation of pixels in the local window, is the normalization constant, Parameters for controlling the threshold adjustment amplitude; According to formula (2), the bright area and dark area in the backlight image are divided and the dark area is extracted. , , (2) in, is the segmentation rule for bright and dark areas, The pixels after filtering of the original dark area ROI The pixel value of .
[0007] Optionally, the dark area obtained Perform multi-scale transformation to enhance dark areas
[0008] Dynamic range, including: Get the dark area , and the dark area Perform Gamma & Log transformation; Perform Gamma transformation according to formula (3): , (3) in, is the output of the Gamma transform, Dark area of the input The original pixel value, is a nonlinear mapping parameter used to adjust Luminance curve; Perform LOG transformation according to formula (4): , (4) in, is the output of LOG transformation, is the logarithmic gain coefficient.
[0009] Optionally, the transformed dark area Input the LightenNet model and generate the corresponding lighting map, including: According to the CNN layer of the LightenNet model, the dark area is transformed Perform convolution to obtain the corresponding illumination map; According to formula (5) and formula (6), the CNN layer parameter model is modified to minimize the loss function. , (5) , (6) in, is the loss value calculated by the loss function, is the sample size, a positive integer, , is the illumination map of a real low-light image, To predict the light map, is a dynamic power function, For the The original error of the samples, is the coefficient, is the hyperbolic tangent activation function.
[0010] Optionally, the dark area is combined with the illumination map to generate an enhanced ,include: According to formula (7) and formula (8), the linear coefficient within the local window of each pixel is obtained. , (7) , (8) in, For the The linear coefficient of the local window, The first The lighting map of a local window, The original dark area No. The pixel value in the local window, For local window Dark area of internal input The covariance between the original pixel value and the light map, For local window inner dark zone The variance of the original pixel values, 、 For local window inner dark zone The original pixel value and the mean of the illumination map, Regularization parameter to prevent the denominator from being zero; According to formula (9), the refined illumination map of each pixel is obtained. , (9) in, Pixels The refined light map, For all pixels containing this The number of windows, For the corresponding pixel The original dark area of the position The pixel value above; According to formula (10), the enhanced , , (10) in, For the enhanced image, Extracted from the original backlight image area; According to formula (11), linear average fusion is performed. , (11) in, To fuse ten numbers of enhanced The image obtained after the image, is the index, For the Enhancement corresponding to Gamma transformation result, For the Enhancement corresponding to the Log transformation result.
[0011] Optionally, performing traffic violation detection on the first image and issuing an alarm when a traffic violation is detected includes: Acquire the first image and text data of the first image; Preprocessing the first image and the text data; According to formula (12), the pre-processed first image is fed into the RT-DETR backbone network ResNet18 network to extract the three layers of image features. , (12) in, is the image feature of each layer, is the first image, Indicates that the first image is sent to the Resnet18 neural network for processing. is the number of image layers; The text data is encoded into a text embedding vector according to formula (13), , (13) in, is the text embedding vector, For text encoder For text sequences To encode, is a text sequence, is the length of the text; The image features of each layer are modulated by the PAC-APN module through formula (14), , (14) in, is the image feature after each layer is modulated by the PAC-APN module, is the expansion rate, is the dilated convolution, Indicates mapping the text embedding vector t to a weight consistent with the number of image channels, is broadcast multiplication; The image features and text vectors are fused at different layers using formula (15) to obtain multimodal features. , (15) in, is the feature after the fusion of image features and text vectors, is the gating factor, is the text embedding vector Perform dimension expansion operations.
[0012] Optionally, performing traffic violation detection on the first image and issuing an alarm when a traffic violation is detected includes: Sending the fused multimodal features to the decryptor part; The decoder converts the multimodal features into a specific detection target location; Performing state recognition on the target position; Send out alerts when violations are identified.
[0013] Optionally, the detection system further includes: When the first image is blocked, acquiring the first image; The blocked portion is compensated according to the Kalman filter algorithm to obtain a compensated first image, and the process returns to the step of detecting the first image.
[0014] Optionally, the occluded portion is compensated according to a Kalman filter algorithm, including: According to formula (18) to formula (20), the state prediction matrix is obtained. , (18) , (19) , (20) in, is the predicted state at the next moment, is the current state, is the center coordinate of the horizontal protective equipment, is the width and height of the detection box, is the center point movement speed, is the center point acceleration, is the state transition matrix, is the frame time interval, is the process noise.
[0015] Optionally, the occluded portion is compensated according to a Kalman filter algorithm, including: According to formula (21) and formula (22), the observation model is obtained. ,(twenty one) ,(twenty two) in, is the predicted frame position at the next moment, and the occluded part is compensated according to the virtual detection frame. is the mapping matrix, is the observation noise, According to formula (23), the output accuracy of the control prediction frame is obtained. ,(twenty three) in, To control the accuracy of the prediction frame, the occluded part is compensated according to the virtual detection frame. is the confidence level.
[0016] Through the above technical solution, the present invention provides a method for detecting violations of electric high-altitude operations, which obtains a visual flow, obtains an image to be detected according to the visual flow, obtains a backlight image in the image, and extracts the dark area in the backlight image. , for the dark area obtained Perform multi-scale transformation to enhance dark areas Dynamic range, the transformed dark area Input the LightenNet model to generate the corresponding illumination map, and convert the dark area Combined with the light map to generate enhanced , the unbacklit image and the enhanced The backlit image is used as the first image to determine whether the first image is occluded. If the first image is not occluded, the first image is detected for violation, and an alarm is issued when a violation is detected. The present invention improves the quality of the image to be detected by performing image enhancement on the backlit image, and introduces the PAC-APN module in the cross-modal fusion stage to effectively utilize text information to enhance detection features, and designs a gated fusion strategy to achieve multi-scale semantic matching and fusion, thereby realizing the recognition and judgment of image features. When it is determined that the graphic is occluded, the occlusion compensation of the Kalman filter can predict the change of the target size, adapt to the deformation or scale change of the detection frame caused by the occlusion, compensate for the image, and effectively improve the detection accuracy of safety protection equipment under complex working conditions.
[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings: Figure 1 This is a flow chart of a method for detecting violations of electrical overhead work according to an embodiment of the present invention; Figure 2 This is an embodiment of the present invention to extract dark areas in backlit images flow chart; Figure 3 An embodiment of the present invention is to enhance the dark area Dynamic range flow chart; Figure 4 is a flow chart of generating a corresponding illumination map according to one embodiment of the present invention; Figure 5 An embodiment of the present invention generates enhanced flow chart; Figure 6 is a flow chart of detecting traffic violations on a first image according to an embodiment of the present invention; Figure 7 This is a flowchart of compensating for an occluded portion according to an embodiment of the present invention. DETAILED DESCRIPTION
[0019] The following describes the specific implementation of the embodiment of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention and is not used to limit the embodiment of the present invention.
[0020] In the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0021] like Figure 1 The figure shows a flow chart of a method for detecting violations of electrical high-altitude operations according to an embodiment of the present invention. Figure 1 In the present invention, the detection method may include: In step S1, a visual flow is obtained, and an image to be detected is obtained according to the visual flow; In step S2, a backlight image is obtained from the image, and a dark area in the backlight image is extracted. ; In step S3, the dark area is obtained Perform multi-scale transformation to enhance dark areas Dynamic range; In step S4, the transformed dark area Input the LightenNet model and generate the corresponding lighting map; In step S5, the dark area Combined with the light map, an enhanced ; In step S6, the non-backlit image and the enhanced The image set after is the first image; In step S7, it is determined whether the first image is blocked; In step S8 , when there is no occlusion in the first image, a traffic violation detection is performed on the first image, and an alarm is issued when a traffic violation is detected.
[0022] In step S9, when the first image is blocked, the first image is acquired; In step S10, the blocked portion is compensated according to the Kalman filter algorithm to obtain a compensated first image, and the process returns to the step of detecting the first image.
[0023] In this Figure 1 In the method shown, step S1 can be used to obtain a visual flow and obtain an image to be detected based on the visual flow. Step S2 can be used to obtain a backlight image in the image and extract the dark area in the backlight image. For extracting dark areas in backlit images The specific method can be various forms known to those skilled in the art. In one example of the present invention, the dark area in the backlight image is extracted. Methods may include Figure 2 The steps shown. Specifically: In step S11, the backlight image is preprocessed according to the Gaussian filtering algorithm; In step S12, pixels in the backlight image are obtained, and the segmentation threshold in the backlight image is obtained according to formula (1). , (1) in, is the calculated segmentation threshold, Pixel The mean value of the pixels in the local window centered at is the standard deviation of pixels in the local window, is the normalization constant, Parameters for controlling the threshold adjustment amplitude; In step S13, the bright and dark areas in the backlight image are segmented according to formula (2), and the dark area is extracted. , , (2) in, is the segmentation rule for bright and dark areas, The pixels after filtering of the original dark area ROI The pixel value of .
[0024] In this Figure 2 In the method shown, step S11 can be used to pre-process the backlight image according to the Gaussian filtering algorithm. The image is smoothed and denoised by the Gaussian filtering algorithm to avoid the introduction of too many isolated points by threshold segmentation. The Gaussian filter is calculated according to formula (24) and formula (25): ,(twenty four) , (25) in, is the normalization factor to maintain the consistency of the brightness of the filtered image, is the radius of the Gaussian kernel, Can be 2, kernel size is ,Right now . For the input image at position The pixel value of is the position of a point in the nucleus relative to the center (centered at the origin). In one embodiment of the present invention, Can be , is the weight value of the Gaussian kernel at position (𝑖,𝑗). The Gaussian kernel is generated by a weight function that combines spatial distance, gradient sensitivity, and directional regulation. It can be calculated according to formula (26): , (26) in, and is the Gaussian kernel in and The standard deviation in the direction is used to control the degree of diffusion. In order to better meet the needs of high-altitude working scenarios, for vertical objects such as vertical safety ropes that need to be identified in the subsequent target detection process, because they are easily confused with cables, it is necessary to strengthen the features and make them more inclined to longitudinal smoothness, that is, , in one embodiment of the present invention, It can be 0.8, It can be 2.0. is the gradient response coefficient, which controls the proportion of edge information in the weight. The larger it is, the more edges are retained. In combination with the requirements of high-altitude operations, in one embodiment of the present invention, , preserving the edge structure. is the local gradient amplitude of the image, which describes whether the pixel point is an edge (dramatically changing) area. It can be calculated according to formula (27): , (27) in, is the local gradient amplitude of the image, and They are and Directional gradient.
[0025] In order to dynamically adjust the threshold according to the mean and standard deviation of the local neighborhood pixels, which is suitable for scenes with uneven illumination, adaptive threshold calculation is required. Step S12 can be used to obtain pixels in the backlight image and obtain the segmentation threshold in the backlight image according to formula (1). , (1) in, is the calculated segmentation threshold, Pixel The average value of the pixels in the local window centered at . In one example of the present invention, the local window can be 15×15 in size and needs to cover a large enough area to capture local illumination changes. is the standard deviation of pixels in the local window, reflecting the regional contrast, is the normalization constant, The value of can be 128, which corresponds to the maximum standard deviation of 8-bit images. is the parameter that controls the threshold adjustment amplitude. The value of can be 0.2.
[0026] Step S13 can be used to segment the bright area and dark area in the backlight image according to formula (2) and extract the dark area , , (2) in, is the segmentation rule for bright and dark areas, The original dark area Filtered pixels The pixel value of .
[0027] Step S3 can be used to obtain the dark area Perform multi-scale transformation to enhance dark areas Dynamic range. For enhanced dark areas The specific method of improving the dynamic range can be various forms known to those skilled in the art. In one embodiment of the present invention, the dark area is enhanced. Dynamic range methods can include Figure 3 The steps shown. Specifically: In step S21, the dark area is obtained , and for dark areas Perform Gamma & Log transformation; In step S22, Gamma transformation is performed according to formula (3): , (3) in, is the output of the Gamma transform, Dark area of the input The original pixel value, It is a nonlinear mapping parameter used to adjust the ROI brightness curve; In step S23, LOG transformation is performed according to formula (4): , (4) in, is the output of LOG transformation, is the logarithmic gain coefficient.
[0028] In this Figure 3 In the method shown, step S21 can be used to obtain the dark area , and for dark areas Perform multiple sets of multi-scale Gamma & Log transformations. Step S22 can be used to perform Gamma transformation according to formula (3), , (3) in, is the output of the Gamma transform, Dark area of the input The original pixel value, as the input of the Gamma transformation, represents the original brightness value of the dark area that needs to be enhanced. is a nonlinear mapping parameter used to adjust Brightness curve, controls the nonlinear mapping of input pixel values through different Value Adjustment Dark Area The brightness curve of the image is stretched to stretch the dynamic range of the dark part and generate images with different brightness levels. The smaller one in formula (3) A value of 0.6 for example can significantly enhance dark areas, while =1.0 to retain the original brightness.
[0029] Step S23 can be used to perform LOG transformation according to formula (4), , (4) in, is the output of LOG transformation, is the logarithmic gain coefficient, which is used to adjust the steepness of the logarithmic curve and control the expansion of the dark area. The smaller it is, the more obvious the expansion of dark area details will be.
[0030] Step S4 can be used to convert the transformed dark area Input the LightenNet model and generate the corresponding illumination map. The specific method for generating the corresponding illumination map can be various forms known to those skilled in the art. In one example of the present invention, the method for generating the corresponding illumination map may include: Figure 4 The steps shown. Specifically: In step S31, the transformed dark area is processed according to the CNN layer of the LightenNet model. Perform convolution to obtain the corresponding illumination map; In step S32, the CNN layer parameter model is modified by minimizing the loss function according to formula (5) and formula (6). , (5) , (6) in, is the loss value calculated by the loss function, is the sample size, a positive integer, , is the illumination map of a real low-light image, To predict the light map, is a dynamic power function, For the The original error of the samples, is a coefficient used to control the speed at which the power decreases from 2 to 1. is the hyperbolic tangent activation function.
[0031] Step S31 can be used to transform the dark area according to the CNN layer of the LightenNet model Convolution is performed to obtain the corresponding illumination map. The LightenNet model is pre-trained on a low-light dataset to learn the mapping from low-light input to normal illumination, and the dark area after input Gamma transformation and Log transformation is , generating the corresponding illumination map. The model trains a total of four CNN layers. Different convolutional layers are responsible for the following feature extraction as shown in Table 1: Table 1
[0032] Step S32 can be used to modify the CNN parameter model by minimizing the loss function according to formula (5) and formula (6), by using an improved loss function based on mean square error (MSE): a dynamic power MSE loss function to minimize the difference between the predicted illumination map and the illumination map of the real low-light image. , (5) , (6) in, is the loss value calculated by the loss function, is the sample size, a positive integer, , is the illumination map of a real low-light image, To predict the light map, is a dynamic power function used to control the penalty intensity of the error term. For the The original error of the samples, is a coefficient used to control the speed at which the power decreases from 2 to 1. is the hyperbolic tangent activation function.
[0033] In one embodiment of the present invention, when the error is small is approximately equal to 0, at this time is about 2, which keeps the square penalty for small errors and is conducive to fine fitting. tends to 1, then It is approximately 1, reducing sensitivity to large errors and the risk of gradient explosion. To control the speed at which the power decreases from 2 to 1.
[0034] In order to remove noise and smooth discontinuous areas in the illumination map while preserving edge structures, guided filtering is performed. The image is the guide image, and the original illumination map (the illumination map generated by the CNN layer) is filtered to obtain the refined illumination map. Step S5 can be used to combine the dark area and the illumination map to generate an enhanced For generating enhancements The specific method can be various forms known to those skilled in the art. In one example of the present invention, the enhanced This can include Figure 5 The steps shown. Specifically: In step S41, the linear coefficient within the local window of each pixel is obtained according to formula (7) and formula (8), , (7) , (8) in, For the The linear coefficient of the local window, The first The lighting map of a local window, For the original dark area ROI The pixel value in the local window, For local window Dark area of internal input The covariance between the original pixel value and the light map, For local window inner dark zone The variance of the original pixel values, 、 For local window inner dark zone The original pixel value and the mean of the illumination map, To prevent the regularization parameter from having a denominator of zero, in one example of the present invention, The value of can be , the local window can be 15×15 in size; In step S42, the refined illumination map of each pixel is obtained according to formula (9): , (9) in, Pixels The refined light map, For all pixels containing this The number of windows, For the corresponding pixel The original dark area of the position The pixel value above; In step S43, the enhanced , , (10) in, For the enhanced image, Extracted from the original backlight image area, In step S44, linear average fusion is performed according to formula (11): , (11) in, To fuse ten numbers of enhanced The image obtained after the image, is the index, For the Enhancement corresponding to Gamma transformation result, For the Enhancement corresponding to the Log transformation result.
[0035] In such Figure 5 In the method shown, step S41 is used to obtain each pixel In the local window The linear coefficients within the step S42 are used to calculate the linear coefficients of each pixel. , aggregate all windows containing it to obtain a smooth and edge-preserving illumination map, and step S43 is used to use the refined illumination map Remove previously generated and The illumination component of the reflection component is retained to achieve local contrast improvement. Step S44 is used to fuse the enhancement results of multiple transformations through linear averaging, balance the advantages of different illumination maps, suppress noise and over-enhancement, and comprehensively improve the contrast and detail.
[0036] Step S6 can be used to combine the non-backlit image and the enhanced The image after the image is taken as the first image. Step S7 can be used to determine whether there is any occlusion in the first image. Step S8 can be used to perform traffic violation detection on the first image when there is no occlusion in the first image, and to issue an alarm when a traffic violation is detected. The specific method for detecting traffic violations on the first image can be various forms known to those skilled in the art. In one example of the present invention, the method for detecting traffic violations on the first image may include: Figure 6 The steps shown. Specifically: In step S51, a first image and text data of the first image are obtained; In step S52, the first image and text data are pre-processed; In step S53, the pre-processed first image is fed into the RT-DETR backbone network ResNet18 network to extract the last three layers of image features according to formula (12). , (12) in, is the image feature of each layer, is the first image, , is a set, representing the first input image The size of the three channels is 640 640 images, To indicate that the first image is fed into the Resnet18 neural network for processing, is the number of image layers; In step S54, the text data is encoded into a text embedding vector according to formula (13), , (13) in, is the text embedding vector, For text encoder For text sequences To encode, is a text sequence, is the length of the text; In step S55, the image features of each layer are modulated by the PAC-APN module through formula (14), , (14) in, is the image feature after each layer is modulated by the PAC-APN module, is the expansion rate, is the dilated convolution, Indicates mapping the text embedding vector t to a weight consistent with the number of image channels, is broadcast multiplication; In step S56, the image features and text vectors are fused at different layers using formula (15) to obtain multimodal features. , (15) in, is the feature after the fusion of image features and text vectors, is the gating factor, It is an operation to expand the dimension of the text embedding vector t; In step S57, the fused multimodal features are fed into the decryptor part; In step S58, the decoder converts the multimodal features into a specific detection target location; In step S59, the target position is identified in state; In step S60 , an alarm is issued when a violation is detected.
[0037] In this Figure 6 In the illustrated method, step S51 can be used to obtain a first image and text data associated with the first image. Step S52 can be used to preprocess the first image and text data. Five labels are assigned to the first image dataset: vertical protection normal, vertical protection abnormal, horizontal protection normal, horizontal protection abnormal, and person. Furthermore, all images are resized to 640×640 using the resize function in OpenCV, completing the preprocessing of the first image. The text data contains a text description of the first image, such as "A person working at height," "Vertical protection normal," and "Horizontal protection normal." There are three types of vertical protection: no vertical protection (i.e., no vertical protection equipment is worn); abnormal vertical protection (i.e., a vertical protection equipment is worn but used incorrectly, such as not being attached to the pole or being attached too low and used too high (i.e., attached at a position lower than the person's height, requiring overhead attachment); and normal vertical protection (i.e., a vertical protection equipment is worn and used correctly). There are three types of horizontal protection: no horizontal protection: that is, no horizontal protection device is worn; abnormal horizontal protection: that is, horizontal protection device is worn, but used incorrectly, such as not hanging it on the pole tower, or hanging it low and using it high (that is, the hanging position is lower than the position of the person, and it needs to be hung on a plane parallel to the waist); normal horizontal protection: that is, horizontal protection device is worn and used correctly.
[0038] Step S53 can be used to send the pre-processed first image to the RT-DETR backbone network ResNet18 network according to formula (12) to extract the three layers of image features, including visual features at different semantic levels. , (12) in, is the image feature of each layer, is the first image, , is a set, representing the first input image The size of the three channels is 640 640 images, To indicate that the first image is fed into the Resnet18 neural network for processing, is the number of image layers; Step S54 can be used to encode the text data into a text embedding vector according to formula (13), , (13) in, is the text embedding vector, which serves as the semantic anchor for subsequent image guidance. For text encoder For text sequences To encode, is a text sequence, is the length of the text; Step S55 can be used to modulate the image features of each layer by the PAC-APN module through formula (14). PAC-APN uses dilated convolution with different expansion rates to process image features and guides its channel selective response with the text embedding vector. , (14) in, is the image feature after each layer is modulated by the PAC-APN module, is the expansion rate, is the dilated convolution, Indicates embedding text into vector Mapped to weights consistent with the number of image channels, is broadcast multiplication; Step S56 can be used to fuse the image features and text vectors at different layers through formula (15) to obtain multimodal features. , (15) in, is the feature after the fusion of image features and text vectors, is the gating factor, This is an operation to expand the dimension of the text embedding vector t, which is used to copy the text embedding to be consistent with the shape of the feature map. A multi-layer perceptron, a fully connected neural network module, is used to generate dynamic weights to identify the relationship between text features and image features. is global average pooling.
[0039] Step S57 can be used to feed the fused multimodal features into the decryptor. Step S58 can be used to convert the multimodal features into specific detection target locations through the decoder, marking the location of the target to be detected. Step S59 can be used to identify the status of the target location. Step S60 can be used to issue an alarm when a violation is identified. In one example of the present invention, the decoder identifies the location of the horizontal safety rope buckle after processing, and then compares it with the operator's waist position to determine whether they are at the same horizontal position, thereby determining compliance. If a violation is detected, an alarm is issued.
[0040] Step S9 can be used to obtain the first image when there is occlusion in the first image. Step S10 can be used to compensate the occluded portion according to the Kalman filter algorithm to obtain the compensated first image, and return to the step of detecting the first image. The specific method of compensating the occluded portion can be various forms known to those skilled in the art. In one example of the present invention, the method of compensating the occluded portion can include: Figure 7 The steps shown. Specifically: In step S71, the state prediction matrix is obtained according to formula (18) to formula (20), , (18) , (19) , (20) in, is the predicted state at the next moment, is the current state, is the center coordinate of the horizontal protective equipment, is the width and height of the detection box, is the center point movement speed, is the center point acceleration, is the state transition matrix, is the frame time interval. In one example of the present invention, The value of can be , is the process noise, , in one example of the present invention, ; In step S72, the observation model is obtained according to formula (21) and formula (22), ,(twenty one) ,(twenty two) in, is the predicted frame position at the next moment, and the occluded part is compensated according to the virtual detection frame. is a mapping matrix used to map the position of the prediction frame at the next moment. is the observation noise, , in one example of the present invention, ; In step S73, the output accuracy of the control prediction frame is obtained according to formula (23), ,(twenty three) in, is the prediction box after controlling the accuracy, is the confidence level. In one example of the present invention, It can be 0.6, which is the manually assigned confidence level.
[0041] When the horizontal protection buckle is not detected in step S59 or the confidence of the detection frame is lower than 0.3, it is determined that an occlusion has occurred. Then, step S10 is required to compensate for the occlusion. The state prediction matrix obtained in step S71 is used to predict the future position of the target based on its speed and acceleration. The position of the next frame is predicted based on the position of the previous frame. The observation model is obtained according to step S72, and the position information is extracted from the predicted state of the next frame. The first four dimensions of the matrix are used to predict the position of the prediction frame at the next moment, compensate for the occluded part according to the virtual detection frame, and control the output accuracy of the prediction frame through step S73. In one example of the present invention, the output accuracy of the prediction frame can be 0.6.
[0042] Through the above technical solution, the present invention provides a method for detecting violations of electric high-altitude operations, which obtains a visual flow, obtains an image to be detected according to the visual flow, obtains a backlight image in the image, and extracts the dark area in the backlight image. , for the dark area obtained Perform multi-scale transformation to enhance dark areas Dynamic range, the transformed dark area Input the LightenNet model to generate the corresponding illumination map, and convert the dark area Combined with the light map to generate enhanced , the unbacklit image and the enhanced The backlit image is used as the first image to determine whether the first image is occluded. If the first image is not occluded, the first image is detected for violation, and an alarm is issued when a violation is detected. The present invention improves the quality of the image to be detected by performing image enhancement on the backlit image, and introduces the PAC-APN module in the cross-modal fusion stage to effectively utilize text information to enhance detection features, and designs a gated fusion strategy to achieve multi-scale semantic matching and fusion, thereby realizing the recognition and judgment of image features. When it is determined that the graphic is occluded, the occlusion compensation of the Kalman filter can predict the change of the target size, adapt to the deformation or scale change of the detection frame caused by the occlusion, compensate for the image, and effectively improve the detection accuracy of safety protection equipment under complex working conditions.
[0043] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0044] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0045] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0046] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0047] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
[0048] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0049] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.
[0050] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.
[0051] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.
Claims
1. A method for detecting violations of electrical high-altitude operations, characterized in that: The detection system comprises: Acquire a visual stream, and acquire an image to be detected according to the visual stream; Acquire a backlight image in the image and extract a dark area in the backlight image ; The dark area obtained Perform multi-scale transformation to enhance dark areas Dynamic range; The dark area after transformation Input the LightenNet model and generate the corresponding lighting map; The dark area Combined with the light map, an enhanced ; The non-backlit image and the enhanced The image set after is the first image; Determining whether the first image is blocked; In the case that there is no occlusion in the first image, a traffic violation detection is performed on the first image, and an alarm is issued when a traffic violation is detected.
2. The detection method according to claim 1, characterized in that Acquire a backlight image in the image and extract a dark area in the backlight image ,include: Preprocessing the backlight image according to a Gaussian filtering algorithm; Obtain pixels in the backlight image, and obtain the segmentation threshold in the backlight image according to formula (1), ，（1） in, is the calculated segmentation threshold, Pixel The mean value of the pixels in the local window centered at is the standard deviation of pixels in the local window, is the normalization constant, Parameters for controlling the threshold adjustment amplitude; According to formula (2), the bright area and dark area in the backlight image are segmented and the dark area ROI is extracted. ，（2） in, is the segmentation rule for bright and dark areas, The pixels after filtering of the original dark area ROI The pixel value of .
3. The detection method according to claim 1, wherein The dark area obtained Perform multi-scale transformation to enhance dark areas Dynamic range, including: Get the dark area , and the dark area Perform Gamma & Log transformation; According to formula (3), Gamma transformation is performed. ，（3） in, is the output of the Gamma transform, Dark area of the input The original pixel value, is a nonlinear mapping parameter used to adjust Luminance curve; Perform LOG transformation according to formula (4): ，（4） in, is the output of LOG transformation, is the logarithmic gain coefficient.
4. The detection method according to claim 3, characterized in that The dark area after transformation Input the LightenNet model and generate the corresponding lighting map, including: According to the CNN layer of the LightenNet model, the dark area is transformed Perform convolution to obtain the corresponding illumination map; According to formula (5) and formula (6), the CNN layer parameter model is modified to minimize the loss function. ，（5） ，（6） in, is the loss value calculated by the loss function, is the sample size, a positive integer, , is the illumination map of a real low-light image, To predict the light map, is a dynamic power function, For the The original error of the samples, is the coefficient, is the hyperbolic tangent activation function.
5. The detection method according to claim 4, characterized in that Combine the dark area with the light map to generate an enhanced ,include: According to formula (7) and formula (8), the linear coefficient within the local window of each pixel is obtained. ，（7） ，（8） in, For the The linear coefficient of the local window, The first The lighting map of a local window, The original dark area No. The pixel value in the local window, For local window Dark area of internal input The covariance between the original pixel value and the light map, For local window inner dark zone The variance of the original pixel values, 、 For local window inner dark zone The original pixel value and the mean of the illumination map, Regularization parameter to prevent the denominator from being zero; According to formula (9), the refined illumination map of each pixel is obtained. ，（9） in, Pixels The refined light map, For all pixels containing this The number of windows, For the corresponding pixel The original dark area of the position The pixel value above; According to formula (10), the enhanced , ，（10） in, For the enhanced image, Extracted from the original backlight image area; According to formula (11), linear average fusion is performed. ，（11） in, To fuse ten numbers of enhanced The image obtained after the image, is the index, For the Enhancement corresponding to Gamma transformation result, For the Enhancement corresponding to the Log transformation result.
6. The detection method according to claim 1, characterized in that Performing traffic violation detection on the first image and issuing an alarm when a traffic violation is detected includes: Acquire the first image and text data of the first image; Preprocessing the first image and the text data; According to formula (12), the pre-processed first image is fed into the RT-DETR backbone network ResNet18 network to extract the three layers of image features. ，（12） in, is the image feature of each layer, is the first image, Indicates that the first image is sent to the Resnet18 neural network for processing. is the number of image layers; The text data is encoded into a text embedding vector according to formula (13), ，（13） in, is the text embedding vector, For text encoder For text sequences To encode, is a text sequence, is the length of the text; The image features of each layer are modulated by the PAC-APN module through formula (14), ，（14） in, is the image feature after each layer is modulated by the PAC-APN module, is the expansion rate, is the dilated convolution, Indicates mapping the text embedding vector t to a weight consistent with the number of image channels, is broadcast multiplication; The image features and text vectors are fused at different layers using formula (15) to obtain multimodal features. ，（15） in, is the feature after the fusion of image features and text vectors, is the gating factor, It is an operation to expand the dimension of the text embedding vector t.
7. The detection method according to claim 6, characterized in that Performing traffic violation detection on the first image and issuing an alarm when a traffic violation is detected includes: The fused multimodal features are fed into the decryptor part: The decoder converts the multimodal features into a specific detection target location; Performing state recognition on the target position; Send out alerts when violations are identified.
8. The detection method according to claim 1, wherein The detection system also includes: When the first image is blocked, acquiring the first image; The blocked portion is compensated according to the Kalman filter algorithm to obtain a compensated first image, and the process returns to the step of detecting the first image.
9. The detection method according to claim 8, characterized in that The occlusion is compensated based on the Kalman filter algorithm, including: According to formula (18) to formula (20), the state prediction matrix is obtained. ，（18） ，（19） ，（20） in, is the predicted state at the next moment, is the current state, is the center coordinate of the horizontal protective equipment, is the width and height of the detection box, is the center point movement speed, is the center point acceleration, is the state transition matrix, is the frame time interval, is the process noise.
10. The detection method according to claim 9, characterized in that: The occlusion is compensated based on the Kalman filter algorithm, including: According to formula (21) and formula (22), the observation model is obtained. ，（21） ，（22） in, is the predicted frame position at the next moment, and the occluded part is compensated according to the virtual detection frame. is the mapping matrix, is the observation noise, According to formula (23), the output accuracy of the control prediction frame is obtained. ，（23） in, is the prediction box after controlling the accuracy, is the confidence level.