Data cleaning method and system for power distribution safety supervision operation

Abstract

The invention discloses a data cleaning method and system for power distribution safety supervision operation. The method comprises the following steps: collecting a video of the power distribution safety supervision operation; extracting a plurality of frame images from the video of the power distribution safety supervision operation, and forming a frame image sequence; any two frame images are continuously selected from the frame image sequence, the Jaccard distance between the composite Hash codes of the two frame images is obtained through calculation, then whether the Jaccard distance is smaller than a preset threshold value epsilon or not is judged, if yes, one frame image and the corresponding composite Hash code are deleted, if not, the two frame images are reserved, the process is repeated, and if not, the Jaccard distance between the composite Hash codes of the two frame images is not smaller than the preset threshold value epsilon, and the Jaccard distance between the composite Hash codes of the two frame images is not smaller than the preset threshold value epsilon. All the frame images in the frame image sequence are compared pairwise and are correspondingly processed. The image deduplication method solves the problem that the traditional image deduplication algorithm mostly pays attention to the whole of the image instead of the local content of the image, so that the deduplication efficiency and accuracy are not good in the application scene needing to pay attention to the local content of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of power distribution safety and monitoring operations, and in particular to a data cleaning method and system for power distribution safety and monitoring operations. Background Art

[0002] In the process of power distribution safety supervision, in order to ensure the safety of the operation, monitoring equipment is usually deployed for real-time monitoring and video recording. These monitoring devices can capture the details of the operation site, providing an important basis for post-analysis and the formulation of safety regulations. Extracting frame images from monitoring videos and checking and marking them for safety regulations are important steps to ensure the safety of operations. However, since monitoring equipment usually shoots at a high frequency, the amount of frame image data generated is huge and contains a large number of similar images. The information carried by these images is the same. If they are not screened, a lot of time will be spent on repetitive and meaningless work, and useless image data with a high degree of overlap will be generated.

[0003] Therefore, it is necessary to clean these frame image data. The existing cleaning method is generally to first use the OpenCV library and other methods to extract frames from the video, and then use the image deduplication algorithm to deduplicate several video frames. The traditional image deduplication algorithm is generally AVG-hash, DCT-hash or BDCT-hash algorithm. These algorithms pay more attention to the overall image rather than the local content of the image. Therefore, these algorithms have certain limitations in the application process. For application scenarios that need to pay attention to the local content of the image, these algorithms have poor efficiency and accuracy in image deduplication. Summary of the invention

[0004] In response to the above-mentioned defects, the present invention proposes a data cleaning method and system for power distribution safety and monitoring operations, aiming to solve the problem that traditional image deduplication algorithms focus more on the overall image rather than the local content of the image, resulting in poor deduplication efficiency and accuracy in application scenarios that require attention to the local content of the image.

[0005] To achieve this object, the present invention adopts the following technical solutions:

[0006] A data cleaning method for power distribution safety supervision operation includes the following steps:

[0007] Step S1: Collecting videos of power distribution safety supervision operations;

[0008] Step S2: extracting a number of frame images from the video of the power distribution safety supervision operation and forming a frame image sequence;

[0009] Step S3: continue to select any two frame images from the frame image sequence, and perform steps S4 to S7 on them, and repeat this process until all frame images in the frame image sequence have been compared two by two and processed accordingly;

[0010] Step S4: performing gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain global texture feature hash codes of the two frame images;

[0011] Step S5: using the improved OpenPose algorithm to extract local posture features from the two frame images respectively, and performing hash calculations on the two local posture features respectively to obtain local posture feature hash codes of the two frame images;

[0012] Step S6: Combining the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images;

[0013] Step S7: Calculate the Jaccard distance between the composite hash codes of the two frame images, and determine whether the Jaccard distance between the composite hash codes of the two frame images is less than a preset threshold ε. If so, delete one of the frame images and its corresponding composite hash code. If not, retain the two frame images to complete the cleaning of the frame image data.

[0014] Preferably, in step S4, gray level co-occurrence matrix GLCM hash processing is performed on the frame image to obtain a global texture feature hash code of the frame image, which specifically includes the following sub-steps:

[0015] Step S41: converting the frame image into a grayscale image;

[0016] Step S42: Calculate the gray level co-occurrence frequency of pixel pairs in the gray image at a given distance and direction to construct a GLCM;

[0017] Step S43: extracting texture features from GLCM, the texture features including energy, contrast, entropy and correlation;

[0018] Step S44: Perform hash processing on the texture features to generate a global texture feature hash code.

[0019] Preferably, in step S5, the improved OpenPose algorithm is used to extract local posture features from the frame image, and the local posture features are hashed to obtain the local posture feature hash code of the frame image, which specifically includes the following sub-steps:

[0020] Step S51: extracting information of 14 skeleton joints from the frame image using the improved OpenPose algorithm, and arranging the information of the 14 skeleton joints in a certain order to form a skeleton joint sequence;

[0021] Step S52: Establishing a long short-term memory network LSTM model;

[0022] Step S53: input the skeleton joint point sequence into the LSTM model for processing, and output a local posture feature vector;

[0023] Step S54: Use supervised deep hashing SDH to convert the local posture feature vector into a binary local posture feature hash code.

[0024] Preferably, in step S6, the global texture feature hash code of the frame image is combined with the local posture feature hash code of the frame image to generate a composite hash code of the frame image, which specifically includes the following sub-steps:

[0025] Step S61: combining the global texture feature hash code T of the frame image and the local posture feature hash code B of the frame image into a hash code F=[TB], wherein the length of the hash code F is 80;

[0026] Step S62: using a key to generate a pseudo-random secret stream with a length of 80, and sorting the elements in the pseudo-random secret stream and recording the original position of each element to generate a vector Z;

[0027] Step S63: Use the vector Z to reorder the elements in the hash code F to obtain a new hash code H, wherein the relationship between the elements in the new hash code H and the elements in the vector Z satisfies the following formula:

[0028] h(q)=f(z(q));

[0029] Wherein, h(q) represents the qth element in the new hash code H, z(q) represents the qth element in the vector Z, and f() represents the transformation function that reorders the vector Z;

[0030] Step S64: quantize the elements in the new hash code H to obtain a quantized new hash code H, which is a composite hash code of the frame image.

[0031] Preferably, in step S7, the Jaccard distance between the composite hash codes of the two frame images is calculated, which specifically includes the following sub-steps:

[0032] Step S71: converting the composite hash codes of the two frame images into two sets by using a mapping method;

[0033] Step S72: Calculate the size of the intersection and union of the two sets;

[0034] Step S73: Calculate the Jaccard distance L between the composite hash codes of the two frame images according to the size of the intersection and union of the two sets. The specific calculation formula is as follows:

[0035]

[0036] Another aspect of the present application provides a data cleaning system for power distribution safety and supervision operations, the system comprising:

[0037] The acquisition module is used to collect videos of power distribution safety and supervision operations;

[0038] An extraction module is used to extract a number of frame images from the video of power distribution safety supervision operation and form a frame image sequence;

[0039] A selection and execution module is used to continuously select any two frame images from the frame image sequence, and execute the texture feature hash code calculation module, the posture feature hash code calculation module, the composite hash code generation module, the calculation module and the judgment module on them, and repeat this process until all the frame images in the frame image sequence have been compared two by two and processed accordingly;

[0040] The texture feature hash code calculation module is used to perform gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain the global texture feature hash codes of the two frame images;

[0041] A posture feature hash code calculation module is used to extract local posture features from two frame images respectively using an improved OpenPose algorithm, and perform hash calculations on the two local posture features respectively to obtain local posture feature hash codes of the two frame images;

[0042] A composite hash code generation module is used to combine the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images;

[0043] A calculation module, used for calculating the Jaccard distance between the composite hash codes of two frame images;

[0044] The judging module is used to judge whether the Jaccard distance between the composite hash codes of two frame images is less than a preset threshold ε. If so, one of the frame images and its corresponding composite hash code is deleted; if not, the two frame images are retained to complete the cleaning of the frame image data.

[0045] Preferably, the texture feature hash code calculation module includes:

[0046] A first conversion submodule, used for converting a frame image into a grayscale image;

[0047] A first calculation submodule is used to calculate the gray level co-occurrence frequency of pixel pairs in the gray image at a given distance and direction to construct a GLCM;

[0048] The first extraction submodule extracts texture features from GLCM, and the texture features include energy, contrast, entropy and correlation;

[0049] The first generation submodule is used to perform hash processing on the texture feature to generate a global texture feature hash code.

[0050] Preferably, the posture feature hash code calculation module includes:

[0051] The second extraction submodule is used to extract information of 14 skeleton joints from the frame image using the improved OpenPose algorithm;

[0052] The first sorting submodule is used to arrange the information of the 14 skeleton joint points in a certain order to form a skeleton joint point sequence;

[0053] Establish a submodule to build a long short-term memory network LSTM model;

[0054] The input submodule is used to input the skeleton joint point sequence into the LSTM model for processing;

[0055] Output submodule, used to output local posture feature vector;

[0056] The second conversion submodule is used to convert the local posture feature vector into a binary local posture feature hash code by using supervised deep hashing SDH.

[0057] Preferably, the composite hash code generation module includes:

[0058] A combination submodule, used for combining the global texture feature hash code T of the frame image and the local posture feature hash code B of the frame image into a hash code F=[TB], wherein the length of the hash code F is 80;

[0059] The second generating submodule is used to generate a pseudo-random secret stream with a length of 80 using a key;

[0060] A second sorting submodule, for sorting the elements in the pseudo-random secret stream;

[0061] A recording submodule, used for recording the original position of each element in the pseudo-random secret stream to generate a vector Z;

[0062] The third sorting submodule is used to reorder the elements in the hash code F using the vector Z to obtain a new hash code H, wherein the relationship between the elements in the new hash code H and the elements in the vector Z satisfies the following formula:

[0063] h(q)=f(z(q));

[0064] Wherein, h(q) represents the qth element in the new hash code H, z(q) represents the qth element in the vector Z, and f() represents the transformation function that reorders the vector Z;

[0065] The quantization submodule is used to quantize the elements in the new hash code H to obtain the quantized new hash code H, which is the composite hash code of the frame image.

[0066] Preferably, the calculation module includes:

[0067] A third conversion submodule is used to convert the composite hash codes of the two frame images into two sets by using a mapping method;

[0068] The second calculation submodule is used to calculate the size of the intersection and union of two sets;

[0069] The third calculation submodule is used to calculate the Jaccard distance L between the composite hash codes of the two frame images according to the size of the intersection and union of the two sets. The specific calculation formula is as follows:

[0070]

[0071] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0072] This solution uses the OpenPose-GLCM-hash algorithm to deduplicate several frames extracted from the video of power distribution safety supervision operations. The OpenPose-GLCM-hash algorithm effectively integrates the OpenPose human posture detection and gray-level co-occurrence matrix GLCM feature extraction technology, which not only captures the overall structural information of the image, but also enhances the recognition ability of the local dynamic features of the characters in the image. This enables the algorithm to more accurately identify local details in the scene when facing application scenarios that need to pay attention to the local content of the image, effectively reducing the misjudgment rate and improving the efficiency and accuracy of deduplication. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 The invention is a flowchart of the steps of a data cleaning method for power distribution safety and supervision operations. DETAILED DESCRIPTION

[0074] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0075] A data cleaning method for power distribution safety supervision operation includes the following steps:

[0076] Step S1: Collecting videos of power distribution safety supervision operations;

[0077] Step S2: extracting a number of frame images from the video of the power distribution safety supervision operation and forming a frame image sequence;

[0078] Step S3: continue to select any two frame images from the frame image sequence, and perform steps S4 to S7 on them, and repeat this process until all frame images in the frame image sequence have been compared two by two and processed accordingly;

[0079] Step S4: performing gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain global texture feature hash codes of the two frame images;

[0080] Step S5: using the improved OpenPose algorithm to extract local posture features from the two frame images respectively, and performing hash calculations on the two local posture features respectively to obtain local posture feature hash codes of the two frame images;

[0081] Step S6: Combining the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images;

[0082] Step S7: Calculate the Jaccard distance between the composite hash codes of the two frame images, and determine whether the Jaccard distance between the composite hash codes of the two frame images is less than a preset threshold ε. If so, delete one of the frame images and its corresponding composite hash code. If not, retain the two frame images to complete the cleaning of the frame image data.

[0083] This solution is a data cleaning method for power distribution safety supervision operations, such as Figure 1As shown, the first step is to collect the video of the power distribution safety and supervision operation. In this embodiment, the video of the power distribution safety and supervision operation is collected in real time by deploying monitoring equipment at the power distribution safety and supervision operation site, which is conducive to capturing the situation at the operation site in real time, such as the operation of the staff, the operating status of the equipment, the compliance with safety measures, etc. The second step is to extract a number of frame images from the video of the power distribution safety and supervision operation and form a frame image sequence. In this embodiment, by extracting frames from the video of the power distribution safety and supervision operation, it is conducive to improving the processing efficiency of the video. The third step is to continuously select any two frame images from the frame image sequence, and perform steps S4 to S7 on them, repeating this process until all frame images in the frame image sequence have been compared and processed accordingly. In this embodiment, by selecting two unprocessed frame images from the frame image sequence, and performing similarity analysis on the two frame images, if the similarity of the two frame images is higher than the set threshold, they are regarded as the same image, and only one of the frame images is retained to achieve the effect of deduplication; if the similarity of the two frame images is lower than the set threshold, both frame images are retained. When all the frame images in the frame image sequence have been compared and processed accordingly, the final frame image sequence is a deduplicated sequence. The fourth step is to perform gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain the global texture feature hash codes of the two frame images. In this embodiment, by performing GLCM hash processing on the frame images, the global texture feature hash code representing the overall content of the image can be obtained. GLCM displays the comprehensive information of the image in terms of direction, interval and amplitude changes by calculating the frequency of occurrence of pixel pairs with specific values ​​and spatial relationships in the image. In this embodiment, the four GLCM parameters of contrast, correlation, energy and uniformity are used to describe the global texture features. The fifth step is to use the improved OpenPose algorithm to extract local posture features from the two frame images respectively, and perform hash calculation on the two local posture features respectively to obtain the local posture feature hash codes of the two frame images. In this embodiment, the improved OpenPose algorithm is used to extract the posture features of 14 skeleton joints of the human body from the frame images, and then these posture features are converted into binary feature codes for efficient posture matching and retrieval. Further explanation: the improved OpenPose algorithm is to replace the first ten layers of the feature extraction network of the traditional OpenPose model from the VGG19 architecture to the MobileNet architecture. The MobileNet architecture greatly reduces the amount of computation and the number of parameters of the OpenPose model by using deep separable convolution, while maintaining good performance. The sixth step is to combine the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images. In this embodiment, by combining the global texture feature hash code and the local posture feature hash code of the frame image, the composite hash code finally obtained can reflect the overall content and posture characteristics of the frame image.The seventh step is to calculate the Jaccard distance between the composite hash codes of the two frame images, and determine whether the Jaccard distance between the composite hash codes of the two frame images is less than the preset threshold ε. If so, one of the frame images and its corresponding composite hash code is deleted. If not, the two frame images are retained to complete the cleaning of the frame image data. In this embodiment, the preset threshold ε is 0.5. The Jaccard distance can be used as a similarity metric to detect the similarity of the two frame images. By calculating the Jaccard distance between the composite hash codes of the two frame images, the similarity between the images can be effectively determined. Further explanation: when the Jaccard distance is less than the preset threshold ε, it means that the overall content and posture features of the two frame images are highly similar, and image deduplication processing is required.

[0084] This solution uses the OpenPose-GLCM-hash algorithm to deduplicate several frames extracted from the video of power distribution safety supervision operations. The OpenPose-GLCM-hash algorithm effectively integrates the OpenPose human posture detection and gray-level co-occurrence matrix GLCM feature extraction technology, which not only captures the overall structural information of the image, but also enhances the recognition ability of the local dynamic features of the characters in the image. This enables the algorithm to more accurately identify local details in the scene when facing application scenarios that need to pay attention to the local content of the image, effectively reducing the misjudgment rate and improving the efficiency and accuracy of deduplication.

[0085] Preferably, in step S4, gray level co-occurrence matrix GLCM hash processing is performed on the frame image to obtain a global texture feature hash code of the frame image, which specifically includes the following sub-steps:

[0086] Step S41: converting the frame image into a grayscale image;

[0087] Step S42: Calculate the gray level co-occurrence frequency of pixel pairs in the gray image at a given distance and direction to construct a GLCM;

[0088] Step S43: extracting texture features from GLCM, the texture features including energy, contrast, entropy and correlation;

[0089] Step S44: Perform hash processing on the texture features to generate a global texture feature hash code.

[0090] Specifically, in step S41, by converting the original frame image into a grayscale image, it is helpful to simplify the complexity of image processing. In step S42, GLCM is a matrix whose elements represent the frequency of occurrence of two pixel grayscale value pairs at a given direction and distance. In this embodiment, the given distance d is 1 and the given direction θ is 45°. In step S43, a variety of texture features can be extracted from GLCM, namely energy, contrast, entropy and correlation. The energy can be obtained by calculating the sum of squares of each element of the GLCM matrix, which is used to measure the uniformity of the image texture. The higher the energy, the more uniform the texture; the contrast is obtained by calculating the variance of each element in the GLCM matrix, which is used to reflect the contrast of the image grayscale. The higher the contrast, the clearer the texture; entropy is used to describe the randomness of the image texture. The larger the entropy, the more complex the texture; correlation can be obtained by calculating the correlation coefficient of each pair of elements in the GLCM matrix, which is used to reflect the linear correlation between the grayscale levels of the image. In step S44, hash processing is usually a function that maps data of any length to shorter fixed-length data, converting texture features into global texture feature hash codes, which can reflect the overall content of the image.

[0091] Preferably, in step S5, the improved OpenPose algorithm is used to extract local posture features from the frame image, and the local posture features are hashed to obtain the local posture feature hash code of the frame image, which specifically includes the following sub-steps:

[0092] Step S51: extracting information of 14 skeleton joints from the frame image using the improved OpenPose algorithm, and arranging the information of the 14 skeleton joints in a certain order to form a skeleton joint sequence;

[0093] Step S52: Establishing a long short-term memory network LSTM model;

[0094] Step S53: input the skeleton joint point sequence into the LSTM model for processing, and output a local posture feature vector;

[0095] Step S54: Use supervised deep hashing SDH to convert the local posture feature vector into a binary local posture feature hash code.

[0096] In this embodiment, in step S51, 14 skeleton joints extracted from the frame image include positions such as the head, shoulder, elbow, wrist, hip, knee and ankle of the human body. In step S52, by establishing an LSTM model, it is beneficial to the subsequent processing of the skeleton joint sequence. In step S53, after the LSTM model receives the skeleton joint sequence, the LSTM captures the dynamic changes of the skeleton joints in the time series through the mechanism of forget gate, input gate and output gate. Specifically, for the position of each frame of skeleton joints, LSTM generates corresponding meta features, which describe the changes of human posture in the time series, so that complex motion patterns can be captured. In one embodiment, when it is necessary to identify whether a person is doing a specific hand movement, a sequence of wrist and palm joint coordinates containing multiple time steps is used as an input feature, and the LSTM model is used to process these sequence data. The LSTM model includes an LSTM layer, and each layer in the LSTM layer outputs a hidden state vector. The last layer output hidden state vector in the LSTM layer is extracted as a local posture feature vector, which captures key information related to the hand movement. In step S54, SDH optimizes the hashing process by relaxing the loss function to ensure that similar posture features are mapped to similar binary codes. By converting the high-dimensional posture feature vector into a compact binary hash code, subsequent storage, transmission and retrieval operations are facilitated. In one embodiment, a local posture feature vector f = [f1, f2, ..., f a ], where a is the dimension of the feature, which can be converted into a binary hash code h = [h1, h2, ..., h b ], where a is much smaller than b. The specific conversion steps are as follows: the first step is to establish the SDH model; the second step is to input the local posture feature vector f into the SDH model for processing; the third step is to output the real vector z = [z1, z2, ..., z m ]; The fourth step is to convert each element in z using the sign function to obtain the binary value set s = [sign(z1), sign(z2), ..., sign(z m )], where the sign function maps positive numbers to 1 and negative numbers to 0.

[0097] Preferably, in step S6, the global texture feature hash code of the frame image is combined with the local posture feature hash code of the frame image to generate a composite hash code of the frame image, which specifically includes the following sub-steps:

[0098] Step S61: combining the global texture feature hash code T of the frame image and the local posture feature hash code B of the frame image into a hash code F=[TB], wherein the length of the hash code F is 80;

[0099] Step S62: using a key to generate a pseudo-random secret stream with a length of 80, and sorting the elements in the pseudo-random secret stream and recording the original position of each element to generate a vector Z;

[0100] Step S63: Use the vector Z to reorder the elements in the hash code F to obtain a new hash code H, wherein the relationship between the elements in the new hash code H and the elements in the vector Z satisfies the following formula:

[0101] h(q)=f(z(q));

[0102] Wherein, h(q) represents the qth element in the new hash code H, z(q) represents the qth element in the vector Z, and f() represents the transformation function that reorders the vector Z;

[0103] Step S64: quantize the elements in the new hash code H to obtain a quantized new hash code H, which is a composite hash code of the frame image.

[0104] In this embodiment, in step S62, a pseudo-random secret stream of length 80 is generated using a key to construct a secure hash. By generating a vector Z, it represents the index of the sorted element in the original sequence. In step S63, the elements in the hash code F are reordered using the vector Z to obtain a new hash code H, which means that if the qth element of the vector Z is m, then the qth element of the new hash code H is the mth element in the hash code F. In step S64, the elements in the new hash code H are quantized, which is beneficial to saving storage space and improving security.

[0105] Preferably, in step S7, the Jaccard distance between the composite hash codes of the two frame images is calculated, which specifically includes the following sub-steps:

[0106] Step S71: converting the composite hash codes of the two frame images into two sets by using a mapping method;

[0107] Step S72: Calculate the size of the intersection and union of the two sets;

[0108] Step S73: Calculate the Jaccard distance L between the composite hash codes of the two frame images according to the size of the intersection and union of the two sets. The specific calculation formula is as follows:

[0109]

[0110] In this embodiment, when calculating the Jaccard distance between two composite hash codes, since the Jaccard distance is an indicator used to measure the difference between two sets, it is calculated based on the intersection and union of sets, and the composite hash code is not a set, it is necessary to convert the composite hash code into a set. By calculating the Jaccard distance, it is convenient to subsequently detect the similarity of two frame images.

[0111] Another aspect of the present application provides a data cleaning system for power distribution safety and supervision operations, the system comprising:

[0112] The acquisition module is used to collect videos of power distribution safety and supervision operations;

[0113] An extraction module is used to extract a number of frame images from the video of power distribution safety supervision operation and form a frame image sequence;

[0114] A selection and execution module is used to continuously select any two frame images from the frame image sequence, and execute the texture feature hash code calculation module, the posture feature hash code calculation module, the composite hash code generation module, the calculation module and the judgment module on them, and repeat this process until all the frame images in the frame image sequence have been compared two by two and processed accordingly;

[0115] The texture feature hash code calculation module is used to perform gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain the global texture feature hash codes of the two frame images;

[0116] A posture feature hash code calculation module is used to extract local posture features from two frame images respectively using an improved OpenPose algorithm, and perform hash calculations on the two local posture features respectively to obtain local posture feature hash codes of the two frame images;

[0117] A composite hash code generation module is used to combine the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images;

[0118] A calculation module, used for calculating the Jaccard distance between the composite hash codes of two frame images;

[0119] The judging module is used to judge whether the Jaccard distance between the composite hash codes of two frame images is less than a preset threshold ε. If so, one of the frame images and its corresponding composite hash code is deleted; if not, the two frame images are retained to complete the cleaning of the frame image data.

[0120] This scheme is a data cleaning system for power distribution safety and supervision operations. Through the cooperation of the acquisition module, extraction module, selection and execution module, texture feature hash code calculation module, posture feature hash code calculation module, composite hash code generation module, calculation module and judgment module, it realizes the cleaning of several frame images extracted from the video of power distribution safety and supervision operations. This scheme uses the OpenPose-GLCM-hash algorithm to perform deduplication processing on several frame images extracted from the video of power distribution safety and supervision operations. The OpenPose-GLCM-hash algorithm effectively integrates the OpenPose human posture detection and gray level co-occurrence matrix GLCM feature extraction technology, which not only captures the overall structural information of the image, but also enhances the recognition ability of the local dynamic features of the characters in the image. This enables the algorithm to more accurately identify the local details in the scene when facing application scenarios that need to pay attention to the local content of the image, effectively reducing the misjudgment rate and improving the efficiency and accuracy of deduplication.

[0121] Preferably, the texture feature hash code calculation module includes:

[0122] A first conversion submodule, used for converting a frame image into a grayscale image;

[0123] A first calculation submodule is used to calculate the gray level co-occurrence frequency of pixel pairs in the gray image at a given distance and direction to construct a GLCM;

[0124] The first extraction submodule extracts texture features from GLCM, and the texture features include energy, contrast, entropy and correlation;

[0125] The first generation submodule is used to perform hash processing on the texture feature to generate a global texture feature hash code.

[0126] Specifically, in the first conversion submodule, by converting the original frame image into a grayscale image, it is helpful to simplify the complexity of image processing. In the first calculation submodule, GLCM is a matrix whose elements represent the frequency of occurrence of two pixel grayscale value pairs at a given direction and distance. In this embodiment, the given distance d is 1 and the given direction θ is 45°. In the first extraction submodule, a variety of texture features can be extracted from GLCM, namely energy, contrast, entropy and correlation. Energy can measure the uniformity of image texture. The higher the energy, the more uniform the texture; contrast can reflect the contrast of image grayscale. The higher the contrast, the clearer the texture; entropy can describe the randomness of image texture. The larger the entropy, the more complex the texture; correlation can reflect the linear correlation between image grayscale. In the first generation submodule, hash processing is usually a function that maps data of arbitrary length to shorter fixed-length data, converting texture features into global texture feature hash codes, which can reflect the overall content of the image.

[0127] Preferably, the posture feature hash code calculation module includes:

[0128] The second extraction submodule is used to extract information of 14 skeleton joints from the frame image using the improved OpenPose algorithm;

[0129] The first sorting submodule is used to arrange the information of the 14 skeleton joint points in a certain order to form a skeleton joint point sequence;

[0130] Establish a submodule to build a long short-term memory network LSTM model;

[0131] The input submodule is used to input the skeleton joint point sequence into the LSTM model for processing;

[0132] Output submodule, used to output local posture feature vector;

[0133] The second conversion submodule is used to convert the local posture feature vector into a binary local posture feature hash code by using supervised deep hashing SDH.

[0134] In the present embodiment, in the second extraction submodule, the 14 skeleton joints extracted from the frame image include the positions of the human head, shoulders, elbows, wrists, hips, knees and ankles. In the establishment submodule, by establishing the LSTM model, it is conducive to the subsequent processing of the skeleton joint sequence. In the input submodule, after the LSTM model receives the skeleton joint sequence, the LSTM captures the dynamic changes of the skeleton joints in the time series through the mechanism of the forget gate, the input gate and the output gate. Specifically, for the position of each frame of the skeleton joint, the LSTM generates corresponding meta-features, which describe the changes of the human body posture in the time series, so that complex motion patterns can be captured. In the second conversion submodule, SDH optimizes the hash process by relaxing the loss function to ensure that similar posture features are mapped to similar binary codes. By converting the high-dimensional posture feature vector into a compact binary hash code, subsequent storage, transmission and retrieval operations are facilitated.

[0135] Preferably, the composite hash code generation module includes:

[0136] A combination submodule, used for combining the global texture feature hash code T of the frame image and the local posture feature hash code B of the frame image into a hash code F=[TB], wherein the length of the hash code F is 80;

[0137] The second generating submodule is used to generate a pseudo-random secret stream with a length of 80 using a key;

[0138] A second sorting submodule, for sorting the elements in the pseudo-random secret stream;

[0139] A recording submodule, used for recording the original position of each element in the pseudo-random secret stream to generate a vector Z;

[0140] The third sorting submodule is used to reorder the elements in the hash code F using the vector Z to obtain a new hash code H, wherein the relationship between the elements in the new hash code H and the elements in the vector Z satisfies the following formula:

[0141] h(q)=f(z(q));

[0142] Wherein, h(q) represents the qth element in the new hash code H, z(q) represents the qth element in the vector Z, and f() represents the transformation function that reorders the vector Z;

[0143] The quantization submodule is used to quantize the elements in the new hash code H to obtain the quantized new hash code H, which is the composite hash code of the frame image.

[0144] In this embodiment, in the second generation module, a key is used to generate a pseudo-random secret stream of length 80 for constructing a secure hash. In the third sorting submodule, the elements in the hash code F are reordered using the vector Z to obtain a new hash code H, which means that if the qth element of the vector Z is m, then the qth element of the new hash code H is the mth element in the hash code F. In the quantization submodule, the elements in the new hash code H are quantized, which is conducive to saving storage space and improving security.

[0145] Preferably, the calculation module includes:

[0146] A third conversion submodule is used to convert the composite hash codes of the two frame images into two sets by using a mapping method;

[0147] The second calculation submodule is used to calculate the size of the intersection and union of two sets;

[0148] The third calculation submodule is used to calculate the Jaccard distance L between the composite hash codes of the two frame images according to the size of the intersection and union of the two sets. The specific calculation formula is as follows:

[0149]

[0150] In this embodiment, the Jaccard distance between two composite hash codes can be calculated through the cooperation between the third conversion submodule, the second calculation submodule and the third calculation submodule, so as to facilitate the subsequent detection of the similarity between two frame images.

[0151] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0152] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A data cleaning method for power distribution safety monitoring operations, characterized in that: The following steps are involved: Step S1: Collecting videos of power distribution safety supervision operations; Step S2: extracting a number of frame images from the video of the power distribution safety supervision operation and forming a frame image sequence; Step S3: continue to select any two frame images from the frame image sequence, and perform steps S4 to S7 on them, and repeat this process until all frame images in the frame image sequence have been compared two by two and processed accordingly; Step S4: performing gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain global texture feature hash codes of the two frame images; Step S5: using the improved OpenPose algorithm to extract local posture features from the two frame images respectively, and performing hash calculations on the two local posture features respectively to obtain local posture feature hash codes of the two frame images; Step S6: Combining the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images; Step S7: Calculate the Jaccard distance between the composite hash codes of the two frame images, and determine whether the Jaccard distance between the composite hash codes of the two frame images is less than a preset threshold ε. If so, delete one of the frame images and its corresponding composite hash code. If not, retain the two frame images to complete the cleaning of the frame image data.

2. A data cleaning method for power distribution safety monitoring operations according to claim 1, characterized in that: In step S4, gray level co-occurrence matrix GLCM hash processing is performed on the frame image to obtain a global texture feature hash code of the frame image, which specifically includes the following sub-steps: Step S41: converting the frame image into a grayscale image; Step S42: Calculate the gray level co-occurrence frequency of pixel pairs in the gray image at a given distance and direction to construct a GLCM; Step S43: extracting texture features from GLCM, the texture features including energy, contrast, entropy and correlation; Step S44: Perform hash processing on the texture features to generate a global texture feature hash code.

3. A data cleaning method for power distribution safety monitoring operations according to claim 1, characterized in that: In step S5, the improved OpenPose algorithm is used to extract local posture features from the frame image, and the local posture features are hashed to obtain the local posture feature hash code of the frame image, which specifically includes the following sub-steps: Step S51: extracting information of 14 skeleton joints from the frame image using the improved OpenPose algorithm, and arranging the information of the 14 skeleton joints in a certain order to form a skeleton joint sequence; Step S52: Establishing a long short-term memory network LSTM model; Step S53: input the skeleton joint point sequence into the LSTM model for processing, and output a local posture feature vector; Step S54: Use supervised deep hashing SDH to convert the local posture feature vector into a binary local posture feature hash code.

4. A data cleaning method for power distribution safety monitoring operations according to claim 1, characterized in that: In step S6, the global texture feature hash code of the frame image is combined with the local posture feature hash code of the frame image to generate a composite hash code of the frame image, which specifically includes the following sub-steps: Step S61: combining the global texture feature hash code T of the frame image and the local posture feature hash code B of the frame image into a hash code F=[TB], wherein the length of the hash code F is 80; Step S62: using a key to generate a pseudo-random secret stream with a length of 80, and sorting the elements in the pseudo-random secret stream and recording the original position of each element to generate a vector Z; Step S63: Use the vector Z to reorder the elements in the hash code F to obtain a new hash code H, wherein the relationship between the elements in the new hash code H and the elements in the vector Z satisfies the following formula: h(q)=f(z(q)); Wherein, h(q) represents the qth element in the new hash code H, z(q) represents the qth element in the vector Z, and f() represents the transformation function that reorders the vector Z; Step S64: quantize the elements in the new hash code H to obtain a quantized new hash code H, which is a composite hash code of the frame image.

5. The data cleaning method for power distribution safety monitoring operation according to claim 1 is characterized in that: In step S7, the Jaccard distance between the composite hash codes of the two frame images is calculated, which specifically includes the following sub-steps: Step S71: converting the composite hash codes of the two frame images into two sets by using a mapping method; Step S72: Calculate the size of the intersection and union of the two sets; Step S73: Calculate the Jaccard distance L between the composite hash codes of the two frame images according to the size of the intersection and union of the two sets. The specific calculation formula is as follows:

6. A data cleaning system for power distribution safety monitoring operations, characterized by: Using the data cleaning method for power distribution safety and supervision operations as described in any one of claims 1 to 5, the system includes: The acquisition module is used to collect videos of power distribution safety and supervision operations; An extraction module is used to extract a number of frame images from the video of power distribution safety supervision operation and form a frame image sequence; A selection and execution module is used to continuously select any two frame images from the frame image sequence, and execute the texture feature hash code calculation module, the posture feature hash code calculation module, the composite hash code generation module, the calculation module and the judgment module on them, and repeat this process until all the frame images in the frame image sequence have been compared two by two and processed accordingly; The texture feature hash code calculation module is used to perform gray level co-occurrence matrix GLCM hash processing on the two frame images respectively to obtain the global texture feature hash codes of the two frame images; A posture feature hash code calculation module is used to extract local posture features from two frame images respectively using an improved OpenPose algorithm, and perform hash calculations on the two local posture features respectively to obtain local posture feature hash codes of the two frame images; A composite hash code generation module is used to combine the global texture feature hash codes of the two frame images with the local posture feature hash codes of the corresponding frame images to generate a composite hash code of the two frame images; A calculation module, used for calculating the Jaccard distance between the composite hash codes of two frame images; The judgment module is used to judge whether the Jaccard distance between the composite hash codes of two frame images is less than a preset threshold ε. If so, one of the frame images and its corresponding composite hash code is deleted. If not, the two frame images are retained to complete the cleaning of the frame image data.

7. A data cleaning system for power distribution safety monitoring operations according to claim 6, characterized in that: The texture feature hash code calculation module includes: A first conversion submodule, used for converting a frame image into a grayscale image; A first calculation submodule is used to calculate the gray level co-occurrence frequency of pixel pairs in the gray image at a given distance and direction to construct a GLCM; The first extraction submodule extracts texture features from GLCM, and the texture features include energy, contrast, entropy and correlation; The first generation submodule is used to perform hash processing on the texture feature to generate a global texture feature hash code.

8. A data cleaning system for power distribution safety monitoring operations according to claim 6, characterized in that: The posture feature hash code calculation module includes: The second extraction submodule is used to extract information of 14 skeleton joints from the frame image using the improved OpenPose algorithm; The first sorting submodule is used to arrange the information of the 14 skeleton joint points in a certain order to form a skeleton joint point sequence; Establish a submodule to build a long short-term memory network LSTM model; The input submodule is used to input the skeleton joint point sequence into the LSTM model for processing; Output submodule, used to output local posture feature vector; The second conversion submodule is used to convert the local posture feature vector into a binary local posture feature hash code by using supervised deep hashing SDH.

9. A data cleaning system for power distribution safety monitoring operations according to claim 6, characterized in that: The composite hash code generation module comprises: A combination submodule, used for combining the global texture feature hash code T of the frame image and the local posture feature hash code B of the frame image into a hash code F=[TB], wherein the length of the hash code F is 80; The second generating submodule is used to generate a pseudo-random secret stream with a length of 80 using a key; A second sorting submodule, for sorting the elements in the pseudo-random secret stream; A recording submodule, used for recording the original position of each element in the pseudo-random secret stream to generate a vector Z; The third sorting submodule is used to reorder the elements in the hash code F using the vector Z to obtain a new hash code H, wherein the relationship between the elements in the new hash code H and the elements in the vector Z satisfies the following formula: h(q)=f(z(q)); Wherein, h(q) represents the qth element in the new hash code H, z(q) represents the qth element in the vector Z, and f() represents the transformation function that reorders the vector Z; The quantization submodule is used to quantize the elements in the new hash code H to obtain the quantized new hash code H, which is the composite hash code of the frame image.

10. A data cleaning system for power distribution safety monitoring operations according to claim 6, characterized in that: The calculation module comprises: A third conversion submodule is used to convert the composite hash codes of the two frame images into two sets by using a mapping method; The second calculation submodule is used to calculate the size of the intersection and union of two sets; The third calculation submodule is used to calculate the Jaccard distance L between the composite hash codes of the two frame images according to the size of the intersection and union of the two sets. The specific calculation formula is as follows:

Images