A water conservancy pipeline construction personnel behavior anomaly detection method

Abstract

The application relates to the technical field of video image processing, in particular to a water conservancy pipeline construction personnel behavior abnormality detection method. The method acquires local feature information of pixel points in a gray video frame; acquires similar feature values according to the similarity degrees of the local feature information of different pixel points; divides the gray video frame into a water conservancy pipeline region and a personnel information region, screens out effective video frames, acquires abnormal change values of the effective video frames according to the differences between the complex values of the similar feature values in the water conservancy pipeline region and the personnel information region in the effective video frames; acquires abnormal motion values of the effective video frames based on the abnormal change values, and extracts key frames of personnel abnormalities in a water conservancy pipeline construction site video based on the abnormal motion values. The application characterizes abnormal motion information in a video frame, extracts key frames of personnel abnormalities in a water conservancy pipeline construction site video through the abnormal motion values between frames, and improves the accuracy of key frame extraction.

Description

Technical Field

[0001] This invention relates to the field of video image processing technology, specifically to a method for detecting abnormal behavior of water conservancy pipeline construction workers. Background Technology

[0002] The construction of water conservancy pipelines involves numerous procedures, typically including trench excavation, pipeline laying, and pipeline connection and sealing. Due to the complex environment of the construction site, accidents caused by personnel violations during deep foundation pit operations, high-altitude work, and machinery operation can lead to injuries and fatalities. Especially in large-scale construction projects with numerous large construction machines and poor emergency response capabilities, the most common cause of dangerous accidents is the violation of regulations by construction workers.

[0003] Currently, computer vision and deep learning technologies are widely used in construction sites to provide more targeted identification solutions for unsafe behaviors. However, because video surveillance at construction sites captures a large amount of video data, proactive warnings cannot be provided before accidents occur. Furthermore, it is impossible to accurately pinpoint the critical time of an accident from the vast amount of video surveillance data after it happens. This results in limited on-site control in water conservancy construction projects, and low efficiency and accuracy in extracting and locating irregular and abnormal behaviors. Summary of the Invention

[0004] To address the technical problem of inaccurate keyframes for abnormal personnel behavior in water conservancy pipeline videos caused by using random, equally spaced keyframe extraction algorithms, this invention aims to provide a method for detecting abnormal behavior of water conservancy pipeline construction personnel. The specific technical solution adopted is as follows:

[0005] This invention proposes a method for detecting abnormal behavior of construction workers in water conservancy pipelines, the method comprising:

[0006] Acquire video footage from the construction site of the water conservancy pipeline and process it to obtain grayscale video frames;

[0007] Based on the local texture information and grayscale distribution of each pixel within a preset window in the grayscale video frame, local feature information of each pixel is obtained; based on the similarity of the local feature information of each pixel with the pixels within the preset window, similarity feature values ​​of each pixel in the grayscale video frame are obtained.

[0008] Based on the similarity feature values, each grayscale video frame is divided into a water conservancy pipeline area and a personnel information area; at least two valid video frames are selected based on the number of pixels in the water conservancy pipeline area.

[0009] According to the difference between the chaos degree of the similar feature values in the water conservancy pipeline area and the personnel information area in each valid video frame, an abnormal change value of each valid video frame is obtained; based on the difference of the abnormal change values of the continuous valid video frames and the position change of the pixel points, an abnormal motion value of each valid video frame is obtained;

[0010] Based on the abnormal motion value of the valid video frame, a key frame of personnel abnormality in the water conservancy pipeline camera video is extracted.

[0011] Preferably, the water conservancy pipeline construction site camera video is obtained and processed to obtain a gray video, comprising:

[0012] The frame rate of the camera is set to 25, each frame of the water conservancy pipeline construction site camera video is taken as the input of the Gaussian filter, and the weighted average method is used to process the output result, and the processed result is recorded as a gray video frame.

[0013] Preferably, the method for obtaining the local feature information comprises:

[0014] For each pixel point in each gray video frame, the average value of the gray values of the pixel points in the preset window of each pixel point is taken as the comprehensive gray value of each pixel point; the LBP value of each pixel point is obtained, and the average value of the LBP values of all pixel points in the preset window of each pixel point is taken as the texture value of each pixel point;

[0015] The feature binary tuple of the corresponding pixel point is composed of the comprehensive gray value and the texture value of each pixel point; the feature binary tuple of each pixel point in each gray video frame is taken as the local feature information of the corresponding pixel point.

[0016] Preferably, the method for obtaining the similar feature value comprises:

[0017] For each pixel point in each gray video frame, the cosine similarity of the feature binary tuple of the pixel point and each pixel point in the preset window of the pixel point is calculated as the similarity value between the corresponding two pixel points; the similarity values between the pixel point and each pixel point in the preset window of the pixel point are accumulated to obtain the similar feature value of each pixel point in each gray video frame.

[0018] Preferably, the method for dividing each gray video frame into a water conservancy pipeline area and a personnel information area based on the similar feature value is:

[0019] The maximum inter-class variance method is used for the similar feature value of each pixel point in each gray video frame to obtain a segmentation threshold value of each gray video frame; the region composed of the pixel points with the similar feature value greater than or equal to the segmentation threshold value in each gray video frame is taken as the water conservancy pipeline area, and the region composed of the pixel points with the similar feature value less than the segmentation threshold value is taken as the personnel information area.

[0020] Preferably, the method for obtaining the effective video frame comprises:

[0021] counting the number of pixels in the water pipeline region in each grayscale video frame, and performing negative correlation and normalization on the number of pixels in the water pipeline region to obtain an effective information coefficient of each grayscale video frame;

[0022] taking the grayscale video frame with the effective information coefficient greater than a preset abnormal pixel threshold value as the effective video frame.

[0023] Preferably, the method for obtaining the abnormal change value comprises:

[0024] for each effective video frame, taking the water pipeline region and the personnel information region in the effective video frame as an analysis region;

[0025] for any one of the analysis regions, substituting the probability of the value of each similar feature value in the analysis region into an information entropy formula to obtain a complexity value of the analysis region; taking the ratio of the similar feature value of each pixel in the analysis region to the maximum value of the similar feature value as a similar ratio of each pixel; and accumulating the similar ratios of each pixel in the analysis region to obtain a comprehensive similarity value of the analysis region;

[0026] performing negative correlation mapping on the comprehensive similarity value, and taking the product of the mapping result and the complexity value as a structure disorder degree; and taking the ratio of the structure disorder degree of the personnel information region to the water pipeline region in each effective video frame as an abnormal change value of each effective video frame.

[0027] Preferably, the method for obtaining the abnormal motion value of each effective video frame based on the difference of the abnormal change value and the position change of the pixel point of the continuous effective video frame comprises:

[0028] using a dense optical flow algorithm for each effective video frame to obtain the offset of each pixel point in the x direction and the y direction in each effective video frame;

[0029] combining the offset of the pixel point in the x direction and the y direction in the effective video frame and the difference of the abnormal change value of the continuous effective video frame to obtain an abnormal motion value of each effective video frame.

[0030] Preferably, the method for obtaining the abnormal motion value comprises:

[0031] The calculation formula of the abnormal motion value is as follows:

[0032]

[0033] In the formula, x is the abnormal motion value of the effective video frame, and y is the difference of the abnormal change value of the continuous effective video frame. an abnormal motion value of the i th effective video frame; a preset analysis number; a number of pixel points in a personnel information region in the i th effective video frame; an offset amount of the k th pixel point in the x direction in the t th effective video frame; an offset amount of the k th pixel point in the y direction in the t th effective video frame; an abnormal change value of the t th effective video frame; an abnormal change value of the t+1 th effective video frame; Norm is a normalization function.

[0034] Preferably, the key frame acquisition method comprises the following steps.

[0035] taking the average value of the abnormal motion values of the effective video frames as an abnormal key threshold value;

[0036] when the abnormal motion value of an effective video frame is greater than the abnormal key threshold value, the effective video frame is a key frame of personnel abnormality.

[0037] The present application has the following beneficial effects:

[0038] In the embodiment of the present application, for the gray video frame of the water pipeline, the local texture information and the gray distribution in the preset window of the pixel points are used to accurately present the local feature information of the pixel points from the aspects of texture and gray information, and the similarity of the pixel points at different positions in the gray video frame is analyzed according to the local feature information, and the similarity feature value obtained represents the data characteristics of different categories of pixel points; the gray video frame is divided into a water pipeline region and a personnel information region based on the similarity feature value, in order to reduce the inaccuracy of subsequent analysis caused by less personnel abnormal information in the gray video frame and reduce the analysis workload, the effective video frame is screened out based on the number of pixel points in the water pipeline region, and subsequent analysis is carried out based on the effective video frame; if there is personnel abnormal behavior motion change information, the personnel information region and the water pipeline region have relatively obvious difference change information, and the abnormal change value obtained presents the change of the complexity of the similarity feature value in the two regions; the present application represents the motion feature information between the continuous effective video frames, and extracts the key frame of personnel abnormality in the water pipeline camera video based on the obtained abnormal motion value; the traditional algorithm directly uses the RGB color and other features to accurately represent the abnormal behavior feature of the water pipeline camera video data frame, and the accuracy of the key frame extraction of personnel abnormality in the water pipeline construction site camera video is improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0040] Figure 1 A method flow chart of a water pipeline construction personnel behavior anomaly detection method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purpose, the specific implementation, structure, features and effects of the water pipeline construction personnel behavior anomaly detection method according to the present application will be described in detail below in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0043] The specific scenario to which the present application is directed is personnel abnormal behavior key frame detection in water pipeline camera video.

[0044] The specific scheme of the water pipeline construction personnel behavior anomaly detection method provided by the present application will be specifically described below in combination with the drawings.

[0045] Please refer to Figure 1 which shows a method flow chart of a water pipeline construction personnel behavior anomaly detection method provided by an embodiment of the present application, and the method comprises:

[0046] Step S1: Obtain water pipeline construction site camera video and process to obtain gray video frame.

[0047] Industrial CCD camera is used to obtain video data around the water pipeline construction site. In order to ensure smooth display of water pipeline construction site video changes, combined with the human eye visual residual characteristics, the frame rate FPS of the CCD camera is 25, that is, there are 25 frames of water pipeline video data frames per second.

[0048] The video data of the water pipeline construction site obtained by collection is a color video frame image in RGB color space. In order to avoid the influence of noise points in the collection process on the accuracy of subsequent calculation, the video frame data of the water pipeline construction site in the three different channels of RGB is processed by using the Gaussian filtering method, so as to weaken or even eliminate the influence of random noise in the collection process. At the same time, in order to avoid repeated calculation in the three different channels of RGB in the subsequent calculation process, the processed video data frame of the water pipeline construction site is converted into a corresponding gray video frame by using the weighted average method.

[0049] Step S2: obtaining local feature information of each pixel point according to local texture information and gray distribution in a preset window of each pixel point in the gray video frame; and obtaining a similar feature value of each pixel point in the gray video frame according to a similarity degree of the local feature information of each pixel point and pixel points in the preset window of the pixel point.

[0050] Specifically, in the collected video data of the water pipeline construction site, if there is abnormal behavior motion change information of personnel, there will be relatively obvious difference change information with the static information of the water pipeline construction site. Therefore, the motion change data of different frames in the video data of the water pipeline construction site is analyzed.

[0051] The local texture information in the preset window of the pixel point reflects the texture feature information in the local area of the pixel point, the gray distribution presents the gray feature information in the local area of the pixel point, and the combination of the two makes the local feature information of the pixel point more accurate.

[0052] Preferably, the method for obtaining the local feature information is that: for each pixel point in each gray video frame, taking the mean value of the gray values of the pixel points in the preset window of each pixel point as the comprehensive gray value of each pixel point; obtaining the LBP value of each pixel point, and taking the mean value of the LBP values of all pixel points in the preset window of each pixel point as the texture value of each pixel point; constructing a feature pair of the corresponding pixel point from the comprehensive gray value and the texture value of each pixel point; and taking the feature pair of each pixel point in each gray video frame as the local feature information of the corresponding pixel point.

[0053] As an example, for each gray video frame, if there is abnormal behavior motion of personnel, there will be relatively obvious difference change with the static information of the water pipeline construction site. A preset window is constructed with each pixel point in the gray video frame as the center, and the size of the preset window in the embodiment of the application is empirically taken as 5 5, and the implementer can set it according to the specific situation.

[0054] The local texture information within a preset window of a pixel reflects the texture features of a local area of ​​the pixel, while the grayscale distribution presents the grayscale features of a local area of ​​the pixel. Combining the two yields the local feature information of the pixel. Taking pixel j in a grayscale video frame as an example, the average grayscale value of all pixels within the preset window of pixel j is taken as the comprehensive grayscale value of pixel j. Calculate the LBP value of each pixel within a preset window of pixel j, and use the average LBP value of each pixel within the preset window of pixel j as the texture value of pixel j. The combined grayscale value of pixel j and texture values The feature tuple that makes up pixel j The method for obtaining the LBP value is a well-known technique among those skilled in the art and will not be elaborated upon here.

[0055] The degree of similarity of local feature information between a pixel and the pixels within its preset window represents the probability that pixels within a local area belong to the same type of pixel.

[0056] Preferably, the method for obtaining similar feature values ​​is as follows: for each pixel in each grayscale video frame, calculate the cosine similarity between the feature tuple of the pixel and each pixel in the preset window, and use it as the similarity value between the corresponding two pixels; accumulate the similarity values ​​between the pixel and each pixel in the preset window to obtain the similar feature value of each pixel in each grayscale video frame.

[0057] The larger the similarity feature value of a pixel, the more similar the pixel is to the features of its surrounding pixels. Since the features of different parts of a water conservancy pipeline are relatively similar, while the features of different locations of personnel information are quite different, the similarity feature values ​​of pixels representing water conservancy pipeline information are larger, while the similarity feature values ​​of pixels representing personnel information are smaller.

[0058] The higher the cosine similarity between the feature pairs of two pixels, the greater the similarity between the feature pairs of the two pixels. Calculate the cosine similarity between the feature pairs of pixel j and pixel k within a preset window. Let be the similarity value between pixel j and pixel k. Assuming pixel j is a water pipe pixel, if the similarity value... The larger the similarity value, the more similar the texture features of pixel j are to the local region of pixel k within the preset window, thus increasing the likelihood that pixel k is a water conservancy pipeline pixel; if the similarity value is... The smaller, the greater the possibility that the pixel point k is a personnel information pixel point. The similarity values between the pixel point j and each pixel point in the preset window of the pixel point j are accumulated to obtain the similarity feature value of the pixel point j.

[0059] In other embodiments of the present application, the Euclidean distance between the feature pair of the pixel point j and each pixel point in the preset window of the pixel point j can be calculated; the smaller the Euclidean distance between the two feature pairs, the higher the similarity between the two feature pairs. The Euclidean distances between the feature pairs corresponding to the pixel point j and each pixel point in the preset window of the pixel point j are accumulated, and the accumulated result is negatively correlated to obtain the similarity feature value of the pixel point j.

[0060] Taking the pixel point j as an example, the similarity feature value of the pixel point j is obtained according to the similarity between the feature pair of the pixel point j and each pixel point in the preset window of the pixel point j. The calculation formula of the similarity feature value is as follows:

[0061]

[0062] In the formula, is the similarity feature value of the jth pixel point in the gray video frame; n is the number of pixel points in the preset window of the jth pixel point in the gray video frame; is the feature pair of the jth pixel point in the gray video frame; is the feature pair of the kth pixel point in the local window of the jth pixel point in the gray video frame; is the similarity value between the jth pixel point and the kth pixel point in the preset window of the jth pixel point in the gray video frame.

[0063] It should be noted that when the similarity value The greater, the higher the similarity between the texture features and the gray distribution of the local region of the pixel point j and the pixel points in the preset window of the pixel point j, and the higher the possibility that the pixel point j and the pixel points in the preset window of the pixel point j are the same type of pixel points; that is, if the pixel point j is a water pipeline pixel point, the greater the possibility that the pixel points in the preset window of the pixel point j are water pipeline pixel points, and the greater the similarity feature value of the pixel point j. The greater.

[0064] Step S3: dividing each gray video frame into a water pipeline region and a personnel information region based on the similarity feature value; and screening at least two effective video frames based on the number of pixel points in the water pipeline region.

[0065] The similarity feature value presents the similarity between the pixel points in the local region, and based on the similarity feature value, the feature information of different pixel points in the gray video frame of the water pipeline can be effectively distinguished and represented.

[0066] In the embodiments of the present application, the maximum between-cluster variance method is used for the similar feature values of the pixel points in each gray video frame to obtain the segmentation threshold of each gray video frame; the region composed of the pixel points with the similar feature values greater than or equal to the segmentation threshold in each gray video frame is taken as the water pipeline region, and the region composed of the pixel points with the similar feature values less than the segmentation threshold is taken as the personnel information region. The maximum between-cluster variance method is a known technology for those skilled in the art, and will not be described here.

[0067] It should be noted that the greater the similar feature value of a pixel point, the more similar the features of the pixel point and its surrounding pixel points. The pipeline is usually made of uniform material, has a smooth surface, and the color and texture change gently. The gray scale and texture of the pixel points on the surface of the pipeline are relatively similar, and thus the features of the pixel points on the surface of the pipeline and the surrounding pixel points are more similar, i.e., the similar feature value is greater. The clothes worn by the personnel can have different colors and patterns, and the edge exists at the junction of the skin region and the clothes region. In addition, the posture of the personnel is different, which produces complex wrinkles and shadows, so that the similarity of the gray scale and texture between the pixel points in the personnel region is low, and thus the similarity of the features of the pixel points in the personnel region and the surrounding pixel points is poor, i.e., the similar feature value is smaller. Therefore, the pixel points with the similar feature value greater than or equal to the segmentation threshold are selected to form the water pipeline region, and the pixel points with the similar feature value less than the segmentation threshold form the personnel information region.

[0068] In other embodiments of the present application, the K-Means clustering algorithm can be used for the similar feature values of the pixel points in each gray video frame, wherein K=2, two clustering clusters are obtained. The mean values of the similar feature values in the two clustering clusters are calculated respectively, the region composed of the pixel points corresponding to the similar feature values in the clustering cluster with the greater mean value of the similar feature values is taken as the water pipeline region, and the region composed of the pixel points corresponding to the similar feature values in the clustering cluster with the smaller mean value of the similar feature values is taken as the personnel information region. The K-Means clustering algorithm is a known technology for those skilled in the art, and will not be described here.

[0069] The number of pixel points in the water pipeline region reflects the proportion of the personnel information region in the frame in the gray video frame, and thus whether the frame contains sufficient personnel information for analysis is determined.

[0070] Preferably, the method for obtaining the effective video frame is: the number of pixel points in the water pipeline region in each gray video frame is counted, the number of pixel points in the water pipeline region is negatively correlated and normalized to obtain an effective information coefficient of each gray video frame; and the gray video frame with the effective information coefficient greater than a preset abnormal pixel threshold is taken as the effective video frame.

[0071] (1) Obtain the effective information coefficient of the gray video frame.

[0072] As an example, the similarity feature value of the pixel point in the water pipeline region is larger than the similarity feature value of the pixel point in the personnel information region. The number N of pixel points in each gray video frame and the number of pixel points in the water pipeline region are counted respectively , the ratio of the number of pixel points in the water pipeline region to the number N of pixel points in the gray video frame is calculated . Since the subsequent effective information coefficient of the gray video frame needs to be judged with the preset abnormal pixel threshold value in the value range [0, 1], the value range of the effective information coefficient is ensured to be [0, 1], and therefore the constant 1 is subtracted from the number of pixel points in the water pipeline region , which is negatively correlated and normalized to obtain the effective information coefficient rd of the gray video frame.

[0073] In other embodiments of the present application, the number of pixel points in the personnel information region in the gray video frame can be directly normalized, and the normalized result is taken as the effective information coefficient of the gray video frame.

[0074] According to the number of pixel points in the water pipeline region in each gray video frame, the effective information coefficient of each gray video frame is obtained. The calculation formula of the effective information coefficient is as follows:

[0075]

[0076] In the formula, is the effective information coefficient of the i-th gray video frame; is the number of pixel points in the water pipeline region in the i-th gray video frame, and N is the number of pixel points in the i-th gray video frame.

[0077] It should be noted that in the fixed background water pipeline monitoring scene, when the construction personnel has abnormal behaviors such as shielding or moving, the personnel will shield part of the pipeline region, resulting in a decrease in the pixel of the pipeline region in the video frame and an increase in the pixel of the personnel region. Because the purpose of the present scheme is to detect the abnormal behavior of the personnel, it is necessary to first ensure that there is enough personnel information region in the video frame for analysis. If the effective information coefficient is larger, the number of pixel points in the water pipeline region in the gray video frame is smaller, which means that the water pipeline information in the gray video frame is less, indicating that the frame is more likely to contain enough personnel information for subsequent behavior anomaly analysis, and has higher analysis value, so the frame is more likely to be an effective prerequisite for subsequent abnormal behavior detection. On the contrary, the personnel information region is too small (for example, no one in the frame), and the subsequent abnormal motion analysis will not make sense.

[0078] According to the above calculation method of the effective information coefficient of the gray video frame, the effective information coefficient of each gray video frame is obtained.

[0079] (2) screening out effective video frames.

[0080] When the effective information coefficient is greater than the preset abnormal pixel threshold, it means that the proportion of the personnel information region in the gray video frame is large enough to meet the minimum requirement for subsequent behavior anomaly analysis, that is, the frame has analysis value, and the greater the possibility of the gray video frame being an effective video frame. When the effective information coefficient is less than or equal to the preset abnormal pixel threshold, it means that the proportion of the personnel information region in the gray video frame is too small (there may be no person or the proportion of the person is extremely low), which does not have statistical significance or reference value for subsequent abnormal motion analysis, and therefore the gray video frame with too little information is removed.

[0081] It should be noted that the preset abnormal pixel threshold is an empirical value of 0.7, which can be set by the implementer according to the specific circumstances.

[0082] At this point, the effective video frames in the water pipeline construction site camera video are screened out.

[0083] Step S4: obtaining an abnormal change value of each effective video frame according to the difference between the chaotic degree of similar feature values in the water pipeline region and the personnel information region in each effective video frame; and obtaining an abnormal motion value of each effective video frame based on the difference of the abnormal change values of the continuous effective video frames and the position change of the pixel points.

[0084] In the collected video data of the water pipeline construction site, if there is personnel abnormal behavior motion change information, the personnel information region and the water pipeline region have relatively obvious difference change information; and the change of the complexity of the similar feature values in the two regions presents the information change in the region.

[0085] Preferably, the method for obtaining the abnormal change value is: for each effective video frame, taking the water pipeline region and the personnel information region in the effective video frame as an analysis region; for any one analysis region, substituting the probability of the value of each similar feature value in the analysis region into the information entropy formula to obtain a complexity value of the analysis region; taking the ratio of the similar feature value of each pixel point in the analysis region to the maximum value of the similar feature value as the similar ratio of each pixel point; accumulating the similar ratio of each pixel point in the analysis region to obtain a comprehensive similar value of the analysis region; performing a negative correlation mapping on the comprehensive similar value, and taking the product of the mapping result and the complexity value as the structure disorder degree; and taking the ratio of the structure disorder degrees of the personnel information region and the water pipeline region of each effective video frame as the abnormal change value of each effective video frame.

[0086] As an example, for each valid video frame, since the analysis method of the water pipeline region and the personnel information region in the valid video frame is the same, for the convenience of analysis, the water pipeline region and the personnel information region in the valid video frame are taken as the analysis region.

[0087] (1) Obtain the complexity value of the analysis region.

[0088] The change of the complexity degree of the similar feature value in the analysis region presents the information change condition in the region, and the entropy reflects the complexity degree of data, so the entropy of the similar feature value in the analysis region is taken as the complexity value. The calculation formula of the complexity value is as follows:

[0089]

[0090] In the formula, is the complexity value of each analysis region of the i-th valid video frame; is the kind of the similar feature value in each analysis region of the i-th valid video frame; is the probability of the value of the a-th similar feature value in each analysis region of the i-th valid video frame; is the logarithm function with as the base.

[0091] It should be noted that the feature information of each part in the water pipeline region is relatively similar, and the feature information of each part in the personnel information region is quite different; since the greater the information difference in the region is, the greater the information entropy is, and therefore, compared with the personnel information region, the complexity value of the water pipeline region is smaller, and the complexity value of the personnel information region is larger.

[0092] In another embodiment of the present application, the variance of the similar feature value of the pixel point in the analysis region can be calculated as the complexity value of the analysis region. In other embodiments of the present application, the complexity value of the analysis region can be obtained by various basic mathematical operations and basic data processing algorithms such as standard deviation, which is not limited here.

[0093] (2) Obtain the abnormal change value of the valid video frame.

[0094] The ratio of the similar feature value of each pixel point in the analysis region to the maximum value of the similar feature value in the analysis region is taken as the similar ratio of each pixel point. The similar ratio of each pixel point in the analysis region is accumulated, and the accumulation result can reflect the comprehensive situation of the similar feature of the analysis region, so the comprehensive similar value of the analysis region is obtained; the comprehensive similar value is negatively correlated and mapped, and the product of the mapping result and the complexity value is taken as the structural disorder degree. In the embodiment of the present application, the structural disorder degree is expressed by the formula as follows:

[0095]

[0096] In the formula, is the structure disorder degree of the analysis region in the i-th valid video frame; is the complexity value of the analysis region in the i-th valid video frame; is the number of pixel points in the analysis region in the i-th valid video frame; is the similar feature value of the u-th pixel point in the analysis region in the i-th valid video frame; is the maximum value of the similar feature value of the pixel points in the analysis region in the i-th valid video frame; is the similar ratio of the analysis region in the i-th valid video frame; is the comprehensive similar value of the analysis region in the i-th valid video frame. The embodiment realizes negative correlation mapping through the exp function, and other methods such as taking the reciprocal can also be used, which is not limited here.

[0097] It should be noted that in the case where the personnel behavior in the video frame is not abnormal, the comprehensive similar value of the personnel information region is small and the complexity value is large, the overall gray scale of the personnel information region is not similar and the mode is disorderly, and the structure disorder degree of the personnel information region is greater; the comprehensive similar value of the water pipeline region is large and the complexity value is small, the overall gray scale of the water pipeline region is highly similar and the mode is single, and the structure disorder degree of the water pipeline region is smaller.

[0098] In the case where the personnel behavior in the video frame is not abnormal, since the water pipeline region is usually not affected by the abnormal behavior of the personnel, the complexity value and the comprehensive similar value of the water pipeline region are basically unchanged when the personnel is normal and abnormal, that is, the structure disorder degree of the water pipeline region is basically consistent when the personnel is in normal behavior and abnormal behavior; the abnormal behavior of the personnel will increase the texture disorder degree of the personnel region, most of the pixel points are less similar to the similar gray scale and texture features around them, resulting in that the similar feature value of the pixel points in the personnel region is reduced and more randomized, resulting in that the complexity value is larger, that is, the structure disorder degree of the personnel information region is greater when the personnel is in abnormal behavior than in normal behavior.

[0099] If the personnel changes from normal behavior to abnormal behavior, the structure disorder degree of the water pipeline region is basically unchanged, and the structure disorder degree of the personnel information region is increased, so the change degree of the structure disorder degree of the personnel information region relative to the water pipeline region can represent the degree of abnormal behavior of the personnel. Therefore, the ratio of the structure disorder degree of the personnel information region to the water pipeline region in the valid video frame is taken as an abnormal change value, and the greater the abnormal change value, the more abnormal the personnel behavior in the video frame. Among them, the water pipeline region and the personnel information region are in the same imaging condition in the same video frame, and the structure disorder degree of the water pipeline region can be regarded as the disorder reference line in the current environment, avoiding the interference caused by the change of light.

[0100] It should be noted that although the water pipeline is relatively smooth, the natural light or artificial lighting and other light sources at the water construction site cannot be absolutely uniform, resulting in differences in the similarity feature values of all pixels in the water pipeline area, and thus the structure clutter degree of the water pipeline area will not be zero.

[0101] When the personnel abnormal behavior information appears in the water pipeline video, i.e., the personnel abnormal behavior appears in the continuous effective video frames. In the continuously changing effective video frames, the gap between the distribution entropy of the similarity feature values in the water pipeline area and the personnel information area will change with the corresponding numerical gap change of the personnel abnormal motion; and the similarity feature index of different pixel points in the effective video frame will change accordingly; at this time, the abnormal change value of the effective video frame will change accordingly.

[0102] Preferably, the method for obtaining the abnormal motion value of the effective video frame is: using the dense optical flow algorithm for each effective video frame to obtain the offset amount of each pixel point in the x direction and the y direction in each effective video frame; combining the offset amount of the pixel point in the x direction and the y direction in the effective video frame, and the difference of the abnormal change value of the continuous effective video frames, to obtain the abnormal motion value of each effective video frame.

[0103] First, using the dense optical flow algorithm for each effective video frame to obtain the offset amount of each pixel point in the x direction and the offset amount of each pixel point in the y direction . Taking the pixel point k in the i-th effective video frame as an example for analysis, since when the personnel abnormal behavior information appears in the water pipeline video, the personnel abnormal behavior will appear in multiple continuous effective video frames, it is necessary to analyze multiple continuous effective video frames. The present application analyzes a preset analysis number of continuous effective video frames, and in the embodiment of the present application, the preset analysis number num takes the experience value 3, and the implementer can set it according to the specific circumstances. The dense optical flow algorithm is a known technology to those skilled in the art, which will not be described here.

[0104] For the continuous i-th, i+1-th and i+2-th effective video frames, the offset amount of the pixel point k in the x direction , and is obtained in turn, and the offset amount of the pixel point k in the y direction , and . The offset amount of the pixel point k in the x direction and the y direction in the i-th, i+1-th and i+2-th effective video frames is added respectively, and , and are obtained in turn, then the comprehensive offset amount of the pixel point k is the mean value of the sum of the three offset amounts .

[0105] Secondly, when there is personnel abnormal behavior information, the personnel information area in the effective video frame has movement, while the water pipeline area is in a static state, so the dense optical flow algorithm can be used to obtain the offset of each pixel point in the personnel information area in the x direction and the y direction, while the pixel points in the water pipeline area do not have offset. Therefore, the corresponding comprehensive offset of the pixel points in the personnel information area in the ith effective video frame is accumulated, and the accumulated result is normalized to obtain the optical flow feature value of the ith effective video frame.

[0106] After that, when the personnel abnormal behavior information appears in the water pipeline construction site video, the abnormal change value of the effective video frame will become larger, resulting in that the inter-frame motion abnormality value of the continuous effective video frame will also become larger. The abnormal change values of the ith, i+1 and i+2 effective video frames are , and respectively. The method for obtaining the inter-frame motion abnormality value is: calculating and ; the absolute value of the difference between the two abnormal change values is accumulated to obtain the motion abnormality value of the ith effective video frame.

[0107] The product of the optical flow feature value and the motion abnormality value of the ith effective video frame is taken as the abnormal motion value of the ith effective video frame. The calculation formula of the abnormal motion value is as follows:

[0108]

[0109] In the formula, is the abnormal motion value of the ith effective video frame; is a preset analysis quantity, which is an experience value 3; is the number of pixel points in the personnel information area in the ith effective video frame; is the offset of the kth pixel point in the x direction in the tth effective video frame; is the offset of the kth pixel point in the y direction in the tth effective video frame; is the abnormal change value of the tth effective video frame; is the abnormal change value of the t+1th effective video frame; is an absolute value function; Norm is a normalization function.

[0110] It should be noted that the more serious the personnel abnormal behavior in the water pipeline construction site video is, the more serious the offset of the pixel points in the personnel information area in the effective video frame in the x direction and in the y direction The greater, the abnormal motion value The greater, the abnormal motion value The greater, the abnormal motion value The greater, the abnormal motion value The greater, the abnormal motion value The greater, the abnormal motion value

[0111] At this point, the abnormal motion value of each effective video frame is obtained.

[0112] Step S5: Based on the abnormal motion value of the effective video frame, the key frame of the personnel anomaly in the water pipeline camera video is extracted.

[0113] Suppose the number of effective video frames obtained after the initial screening is F, the abnormal change value represents the abnormal motion change difference between different effective video frames in the water pipeline video, and the mean value of the abnormal motion value of the effective video frame is taken as the abnormal key threshold. The calculation formula of the abnormal key threshold is as follows:

[0114]

[0115] In the formula, T is the abnormal key threshold, F is the number of effective video frames; is the abnormal motion value of the gth effective video frame.

[0116] It should be noted that when the abnormal motion value of the effective video frame The greater, the abnormal motion value

[0117] When the abnormal motion value of the effective video frame is greater than the abnormal key threshold, the effective video frame is the key frame of the personnel anomaly, the key frame of the personnel anomaly is output, and the monitoring of the abnormal personnel behavior in the water pipeline construction site video is completed.

[0118] At this point, the present application is completed.

[0119] To sum up, in the embodiment of the present application, the local feature information of the pixel points in the gray video frame is acquired; the similar feature values are acquired according to the similarity of the local feature information of different pixel points; the gray video frame is divided into a water pipeline region and a personnel information region, and the effective video frame is screened out; the abnormal change value of the effective video frame is acquired according to the difference between the complex values of the similar feature values in the water pipeline region and the personnel information region in the effective video frame; the abnormal motion value of the effective video frame is acquired based on the abnormal change value, and the key frame of personnel abnormality in the water pipeline construction site video is extracted based on the abnormal motion value. The present application characterizes the abnormal motion information in the video frame, extracts the key frame of personnel abnormality in the video of the water pipeline construction site through the abnormal motion value between frames, and improves the accuracy of key frame extraction.

[0120] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0121] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments.

Claims

1. A method for detecting abnormal behavior of a water pipeline construction worker, characterized in that, The method comprises: Acquiring and processing the water pipeline construction site camera video to obtain a gray video frame; According to the local texture information and the gray distribution of each pixel point in the preset window, the local feature information of each pixel point is obtained; according to the similarity degree of each pixel point and the local feature information of the pixel points in the preset window, the similarity feature value of each pixel point in the gray video frame is obtained; Based on the similarity feature value, each gray video frame is divided into a water pipeline region and a personnel information region; based on the number of pixel points in the water pipeline region, at least two effective video frames are screened out; According to the difference between the chaos degrees of the similarity feature values in the water pipeline region and the personnel information region in each effective video frame, the abnormal change value of each effective video frame is obtained; based on the difference between the abnormal change values of the continuous effective video frames and the position change of the pixel points, the abnormal motion value of each effective video frame is obtained; Based on the abnormal motion value of the effective video frame, the key frame of the personnel anomaly in the water pipeline camera video is extracted; The method for obtaining the local feature information comprises: For each pixel point in each gray video frame, the average value of the gray values of the pixel points in the preset window of each pixel point is taken as the comprehensive gray value of each pixel point; the LBP value of each pixel point is obtained, and the average value of the LBP values of all the pixel points in the preset window of each pixel point is taken as the texture value of each pixel point; The feature binary tuple of the corresponding pixel point is formed by the comprehensive gray value and the texture value of each pixel point; the feature binary tuple of each pixel point in each gray video frame is taken as the local feature information of the corresponding pixel point; The method for obtaining the abnormal change value comprises: For each effective video frame, the water pipeline region and the personnel information region in the effective video frame are taken as an analysis region; For any one of the analysis regions, the probability of the value of each similarity feature value in the analysis region is substituted into an information entropy formula to obtain a complexity value of the analysis region; the ratio of the similarity feature value of each pixel point in the analysis region to the maximum value of the similarity feature value is taken as the similarity ratio of each pixel point; the similarity ratios of each pixel point in the analysis region are accumulated to obtain a comprehensive similarity value of the analysis region; The comprehensive similarity value is negatively correlated and mapped, and the product of the mapping result and the complexity value is taken as the structure disorder degree; the ratio of the structure disorder degrees of the personnel information region and the water pipeline region of each effective video frame is taken as the abnormal change value of each effective video frame; The method for obtaining the abnormal motion value of each effective video frame is as follows: For each effective video frame, a dense optical flow algorithm is used to obtain the offset amount of each pixel point in the x direction and the y direction in each effective video frame; The offset amount of the pixel points in the x direction and the y direction in the effective video frame and the difference between the abnormal change values of the continuous effective video frames are combined to obtain the abnormal motion value of each effective video frame.

2. The method according to claim 1, wherein The method for acquiring and processing the water pipeline construction site camera video to obtain a gray video comprises: The camera frame rate is set to 25, each frame of image of the water conservancy pipeline construction site camera video is taken as an input of a Gaussian filter, and a weighted average method is used to process the output result, and the processed result is recorded as a gray video frame.

3. The method according to claim 1, wherein The method for obtaining the similar feature value comprises: For each pixel point in each gray video frame, the cosine similarity of the feature pair between the pixel point and each pixel point in the preset window of the pixel point is calculated as the similarity value between the two corresponding pixel points, and the similarity values between the pixel point and each pixel point in the preset window of the pixel point are accumulated to obtain the similar feature value of each pixel point in each gray video frame.

4. The method according to claim 1, wherein The method for dividing each gray video frame into a water pipeline region and a personnel information region based on the similar feature value comprises: The maximum inter-class variance method is used for the similar feature value of each pixel point in each gray video frame to obtain a segmentation threshold of each gray video frame, a region composed of pixel points with the similar feature value greater than or equal to the segmentation threshold in each gray video frame is taken as the water pipeline region, and a region composed of pixel points with the similar feature value less than the segmentation threshold is taken as the personnel information region.

5. The method according to claim 1, wherein The method for obtaining the effective video frame comprises: The number of pixel points in the water pipeline region in each gray video frame is counted, the number of pixel points in the water pipeline region is negatively correlated and normalized to obtain an effective information coefficient of each gray video frame; A gray video frame with the effective information coefficient greater than a preset abnormal pixel threshold is taken as an effective video frame.

6. The method according to claim 1, wherein The method for obtaining the abnormal motion value comprises: The calculation formula of the abnormal motion value is as follows: wherein, is the abnormal motion value of the i-th valid video frame; is a preset analysis number; is the number of pixel points in the personnel information region in the i-th valid video frame; is the offset of the k-th pixel point in the x direction in the t-th valid video frame; is the offset of the k-th pixel point in the y direction in the t-th valid video frame; is the abnormal change value of the t-th valid video frame; is the abnormal change value of the t+1-th valid video frame; is an absolute value function; Norm is a normalization function.

7. The method according to claim 1, wherein The method for obtaining the key frame comprises: The mean value of the abnormal motion value of the effective video frame is taken as an abnormal key threshold; When the abnormal motion value of the effective video frame is greater than the abnormal key threshold, the effective video frame is a key frame of personnel anomaly.

Images