A real-time video monitoring and identification method for safety behavior of power construction workers

Abstract

The present application relates to the field of image data processing technology, and in particular to a method for real-time video surveillance and identification of safety behaviors of electric power construction workers. The method comprises: obtaining a grayscale image of the head of an electric power construction worker in video surveillance, and determining a strip area from the grayscale image based on the information entropy of the grayscale values ​​of pixels within the neighborhood of a target pixel in the grayscale image; determining a first edge line based on the strip area; determining a second edge line where the jaw line of the electric power construction worker is located from the grayscale image, and determining a fit value between the first edge line and the second edge line. When the fit value is less than a second preset threshold, it is determined that the electric power construction worker is not wearing the helmet strap correctly. Through the above technical solution, it is possible to identify the helmet strap of an electric power construction worker who is not wearing the helmet strap correctly, thereby ensuring the safety of the electric power construction worker.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to a method for real-time video monitoring and identification of safety behaviors of electric power construction personnel. Background Art

[0002] Safety helmets can effectively ensure the safety of the heads of power construction workers during the power construction process. For example, safety helmets can effectively prevent head injuries caused by collisions between power construction workers and surrounding equipment during operation, and can also reduce the threat to the heads of power construction workers from falling objects that may exist during operation.

[0003] The related technology can detect whether the power construction workers are wearing safety helmets, so as to remind the power operators when they are not wearing safety helmets, and encourage the power construction workers to wear safety helmets; however, when the power construction workers wear safety helmets but do not wear the helmet straps correctly, it is difficult to effectively protect the heads of the power construction workers.

[0004] For example, when the chin strap of a safety helmet is in a loose state, the safety helmet of the electric power construction worker may collide with the electrical equipment during operation and fall off, causing potential danger to the head of the electric power construction worker.

[0005] When the safety helmet falls off, the power construction workers may be distracted by picking up the safety helmet, which will not only affect the safety of the electrical equipment operated by the power operators, but also affect the safety of the power operators themselves due to improper operation. Therefore, it is necessary to identify the behavior of power construction workers who do not wear the safety helmet strap correctly based on the wearing inspection of the safety helmets of power construction workers to ensure the safety of power construction workers. Summary of the Invention

[0006] In order to identify the behavior of electric power construction workers who fail to wear the chin strap of their safety helmets correctly, thereby ensuring the safety of electric power construction workers, the present application provides a real-time video monitoring and identification method for electric power construction workers' safety behavior, including: obtaining a grayscale image of the electric power construction worker's head in video monitoring, and using the information entropy of the grayscale values ​​of pixels in the neighborhood range of a target pixel point in the grayscale image as the characteristic value of the target pixel point; using pixels whose characteristic values ​​are less than a first preset threshold as candidate pixels, and determining a strip area based on the area composed of adjacent candidate pixels; the strip area at least passes through the target range at the bottom of the grayscale image; the grayscale value of the boundary of the strip area is within the preset grayscale range; determining a first position point located at the left boundary of the strip area and a second position point located at the right boundary, and using the shortest edge line in the boundary of the strip area passing through the first position point and the second position point as the first edge line; determining a second edge line where the electric power construction worker's jaw line is located from the grayscale image, and determining a fit value between the first edge line and the second edge line. When the fit value is less than the second preset threshold, it is determined that the electric power construction worker fails to wear the chin strap of his safety helmet correctly.

[0007] In this way, it is possible to identify the chin strap of the electric power construction worker who is not wearing the safety helmet correctly, thereby ensuring the safety of the electric power construction worker.

[0008] Optionally, the degree of fit value is determined in the following manner: determining the average value of the shortest distance from the pixel points on the first edge line to the second edge line, and determining the average value of the shortest distance from the pixel points on the first edge line to the second edge line to determine the distance matching degree value between the first edge line and the second edge line; dividing the first edge line and the second edge line by the same number of times, and determining the angle matching degree value between the first edge line and the second edge line based on the difference in the average tangent angles of the edge line segments corresponding to the first edge line and the second edge line obtained after the division; determining the degree of fit value based on the distance matching degree value and the angle matching degree value.

[0009] Optionally, the method also includes: when it is determined that the power construction personnel are not wearing the chin strap of the safety helmet correctly, obtaining the color characteristics of the power construction personnel's clothing from video surveillance; outputting a first prompt information based on the color characteristics of the power construction personnel's clothing; the first prompt is used to prompt the power construction personnel to wear the chin strap of the safety helmet correctly.

[0010] In this way, the color characteristics of the power construction workers' clothes can be combined to provide targeted reminders to the power construction workers.

[0011] Optionally, the second edge line is determined by: performing facial key point detection on the grayscale image of the power construction worker's head, obtaining multiple mandibular contour points of the power construction worker, and performing curve fitting on the multiple mandibular contour points, and using the curve obtained by fitting that passes through the multiple mandibular contour points as the second edge line.

[0012] In this way, a second edge line that is smoother and more closely matches the actual jaw line of the power construction worker can be obtained.

[0013] Optionally, the method also includes: obtaining the strip areas corresponding to the power construction workers at different times in the video surveillance, and matching the strip areas corresponding to different times to obtain consistency values; the consistency values ​​are used to characterize the degree of consistency of the strip areas at different times; and determining whether the power construction workers are wearing the helmet straps correctly based on the consistency values.

[0014] Optionally, the method also includes: determining the first device currently operated by the target power construction personnel based on video surveillance; obtaining the second device currently required to be operated as indicated by the target power construction personnel's operation task, and outputting a second prompt message when the first device is different from the second device; the second prompt message is used to prompt the target power construction personnel that the device currently operated is incorrect.

[0015] In this way, it is possible to prevent power construction personnel from operating equipment that is inconsistent with the operation tasks indicated, thereby ensuring the safety of the equipment operated by the power construction personnel.

[0016] Optionally, the method further includes: when the strip-shaped area does not exist in the grayscale image of the electric power construction worker's head, determining that the electric power construction worker does not wear the helmet strap correctly.

[0017] Optionally, the method also includes: determining the pixel mean and variance value of the local range of pixel points in the video surveillance picture, and adjusting the pixel values ​​of the pixel points in the video surveillance picture according to the pixel mean and variance value of the local range of pixel points; and re-using the image after the pixel values ​​of the pixel points are adjusted as the video surveillance picture.

[0018] In this way, a clearer video surveillance picture can be obtained after adjustment, and the contrast between the hatband and other parts in the video surveillance picture can be improved.

[0019] Optionally, the method also includes: determining the average duration of a single blink and the total number of blinks of the power construction personnel within a preset time period based on the real-time video surveillance footage; and performing fatigue detection on the power construction personnel based on the average duration of a single blink and the total number of blinks within the preset time period.

[0020] Optionally, the grayscale image of the head of the power construction worker is obtained by: inputting the real-time video surveillance image into a pre-trained YOLO network model to obtain the grayscale image of the head of the power construction worker output by the YOLO network model; the YOLO network model is used to output the grayscale image of the head of the person in the input image.

[0021] The technical solution provided by the embodiments of the present application may include the following beneficial effects: obtaining a grayscale image of the head of an electric power construction worker in video surveillance, and since there are at least pixels with high consistency and adjacent to each other inside the hatband, the information entropy of the grayscale values ​​of the pixels in the neighborhood range can be used to screen out a strip area that meets the characteristics of the hatband from the grayscale image, thereby determining the first edge line where the hatband is located; determining the second edge line where the jaw line of the electric power construction worker is located, and comparing the first edge line with the second edge line, so as to detect whether the electric power construction worker is wearing the hatband of the safety helmet correctly. Compared with only determining whether the electric power construction worker is wearing a safety helmet, it can better avoid the identification of the hatband of the electric power construction worker who is not wearing the safety helmet correctly, thereby ensuring the safety of the electric power construction worker.

[0022] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The present invention is a flowchart showing a method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to an exemplary embodiment. DETAILED DESCRIPTION

[0024] First, a brief introduction is given to the application scenario of the embodiment of the present application. In the application scenario of the present application, it is possible to detect whether the power construction personnel are wearing a safety helmet. However, when determining whether the power construction personnel are wearing a safety helmet, if the chin strap of the safety helmet worn by the power construction personnel is in a loose state, the safety helmet worn by the power construction personnel is difficult to effectively protect the safety of the power construction personnel. Therefore, it is difficult to identify the wearing behavior of the chin strap of the safety helmet of the power construction personnel, and it is difficult to effectively ensure the safety of the power construction personnel.

[0025] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0026] In order to solve the above technical problems, the present invention provides a method for real-time video monitoring and identification of safety behaviors of electric power construction personnel. Figure 1FIG. 1 is a flow chart showing a method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps.

[0027] In step S101, a grayscale image of the head of the electric power construction worker in the video surveillance is obtained, and the information entropy of the grayscale values ​​of the pixels in the neighborhood of the target pixel in the grayscale image is used as the feature value of the target pixel.

[0028] Video surveillance equipment can be installed at key locations on power construction sites to capture video surveillance footage of the entire construction process, so as to identify unsafe behaviors such as construction workers not wearing helmet straps correctly.

[0029] Key locations of an electric power construction site may include, for example, an exit location, an entrance location, and locations where electrical equipment is located.

[0030] In the obtained grayscale image of the electric power construction worker's head, the color characteristics of the electric power construction worker's helmet strap are usually significantly different from the electric power construction worker's skin color; for example, the color of the helmet strap is usually gray, black, or the edge is black.

[0031] There are at least a plurality of pixels with relatively consistent colors and adjacent to each other inside the chin strap of the safety belt. Therefore, there are at least a plurality of pixels with relatively consistent colors at the edge of the chin strap of the safety helmet. The information entropy can reflect the complexity of the grayscale values ​​in the neighborhood range of the pixel points. Therefore, the information entropy of the chin strap of the safety helmet is smaller at least at the edge.

[0032] Compared with the pores, hair, and skin texture that may exist on the face of power construction workers, the color characteristics of the helmet strap are more consistent. Therefore, using the information entropy of the grayscale values ​​of pixels in the neighborhood of the target pixel in the grayscale image as the eigenvalue of the target pixel can facilitate the determination of the area where the helmet strap is located from the grayscale image.

[0033] The neighborhood range may be, for example, a range of 3×3, 5×5, or 7×7 centered on the target pixel; the target pixel may be any pixel in the grayscale image.

[0034] The information entropy of the grayscale values ​​of pixels in the neighborhood of a target pixel in a grayscale image can be determined in the following way: determine the frequency ratio of different grayscale levels in the neighborhood respectively, and determine the information entropy of the grayscale values ​​of pixels in the neighborhood of the target pixel based on the frequency ratio of different grayscale levels. The information entropy can characterize the complexity of the grayscale values ​​of pixels in the neighborhood of the target pixel.

[0035] In one embodiment, a grayscale image of the head of a power construction worker is obtained by inputting a real-time video surveillance image into a pre-trained YOLO network model to obtain a grayscale image of the head of the power construction worker output by the YOLO network model; the YOLO network model is used to output a grayscale image of the head of the person in the input image.

[0036] The YOLO (You Only Look Once) network model can effectively locate targets in images. The YOLO network model can utilize existing network models in existing technologies for detecting the position of the operator's head without the need to retrain the network model.

[0037] Those skilled in the art may also train the YOLO network model according to actual needs. For example, the YOLO network model may be obtained by taking a sample image as input and taking the labeling result of the image area where the head of the person in the sample image is located as output for training. The YOLO network model is a relatively mature model in the prior art and will not be described in detail in the embodiments of the present application.

[0038] The real-time video surveillance footage is input into the pre-trained YOLO network model to obtain the grayscale image of the power construction worker's head output by the YOLO network model. This can avoid the interference of parts other than the head in the subsequent calculation process and can also reduce the time or difficulty required for subsequent processing of the grayscale image.

[0039] In one embodiment, the pixel mean and variance value of the local range of pixel points in the video surveillance picture can also be determined, and the pixel values ​​of the pixel points in the video surveillance picture can be adjusted according to the pixel mean and variance value of the local range of pixel points; the image after the pixel values ​​of the pixel points are adjusted is used as the video surveillance picture again.

[0040] The local range of the pixel point can be a range of 3×3, 5×5 or 7×7 centered on the pixel point, and the pixel value of the pixel point in the video surveillance image is adjusted respectively. The pixel value of the pixel point can be adjusted in combination with the local range of the pixel point.

[0041] In the scenario of monitoring the behavior of power construction workers, there may be uneven lighting, resulting in different light intensities in different parts of the same image. By adjusting the pixel values ​​of the local pixel mean and variance values, image quality problems caused by possible uneven lighting can be corrected.

[0042] For example, in a shadow area with a lower local pixel mean, a higher pixel value can be obtained after adjustment, making the dark details in the grayscale image clearer; in an overexposed area with a higher local pixel mean, the pixel value of the pixel point can be reduced accordingly after adjustment to prevent the loss of details that may be caused by overexposure.

[0043] By adjusting the pixel values ​​of the pixels in the grayscale image, the contrast between the chin strap of the helmet and other parts can be improved in the grayscale image obtained after adjustment, so as to determine the area where the chin strap is located from the grayscale image, thereby determining the effectiveness of the power construction personnel's wearing behavior of the chin strap of the helmet.

[0044] For example, the process of adjusting the pixel value of a pixel in a grayscale image can be performed in the following way: ;in, is the pixel value of the pixel The pixel value obtained after adjustment, k is a preset gain coefficient, and the value of the gain coefficient can be between 0.5 and 1; is the pixel mean of the local range of the pixel point, is the variance of the pixel values ​​within the local range of the pixel point, is a preset reference variance, which may be equal to the maximum value of variance of pixel values ​​within a local range of different pixel points in the grayscale image.

[0045] In step S102 , pixels whose characteristic values ​​are smaller than a first preset threshold are taken as candidate pixels, and a strip area is determined based on an area formed by adjacent candidate pixels.

[0046] The strip area at least passes through the target range at the bottom of the grayscale image; and the grayscale value of the boundary of the strip area is within a preset grayscale range.

[0047] The preset grayscale range can be determined based on the grayscale value of the helmet's strap; the color of the helmet's strap is usually gray or black, or the helmet's strap is black or gray at least in the edge area. Therefore, for a grayscale image with a grayscale value ranging from 0 to 255, the preset grayscale range can be in the range of 0 to 50.

[0048] Compared with the hair, pores and skin texture on the skin surface of power construction workers, the chin strap of the safety helmet has at least a relatively continuous and uniformly colored area. These areas have a band-like feature. When power construction workers wear safety helmets, the chin strap will pass underneath when they are worn correctly or when they are in a relaxed state, causing the chin strap of the safety helmet to appear in the lower area of ​​the grayscale image of the power construction workers' heads.

[0049] For example, the strap of the safety helmet passes through at least 1 / 3 of the bottom of the grayscale image of the head of the power construction worker, and the pixel points with characteristic values ​​less than the first preset threshold are taken as candidate pixel points. In addition, combined with the characteristic that the strap at least passes through the chin area of ​​the power construction worker, and the color of the strap is mainly black, gray, or at least black in the edge area, a strip area that is more consistent with the characteristics of the strap of the safety helmet can be obtained.

[0050] The first preset threshold value may be determined according to the value range of the eigenvalue; the eigenvalue may be normalized to be within the range of 0 to 1; the first preset threshold value may be between 0 and 0.4.

[0051] In the embodiment of the present application, the chin strap of the safety helmet can be combined with the relatively universal characteristics of the chin strap of the safety helmet to realize the recognition of the chin strap of the safety helmet; compared with using the image of the chin strap to train the network model, the required deployment cost is lower, and the chin strap of the safety helmet worn by the power construction personnel can be recognized at a lower cost.

[0052] In step S103 , a first position point located at the left boundary of the strip area and a second position point located at the right boundary are determined, and the shortest edge line passing through the first position point and the second position point on the boundary of the strip area is used as the first edge line.

[0053] When the power construction worker wears the chin strap of the safety helmet correctly or the chin strap is in a loose state, the chin strap of the safety helmet is U-shaped or V-shaped. By determining the first position point located at the left boundary of the strip area and the second position point located at the right boundary, the position points of different sides of the chin strap of the safety helmet can be determined, so as to determine whether the power construction worker wears the chin strap correctly.

[0054] The shortest edge line in the boundary of the strip area passing through the first position point and the second position point corresponds to the edge line of the side of the chin strap of the safety helmet that fits the skin of the power construction worker better, which can facilitate determining whether the chin strap is worn correctly.

[0055] All candidate edge lines passing through the first position point and the second position point in the boundary of the strip area may be determined, and the candidate edge line with the shortest length among all the candidate edge lines may be used as the shortest edge line.

[0056] In step S104, the second edge line where the jaw line of the electric power construction worker is located is determined from the grayscale image, and the fit value between the first edge line and the second edge line is determined. When the fit value is less than the second preset threshold, it is determined that the electric power construction worker does not wear the helmet strap correctly.

[0057] The jaw line of the electric power construction worker is the boundary line of the chin of the electric power construction worker's face. Since the jaw line and other parts will show relatively obvious boundary features in the grayscale image, the second edge line where the jaw line of the electric power construction worker is located can be determined based on the results of edge detection of the grayscale image. Alternatively, the technology of facial key point detection in the existing technology can be used to determine the second edge line where the jaw line of the electric power construction worker is located.

[0058] In one embodiment, the second edge line is determined by: performing facial key point detection on the grayscale image of the head of the power construction worker, obtaining multiple mandibular contour points of the power construction worker, and performing curve fitting on the multiple mandibular contour points, and using the curve obtained by fitting and passing through the multiple mandibular contour points as the second edge line.

[0059] Facial key point detection is performed on the grayscale image of the electric power construction worker's head. The facial key point detection model in the existing technology can be used to extract multiple jaw contour points of the electric power construction worker in the grayscale image. The facial key point detection model can be, for example, a facial key point detection model or algorithm in the existing technology such as the Dlib face detector or PFLD (A Practical Facial Landmark Detector, real-time facial key point detection algorithm).

[0060] Compared with using edge detection to extract the edge line of the jaw line in the grayscale image of the construction worker's head, the facial key point detection model realizes the learning of a large number of facial features. The obtained jaw contour points can be more in line with the actual situation of power construction workers. In addition, using the existing facial key point detection model to extract the jaw contour points can avoid the time and computing power required for model training.

[0061] By performing curve fitting on multiple mandibular contour points, a curve that closely matches the actual mandibular line of power construction workers can be obtained, making it easier to determine whether the power construction workers are wearing the chin strap of their helmets correctly. The fitted curve can more naturally and continuously represent the mandibular contour shape, avoiding the jagged or jumpy contour lines caused by discrete points.

[0062] In one embodiment, the degree of fit is determined in the following manner: determining the average value of the shortest distance from the pixel points on the first edge line to the second edge line, and determining the average value of the shortest distance from the pixel points on the first edge line to the second edge line to determine the distance matching value between the first edge line and the second edge line; dividing the first edge line and the second edge line by the same number of times, and determining the angle matching value between the first edge line and the second edge line based on the difference in the average tangent angles of the edge line segments corresponding to the first edge line and the second edge line obtained after the division; determining the degree of fit based on the distance matching value and the angle matching value.

[0063] The fit value between the first edge line and the second edge line can reflect the fit between the chin strap of the safety helmet and the skin of the power construction worker; when the power construction worker wears the chin strap of the safety helmet correctly, the chin strap of the power construction worker will fit the chin of the power construction worker more closely; and when the power construction worker does not wear the chin strap of the safety helmet correctly, the chin strap of the power construction worker and the chin of the power construction worker may be separated from each other. Therefore, the fit value can be better used to determine whether the power construction worker wears the chin strap of the safety helmet correctly.

[0064] For a pixel point on the first edge line, there is another pixel point on the second edge line that is closest to this pixel point on the first edge line; by using the average value of the shortest distances corresponding to different pixel points on the first edge line, combined with the average value of the shortest distances corresponding to different pixel points on the second edge line, the degree of fit between the first edge line and the second edge line can be evaluated from the perspective of the spatial position between the two edge lines.

[0065] The smaller the average value of the shortest distances corresponding to different pixel points on the first edge line, or the smaller the average value of the shortest distances corresponding to different pixel points on the second edge line, the greater the distance matching degree between the first edge line and the second edge line will be, and the higher the degree of fit between the power construction worker's hat strap and the power construction worker's chin will be.

[0066] On the contrary, the larger the average value of the shortest distances corresponding to different pixel points on the first edge line, or the larger the average value of the shortest distances corresponding to different pixel points on the second edge line, the smaller the distance matching degree value between the first edge line and the second edge line will be, the lower the fit between the power construction worker's hat strap and the power construction worker's chin will be, and the more likely the power construction worker's hat strap will be in a loose state.

[0067] By dividing the first edge line and the second edge line into the same number of divisions, the morphological features of the two edge lines can be matched in the same number; for example, the smallest number of pixel points in the first edge line and the second edge line can be used as the number of divisions for the first edge line and the second edge line.

[0068] For the pixel points on the first edge line or the second edge line, the tangent angle of the pixel points on the edge line can be determined, and the average value of the tangent angles of different pixel points in the edge line segments obtained after division is used as the average tangent angle of the edge line segments.

[0069] The angle matching degree between the first edge line and the second edge line can be determined based on the difference in average tangent angles between the edge line segments corresponding to the first edge line and the second edge line obtained after the division.

[0070] For example, the difference in average tangent angle between two edge line segments respectively coming from the first edge line and the second edge line can be determined in order from left to right, thereby determining the angle matching degree value between the first edge line and the second edge line based on the difference in average tangent angle between the edge line segments in different edge line segment pairs; wherein the edge line segment pair includes determining two edge line segments respectively coming from the first edge line and the second edge line and matching each other.

[0071] The distance matching degree value and the angle matching degree value can be normalized to the range of 0 to 1 to avoid the impact of the difference in value range or dimension of the two parameters on the subsequent calculation process; the average value of the distance matching degree value and the angle matching degree value can be used as the fit degree value.

[0072] Combining the two dimensions of distance and angle to determine the degree of fit between the first edge line and the second edge line can avoid misjudgment of the degree of fit between the two edge lines caused by local fitting, compared to determining the degree of fit between the first edge line and the second edge line only from the dimension of distance or only from the dimension of angle, thereby determining a more accurate degree of fit between the first edge line and the second edge line.

[0073] The calculation process of the degree of fit value in the embodiment of the present application is described in more detail below using an exemplary calculation formula as an example: , is the distance matching degree value after normalization, is the angle matching degree value after normalization.

[0074] Among them, the distance matching degree value , exp is an exponential function with a natural constant as the base, is the number of pixels in the first edge line, is the shortest distance from the i-th pixel point in the first edge line to the second edge line, is the number of pixels in the second edge line, is the shortest distance from the jth pixel point in the second edge line to the first edge line.

[0075] By using the shortest distance from a pixel point in the first edge line to the second edge line, and the shortest distance from a pixel point in the first edge line to the second edge line, the degree of fit between the two edge lines can be evaluated from the dimension of distance.

[0076] Angle matching value , exp is an exponential function with a natural constant as the base, is the preset influence coefficient, M is the number of divisions of the first edge line and the second edge line, is the average tangent angle of the mth edge line segment obtained after dividing the first edge line, is the average tangent angle of the mth edge line segment obtained after dividing the second edge line.

[0077] Preset influence coefficient The value can be between 2 and 5. The preset influence coefficient can balance the influence of the distance matching degree value and the angle matching degree value on the fit degree value. The larger the value of the influence coefficient, the smaller the angle matching degree value obtained, and the greater the influence of the angle matching degree value on the fit degree value.

[0078] When the tangent angles at different positions of the edge line are different, the shape of the edge line is different. Therefore, by utilizing the difference in the average tangent angles of the edge line segments corresponding to the first edge line and the second edge line obtained after division, the two edge lines can be compared from the dimension of the edge line shape to determine whether the chin strap of the safety helmet of the power construction worker is worn correctly.

[0079] In one embodiment, when the strip-shaped region does not exist in the grayscale image of the electric power construction worker's head, it can be determined that the electric power construction worker does not wear the chin strap of the safety helmet correctly.

[0080] Since the band-shaped area is the image area in the grayscale image that matches the characteristics of the chin strap worn, when the band-shaped area exists in the grayscale image of the power construction worker's head, it at least indicates that the power construction worker is wearing the chin strap of a safety helmet, for example, the power construction worker is wearing the chin strap of the safety helmet correctly, or the chin strap worn by the power construction worker is in a loose state.

[0081] If the strip-shaped area does not exist in the grayscale image of the electric power construction worker's head, it means that the electric power construction worker is wearing a helmet but has not installed the chin strap, which makes the wearing of the chin strap of the safety helmet in a dangerous state; or, the electric power construction worker may not be wearing a safety helmet, which makes the electric power construction worker's head in a dangerous state.

[0082] When the strip area does not exist in the grayscale image of the electric power construction worker's head, it is determined that the electric power construction worker has not correctly worn the chin strap of the safety helmet, which can help the supervisor or supervisory equipment to remind the electric power construction worker so that the electric power construction worker wears the safety helmet and the chin strap correctly, thereby ensuring the safety of the electric power construction worker's head.

[0083] In one embodiment, when it is determined that the power construction worker is not wearing the chin strap of the safety helmet correctly, the color characteristics of the power construction worker's clothing can be obtained from video surveillance; a first prompt information is output based on the color characteristics of the power construction worker's clothing; the first prompt is used to prompt the power construction worker to wear the chin strap of the safety helmet correctly.

[0084] There may be multiple power construction workers at the power construction site. If the power construction workers are only reminded to wear the chin straps of their safety helmets correctly, they may choose to ignore the output prompts. In the embodiment of the present application, by obtaining the color characteristics of the power construction workers' clothing, the output first prompt information can be made more targeted, thereby achieving targeted reminders to power construction workers who do not wear the chin straps of their safety helmets correctly.

[0085] For example, if it is determined that an electric power construction worker is not wearing the chin strap of a safety helmet correctly, the color characteristics of the electric power construction worker's clothing can be determined based on video surveillance. If the electric power construction worker's top is black and his pants are blue, a prompt can be given: "Construction worker with a black top and blue pants, please wear the chin strap of your safety helmet correctly."

[0086] In one embodiment, the strip areas corresponding to the power construction workers at different times in the video surveillance can also be obtained, and the strip areas corresponding to different times are matched to obtain consistency values; the consistency values ​​are used to characterize the degree of consistency of the strip areas at different times; and whether the power construction workers are wearing the helmet straps correctly is determined based on the consistency values.

[0087] When the power construction workers wear the chin strap of the safety helmet correctly, the chin strap of the power construction workers is in a relatively stable state, and the shape of the chin strap of the power construction workers does not change with the movement of the power construction workers; the shape of the chin strap of the power construction workers has good consistency at different times.

[0088] On the contrary, when the power construction workers do not wear the chin strap of the safety helmet correctly, the chin strap of the power construction workers is in a relatively unstable state, and the shape of the chin strap of the power construction workers will change with the movement of the power construction workers; for example, the chin strap of the safety helmet will sway with the movement of the power construction workers, making the shape of the chin strap of the power construction workers less consistent at different times.

[0089] The matching of strip areas corresponding to different moments can be achieved using matching algorithms such as SIFT (Scale Invariant Feature Transform) and SURF (Speeded-Up Robust Features).

[0090] When matching two strip regions of different lengths, the ratio of the shorter strip region to the longer strip region can be used as the matching ratio between the two strip regions; when matching two strip regions of the same length, the ratio of one strip region to the other strip region can be used as the matching ratio between the two strip regions; the average value of the matching ratios corresponding to different adjacent moments can be used as the consistency value obtained by matching the strip regions corresponding to different moments.

[0091] Determining whether the power construction personnel are wearing the chin strap of the safety helmet correctly based on the consistency value may include: when the consistency value is greater than or equal to a preset consistency threshold, determining that the power construction personnel are wearing the chin strap of the safety helmet correctly; or, when the consistency value is less than the preset consistency threshold, determining that the power construction personnel are not wearing the chin strap of the safety helmet correctly; the value range of the preset consistency threshold may be between 0.7 and 0.8.

[0092] In one embodiment, the first device currently operated by the target power construction personnel can also be determined based on video surveillance; the second device currently required to be operated as indicated by the target power construction personnel's operation task is obtained, and if the first device and the second device are different, a second prompt message is output; the second prompt message is used to prompt the target power construction personnel that the device currently operated is incorrect.

[0093] The target electric power construction worker may be any electric power construction worker among all electric power construction workers. When performing operations on electrical equipment, electric power construction workers generally need to perform the operations in the order in which the operation tasks are assigned.

[0094] When the electrical equipment operated by the power construction personnel is inconsistent with the order assigned to the operation tasks, it may affect the safety of the electrical equipment. Therefore, by comparing the currently operated equipment with the equipment assigned to the operation tasks, it helps to ensure that the power construction personnel perform the operation tasks in the order assigned and ensure the safety of the electrical equipment.

[0095] In one embodiment, the average duration of a single blink and the total number of blinks of the power construction personnel within a preset time period can also be determined based on the real-time video surveillance images; fatigue detection of the power construction personnel can be performed based on the average duration of a single blink and the total number of blinks within the preset time period.

[0096] Eye key point detection can be used to determine whether the eyes of the power construction workers are in an open or closed state in the video surveillance footage at a certain moment. Based on the moments when the power construction workers' eyes are in an open or closed state within the same time period, the duration of each single blink of the power construction workers can be determined.

[0097] The duration of a single blink refers to the time required for the eyes to change from an open state to a closed state, and from a closed state to an open state; for example, the eyes of a power construction worker are in an open state at T1, T2 and T3, in a closed state at T4, and in an open state at T5, T6 and T7. The power construction worker is in a blinking state during the time period of T3, T4 and T5, and the duration of a single blink is the time period corresponding to the time period of T3, T4 and T5.

[0098] When the power construction workers are in a fatigued state, the duration of a single blink of the power construction workers will be longer than the duration of a single blink of the power construction workers in an awake state; and within the same time period, the total number of blinks of the power construction workers in a fatigued state is less than the total number of blinks of the power construction workers in an awake state. Therefore, based on the average value of the single blink duration and the total number of blinks within the preset time period, it is possible to determine whether the power construction workers are in a fatigued state and realize fatigue detection of the power construction workers.

[0099] If the average duration of a single blink within the preset time period is greater than the preset time period, for example, the average duration of a single blink is greater than 0.5 seconds, it can be determined that the power construction personnel are in fatigue detection; or, if the total number of blinks within the preset time period is less than the preset number, for example, the total number of blinks within one minute is less than 10 times, it can be determined that the power construction personnel are in fatigue detection.

[0100] When power construction workers are working in a fatigued state, the probability of them making incorrect operations is higher, and the probability of them colliding with electrical equipment is higher, which not only poses a threat to the electrical equipment they operate, but also affects the safety of the power construction workers. Therefore, fatigue detection of power construction workers helps to ensure the safety of power construction workers and electrical equipment.

[0101] When it is determined that the electric power construction workers are in a fatigue state, the electric power construction workers can be reminded that they are in a fatigue state; for example, the reminder can be made through the terminal device, safety helmet or other wearable device held by the electric power construction workers.

[0102] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A real-time video monitoring and identification method for safety behavior of electric power construction personnel, characterized in that: include: Obtain a grayscale image of the head of a power construction worker in video surveillance, and use the information entropy of the grayscale values ​​of pixels in the neighborhood of a target pixel in the grayscale image as the feature value of the target pixel; Pixels whose characteristic values ​​are less than a first preset threshold are taken as candidate pixels, and a strip area is determined based on an area formed by adjacent candidate pixels; The strip area at least passes through the target range at the bottom of the grayscale image; the grayscale value of the boundary of the strip area is within a preset grayscale range; Determine a first position point located at the left boundary of the strip area and a second position point located at the right boundary, and use the shortest edge line of the strip area passing through the first position point and the second position point as the first edge line; Determining a second edge line where the jaw line of the electric power construction worker is located from the grayscale image, and determining a fit value between the first edge line and the second edge line; if the fit value is less than a second preset threshold, determining that the electric power construction worker is not wearing the helmet strap correctly; The degree of fit is determined by: determining an average value of the shortest distances from pixels on the first edge line to the second edge line, and determining an average value of the shortest distances from pixels on the second edge line to the first edge line, to determine a distance matching degree value between the first edge line and the second edge line; Performing the same number of divisions on the first edge line and the second edge line, and determining an angle matching degree value between the first edge line and the second edge line based on a difference in average tangent angles of edge line segments corresponding to the first edge line and the second edge line obtained after the divisions; The fit degree value is determined according to the distance matching degree value and the angle matching degree value.

2. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The method further comprises: When it is determined that the electric power construction worker is not wearing the chin strap of the helmet correctly, the color characteristics of the electric power construction worker's clothing are obtained from the video surveillance; The first prompt information is output according to the color characteristics of the clothes of the electric power construction personnel; the first prompt is used to remind the electric power construction personnel to wear the chin strap of the safety helmet correctly.

3. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The second margin line is determined as follows: Facial key point detection is performed on the grayscale image of the electric power construction worker's head to obtain multiple mandibular contour points of the electric power construction worker, and curve fitting is performed on the multiple mandibular contour points. The curve obtained by fitting and passing through the multiple mandibular contour points is used as the second edge line.

4. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The method further comprises: Obtain the strip areas corresponding to the power construction workers at different times in the video surveillance, and match the strip areas corresponding to different times to obtain consistency values; the consistency values ​​are used to represent the consistency levels of the strip areas at different times; The consistency value is used to determine whether the power construction workers are wearing the chin strap of the safety helmet correctly.

5. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The method further comprises: Determining, based on video surveillance, a first device currently being operated by a target power construction worker; Obtain the second device currently required to be operated as indicated by the operation task of the target power construction personnel, and output a second prompt message when the first device and the second device are different; the second prompt message is used to prompt the target power construction personnel that the device currently operated is incorrect.

6. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The method further comprises: If the strip-shaped region does not exist in the grayscale image of the electric power construction worker's head, it is determined that the electric power construction worker does not wear the chin strap of the safety helmet correctly.

7. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The method further comprises: Determining a pixel mean and a variance value of a local range of pixel points in a video surveillance image, and adjusting a pixel value of a pixel point in the video surveillance image according to the pixel mean and the variance value of the local range of pixel points; The image after adjusting the pixel values ​​of the pixels is used as the video surveillance screen again.

8. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The method further comprises: Based on the real-time video surveillance footage, determine the average duration of a single blink and the total number of blinks of the power construction workers within a preset time period; Fatigue detection of power construction workers is carried out based on the average duration of single blinks and the total number of blinks within a preset time period.

9. The method for real-time video monitoring and identification of safety behaviors of electric power construction personnel according to claim 1 is characterized in that: The grayscale image of the electric power construction worker's head is obtained in the following way: The real-time video surveillance image is input into a pre-trained YOLO network model to obtain a grayscale image of the head of the power construction worker output by the YOLO network model; the YOLO network model is used to output a grayscale image of the head of the person in the input image.

Images