Video-based workplace safety inspection method and system, and medium

Abstract

The invention belongs to the technical field of machine vision, and particularly relates to a video-based workplace safety inspection method and system and a medium. According to the invention, through the video data of the workplace, standard dressing detection is realized, whether dressing types of all moving targets in each specific operation domain are in a limited range of the specific operation domain to which the moving targets belong is judged, and early warning is carried out on an intrusion cross-domain operation phenomenon of non-operating personnel; a pedestrian target is firstly positioned based on video data, a pedestrian target area is preprocessed and then safe dressing detection is carried out, and early warning is carried out on an intrusion operation phenomenon of non-operating personnel; whether the moving target area in each specific operation area exceeds the standard is judged, and an alarm is timely given when the operating personnel in the specific operation area exceed the standard; and the efficiency and the comprehensiveness of the dressing safety inspection of the oil field workplace are improved.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology, and in particular relates to a video-based workplace safety inspection method, system and medium. Background Art

[0002] The problem of improper dress of workers at oilfield operation sites often occurs. For example, oilfield workers do not wear safety helmets or work clothes when working, which leads to some unsafe incidents from time to time. Therefore, standardized safety clothing testing plays a vital role in promoting intelligent safety management of enterprises and protecting the safety of life and property of personnel.

[0003] On the one hand, for the standardized safety clothing inspection task, the oilfield operation site conducts safety clothing inspections on the operation site personnel manually, which is not comprehensive enough and inefficient. On the other hand, in the oilfield operation site, there is also the phenomenon of non-operating personnel invading and operating in violation of regulations in different operation areas, which not only poses a safety hazard, but also increases the difficulty of the clothing inspection task. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the purpose of the present invention is to provide a video-based workplace safety inspection method, system and medium.

[0005] The technical solution adopted by the present invention is:

[0006] A video-based workplace safety inspection method, wherein the workplace includes one or more specific work areas, and the method comprises the following steps:

[0007] S1: Acquire video data of the workplace, and detect moving targets in the workplace based on the video data. If a moving target is detected in the workplace, proceed to step S2;

[0008] S2: extracting the original image data of the work site and marking the moving target area; in the work time period, determining whether the moving target area on each specific work domain exceeds the standard, if so, triggering an alarm; otherwise, proceeding to step S3;

[0009] S3: Preprocess the original image data of the workplace, and input the preprocessed image data into the trained detection network to perform standard dress detection: detect whether the dress in the head area, body area and foot area of ​​the moving target is in violation of the rules; when the dress in the head area, body area and foot area is not in violation of the rules, proceed to step S4; otherwise, trigger an alarm;

[0010] S4: Determine whether the color of the clothing on the body area of ​​each moving target is within the limited range of the specific work domain to which it belongs. If so, the workplace is safe; otherwise, predict the movement trajectory of the moving target with abnormal clothing color in real time: obtain the predicted movement position of the current moving target, and determine whether the predicted movement position is within the operating position of the current specific work domain. If so, trigger an alarm, otherwise, the workplace is safe.

[0011] As a preferred solution of the present invention, in step S1, the moving target detection includes the following specific steps:

[0012] S11: extracting original image data of the workplace, scaling the pixels of the original image data to 640×640, and dividing the scaled image data slices into S×S grids;

[0013] S12: Input the output data of step S11 into the moving target detection model based on multi-scale feature fusion, and perform independent detection on four feature maps with scales of 20×20, 40×40, 80×80 and 160×160 to obtain four detection bounding boxes;

[0014] S13: Select the detection bounding box with the highest confidence as the moving target area.

[0015] As a preferred solution of the present invention, a mobile target detection model includes a feature extraction network and a Neck structure; wherein the feature extraction network includes a CSP structure and a Focus structure; the CSP structure performs a slicing operation, and the original 640×640×3 image data is first copied to 4 copies, and then the 4 images are cut into 4 320×320×3 slices through a slicing operation; then Concat is used to connect the 4 slices in depth, and the output is 320×320×12; then a convolution layer with a convolution kernel number of 32 is passed to generate an output of 320×320×32;

[0016] Step S12 includes the following sub-steps:

[0017] S121: Four independent detection branches are formed with four feature maps of scales of 20×20, 40×40, 80×80 and 160×160; each detection branch extracts the underlying feature map of the corresponding scale of the input data;

[0018] S122: Each detection branch obtains a deep feature map from the feature extraction network, upsamples the deep feature map in the FPN layer, and fuses the upsampled data with the bottom feature map to obtain a fused feature map;

[0019] S123, each detection branch fuses the fused feature map from bottom to top in the PAN layer and then downsamples it. Finally, each detection branch independently detects the downsampled data.

[0020] The single-stage detection method of the original YOLO series detects target objects through feature maps of three different scales, and outputs feature maps of three scales of 20×20, 40×40, and 80×80, respectively. However, only using features of three scales does not fully utilize shallow information, which will cause some target information to be lost; therefore, the present invention expands the original three-scale detection to four scales, forming a four-detection branch structure with an input size of 640×640, and each detection branch extracts features from the feature extraction network.

[0021] As a preferred solution of the present invention, in step S2, the method for determining whether the moving target area on the specific operation domain exceeds the standard includes:

[0022] Crop the images of each specific operation domain from the original image data;

[0023] Determine whether the number of moving target areas in each specific work domain image is greater than Y, where Y is the threshold number of workers in the specific work domain where the moving target area is located; if so, it is determined that the moving target area on the current specific work domain exceeds the standard; otherwise, it is determined that the moving target area on the current specific work domain does not exceed the standard.

[0024] As a preferred solution of the present invention, the method for determining whether the moving target area on a specific operation domain exceeds the standard further includes:

[0025] Crop the images of each specific operation domain from the original image data;

[0026] Calculate the area and A1 of all moving target areas in the current specific operation domain image, and calculate the area ratio K=A1 / A2, where A2 is the area of ​​the current specific operation domain image;

[0027] It is determined whether the area ratio K exceeds the threshold. If so, it is determined that the moving target area on the current specific operation domain exceeds the standard. Otherwise, it is determined that the moving target area on the current specific operation domain does not exceed the standard.

[0028] As a preferred solution of the present invention, the method for calculating the area and A1 of all moving target regions in the current specific operation domain picture includes:

[0029] T1: Get the area of ​​each moving target area;

[0030] T2: Correct the area of ​​each moving target area: determine whether the area A of the current moving target area is within the single area threshold range. If so, the area A of the current moving target area remains unchanged; otherwise, enter T3;

[0031] T3: Determine the range to which the current moving target area A belongs, the range including the i-th area threshold range, i=2, 3, ...; if the current moving target area A belongs to the i-th area threshold range, then replace the current moving target area A with: B*i; B is the maximum value of a single area threshold range;

[0032] T4: Calculate the area sum A1 using the corrected areas of each moving target region.

[0033] For the same moving target area, there may be multiple workers gathered together. When calculating the area of ​​the moving target area, the overlapping parts in the picture will be omitted, resulting in inaccurate judgment of whether the moving target area exceeds the standard. Therefore, in order to more accurately determine whether the moving target area exceeds the standard, this scheme uses a single area threshold and the i-th area threshold range to represent the working area occupied by an operator and i operators. When the area A of the current moving target area overlaps, the number of overlapping operators is determined, and the working area is calculated according to the actual number of operators to timely correct the area of ​​each moving target area.

[0034] As a preferred embodiment of the present invention, in step S3, the pre-processing method includes:

[0035] Obtain the coordinates of the upper left corner and the lower right corner of the moving target, and cut out the moving target frame from the original image data, first enlarge each moving target frame, and then sharpen and enhance the contrast of the pedestrian target area in each moving target frame;

[0036] Divide the pedestrian target area: first obtain the skeleton key points of the human body posture according to the Alphapose model where k n and c n Respectively represent the position and confidence of the nth skeleton key point;

[0037] The head area, body area and foot area are divided based on the skeleton key points:

[0038] The head area is divided by the nose skeleton key points, eye skeleton key points and ear skeleton key points:

[0039] Take the left shoulder skeleton key point (k LJ , c LJ )、Right shoulder skeleton key point (k RJ , c RJ ), left waist skeleton key points (k LY , c LY ) and the right waist skeleton key points (k RY , c RY ) to divide the body into regions:

[0040] Body Area Center Body area width w = max(k RY-x , k LY-x , k LJ-x , k RJ-x )-min(k RY-x , k LY-x , k LJ-x , k RJ-x ); Body area height h = max(k RY-y , k LY-y , k LJ-y , k RJ-y )-min(k RY-y , k LY-y , k LJ-y , k RJ-y );

[0041] where k RY-x is the x coordinate of the key point of the right waist skeleton, k RY-y is the y coordinate of the key point of the right waist skeleton, k LY-x is the x coordinate of the left waist skeleton key point, k LY-y is the y coordinate of the left waist skeleton key point; k RJ-x is the x coordinate of the key point of the right shoulder skeleton, k RJ-y The y coordinate of the right shoulder skeleton key point, k LJ-y is the y coordinate of the left shoulder skeleton key point, k LJ-x is the x-coordinate of the left shoulder skeleton key point;

[0042] The foot area is divided by the left foot skeleton key points and the right foot skeleton key points.

[0043] As a preferred embodiment of the present invention, step S4 includes the following sub-steps:

[0044] S41: Obtain the body area color of each moving target and the ratio D of the height to width of the body area. When D is less than 2.5, compare the color with the preset color library of the specific work area. If there is a matching color in the preset color library, the work place is safe; otherwise, proceed to step S42;

[0045] S42: Predict the predicted motion position of the current moving target at the future time t+1 based on the skeleton key points of the pedestrian target area:

[0046] The posture estimation is performed based on the skeleton key points of the pedestrian target area to obtain the walking angle a, rotation angle b, walking speed v and current position of pedestrian i at the current time t. Calculate the predicted motion position L at the future time t+1: Where S i is the relative position;

[0047] in is the position at time t-1+n; the future time t+1 is the shortest time for the pedestrian to reach the operating position of the current specific operation domain at the current walking speed v;

[0048] S43: Determine whether the predicted motion position is within the operating position of the current specific work domain. If so, trigger an alarm; otherwise, the workplace is safe.

[0049] Generally speaking, the colors of oil workers' clothing are divided into the following categories: red is the senior color among oil workers, representing the management personnel or senior technicians of oil drilling companies, who are generally responsible for on-site arrangements, commands, management and other work; orange is the intermediate color among oil workers, representing the color of on-site supervisors, who are usually responsible for specific construction work, such as rig operators, engineers, technicians, logging personnel, etc.; yellow is the primary color among oil workers, representing ordinary workers on site, who are generally engaged in some simple work, such as excavation, cleaning, and transportation; blue is the color of oil workers' safety equipment, representing safety clothing, who are generally wearing work protective equipment such as safety helmets, safety boots, and safety glasses; green is a special color among oil workers, representing special identities, such as professionals in environmental protection, water conservancy, geology, etc.; in short, through clothing of different colors, oil workers can quickly identify the identities, responsibilities and safety levels of their colleagues, ensuring work safety and efficiency. In view of this, this scheme uses the color of the oil operator's clothing to identify whether the mobile target belongs to the preset range of the current specific operation domain. In some special specific operation domains, only people with specific identities can work. For different specific operation domains, the clothing colors that can be used for work are pre-set. By identifying whether the color of the mobile target's body position belongs to the pre-set clothing color of the current specific operation domain, the mobile target with abnormal color (not the current specific operation domain personnel) is monitored and predicted in real time to obtain the predicted movement position of the current mobile target at the future t+1 moment. When the predicted movement position is within the operating position of the current specific operation domain, an alarm is triggered to effectively prevent cross-domain operators from invading the operation situation.

[0050] A video-based workplace safety inspection system:

[0051] It includes an acquisition module for acquiring video data of the working place;

[0052] The target detection module is used to detect moving targets in the workplace based on video data;

[0053] The first judgment module is used to extract the original image data of the workplace and mark the moving target when a moving target is detected in the workplace; and judge whether the moving target in each specific work area exceeds the standard during the work period;

[0054] A preprocessing module, used to preprocess the original image data of the workplace;

[0055] The clothing detection module is used to input the pre-processed image data into the trained detection network to perform standardized clothing detection;

[0056] The running trajectory prediction module is used to obtain the predicted moving position of the moving target when the clothing type is not within the limited range of the current specific operation domain, and determine whether the predicted moving position is within the operating position of the current specific operation domain;

[0057] The alarm module is used to give an alarm when the moving target in a specific work domain exceeds the standard, the moving target is not dressed properly, the head position of the moving target is dressed in violation of regulations, the body position of the moving target is dressed in violation of regulations, the clothing color of the moving target is not within the specified range of the current specific work domain, and the predicted motion position is at the operating position of the current specific work domain.

[0058] A computer-readable medium stores a computer program, which is executed by a processor to implement the above-mentioned video-based workplace safety inspection method.

[0059] The beneficial effects of the present invention are:

[0060] 1. The present invention uses video data from the workplace to, on the one hand, detect standardized clothing, and promptly issue an alarm for illegal clothing, thereby improving the efficiency of clothing safety inspections in oilfield workplaces. On the other hand, under the premise of standardized clothing, the present invention also determines whether the clothing types of all mobile targets in each specific work domain are within the limited range of the specific work domain to which they belong, and monitors the movement trajectories of non-operating personnel in the specific work domain in real time. When it is predicted that non-operating personnel are about to reach the operating position of the current specific work domain, an early warning is issued in time for the intrusion of non-operating personnel into cross-domain operations, thereby improving the comprehensiveness of clothing safety inspections in oilfield workplaces.

[0061] 2. Since oilfield operation sites require wearing safety helmets and color-coded work clothes, the present invention provides a new technical concept: first locate pedestrian targets based on video data, then pre-process the pedestrian target area and perform safety clothing detection, then determine the intrusion of non-operating personnel based on the color of the work clothes for pedestrian targets that are in compliance with the clothing, and monitor the movement trajectory of non-operating personnel in real time. When it is predicted that non-operating personnel are about to reach the operating position of the current specific operation domain, timely warn of the intrusion of non-operating personnel, thereby improving the comprehensiveness of safety inspections of clothing at oilfield operation sites.

[0062] 3. The present invention first performs larger target detection of operators, i.e., mobile target detection, and determines whether the mobile target area on each specific operating domain exceeds the standard during the operating time period, and promptly issues an alarm if the operators on the specific operating domain exceed the standard. The present invention then performs clothing detection on the head position and body position based on the mobile target area, and integrates the safety detection of the specific operating domain into the clothing detection process, and promptly issues an early warning for the intrusion of non-operating personnel. This not only effectively alleviates the problem that some targets are difficult to detect, but also improves the efficiency and comprehensiveness of clothing safety inspections in oilfield workplaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flowchart of the method of workplace safety inspection based on video;

[0064] Figure 2 It is a schematic diagram of the Neck network structure of the mobile target detection model;

[0065] Figure 3 It is a schematic diagram of the moving target area. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0067] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0068] The problem of improper dress of workers at oilfield operation sites often occurs. For example, oilfield workers do not wear safety helmets or work clothes when working, which leads to some unsafe incidents from time to time. Therefore, standardized safety clothing testing plays a vital role in promoting intelligent safety management of enterprises and protecting the safety of life and property of personnel.

[0069] On the one hand, for the standardized safety clothing inspection task, the oil field operation site conducts safety clothing inspections on the operation site personnel manually, which is not comprehensive enough and inefficient; on the other hand, in the oil field operation site, there is also the phenomenon of non-operating personnel invading different operation areas to perform cross-domain operations, which not only poses a safety hazard, but also increases the difficulty of the clothing inspection task. In view of this, the present invention provides the following embodiments to solve the above technical problems:

[0070] Embodiment 1:

[0071] This embodiment provides a video-based workplace safety inspection method, wherein the workplace includes one or more specific work areas, such as Figure 1 As shown, the method includes:

[0072] S1: Acquire video data of the workplace, and detect moving targets in the workplace based on the video data. If a moving target is detected in the workplace, proceed to step S2;

[0073] Moving target detection includes methods:

[0074] S11: extracting original image data of the workplace, scaling the pixels of the original image data to 640×640, and dividing the scaled image data slices into S×S grids;

[0075] S12: Input the output data of step S11 into the moving target detection model based on multi-scale feature fusion, and perform independent detection on four feature maps with scales of 20×20, 40×40, 80×80 and 160×160 to obtain four detection bounding boxes;

[0076] Step S12 includes the following sub-steps:

[0077] S121: Four independent detection branches are formed with four feature maps of scales of 20×20, 40×40, 80×80 and 160×160; each detection branch extracts the underlying feature map of the corresponding scale of the input data;

[0078] S122: Each detection branch obtains a deep feature map from the feature extraction network (the mobile target detection model of this embodiment includes a feature extraction network and a Neck structure, wherein the feature extraction network includes a CSP structure and a Focus structure, and the CSP structure performs a slicing operation, and the original 640×640×3 image data is first copied to 4 copies, and then the 4 images are cut into 4 320×320×3 slices through a slicing operation, and then the 4 slices are connected in depth using Concat, and the output is 320×320×12, and then a convolution layer with 32 convolution kernels is passed to generate an output of 320×320×32; the loss of original information is reduced, the amount of calculation is reduced, and the detection speed is accelerated.), and the deep feature map is upsampled in the FPN layer, and the upsampled data is fused with the bottom feature map to obtain a fused feature map;

[0079] S123: Each detection branch fuses the fused feature map from bottom to top in the PAN layer and then downsamples it. Finally, each detection branch independently detects the downsampled data.

[0080] The single-stage detection method of the original YOLO series detects target objects through feature maps of three different scales, and outputs feature maps of three scales of 20×20, 40×40, and 80×80, respectively. However, only using features of three scales does not fully utilize shallow information, which will lead to the loss of some target information. Therefore, this scheme expands the original three-scale detection to four scales, forming a four-detection branch structure with an input size of 640×640. Each detection branch extracts features from the feature extraction network.

[0081] The Neck structure of this solution is the enhanced path aggregation network PANet structure, such as Figure 2As shown in the figure, it is composed of FPN layer + PAN layer. The FPN layer uses a top-down approach to upsample and fuse the feature maps; the PAN layer uses a bottom-up approach to downsample and fuse the feature maps; the deep feature maps extracted from the feature extraction network are upsampled in the FPN layer and fused with the underlying feature maps of the corresponding size in the early stage of the network from deep to shallow into effective information, realizing the Concat connection operation; however, there is a problem in the FPN layer network. When the shallow feature map transmits information to the deep layer, it is difficult to fuse with the high-level feature map. Therefore, the PAN network layer is realized on the basis of the FPN feature pyramid network layer, and a bottom-up path is added. The feature map fused by the FPN layer continues to be downsampled and fused from the bottom to the top to realize the reverse fusion of the feature map, which will obtain richer feature information. The PAN layer is used to fuse the fused feature maps in the FPN network from bottom to top and perform downsampling operations to achieve reverse fusion of feature maps. Finally, independent detection is performed on the fused feature maps of four scales. The reverse fusion path through the PAN layer is fused again so that deep features can be detected more accurately and fine-grained. By fusing shallow feature information of more scales, the feature expression ability of the path aggregation network PANet is enhanced, the target detection accuracy is improved, and the missed detection rate is reduced.

[0082] S13: Select the detection bounding box with the highest confidence as the moving target area.

[0083] S2: extract the original image data of the workplace and mark the moving target area; during the working time period, determine whether the moving target area on each specific working domain exceeds the standard. If so, trigger an alarm, otherwise enter S3;

[0084] Methods for determining whether the moving target area on a specific operation domain exceeds the standard include:

[0085] Crop the images of each specific operation domain from the original image data;

[0086] Determine whether the number of moving target areas in each specific work domain image is greater than Y, where Y is the threshold number of workers in the specific work domain where the moving target area is located; if so, it is determined that the moving target area on the current specific work domain exceeds the standard; otherwise, it is determined that the moving target area on the current specific work domain does not exceed the standard.

[0087] The method for determining whether the moving target area on a specific operation domain exceeds the standard also includes:

[0088] Crop the images of each specific operation domain from the original image data;

[0089] Calculate the area and A1 of all moving target areas in the current specific operation domain image, and calculate the area ratio K=A1 / A2, where A2 is the area of ​​the current specific operation domain image;

[0090] It is determined whether the area ratio K exceeds the threshold. If so, it is determined that the moving target area on the current specific operation domain exceeds the standard. Otherwise, it is determined that the moving target area on the current specific operation domain does not exceed the standard.

[0091] The calculation method of the area and A1 of all moving target areas in the current specific operation domain image includes:

[0092] T1: Get the area of ​​each moving target area;

[0093] T2: Correct the area of ​​each moving target area: determine whether the area A of the current moving target area is within the single area threshold range. If so, the area A of the current moving target area remains unchanged; otherwise, enter T3;

[0094] T3: Determine the range to which the current moving target area A belongs, the range including the i-th area threshold range, i=2, 3, ...; if the current moving target area A belongs to the i-th area threshold range, then replace the current moving target area A with: B*i; B is the maximum value of a single area threshold range;

[0095] T4: Calculate the area sum using the corrected areas of each moving target area.

[0096] For the same moving target area, there may be multiple workers gathered together. When calculating the area of ​​the moving target area, the overlapping parts in the picture will be omitted, resulting in inaccurate judgment of whether the moving target area exceeds the standard. Therefore, in order to more accurately determine whether the moving target area exceeds the standard, this scheme uses a single area threshold and the i-th area threshold range to represent the working area occupied by an operator and i operators. When the area A of the current moving target area overlaps, the number of overlapping operators is determined, and the working area is calculated according to the actual number of operators to timely correct the area of ​​each moving target area.

[0097] like Figure 3 As shown, the moving target area e of this embodiment includes three workers P1, P2 and P3, but in fact the area of ​​the moving target area e is smaller than the sum of the workers P1, P2 and P3. Therefore, in order to accurately determine the workers in the moving target area e, the area of ​​the moving target area e needs to be corrected.

[0098] S3: Preprocess the original image data of the workplace, and input the preprocessed image data into the trained detection network to perform standard dress detection: detect whether the dress in the head area, body area and foot area of ​​the moving target is in violation of the rules; when the dress in the head area, body area and foot area is not in violation of the rules, proceed to step S4; otherwise, trigger an alarm;

[0099] The pretreatment method comprises:

[0100] Obtain the coordinates of the upper left corner and the lower right corner of the moving target, and cut out the moving target frame from the original image data, first enlarge each moving target frame, and then sharpen and enhance the contrast of the pedestrian target area in each moving target frame;

[0101] Divide the pedestrian target area: first obtain the skeleton key points of the human posture according to the Alphapose model (k n , c n ), where k n and c n Respectively represent the position and confidence of the nth skeleton key point;

[0102] The head area, body area and foot area are divided based on the skeleton key points:

[0103] The head area is divided by the nose skeleton key points, eye skeleton key points and ear skeleton key points:

[0104] Take the left shoulder skeleton key point (k RJ , c RJ )、Right shoulder skeleton key point (k LJ , c LJ ), left waist skeleton key points (k RR , c RY ) and the right waist skeleton key points (k RY , c RY ) to divide the body into regions:

[0105] Body Area Center Body area width w = max(k RY-x , k RY-x , k LJ-x , k RJ-x )-min(k RY-x , k RY-x , k LJ-x , k RJ-x ); Body area height h = max(k RY-y , k RY-y , k LJ-y , k RJ-y )-min(k RY-y , k RY-y , k LJ-y , k RJ-y );

[0106] The foot area is divided by the left foot skeleton key points and the right foot skeleton key points.

[0107] In order to effectively alleviate the problem that some smaller targets are difficult to detect, this solution enlarges each moving target frame, and then divides the head position and body position of the pedestrian target area. The body position can be further divided into upper body, lower body, feet and hands as needed; after the detailed division, it is necessary to build models for the upper body, lower body, feet and hands respectively.

[0108] According to the four target frame extraction positions obtained by the preset detailed division (the upper body target frame extraction position, the lower body target frame extraction position, the foot target frame extraction position and the hand target frame extraction position), the moving target frame is subjected to convolution feature extraction to obtain four convolution feature maps, and each target frame extraction position corresponds to a convolution feature map;

[0109] Then, the four convolutional feature maps are compared with the pre-trained convolutional feature maps with the same extraction position to obtain four comparison scores. The scores are compared with the preset thresholds to obtain the inspection results of various parts of the safety clothing of the personnel on the work site.

[0110] S4: Determine whether the color of the clothing on the body area of ​​each moving target is within the limited range of the specific work domain to which it belongs. If so, the workplace is safe. Otherwise, predict the running trajectory of the moving target with abnormal clothing color in real time: obtain the predicted movement position of the current moving target, and determine whether the predicted movement position is within the operating position of the current specific work domain. If so, trigger an alarm; otherwise, the workplace is safe.

[0111] S4 includes the following sub-steps,

[0112] S41: Obtain the body area color of each moving target and the ratio D of the height to width of the body area. When D is less than 2.5, compare the color with the preset color library of the specific work area. If there is a matching color in the preset color library, the work place is safe; otherwise, enter S42;

[0113] S42: Predict the predicted motion position of the current moving target at the future time t+1 based on the skeleton key points of the pedestrian target area:

[0114] Based on the skeleton key points of the pedestrian target area, the walking angle a, rotation angle b, walking speed v and current position of pedestrian i at the current time t are obtained. Calculate the predicted motion position at time t+1 in the future: Where S i is the relative position; is the position at time t-1+n; where time t+1 in the future is the shortest time for a pedestrian to reach the operating position of the current specific operating area at the current walking speed v;

[0115] S43: Determine whether the predicted motion position is within the operating position of the current specific work domain. If so, trigger an alarm; otherwise, the workplace is safe.

[0116] Generally speaking, the colors of oil workers' clothing are divided into the following categories: red is the senior color among oil workers, representing the management personnel or senior technicians of oil drilling companies, who are generally responsible for on-site arrangements, commands, management and other work; orange is the intermediate color among oil workers, representing the color of on-site supervisors, who are usually responsible for specific construction work, such as rig operators, engineers, technicians, logging personnel, etc.; yellow is the primary color among oil workers, representing ordinary workers on site, who are generally engaged in some simple work, such as excavation, cleaning, and transportation; blue is the color of oil workers' safety equipment, representing safety clothing, who are generally wearing work protective equipment such as safety helmets, safety boots, and safety glasses; green is a special color among oil workers, representing special identities, such as professionals in environmental protection, water conservancy, geology, etc.; in short, through clothing of different colors, oil workers can quickly identify the identities, responsibilities and safety levels of their colleagues, ensuring work safety and efficiency. In view of this, this scheme uses the color of the oil operator's clothing to identify whether the mobile target belongs to the preset range of the current specific operation domain. In some special specific operation domains, only people with specific identities can work. For different specific operation domains, the colors of clothing that can be used for work are pre-set. By identifying whether the color of the mobile target's body position belongs to the pre-set clothing color of the current specific operation domain, cross-domain operations and personnel intrusions can be effectively prevented.

[0117] The video-based workplace safety inspection method provided in this embodiment uses the video data of the workplace to, on the one hand, detect standardized clothing and promptly issue alarms for illegal clothing, thereby improving the efficiency of clothing safety inspections in oilfield workplaces. On the other hand, under the premise of dress code, this solution also determines whether the clothing types of all mobile targets in each specific work domain are within the limited range of the specific work domain to which they belong, and promptly issues alarms for the phenomenon of non-operating personnel invading cross-domain operations, thereby improving the comprehensiveness of clothing safety inspections in oilfield workplaces.

[0118] Since oilfield operation sites require wearing safety helmets and color-coded work clothes, this solution provides a new technical concept: first locate pedestrian targets based on video data, then pre-process the pedestrian target area and perform safety clothing detection, and then for pedestrian targets with compliant clothing, determine whether they are non-operating personnel intrusion based on the color of their work clothes, and monitor the movement trajectory of non-operating personnel in real time. When it is predicted that non-operating personnel are about to reach the operating position of the current specific operation domain, timely warn of the intrusion of non-operating personnel, thereby improving the comprehensiveness of safety inspections of clothing at oilfield operation sites.

[0119] Analyzing the actual oilfield operation scenes, the safety clothing detection task has the following characteristics: First, due to the high placement of monitoring cameras in actual oilfield operation scenes, the target of the operator in the shooting picture is small, and the monitoring camera is exposed to the air for a long time, the monitoring picture will be slightly blurred, which will bring certain difficulties to the standardized clothing detection; second, the clothing parts of the operator are smaller than the target size of the operator, such as safety helmets, work tops, work pants and other parts, which will also affect the accuracy of the detection; third, due to the influence of the external environment, such as changes in lighting, weather changes, scene occlusion and changes in monitoring angle Factors such as variability will also make target detection very difficult. Based on the above considerations, this solution first conducts larger target detection of operators, namely mobile target detection. During the operating time period, it determines whether the mobile target area in each specific operating domain exceeds the standard, and promptly issues an alarm for the exceeding of the standard by operators in the specific operating domain. Then, the clothing detection of the head position and body position is performed according to the mobile target area. The safety detection of the specific operating domain is integrated into the clothing detection process, and the intrusion of non-operating personnel is promptly warned. This not only effectively alleviates the problem that some targets are difficult to detect, but also improves the efficiency and comprehensiveness of clothing safety inspections in oilfield workplaces.

[0120] Example 2

[0121] This embodiment provides a video-based workplace safety inspection system, which is used to implement the video-based workplace safety inspection method of embodiment 1, including:

[0122] An acquisition module, used to obtain video data of the workplace;

[0123] The target detection module is used to detect moving targets in the workplace based on video data;

[0124] The first judgment module is used to extract the original image data of the workplace and mark the moving target when a moving target is detected in the workplace; and judge whether the moving target in each specific work area exceeds the standard during the work period;

[0125] A preprocessing module, used to preprocess the original image data of the workplace;

[0126] The clothing detection module is used to input the pre-processed image data into the trained detection network to perform standard clothing detection: first detect whether the clothing of the moving target is standard;

[0127] The running trajectory prediction module is used to obtain the predicted moving position of the moving target when the clothing type is not within the limited range of the current specific operation domain, and determine whether the predicted moving position is within the operating position of the current specific operation domain;

[0128] The alarm module is used to give an alarm when the moving target in a specific work domain exceeds the standard, the moving target is not dressed properly, the head position of the moving target is dressed in violation of regulations, the body position of the moving target is dressed in violation of regulations, the clothing color of the moving target is not within the specified range of the current specific work domain, and the predicted motion position is at the operating position of the current specific work domain.

[0129] Example 3

[0130] This embodiment provides a computer-readable medium having a computer program stored thereon. The computer program is executed by a processor to implement a video-based workplace safety inspection method as described in Embodiment 1.

[0131] The present invention is not limited to the above-mentioned optional implementation modes. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that fall within the scope defined by the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A video-based workplace safety inspection method, characterized in that: The work site includes one or more specific work areas, and the method comprises the following steps: S1: Acquire video data of the workplace, and detect moving targets in the workplace based on the video data. If a moving target is detected in the workplace, proceed to step S2; S2: extracting the original image data of the work site and marking the moving target area; in the work time period, determining whether the moving target area on each specific work domain exceeds the standard, if so, triggering an alarm; otherwise, proceeding to step S3; S3: Preprocess the original image data of the workplace, and input the preprocessed image data into the trained detection network to perform standard dress detection: detect whether the dress in the head area, body area and foot area of ​​the moving target is in violation of the rules; when the dress in the head area, body area and foot area is not in violation of the rules, proceed to step S4; otherwise, trigger an alarm; S4: Determine whether the color of the clothing on the body area of ​​each moving target is within the limited range of the specific work domain to which it belongs. If so, the workplace is safe; otherwise, predict the movement trajectory of the moving target with abnormal clothing color in real time: obtain the predicted movement position of the current moving target, and determine whether the predicted movement position is within the operating position of the current specific work domain. If so, trigger an alarm, otherwise, the workplace is safe.

2. A video-based workplace safety inspection method according to claim 1, characterized in that: In step S1, moving target detection includes the following specific steps: S11: extracting original image data of the workplace, scaling the pixels of the original image data to 640×640, and dividing the scaled image data slices into S×S grids; S12: Input the output data of step S11 into the moving target detection model based on multi-scale feature fusion, and perform independent detection on four feature maps with scales of 20×20, 40×40, 80×80 and 160×160 to obtain four detection bounding boxes; S13: Select the detection bounding box with the highest confidence as the moving target area.

3. A video-based workplace safety inspection method according to claim 1, characterized in that: The mobile target detection model includes a feature extraction network and a Neck structure; the feature extraction network includes a CSP structure and a Focus structure; the CSP structure performs a slicing operation, and the original 640×640×3 image data is first copied to four copies, and then the four images are cut into four 320×320×3 slices through a slicing operation; then Concat is used to connect the four slices in depth, and the output is 320×320×12; then a convolution layer with 32 convolution kernels is used to generate a 320×320×32 output; Step S12 includes the following sub-steps: S121: Four independent detection branches are formed with four feature maps of scales of 20×20, 40×40, 80×80 and 160×160; each detection branch extracts the underlying feature map of the corresponding scale of the input data; S122: Each detection branch obtains a deep feature map from the feature extraction network, upsamples the deep feature map in the FPN layer, and fuses the upsampled data with the bottom feature map to obtain a fused feature map; S123, each detection branch fuses the fused feature map from bottom to top in the PAN layer and then downsamples it. Finally, each detection branch independently detects the downsampled data.

4. A video-based workplace safety inspection method according to claim 1, characterized in that: In step S2, the method for determining whether the moving target area on the specific operation domain exceeds the standard includes: Crop the images of each specific operation domain from the original image data; Determine whether the number of moving target areas in each specific work domain image is greater than Y, where Y is the threshold number of workers in the specific work domain where the moving target area is located; if so, it is determined that the moving target area on the current specific work domain exceeds the standard; otherwise, it is determined that the moving target area on the current specific work domain does not exceed the standard.

5. A video-based workplace safety inspection method according to claim 4, characterized in that: The method for determining whether the moving target area on a specific operation domain exceeds the standard also includes: Crop the images of each specific operation domain from the original image data; Calculate the area and A1 of all moving target areas in the current specific operation domain image, and calculate the area ratio K=A1 / A2, where A2 is the area of ​​the current specific operation domain image; It is determined whether the area ratio K exceeds the threshold. If so, it is determined that the moving target area on the current specific operation domain exceeds the standard. Otherwise, it is determined that the moving target area on the current specific operation domain does not exceed the standard.

6. A video-based workplace safety inspection method according to claim 5, characterized in that: The calculation method of the area and A1 of all moving target areas in the current specific operation domain image includes: T1: Get the area of ​​each moving target area; T2: Correct the area of ​​each moving target area: determine whether the area A of the current moving target area is within the single area threshold range. If so, the area A of the current moving target area remains unchanged; otherwise, enter T3; T3: Determine the range to which the current moving target area A belongs, the range including the i-th area threshold range, i=2, 3, ...; if the current moving target area A belongs to the i-th area threshold range, then replace the current moving target area A with: B*i; B is the maximum value of a single area threshold range; T4: Calculate the area sum A1 using the corrected areas of each moving target region.

7. The video-based workplace safety inspection method according to claim 1, characterized in that: In step S3, the preprocessing method includes: Obtain the coordinates of the upper left corner and the lower right corner of the moving target, and cut out the moving target frame from the original image data, first enlarge each moving target frame, and then sharpen and enhance the contrast of the pedestrian target area in each moving target frame; Divide the pedestrian target area: first obtain the skeleton key points of the human body posture according to the Alphapose model where k n and c n Respectively represent the position and confidence of the nth skeleton key point; The head area, body area and foot area are divided based on the skeleton key points: The head area is divided by the nose skeleton key points, eye skeleton key points and ear skeleton key points: Take the left shoulder skeleton key point (k LJ , c LJ )、Right shoulder skeleton key point (k RJ , c RJ ), left waist skeleton key points (k LY , c LY ) and the right waist skeleton key points (k RY , c RY ) to divide the body into regions: Body Area Center Body area width w = max(k RY-x , k LY-x , k LJ-x , k RJ-x )-min(k RY-x , k LY-x , k LJ-x , k RJ-x ); Body area height h = max(k RY-y , k LY-y , k LJ-y , k RJ-y )-min(k RY-y , k LY-y , k LJ-y , k RJ-y ); where k RY-x is the x coordinate of the key point of the right waist skeleton, k RY-y is the y coordinate of the key point of the right waist skeleton, k LY-x is the x coordinate of the left waist skeleton key point, k LY-y is the y coordinate of the left waist skeleton key point; k RJ-x is the x coordinate of the key point of the right shoulder skeleton, k RJ-y The y coordinate of the right shoulder skeleton key point, k LJ-y is the y coordinate of the left shoulder skeleton key point, k LJ-x is the x-coordinate of the left shoulder skeleton key point; The foot area is divided by the left foot skeleton key points and the right foot skeleton key points.

8. The video-based workplace safety inspection method according to claim 1, characterized in that: Step S4 includes the following sub-steps: S41: Obtain the body area color of each moving target and the ratio D of the height to width of the body area. When D is less than 2.5, compare the color with the preset color library of the specific work area. If there is a matching color in the preset color library, the work place is safe; otherwise, proceed to step S42; S42: Predict the predicted motion position of the current moving target at the future time t+1 based on the skeleton key points of the pedestrian target area: The posture estimation is performed based on the skeleton key points of the pedestrian target area to obtain the walking angle a, rotation angle b, walking speed v and current position of pedestrian i at the current time t. Calculate the predicted motion position L at the future time t+1: Where S i is the relative position; in is the position at time t-1+n; the future time t+1 is the shortest time for the pedestrian to reach the operating position of the current specific operation domain at the current walking speed v; S43: Determine whether the predicted motion position is within the operating position of the current specific work domain. If so, trigger an alarm; otherwise, the workplace is safe.

9. A video-based workplace safety inspection system, characterized by: It includes an acquisition module for acquiring video data of the workplace; The target detection module is used to detect moving targets in the workplace based on video data; A first judgment module is used to extract the original image data of the workplace and mark the moving target when a moving target is detected in the workplace; And during the operation time period, it is determined whether the moving targets in each specific operation domain exceed the standard; A preprocessing module, used to preprocess the original image data of the workplace; The clothing detection module is used to input the pre-processed image data into the trained detection network to perform standardized clothing detection; The running trajectory prediction module is used to obtain the predicted moving position of the moving target when the clothing type is not within the limited range of the current specific operation domain, and determine whether the predicted moving position is within the operating position of the current specific operation domain; The alarm module is used to give an alarm when the moving target in a specific work domain exceeds the standard, the moving target is not dressed properly, the head position of the moving target is dressed in violation of regulations, the body position of the moving target is dressed in violation of regulations, the clothing color of the moving target is not within the specified range of the current specific work domain, and the predicted motion position is at the operating position of the current specific work domain.

10. A computer readable medium, characterized in that: A computer program is stored, and the computer program can be executed by a processor to implement the video-based workplace safety inspection method as described in any one of claims 1 to 8.

Images