A System and Method for Detecting Severe Fatigue of Construction Workers Based on Spatiotemporal Characteristics

Abstract

This invention provides a system and method for detecting severe fatigue in construction workers based on spatiotemporal features. The system includes: a video acquisition end for acquiring videos of construction workers at work; a key point extraction end for inputting the video into a first target network and locating the skeletal key points of the construction workers in the video based on the first target network; an analysis end for analyzing the dynamic posture of the construction workers during the work process based on the skeletal key points and extracting fatigue features based on the dynamic posture; and a fatigue detection end for calculating the dispersion of the fatigue features of the construction workers based on a sliding window, inputting the dispersion of the fatigue features into a second target network for analysis, and determining the fatigue state of the construction workers based on the output results. This system achieves accurate and effective detection of severe fatigue in construction workers through spatiotemporal features, ensuring both accuracy and efficiency in detecting severe fatigue.

Description

Technical Field

[0001] This invention relates to the field of fatigue state technology, and in particular to a system and method for detecting severe fatigue in construction workers based on spatiotemporal characteristics. Background Technology

[0002] Currently, there are two main methods for detecting fatigue among construction workers: one is fatigue detection based on physiological signals, which collects physiological signals through electrodes or sensors worn on the worker's body; the other is fatigue detection based on computer vision, which predicts fatigue by extracting facial behavioral features of the worker, including eye movement features, mouth features, head features, etc. Detecting whether construction workers are severely fatigued helps to ensure the safety of construction workers and the orderly progress of construction.

[0003] However, both of the above methods have the following drawbacks:

[0004] When fatigue detection is based on physiological signals, these signals are collected by electrodes or sensors worn on the body of construction workers. Prolonged use can lead to slight errors in the collected signals and inaccurate data. When workers on construction sites are required to wear sensors, they have no choice.

[0005] When fatigue detection is performed based on computer vision, if the worker's face is obscured or their head posture changes significantly during the detection process, the loss of key facial points will prevent the extraction of fatigue features, thus limiting the accuracy and robustness of fatigue detection.

[0006] Therefore, in order to overcome the above-mentioned defects, the present invention provides a system and method for detecting severe fatigue of construction workers based on spatiotemporal characteristics. Summary of the Invention

[0007] This invention provides a system and method for detecting severe fatigue of construction workers based on spatiotemporal features. It analyzes the work videos of construction workers through a first target network to accurately and effectively determine the fatigue characteristics (spatial features) of the workers. Then, it analyzes the dispersion (temporal features) of the fatigue characteristics through a second target network, achieving accurate and effective detection of severe fatigue states of construction workers through spatiotemporal features, thus ensuring the accuracy and efficiency of severe fatigue state detection.

[0008] This invention provides a system for detecting severe fatigue in construction workers based on spatiotemporal characteristics, comprising:

[0009] The video acquisition terminal is used to acquire videos of construction workers at work.

[0010] The key point extraction end is used to input the operation video into the first target network and locate the skeleton key points of the construction personnel in the operation video based on the first target network;

[0011] The analysis end is used to analyze the dynamic posture of construction workers during the operation based on the skeleton key points, and to extract the fatigue characteristics of construction workers based on the dynamic posture.

[0012] The fatigue detection end is used to calculate the dispersion of the fatigue characteristics of construction workers based on a sliding window, input the dispersion of the fatigue characteristics of construction workers into the second target network for analysis, and determine the fatigue state of construction workers based on the output results.

[0013] Preferably, a construction worker severe fatigue detection system based on spatiotemporal characteristics includes a video acquisition terminal comprising:

[0014] The instruction acquisition unit is used to acquire video acquisition instructions sent by the management terminal, parse the video acquisition instructions, and determine the target video acquisition area;

[0015] The link construction unit is used to determine the terminal address of the video surveillance terminal in the target video acquisition area, and to construct the data transmission link between the video surveillance terminal and the management terminal based on the terminal address;

[0016] The video acquisition unit is used to transmit video acquisition instructions to the video monitoring terminal via the data transmission link, control the video monitoring terminal to acquire video of the work status of construction personnel in the target video acquisition area, obtain work video, and feed back the acquired work video to the management terminal via the data transmission link.

[0017] Preferably, a construction worker severe fatigue detection system based on spatiotemporal characteristics includes a key point extraction end, comprising:

[0018] The first positioning unit is used to read the operation video, perform the first positioning of the construction personnel in the operation video, and obtain the target video segment;

[0019] The second positioning unit is used to input the target video segment into the first target network, and perform second positioning on the key points of the target video segment according to the first target network, thereby determining the skeleton key pixel points of the construction personnel in the target video segment and the coordinate values ​​corresponding to the skeleton key pixel points.

[0020] Preferably, a severe fatigue detection system for construction workers based on spatiotemporal characteristics includes an analysis terminal comprising:

[0021] The grouping unit is used to obtain the key points of the skeleton of the construction personnel, and to group the key points of the skeleton according to the preset grouping rules. Based on the grouping results, the key points of the skeleton in each group are modularized to obtain the target part module. The target part module includes the head module, elbow module, hip module and ankle module.

[0022] The fatigue feature determination unit is used to determine the first fatigue feature, the second fatigue feature, and the third fatigue feature based on the target part module, respectively.

[0023] The fatigue characteristic summarization unit is used to summarize the first fatigue characteristic, the second fatigue characteristic, and the third fatigue characteristic, and to determine the final fatigue characteristic of the construction personnel based on the summarization results.

[0024] Preferably, a severe fatigue detection system for construction workers based on spatiotemporal characteristics includes a fatigue characteristic determination unit, comprising:

[0025] The first fatigue characteristic determination sub-unit is used for:

[0026] Obtain the structural features of each target part module, and construct a rectangular coordinate system based on the structural features. Then, determine the target coordinate values ​​of each skeleton key point in each target part module based on the rectangular coordinate system.

[0027] The module center point of each target part module is determined based on the target coordinate values ​​of each skeleton key point and the number of target skeleton key points in each target part module, and the module center points of each target part module are connected according to the preset connection order.

[0028] Based on the connection results, the module center area enclosed by the modules of each target part of the construction personnel is determined, and the module center area is used as the first fatigue characteristic.

[0029] The second fatigue characteristic determination sub-unit is used for:

[0030] Obtain the center point of the hip module, and determine the first center coordinate value corresponding to the center point of the hip module based on the constructed rectangular coordinate system;

[0031] The target distance between the center of gravity of the construction worker and the ground is determined based on the ordinate of the first center coordinate value, and the change characteristics of the target distance between the center of gravity of the construction worker and the ground are determined based on the operation video.

[0032] Based on the target distance change characteristics, the characteristics of the movement of the center of gravity of the construction workers are determined, and the characteristics of the movement of the center of gravity of the construction workers are used as the second fatigue characteristics.

[0033] The third fatigue characteristic determination sub-unit is used for:

[0034] Obtain the center points of the head module and the ankle module respectively, and determine the second center coordinate value corresponding to the center point of the head module and the third center coordinate value corresponding to the center point of the ankle module based on the constructed rectangular coordinate system.

[0035] The center points of the head module and the ankle module are connected based on the second center coordinate value and the third center coordinate value, and the center line of the construction worker's body is determined based on the connection result.

[0036] Set the ground as the reference plane, and determine the target angle between the human body's centerline and the ground based on the reference plane;

[0037] Based on the target angle, the body tilt characteristics of the construction worker relative to the ground are determined, and the body tilt characteristics are used as the third fatigue characteristic.

[0038] The fatigue feature summarization subunit is used to summarize the first fatigue feature, the second fatigue feature, and the third fatigue feature, and to determine the final fatigue feature of the construction personnel based on the summarization results.

[0039] Preferably, a severe fatigue detection system for construction workers based on spatiotemporal characteristics includes a fatigue detection end comprising:

[0040] A fixed-length sliding window interval unit is used to read the fatigue characteristics of construction workers and fix the length of the sliding window interval.

[0041] The feature division unit is used to divide the fatigue characteristics of construction workers into equal parts based on the length of the sliding window interval, and to determine the target data within each window based on the length of the sliding window interval.

[0042] The discreteness determination unit is used to calculate the data discreteness within the window and determine the discreteness of the fatigue characteristics of construction workers based on the calculation results.

[0043] Preferably, a severe fatigue detection system for construction workers based on spatiotemporal characteristics includes a fatigue detection end comprising:

[0044] The change sequence determination unit is used to determine the change sequence of construction workers' fatigue characteristics based on the degree of dispersion of the fatigue characteristics of construction workers;

[0045] The fatigue state information determination unit is used to input the change sequence of fatigue characteristics of construction workers into the second target network for analysis to determine the fatigue state information of construction workers.

[0046] The fatigue state determination unit is used to output the fatigue state of construction workers based on their fatigue state information.

[0047] Preferably, a fatigue state information determination unit for a construction worker severe fatigue detection system based on spatiotemporal characteristics includes:

[0048] The analysis subunit is used to receive the change sequence of the fatigue characteristics of the construction workers based on the input of the second target network, and to determine the duration of each action feature of the construction workers during the construction process based on the change sequence of the fatigue characteristics of the construction workers.

[0049] The fatigue coefficient determination subunit is used to match the target fatigue evaluation index from the preset fatigue evaluation index library based on each action feature, and to detect the fatigue level of the duration of the corresponding action feature based on the target fatigue evaluation index, so as to obtain the fatigue coefficient of each action feature as a function of duration.

[0050] The fatigue level determination subunit is used to determine the influence weight of different parts of the human body on the total fatigue level based on human structural characteristics, and to determine the current fatigue level of the construction worker based on the influence weight and the fatigue coefficient generated by each action feature over time. At the same time, the current fatigue level of the construction worker is classified into levels based on a preset fatigue level threshold, and the fatigue status information of the construction worker is determined based on the level classification results. The fatigue level includes severe fatigue and non-severe fatigue.

[0051] Preferably, a severe fatigue detection system for construction workers based on spatiotemporal characteristics includes a fatigue degree determination subunit, comprising:

[0052] The result acquisition subunit is used to obtain the level classification result of the current fatigue level of the construction workers, and when the classification result determines that the construction workers are severely fatigued, the identity information of the current construction workers is locked.

[0053] The alarm subunit is used to generate alarm information based on the identity information of the current construction personnel and transmit the alarm information to the management terminal for alarm notification;

[0054] The notification subunit is used to parse alarm information based on the management terminal, determine the terminal communication address of the current construction personnel based on the parsing result, and send a rest notification to the current construction personnel based on the terminal communication address.

[0055] This invention provides a method for detecting severe fatigue in construction workers based on spatiotemporal characteristics, comprising:

[0056] Step 1: Extract the work videos of the construction workers;

[0057] Step 2: Input the operation video into the first target network, and locate the key skeletal points of the construction workers in the operation video based on the first target network;

[0058] Step 3: Analyze the dynamic posture of construction workers during the operation based on the skeleton key points, and extract the fatigue characteristics of construction workers based on the dynamic posture;

[0059] Step 4: Calculate the dispersion of the fatigue characteristics of construction workers based on the sliding window, input the dispersion of the fatigue characteristics of construction workers into the second target network for analysis, and determine the fatigue state of construction workers based on the output results.

[0060] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0061] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0063] Figure 1 This is a structural diagram of a construction worker severe fatigue detection system based on spatiotemporal characteristics, as described in an embodiment of the present invention.

[0064] Figure 2 This is a structural diagram of the key point extraction end in a spatiotemporal feature-based system for detecting severe fatigue of construction workers, as described in an embodiment of the present invention.

[0065] Figure 3 This is a flowchart of a method for detecting severe fatigue of construction workers based on spatiotemporal characteristics, as described in an embodiment of the present invention. Detailed Implementation

[0066] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0067] Example 1:

[0068] This embodiment provides a system for detecting severe fatigue in construction workers based on spatiotemporal characteristics, such as... Figure 1 As shown, it includes:

[0069] The video acquisition terminal is used to acquire videos of construction workers at work.

[0070] The key point extraction end is used to input the operation video into the first target network and locate the skeleton key points of the construction personnel in the operation video based on the first target network;

[0071] The analysis end is used to analyze the dynamic posture of construction workers during the operation based on the skeleton key points, and to extract the fatigue characteristics of construction workers based on the dynamic posture.

[0072] The fatigue detection end is used to calculate the dispersion of the fatigue characteristics of construction workers based on a sliding window, input the dispersion of the fatigue characteristics of construction workers into the second target network for analysis, and determine the fatigue state of construction workers based on the output results.

[0073] In this embodiment, the work video can be a video obtained by construction workers through a video acquisition device during the construction process, used to record the changes in the posture of the construction workers during the work process.

[0074] In this embodiment, the first target network is pre-set and used to determine the skeletal key points of construction workers from the work video. For example, it can be a single-level multi-person key point detector YOLOv7-Pose network.

[0075] In this embodiment, the key points of the skeleton can be specific joint positions that can characterize the posture characteristics of the construction worker, such as the head, elbows, hips and ankles of the construction worker.

[0076] In this embodiment, dynamic posture can be the body posture of construction workers at different times during the construction process, which makes it easier to accurately determine the fatigue characteristics of construction workers.

[0077] In this embodiment, the fatigue characteristics can be the specific posture of the construction worker when the worker is in a state of fatigue during the construction process.

[0078] In this embodiment, the sliding window is used to discretize fatigue characteristics, thereby facilitating the determination of the fatigue degree of construction workers.

[0079] In this embodiment, the second target network is pre-set and used to analyze the fatigue state of construction workers. The second target network can be a long short-term memory recurrent neural network.

[0080] In this embodiment, fatigue state can characterize the specific degree of fatigue corresponding to the fatigue characteristics of construction workers, such as mild fatigue or severe fatigue.

[0081] The beneficial effects of the above technical solution are as follows: by analyzing the operation video of the construction workers through the first target network, the fatigue characteristics (spatial characteristics) of the construction workers can be accurately and effectively determined; secondly, by analyzing the dispersion of the fatigue characteristics (temporal characteristics) through the second target network, the severe fatigue state of the construction workers can be accurately and effectively detected through spatiotemporal characteristics, thus ensuring the accuracy and efficiency of the detection of severe fatigue state.

[0082] Example 2:

[0083] Based on Example 1, this example provides a system for detecting severe fatigue of construction workers based on spatiotemporal characteristics. The video acquisition terminal includes:

[0084] The instruction acquisition unit is used to acquire video acquisition instructions sent by the management terminal, parse the video acquisition instructions, and determine the target video acquisition area;

[0085] The link construction unit is used to determine the terminal address of the video surveillance terminal in the target video acquisition area, and to construct the data transmission link between the video surveillance terminal and the management terminal based on the terminal address;

[0086] The video acquisition unit is used to transmit video acquisition instructions to the video monitoring terminal via the data transmission link, control the video monitoring terminal to acquire video of the work status of construction personnel in the target video acquisition area, obtain work video, and feed back the acquired work video to the management terminal via the data transmission link.

[0087] In this embodiment, the video capture command is sent by the management terminal and is used to capture video images of the target video capture area.

[0088] In this embodiment, the target video acquisition area can be the area where the fatigue state of construction workers needs to be detected.

[0089] In this embodiment, the video surveillance terminal is pre-configured, such as a camera.

[0090] In this embodiment, the terminal address can be a terminal communication address that represents different video surveillance terminals, thereby enabling the acquisition of construction workers' work videos.

[0091] The beneficial effects of the above technical solution are: by effectively collecting the operation videos of construction workers in the target video collection area, it is convenient to accurately and effectively detect the severe fatigue state of construction workers based on the collected operation videos, thus providing convenience and guarantee for the detection of severe fatigue state of construction workers.

[0092] Example 3:

[0093] Based on Example 1, this example provides a system for detecting severe fatigue of construction workers based on spatiotemporal characteristics, such as... Figure 2 As shown, the key point extraction end includes:

[0094] The first positioning unit is used to read the operation video, perform the first positioning of the construction personnel in the operation video, and obtain the target video segment;

[0095] The second positioning unit is used to input the target video segment into the first target network, and perform second positioning on the key points of the target video segment according to the first target network, thereby determining the skeleton key pixel points of the construction personnel in the target video segment and the coordinate values ​​corresponding to the skeleton key pixel points.

[0096] In this embodiment, the first positioning may be to lock onto the image of the construction workers appearing in the work video.

[0097] In this embodiment, the target video segment can be a video segment of the construction workers obtained by locating the image of the construction workers, that is, a video segment that can clearly characterize the action features of the construction workers.

[0098] In this embodiment, the key points can be various parts of the construction workers in the target video segment, such as the head or legs.

[0099] In this embodiment, the second positioning can be to locate the key points of the construction personnel in the target video segment, thereby facilitating the determination of the key skeletal points of the construction personnel.

[0100] In this embodiment, the key pixels of the skeleton can be the image positions of the key points of the skeleton corresponding to the key parts of the construction personnel in the target video segment.

[0101] In this embodiment, the coordinate values ​​can be the specific location information of the skeleton key pixels in the target video segment.

[0102] The beneficial effects of the above technical solution are: by analyzing the operation video, the initial positioning of the construction personnel and the coordinate values ​​corresponding to the key pixels of the construction personnel's skeleton can be locked respectively, which facilitates the accurate and reliable analysis of the key points of the construction personnel's skeleton in the operation video, and provides convenience and guarantee for the detection of severe fatigue status of construction personnel.

[0103] Example 4:

[0104] Based on Example 1, this example provides a system for detecting severe fatigue of construction workers based on spatiotemporal characteristics. The analysis terminal includes:

[0105] The grouping unit is used to obtain the key points of the skeleton of the construction personnel, and to group the key points of the skeleton according to the preset grouping rules. Based on the grouping results, the key points of the skeleton in each group are modularized to obtain the target part module. The target part module includes the head module, elbow module, hip module and ankle module.

[0106] The fatigue feature determination unit is used to determine the first fatigue feature, the second fatigue feature, and the third fatigue feature based on the target part module, respectively.

[0107] The fatigue characteristic summarization unit is used to summarize the first fatigue characteristic, the second fatigue characteristic, and the third fatigue characteristic, and to determine the final fatigue characteristic of the construction personnel based on the summarization results.

[0108] In this embodiment, the preset grouping rules are pre-set and used to group the key points of the construction workers' skeletons, thereby facilitating the accurate determination of the fatigue characteristics of the construction workers.

[0109] In this embodiment, modular processing can be achieved by defining each set of skeletal key points as a module (i.e., a human body structural part).

[0110] In this embodiment, the first fatigue feature can be the module center area formed by connecting the center points of the head module, elbow module, hip module, and ankle module. The change in the module center area can reflect the limb movement of the construction worker under different mental states.

[0111] In this embodiment, the second fatigue characteristic can be the change in the distance between the worker's center of gravity and the ground.

[0112] In this embodiment, the third fatigue feature can be the angle formed between the center line of the construction worker's body and the ground, which is used to characterize the tilt of the construction worker relative to the ground, so as to facilitate the determination of the fatigue level of the construction worker by tilting.

[0113] The beneficial effects of the above technical solution are as follows: by grouping and modularizing the key points of the construction worker's skeleton, the different target parts of the construction worker can be effectively identified. Secondly, by analyzing the target parts of the modular model, the first, second, and third fatigue characteristics of the construction worker can be effectively identified. Finally, by summarizing the first, second, and third fatigue characteristics, the comprehensiveness and reliability of the determination of the fatigue characteristics of the construction worker are ensured, and the accuracy of detecting the severe fatigue state of the construction worker is guaranteed.

[0114] Example 5:

[0115] Based on Example 4, this example provides a severe fatigue detection system for construction workers based on spatiotemporal characteristics, including a fatigue characteristic determination unit, comprising:

[0116] The first fatigue characteristic determination sub-unit is used for:

[0117] Obtain the structural features of each target part module, and construct a rectangular coordinate system based on the structural features. Then, determine the target coordinate values ​​of each skeleton key point in each target part module based on the rectangular coordinate system.

[0118] The module center point of each target part module is determined based on the target coordinate values ​​of each skeleton key point and the number of target skeleton key points in each target part module, and the module center points of each target part module are connected according to the preset connection order.

[0119] Based on the connection results, the module center area enclosed by the modules of each target part of the construction personnel is determined, and the module center area is used as the first fatigue characteristic.

[0120] The second fatigue characteristic determination sub-unit is used for:

[0121] Obtain the center point of the hip module, and determine the first center coordinate value corresponding to the center point of the hip module based on the constructed rectangular coordinate system;

[0122] The target distance between the center of gravity of the construction worker and the ground is determined based on the ordinate of the first center coordinate value, and the change characteristics of the target distance between the center of gravity of the construction worker and the ground are determined based on the operation video.

[0123] Based on the target distance change characteristics, the characteristics of the movement of the center of gravity of the construction workers are determined, and the characteristics of the movement of the center of gravity of the construction workers are used as the second fatigue characteristics.

[0124] The third fatigue characteristic determination sub-unit is used for:

[0125] Obtain the center points of the head module and the ankle module respectively, and determine the second center coordinate value corresponding to the center point of the head module and the third center coordinate value corresponding to the center point of the ankle module based on the constructed rectangular coordinate system.

[0126] The center points of the head module and the ankle module are connected based on the second center coordinate value and the third center coordinate value, and the center line of the construction worker's body is determined based on the connection result.

[0127] Set the ground as the reference plane, and determine the target angle between the human body's centerline and the ground based on the reference plane;

[0128] Based on the target angle, the body tilt characteristics of the construction worker relative to the ground are determined, and the body tilt characteristics are used as the third fatigue characteristic.

[0129] The fatigue feature summarization subunit is used to summarize the first fatigue feature, the second fatigue feature, and the third fatigue feature, and to determine the final fatigue feature of the construction personnel based on the summarization results.

[0130] In this embodiment, the structural features are used to characterize the shape features of different target parts modules, thereby facilitating the determination of the module center point of different target parts modules.

[0131] In this embodiment, the target coordinate values ​​are used to characterize the specific positions of different skeleton key points in the corresponding target part modules.

[0132] In this embodiment, the number of targets can be the total number of skeletal key points in each target part module.

[0133] In this embodiment, the module center point can be the center position within each target part module, and its calculation formula is as follows:

[0134]

[0135] Among them, (C) x C y (X) is the center point of each module. i ,Y i ) represents the coordinates of key points within the module, and N represents the number of key points in the module.

[0136] In this embodiment, the preset connection order is pre-set and is used to characterize the order in which the module center points of each target part module are connected.

[0137] In this embodiment, the module center area can be the area of ​​the closed region formed by connecting the center points of modules from different target locations, and its calculation formula is as follows:

[0138]

[0139] Where S represents the center area of ​​the module; (X i ,Y i ) represents the coordinates of the center point within each target part module.

[0140] In this embodiment, the first center coordinate value can be the specific location of the center point of the hip module within the hip module.

[0141] In this embodiment, the center of gravity of the human body can be the hip module as the support of the center of the human body.

[0142] In this embodiment, the target distance is the distance between the center point of the hip module and the ground (i.e., the height of the human body's center of gravity relative to the ground).

[0143] In this embodiment, the target distance change feature is used to characterize the change in distance between the center of gravity of the construction worker and the ground during the construction process.

[0144] In this embodiment, the movement characteristic of a person's center of gravity can be a specific degree of change in the height of the construction worker's center of gravity above the ground.

[0145] In this embodiment, the second center coordinate value can be the specific location of the module center point of the head module within the head module.

[0146] In this embodiment, the third center coordinate value can be the specific location of the center point of the ankle module within the ankle module.

[0147] In this embodiment, the human body centerline can be a straight line obtained by connecting the center points of the head module and the ankle module.

[0148] In this embodiment, the target angle can be the angle formed by the human body's centerline and the ground, and its calculation formula is as follows:

[0149]

[0150] Where θ is the angle between the human body's centerline and the ground, (x i ,y i (x) represents the coordinates of the center point of the header module. j ,y j ) represents the coordinates of the center point of the ankle module.

[0151] In this embodiment, the body tilt feature is used to characterize the tilt of the construction worker's body relative to the ground.

[0152] The beneficial effects of the above technical solution are: by performing case-by-case analysis on the target part module of the construction personnel, the fatigue characteristics of the construction personnel in different aspects can be effectively analyzed, ensuring the comprehensiveness and reliability of the final fatigue characteristics of the construction personnel, providing convenience and guarantee for the accurate detection of the severe fatigue state of the construction personnel, and also improving the effect of detecting the severe fatigue state of the construction personnel.

[0153] Example 6:

[0154] Based on Example 1, this example provides a system for detecting severe fatigue of construction workers based on spatiotemporal characteristics. The fatigue detection terminal includes:

[0155] A fixed-length sliding window interval unit is used to read the fatigue characteristics of construction workers and fix the length of the sliding window interval.

[0156] The feature division unit is used to divide the fatigue characteristics of construction workers into equal parts based on the length of the sliding window interval, and to determine the target data within each window based on the length of the sliding window interval.

[0157] The discreteness determination unit is used to calculate the data discreteness within the window and determine the discreteness of the fatigue characteristics of construction workers based on the calculation results.

[0158] In this embodiment, the sliding window interval length is used to calculate the dispersion of fatigue features. Specifically, the fatigue features can be divided equally by fixing the sliding window interval length, and the data dispersion within each window can be calculated based on the data within each window after sliding a certain sliding step.

[0159] In this embodiment, the target data can be the specific data content contained in each window after the fatigue features are divided into equal parts.

[0160] In this embodiment, the dispersion is calculated by dividing the fatigue feature into equal parts with a fixed sliding window interval length Δ. For each sliding step, the data dispersion within that window is calculated based on the data within that window. The calculation formula is as follows:

[0161]

[0162] Where P = [S, L, θ] represents different fatigue characteristics, including the module center area S, the distance of the human body's center of gravity from the ground L, and the angle θ between the human body's centerline and the ground; D p The degree of dispersion under different features; Δ is the length of the sliding window; within a sliding window of data length Δ, d Pi , These represent individual values ​​of different fatigue characteristics (S,L,θ) and their average values ​​within a sliding window of Δ.

[0163] The beneficial effects of the above technical solution are: by processing the obtained fatigue features through a sliding window, the dispersion of the matching features can be accurately and effectively analyzed, which facilitates the input of the dispersion of fatigue features into the second target network for fatigue state detection, thus providing convenience and guarantee for the detection of severe fatigue state of users.

[0164] Example 7:

[0165] Based on Example 1, this example provides a system for detecting severe fatigue of construction workers based on spatiotemporal characteristics. The fatigue detection terminal includes:

[0166] The change sequence determination unit is used to determine the change sequence of construction workers' fatigue characteristics based on the degree of dispersion of the fatigue characteristics of construction workers;

[0167] The fatigue state information determination unit is used to input the change sequence of fatigue characteristics of construction workers into the second target network for analysis to determine the fatigue state information of construction workers.

[0168] The fatigue state determination unit is used to output the fatigue state of construction workers based on their fatigue state information.

[0169] In this embodiment, the sequence of changes in fatigue characteristics can be specific data on the fatigue characteristics of construction workers at different time points.

[0170] In this embodiment, fatigue state information can be specific data values ​​characterizing the fatigue state of construction workers.

[0171] The beneficial effects of the above technical solution are: by accurately and effectively judging the change sequence of fatigue characteristics of construction workers based on the dispersion of their fatigue state, and by inputting the change sequence of fatigue characteristics of construction workers into the second target network for analysis, the severe fatigue state of construction workers can be accurately and effectively analyzed, thus ensuring the accuracy of detecting severe fatigue state of construction workers.

[0172] Example 8:

[0173] Based on Example 7, this example provides a severe fatigue detection system for construction workers based on spatiotemporal characteristics, including a fatigue state information determination unit, comprising:

[0174] The analysis subunit is used to receive the change sequence of the fatigue characteristics of the construction workers based on the input of the second target network, and to determine the duration of each action feature of the construction workers during the construction process based on the change sequence of the fatigue characteristics of the construction workers.

[0175] The fatigue coefficient determination subunit is used to match the target fatigue evaluation index from the preset fatigue evaluation index library based on each action feature, and to detect the fatigue level of the duration of the corresponding action feature based on the target fatigue evaluation index, so as to obtain the fatigue coefficient of each action feature as a function of duration.

[0176] The fatigue level determination subunit is used to determine the influence weight of different parts of the human body on the total fatigue level based on human structural characteristics, and to determine the current fatigue level of the construction worker based on the influence weight and the fatigue coefficient generated by each action feature over time. At the same time, the current fatigue level of the construction worker is classified into levels based on a preset fatigue level threshold, and the fatigue status information of the construction worker is determined based on the level classification results. The fatigue level includes severe fatigue and non-severe fatigue.

[0177] In this embodiment, the action feature can be a specific action performed by the construction worker during the work process.

[0178] In this embodiment, the preset fatigue level evaluation index library is pre-set and used to store different types of fatigue level evaluation indexes.

[0179] In this embodiment, the target fatigue level evaluation index can be a basis or reference data applicable to evaluating the fatigue level of the current action characteristics.

[0180] In this embodiment, fatigue detection based on the duration of the corresponding action feature based on the target fatigue evaluation index can be performed by analyzing the duration of each action feature according to the requirements or values ​​of the target fatigue evaluation index. For example, when the arm is straightened, the fatigue level increases with the duration.

[0181] In this embodiment, the fatigue coefficient is used to characterize the degree of fatigue that occurs in each action feature over time. The larger the value, the greater the degree of fatigue that occurs in the current action feature over time.

[0182] In this embodiment, the influence weight is used to characterize the degree of influence of different parts of the human body on the overall fatigue level of the body.

[0183] In this embodiment, the preset fatigue level threshold is pre-set and serves as the boundary between severe fatigue and non-severe fatigue; it is adjustable.

[0184] The beneficial effects of the above technical solution are as follows: by using a second target network to accurately and effectively analyze the change sequence of fatigue characteristics of construction workers, the fatigue level of construction workers can be accurately and effectively judged. Secondly, by classifying the fatigue level of construction workers according to the fatigue level, the management terminal can accurately and effectively understand the fatigue status of construction workers, thereby facilitating the corresponding warning operation and improving the effect of detecting severe fatigue status of construction workers.

[0185] Example 9:

[0186] Based on Example 8, this example provides a system for detecting severe fatigue of construction workers based on spatiotemporal characteristics. The fatigue degree determination subunit includes:

[0187] The result acquisition subunit is used to obtain the level classification result of the current fatigue level of the construction workers, and when the classification result determines that the construction workers are severely fatigued, the identity information of the current construction workers is locked.

[0188] The alarm subunit is used to generate alarm information based on the identity information of the current construction personnel and transmit the alarm information to the management terminal for alarm notification;

[0189] The notification subunit is used to parse alarm information based on the management terminal, determine the terminal communication address of the current construction personnel based on the parsing result, and send a rest notification to the current construction personnel based on the terminal communication address.

[0190] The beneficial effects of the above technical solution are: by determining the identity information of severely fatigued construction workers, the terminal communication address of the construction workers can be locked based on the identity information, thereby sending the corresponding rest notice to the current construction workers, thus ensuring the personal safety of the construction workers and improving the effectiveness of detecting the severe fatigue state of the construction workers.

[0191] Example 10:

[0192] This embodiment provides a method for detecting severe fatigue in construction workers based on spatiotemporal characteristics, such as... Figure 3 As shown, it includes:

[0193] Step 1: Extract the work videos of the construction workers;

[0194] Step 2: Input the operation video into the first target network, and locate the key skeletal points of the construction workers in the operation video based on the first target network;

[0195] Step 3: Analyze the dynamic posture of construction workers during the operation based on the skeleton key points, and extract the fatigue characteristics of construction workers based on the dynamic posture;

[0196] Step 4: Calculate the dispersion of the fatigue characteristics of construction workers based on the sliding window, input the dispersion of the fatigue characteristics of construction workers into the second target network for analysis, and determine the fatigue state of construction workers based on the output results.

[0197] The beneficial effects of the above technical solution are as follows: by analyzing the operation video of the construction workers through the first target network, the fatigue characteristics (spatial characteristics) of the construction workers can be accurately and effectively determined; secondly, by analyzing the dispersion of the fatigue characteristics (temporal characteristics) through the second target network, the severe fatigue state of the construction workers can be accurately and effectively detected through spatiotemporal characteristics, thus ensuring the accuracy and efficiency of the detection of severe fatigue state.

[0198] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A construction worker severe fatigue detection system based on spatio-temporal features, characterized by, The method comprises the following steps: a video acquisition end is configured to acquire a work video of a construction worker; a key point extraction end is configured to input the work video into a first target network and locate skeleton key points of the construction worker in the work video based on the first target network; an analysis end is configured to analyze a dynamic posture of the construction worker in a work process based on the skeleton key points and extract a fatigue feature of the construction worker based on the dynamic posture; a fatigue detection end is configured to calculate a dispersion degree of the fatigue feature of the construction worker based on a sliding window, input the dispersion degree of the fatigue feature of the construction worker into a second target network for analysis, and determine a fatigue state of the construction worker based on an output result; the analysis end comprises: a grouping unit is configured to acquire the skeleton key points of the construction worker, group the skeleton key points based on a preset grouping rule, and modularize the skeleton key points of each group based on a grouping result to obtain target part modules, wherein the target part modules comprise a head module, an elbow module, a crotch module, and an ankle module; a fatigue feature determination unit is configured to determine a first fatigue feature, a second fatigue feature, and a third fatigue feature based on the target part modules respectively; a fatigue feature summarizing unit is configured to summarize the first fatigue feature, the second fatigue feature, and the third fatigue feature and determine a final fatigue feature of the construction worker based on a summarizing result; the fatigue feature determination unit comprises: a first fatigue feature determination subunit is configured to: acquire a structural feature of each target part module, determine a rectangular coordinate system based on the structural feature, and determine target coordinate values of each skeleton key point in each target part module based on the rectangular coordinate system; determine a module center point of each target part module based on the target coordinate values of each skeleton key point and a target number of skeleton key points in each target part module, and connect the module center points of the target part modules based on a preset connection sequence; determine a module center area surrounded by the target part modules of the construction worker based on a connection result, and take the module center area as the first fatigue feature; a second fatigue feature determination subunit is configured to: acquire a module center point of the crotch module, and determine a first center coordinate value corresponding to the module center point of the crotch module based on the constructed rectangular coordinate system; determine a target distance between a body center of gravity of the construction worker and the ground based on a longitudinal coordinate of the first center coordinate value, and determine a target distance change feature of the body center of gravity of the construction worker based on the work video; determine a body center of gravity movement feature of the construction worker based on the target distance change feature, and take the body center of gravity movement feature as the second fatigue feature; a third fatigue feature determination subunit is configured to: acquire the module center points of the head module and the ankle module respectively, and determine a second center coordinate value corresponding to the module center point of the head module and a third center coordinate value corresponding to the module center point of the ankle module based on the constructed rectangular coordinate system respectively; connect the module center points of the head module and the ankle module based on the second center coordinate value and the third center coordinate value, and determine a body center line of the construction worker based on a connection result; set the ground as a reference plane, and determine a target included angle between the body center line and the ground based on the reference plane; Determine the body tilt feature of the construction worker relative to the ground based on the target included angle, and take the body tilt feature as a third fatigue feature; A fatigue feature aggregation subunit is configured to aggregate the first fatigue feature, the second fatigue feature, and the third fatigue feature, and determine the final fatigue feature of the construction worker based on the aggregation result; The fatigue detection end comprises: A change sequence determination unit is configured to determine the change sequence of the fatigue feature of the construction worker based on the dispersion degree of the fatigue feature of the construction worker; A fatigue state information determination unit is configured to input the change sequence of the fatigue feature of the construction worker into the second target network for analysis, and determine the fatigue state information of the construction worker; A fatigue state determination unit is configured to output the fatigue state of the construction worker based on the fatigue state information of the construction worker.

2. The construction worker severe fatigue detection system based on space-time features according to claim 1, characterized in that, The video acquisition end comprises: An instruction acquisition unit is configured to acquire a video collection instruction sent by a management terminal, and analyze the video collection instruction to determine a target video collection area; A link construction unit is configured to determine the terminal address of a video monitoring terminal in the target video collection area, and construct a data transmission link between the video monitoring terminal and the management terminal based on the terminal address; A video acquisition unit is configured to transmit the video collection instruction to the video monitoring terminal based on the data transmission link, control the video monitoring terminal to collect the work state of the construction worker in the target video collection area to obtain a work video, and feed back the collected work video to the management terminal based on the data transmission link.

3. The construction worker severe fatigue detection system based on space-time features of claim 1, wherein The key point extraction end comprises: A first positioning unit is configured to read the work video, first position the construction worker in the work video, and obtain a target video segment; A second positioning unit is configured to input the target video segment into the first target network, and second position the key points of the target video segment according to the first target network, to determine the skeleton key pixel points of the construction worker in the target video segment and the coordinate values corresponding to the skeleton key pixel points.

4. The construction worker severe fatigue detection system based on space-time features of claim 1, wherein The fatigue detection end comprises: A sliding window interval length fixing unit is configured to read the fatigue feature of the construction worker, and fix the length of the sliding window interval; A feature equal division unit is configured to equally divide the fatigue feature of the construction worker based on the length of the sliding window interval, and determine the target data in each window based on the length of the sliding window interval; A dispersion degree determination unit is configured to calculate the dispersion degree of the data in the window, and determine the dispersion degree of the fatigue feature of the construction worker according to the calculation result.

5. The construction worker severe fatigue detection system based on space-time features of claim 1, wherein, The fatigue state information determination unit comprises: An analysis subunit is configured to receive the change sequence of the fatigue feature of the construction worker based on the second target network, and determine the duration of each action feature of the construction worker in the construction process based on the change sequence of the fatigue feature of the construction worker; A fatigue coefficient determination subunit is configured to match a target fatigue degree evaluation index from a preset fatigue degree evaluation index library based on each action feature, and detect the fatigue degree of the duration of the corresponding action feature based on the target fatigue degree evaluation index, to obtain the fatigue coefficient generated by each action feature with the duration. The fatigue degree determination subunit is configured to determine an influence weight of different parts of the human body on the total fatigue degree of the human body based on the human body structure features, determine the current fatigue degree of the construction worker based on the influence weight and a fatigue coefficient generated by each action feature with the duration, grade the current fatigue degree of the construction worker based on a preset fatigue degree threshold, and determine the fatigue state information of the construction worker based on the grading result. The fatigue degree includes severe fatigue and non-severe fatigue.

6. The construction worker severe fatigue detection system based on space-time features according to claim 5, characterized in that, The fatigue degree determination subunit includes: The result acquisition subunit is configured to acquire the grading result of the current fatigue degree of the construction worker, and lock the identity information of the current construction worker when the grading result determines that the construction worker is in severe fatigue; The alarm subunit is configured to generate alarm information based on the identity information of the current construction worker, and transmit the alarm information to the management terminal for alarm reminding; The notification subunit is configured to analyze the alarm information based on the management terminal, determine the terminal communication address of the current construction worker based on the analysis result, and send a rest notification to the current construction worker based on the terminal communication address.

7. A construction worker severe fatigue detection method based on spatiotemporal features, implemented by the construction worker severe fatigue detection system based on spatiotemporal features according to any one of claims 1-6, characterized in that, The method includes: Step 1: extracting the work video of the construction worker; Step 2: inputting the work video into a first target network, and positioning the skeleton key points of the construction worker in the work video based on the first target network; Step 3: analyzing the dynamic posture of the construction worker in the work process based on the skeleton key points, and extracting the fatigue features of the construction worker based on the dynamic posture; Step 4: calculating the dispersion degree of the fatigue features of the construction worker based on the sliding window, inputting the dispersion degree of the fatigue features of the construction worker into a second target network for analysis, and determining the fatigue state of the construction worker based on the output result.

Images