Multi-person three-dimensional work efficiency risk assessment method and device

Abstract

The invention discloses a multi-person three-dimensional work efficiency risk assessment method and device, and belongs to the technical field of machine vision, and the method comprises the steps: obtaining a work image frame of a construction site; wherein the operation image frames are acquired through monocular image acquisition equipment, and each frame of image comprises a plurality of workers; inputting the operation image frame into a pre-trained attitude and depth evaluation model to obtain attitude and depth information of each worker; for each worker, integrating the posture and depth information of the worker, and reconstructing the absolute three-dimensional human body posture of the worker in the world coordinate system, so as to obtain the absolute three-dimensional human body postures of all workers in the same world coordinate system; based on this, the physical ergonomics risk of the worker is evaluated in real time. According to the invention, postures and ergonomics risks of multiple persons in a unified three-dimensional space can be evaluated at the same time from a monocular video, and the problem that a collaborative operation scene is difficult to evaluate in the prior art is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a multi-person three-dimensional ergonomics risk assessment method and device. BACKGROUND

[0002] Construction operations usually involve high-intensity physical labor, which makes the practitioners face serious musculoskeletal injury risks. Data shows that such occupational injuries are one of the main health threats in the industrial field. Traditional ergonomics risk assessment methods, such as direct observation or self-assessment questionnaires, are not only subjective, time-consuming and labor-intensive, but also difficult to continuously and objectively monitor in the dynamically changing construction site.

[0003] With the development of artificial intelligence technology, human pose estimation technology based on computer vision provides new possibilities for realizing automatic ergonomics risk assessment. However, existing visual assessment schemes are mostly limited to two-dimensional pose analysis, and their evaluation accuracy is limited due to the lack of depth information. Although subsequent three-dimensional pose estimation technology has been developed, these three-dimensional assessment methods have achieved good results in analyzing the poses of individual workers, but when applied to real construction sites, their limitations become apparent: first, existing methods are mainly designed for single-person scenarios and cannot meet the monitoring needs of multi-person collaborative work that is prevalent in construction sites; second, existing methods mostly predict "person-centered" relative coordinates and cannot obtain the real spatial position relationship between different workers, making it difficult to analyze collaborative interaction behaviors.

[0004] At present, there is still a lack of a scheme that can simultaneously accurately position and reconstruct the poses of multiple workers in three-dimensional space using only a conventional monocular camera, and perform collaborative ergonomics risk assessment on this basis. SUMMARY

[0005] The present application provides a multi-person three-dimensional ergonomics risk assessment method and device to solve the technical problem that the prior art cannot meet the monitoring needs of multi-person collaborative work that is prevalent in construction sites.

[0006] To solve the above technical problems, the present application provides the following technical solutions: On the one hand, the present application provides a multi-person three-dimensional ergonomics risk assessment method, comprising: obtaining an operation image frame of a construction site; wherein the operation image frame is obtained by a monocular image acquisition device, and each image frame in the operation image frame contains multiple workers; inputting the operation image frame into a pre-trained pose and depth assessment model to obtain the pose information and depth information of each worker in the operation image frame; For each worker, integrate the posture information and the depth information, reconstruct the absolute three-dimensional human posture of the worker in the world coordinate system, and thus obtain the absolute three-dimensional human postures of all workers in the same world coordinate system. Based on the absolute three-dimensional human posture of the worker in the world coordinate system, the ergonomics risk of the worker is evaluated in real time, and the ergonomics risk evaluation result of each worker is obtained.

[0007] Further, the posture and depth evaluation model comprises a detection module, a posture estimation module and a depth estimation module; wherein, The detection module is used to identify and locate each worker in the work image frame. The posture estimation module is used to identify the relative three-dimensional human posture of each worker located by the detection module with the worker's own pelvis as the origin, denoted as posture information; and extract the corresponding posture features of the worker. The depth estimation module is used to fuse the posture features extracted by the posture estimation module, and predict the absolute depth of each worker located by the detection module in the three-dimensional space, denoted as depth information.

[0008] Further, the depth information is represented as:

[0009] wherein, k is a dynamic adjustment factor related to the real-time posture of the worker learned and predicted by the depth estimation module; and is the focal length of the camera pixel; A is a preset human standard space area; W and H are the pixel width and height of the image bounding box of the worker wrapped with the to-be-predicted depth information, respectively.

[0010] Further, the detection module identifies and locates each worker in the work image frame in the following manner: For the first frame of image in the work image frame, a Faster R-CNN target detection model is used for worker identification and positioning; for the subsequent frames after the first frame of image in the work image frame, a posture key point-based tracking method is used to directly locate the worker area of the current frame using the posture information estimated in the previous frame.

[0011] Further, the backbone network of the posture estimation module adopts a ResNet network architecture.

[0012] Further, the backbone network of the depth estimation module is composed of a plurality of sequentially connected blocks, and a depth separable convolution is used instead of a standard convolution operation in each block.

[0013] Further, the training of the posture and depth evaluation model includes two stages: in the first stage, the posture estimation module is trained; in the second stage, the posture estimation module is frozen and the depth estimation module is trained.

[0014] Further, the training set of the posture estimation module is an enhanced training set that fuses two-dimensional and three-dimensional human posture public data sets; during training, a two-dimensional-three-dimensional collaborative learning technique is adopted, and when calculating the loss function, the depth dimension from the two-dimensional data set sample is marked as invisible, so that it does not participate in loss calculation; and during the training process, a residual log-likelihood estimation method is adopted to calibrate the probability distribution of the predicted human key point coordinates.

[0015] Further, based on the absolute three-dimensional human posture of the workers in the world coordinate system, the ergonomics risk of the workers is evaluated in real time, and the ergonomics risk evaluation result of each worker is obtained, including: For each worker, based on the absolute three-dimensional human posture in the world coordinate system, the ergonomics parameters of the human body are calculated, including the bending and twisting angles of the torso, the forward inclination and lateral bending angles of the neck, the support mode of the legs, the lifting and abduction angles of the upper arms, the flexion and extension angles of the forearms, and the flexion and deviation angles of the wrists; The calculated ergonomics parameters of the human body are input into a preset rapid whole-body evaluation system to obtain a quantitative posture fatigue risk score, and the fatigue risk of the workers is classified according to the obtained posture fatigue risk score; Wherein, the risk levels of worker fatigue include five levels of negligible risk, low risk, medium risk, high risk and extremely high risk; after classifying the fatigue risk of the workers according to the obtained posture fatigue risk score, the multi-person three-dimensional ergonomics risk evaluation method further includes: When the fatigue risk level of any worker reaches high risk or extremely high risk, a preset warning signal is triggered.

[0016] Further, after the ergonomics risk of the workers is evaluated in real time based on the absolute three-dimensional human posture of the workers in the world coordinate system, the method further includes: based on the absolute three-dimensional human posture of the workers in the world coordinate system and the individual risk evaluation result, the collective ergonomics risk distribution of the workers is determined and analyzed.

[0017] The absolute spatial distance between any two workers in the world coordinate system in the work image frame is calculated i And j The distance D ij The distance D ij can be calculated as follows:

[0018] in,( x k , y k , z k ) indicates workers k The three-dimensional coordinates of a specific body part (e.g., the pelvic joint in the three-dimensional pose) in the world coordinate system. If the absolute spatial distance... D ij Less than the preset collaboration distance threshold T coop Then the worker is judged i and j In the same collaborative work group G .

[0019] The cooperative distance threshold T coop This is a preset parameter, determined based on ergonomic or geometric constraints of the actual work scenario. The determination criteria may include, but are not limited to: the worker's maximum comfortable reach distance, the maximum effective length of tools or materials involved in collaborative work, and the minimum safe distance required for a specific task. T coop It can be determined through the experience of domain experts or based on statistical analysis of historical operation data.

[0020] Furthermore, the determination and analysis of the collective ergonomic risk distribution of workers also includes: For the aforementioned collaborative work group G The individual postural fatigue risk scores of all workers in the group were integrated. S n Calculate the group's collective risk index. S G The aforementioned collective risk indicators S G It can be calculated using the following formula:

[0021] in, Indicates the collaborative work group G The total number of workers in the country.

[0022] When the collective risk indicator S G Reaching or exceeding the preset collective risk threshold T coop In this case, the multi-person three-dimensional ergonomics risk assessment method triggers a collective early warning signal or outputs suggestions for optimizing task allocation. On the other hand, the present invention also provides a multi-person three-dimensional ergonomics risk assessment device, comprising: The data acquisition module is configured to acquire work image frames of a construction site, wherein each of the work image frames is acquired by a monocular image acquisition device and contains a plurality of workers. The data processing module is configured to: input the work image frames into a pre-trained posture and depth evaluation model to acquire posture information and depth information of each worker in the work image frames; for each worker, integrate the posture information and the depth information to reconstruct an absolute three-dimensional human posture of the worker in a world coordinate system, thereby obtaining absolute three-dimensional human postures of all the workers in the same world coordinate system; based on the absolute three-dimensional human postures of the workers in the world coordinate system, perform real-time ergonomic risk assessment on the workers to obtain ergonomic risk assessment results of each worker.

[0023] In another aspect, the present application further provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above method.

[0024] In another aspect, the present application further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, which is loaded and executed by the processor to implement the above method.

[0025] The technical scheme provided by the present application has at least the following beneficial effects: The multi-person three-dimensional ergonomic risk assessment method and device provided by the present application overcome the limitations of the prior art that most of them can only handle single-person scenarios and can only obtain relative three-dimensional postures centered on individuals. Through the innovative architecture of posture and depth separation fusion and the two-stage decoupling training strategy, the present application can simultaneously and accurately reconstruct the absolute three-dimensional postures of multiple persons in the same picture in a unified three-dimensional coordinate system only by using a monocular camera. This makes it possible to analyze the ergonomics of multi-person collaborative work commonly existing in construction and other scenarios, not only individual risks but also collective risk distribution can be assessed, providing comprehensive and objective technical support for optimizing task allocation and protecting the occupational health of multiple workers. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1is a flowchart of a multi-person three-dimensional ergonomics risk assessment method provided by an embodiment of the present application; Figure 2 is a model structure diagram provided by an embodiment of the present application; wherein (a) is the overall architecture of the model; (b) is a pose estimation module; and (c) is a depth estimation module; Figure 3 is a flowchart of a two-stage decoupling training strategy provided by an embodiment of the present application; Figure 4 is an application effect example diagram of risk assessment on multiple workers provided by an embodiment of the present application; wherein (a) shows the pose reconstruction and risk assessment result of multiple workers under a first representative frame; and (b) shows the pose reconstruction and risk assessment result of multiple workers under a second representative frame; Figure 5 is a system block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0029] First of all, it should be noted that in the embodiments of the present application, the words such as "exemplarily", "for example" are used to represent as an example, illustration or explanation. Any embodiment or design scheme described as "exemplary" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "exemplarily" is intended to present the concept in a specific way. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0030] First embodiment

[0031] To solve the technical problem that the prior art solutions are mostly limited to single-person scenarios and cannot obtain the real three-dimensional spatial position relationship between multiple persons, the present embodiment provides a multi-person three-dimensional ergonomics risk assessment method, which can be implemented by an electronic device. The execution flow of the method is as shown in Figure 1 , which includes the following steps: S1, obtaining a work image frame of a construction site; wherein the work image frame is obtained by a monocular image acquisition device, and each image frame in the work image frame contains multiple workers; Specifically, in the present embodiment, the acquisition method of the work image frame can be: using a smart phone to video a real work scene containing multiple workers cooperatively performing a steel bar installation work at a frame rate of 30 frames / second and a resolution of 1920x1080 pixels, and decomposing the video into continuous work image frames.

[0032] S2, input the work image frame into the pre-trained pose and depth evaluation model to obtain the pose and depth information of each worker in the work image frame; Specifically, in this embodiment, the structure of the pose and depth evaluation model is as follows: Figure 2 As shown, it includes a detection module, a pose estimation module, and a depth estimation module.

[0033] The detection module identifies and locates each worker in the work image frames. This module employs an efficient strategy combining detection and tracking. Specifically, for the first frame of the work image frame sequence, the Faster R-CNN object detection model is used for worker identification and localization. For subsequent frames of the work image frame sequence, a pose keypoint-based tracking method is used, utilizing the pose information estimated in the previous frame to directly locate the worker region in the current frame. This avoids performing computationally intensive object detection for each frame, thus significantly improving processing efficiency.

[0034] The pose estimation module identifies the relative 3D human pose of each worker located by the detection module, with their pelvis as the origin, and records this as pose information. It also extracts the corresponding pose features. The backbone network of this module adopts a ResNet50 network architecture. The pose features refer to the high-dimensional feature vectors extracted from the backbone network (ResNet50) of the pose estimation module. These features capture the worker's pose semantic information, including the spatial relationships of key points and pose context. The extraction process is as follows: After the detection module locates the image region of each worker, it inputs the image of that region into the ResNet50 network for forward propagation. In the network, the image passes through multiple convolutional layers and residual blocks, and finally, the high-dimensional feature representation of the intermediate layers is extracted before the fully connected layers, which is the pose feature. Simultaneously, the network outputs the relative 3D human pose coordinates with the pelvis as the origin through the fully connected layers. Therefore, inputting the personnel localization results from the detection module into ResNet50 simultaneously yields both the 3D human pose and pose features.

[0035] The depth estimation module fuses the pose features extracted by the pose estimation module to predict the absolute depth of each worker located by the detection module in three-dimensional space, denoted as depth information. This module employs a lightweight network architecture as its backbone. Specifically, the backbone consists of multiple sequentially connected blocks. Each block uses depthwise separable convolutions instead of standard convolution operations, combined with batch normalization and the ReLU6 activation function to significantly reduce network parameters. Although the backbone contains 17 blocks, its total number of parameters is approximately 3 million, far fewer than the approximately 25 million parameters of the ResNet50 network used in the pose estimation module, thus ensuring both performance and computational efficiency.

[0036] The process of worker pose and depth information extraction using the model is as follows: first, the pose estimation module is used to process the image of each person located, predict the relative three-dimensional human pose with the pelvis as the coordinate origin, and simultaneously extract high-dimensional pose features for depth estimation. Then, the depth estimation module is used to fuse the pose features extracted in the previous step with the features extracted from the image by the module itself to predict the absolute root depth of the person in the three-dimensional space d . The calculation of the absolute root depth is based on the following relationship:

[0037] wherein, k is a dynamic adjustment factor related to the real-time pose of the worker learned and predicted by the depth estimation module; and is the focal length of the camera pixel; A is a preset standard human body space area; W and H are the pixel width and height of the image bounding box of the worker wrapped with the depth information to be predicted, respectively.

[0038] As shown in Figure 2 and Figure 3 , the pose and depth evaluation model training adopts a two-stage decoupling training strategy, including the following two stages: In the first stage, the pose estimation module is trained. To improve the generalization ability, its training set is an enhanced training set that combines large-scale two-dimensional and three-dimensional human pose public datasets, wherein the two-dimensional datasets include COCO and MPII, and the three-dimensional datasets include H36M, UP3D, MuCo and 3DPW. During training, a two-dimensional-three-dimensional collaborative learning technique is used, and when calculating the loss function, the depth dimension from the two-dimensional dataset sample is marked as invisible, so that it does not participate in the loss calculation, which enables the model to learn the scene diversity of two-dimensional data and the depth information of three-dimensional data at the same time. To further improve the prediction accuracy of key point coordinates, a residual log-likelihood estimation method is used in the training process to calibrate the probability distribution of the predicted human key point coordinates.

[0039] In the second stage, the pose estimation module is frozen and the depth estimation module is trained. Specifically, after the first stage of training is completed, all network parameters of the pose estimation module are "frozen", that is, they remain unchanged during the second stage of training. When training the depth estimation module, its input not only includes image information, but also fuses high-dimensional pose features extracted by the frozen pose estimation module through splicing processing, which significantly improves the accuracy of depth prediction.

[0040] Compared with the embodiment, if the pose features extracted by the pose estimation module are not fused in the depth estimation module, the average root depth error of the workers will be significantly increased. For example, the error value can be increased from 230 mm to about 260 mm or from 473 mm to about 618 mm on different test data sets. The results strongly prove that the effective cooperation of pose features and depth features is a key technical feature to achieve high-precision multi-person absolute three-dimensional space positioning, thereby ensuring the accuracy and reliability of the method in ergonomics risk assessment.

[0041] S3, for each worker, integrating its pose information and depth information to reconstruct its absolute three-dimensional human pose in the world coordinate system to obtain the absolute three-dimensional human poses of all workers in the same world coordinate system; Specifically, S3 is a coordinate transformation and integration of the relative three-dimensional pose coordinates with the pelvis as the origin obtained by S2 and the absolute root depth as the absolute three-dimensional space coordinates, thereby reconstructing the absolute three-dimensional poses of all personnel in the same world coordinate system in the scene.

[0042] S4, based on the absolute three-dimensional human pose of the worker in the world coordinate system, real-time ergonomics risk assessment of the worker's body is performed to obtain the ergonomics risk assessment result of each worker.

[0043] Specifically, in the embodiment, the implementation process of S4 is as follows: S41, based on the reconstructed absolute three-dimensional human pose, a set of ergonomics parameters required for rapid whole-body assessment is calculated, including the bending and twisting angles of the torso, the forward inclination and lateral bending angles of the neck, the support mode of the legs, the lifting and abduction angles of the upper arms, the flexion and extension angles of the forearms, and the flexion and extension and deflection angles of the wrists.

[0044] S42, input the calculated angle parameters into the rapid whole-body assessment system to obtain a quantitative posture fatigue risk score, and divide the fatigue risk into five levels of negligible risk, low risk, medium risk, high risk, and extremely high risk according to the score.

[0045] S43, when the system determines that the fatigue risk level of any person reaches the preset high risk level, a preset warning signal is triggered. Figure 4 The application effect of the method in a real scene is shown. For example, in frame A ( Figure 4 , (a)), worker 2 in a squatting position has a rapid whole-body assessment score of 9, and the risk level is determined to be high risk; worker 4 in a bending position has a score of 7, and the risk level is medium risk. In frame B ( Figure 4In (b) of the example, workers 1 and 2, who are also in a squatting position, scored 10 and 9 respectively, both reaching the high-risk or very high-risk level, and the system will trigger an alert for this.

[0046] Furthermore, in this embodiment, the method may also include the following steps: Based on the absolute three-dimensional human posture of the workers in the aforementioned world coordinates and the results of individual risk assessments, the collective ergonomic risk distribution of the workers is determined and analyzed, as follows: Based on the absolute three-dimensional human posture, a collaborative work group is determined. The calculation is performed on any two workers in the work image frame. i and j The absolute spatial distance between them in the world coordinate system D ij The distance D ij It can be calculated using the following formula:

[0047] in,( x k , y k , z k ) indicates workers k The three-dimensional coordinates of a specific body part (e.g., the pelvic joint in the three-dimensional pose) in the world coordinate system. If the absolute spatial distance... D ij Less than the preset collaboration distance threshold T coop Then the worker is judged i and j In the same collaborative work group G .

[0048] The cooperative distance threshold T coop This is a preset parameter, determined based on ergonomic or geometric constraints of the actual work scenario. The determination criteria may include, but are not limited to: the worker's maximum comfortable reach distance, the maximum effective length of tools or materials involved in collaborative work, and the minimum safe distance required for a specific task. T coop It can be determined through the experience of domain experts or based on statistical analysis of historical operation data.

[0049] Integrate individual risk assessment results to quantify collective risk indicators. This applies to the aforementioned collaborative work group. G The individual postural fatigue risk scores of all workers in the group were integrated. S n(i.e., the evaluation results of step S4), calculate the group's collective risk index. S G The aforementioned collective risk indicators S G It can be calculated using the following formula:

[0050] in, Indicates the collaborative work group G The total number of workers in the country.

[0051] Based on the aforementioned collective risk indicator, a collective early warning is triggered. When the aforementioned collective risk indicator... S G Reaching or exceeding the preset collective risk threshold T coop At that time, the multi-person three-dimensional ergonomic risk assessment method triggers a collective early warning signal or outputs suggestions for optimizing task allocation.

[0052] In summary, this embodiment provides a method for assessing the ergonomic risks of multiple workers in 3D, overcoming the limitations of existing technologies that mostly only handle single-person scenarios and can only acquire relative 3D poses centered on the individual. This method, through an innovative architecture that separates and fuses pose and depth, and a two-stage decoupled training strategy, can synchronously and accurately reconstruct the absolute 3D poses of multiple workers in the same frame within a unified 3D coordinate system using only a single camera. This makes it possible to perform ergonomic analysis on collaborative work involving multiple workers, commonly found in scenarios such as construction, enabling not only the assessment of individual risks but also the analysis of collective risk distribution, providing comprehensive and objective technical support for optimizing task allocation and ensuring the occupational health of multiple workers.

[0053] Second Embodiment

[0054] This embodiment provides a multi-person three-dimensional ergonomic risk assessment device, which includes the following modules: The data acquisition module is used to acquire operation image frames at the construction site; wherein, the operation image frames are acquired through a monocular image acquisition device, and each image frame contains multiple workers; The data processing module is used for: The work image frame is input into a pre-trained pose and depth evaluation model to obtain the pose and depth information of each worker in the work image frame; For each worker, their posture and depth information are integrated to reconstruct their absolute three-dimensional human posture in the world coordinate system, thus obtaining the absolute three-dimensional human posture of all workers in the same world coordinate system. Based on the worker's absolute three-dimensional human posture in the world coordinates, the worker's ergonomic risks are assessed in real time, and the ergonomic risk assessment results for each worker are obtained.

[0055] It should be noted that the multi-person three-dimensional ergonomic risk assessment device in this embodiment corresponds to the multi-person three-dimensional ergonomic risk assessment method in the first embodiment described above; the functions implemented by each functional module in the multi-person three-dimensional ergonomic risk assessment device in this embodiment correspond one-to-one with the process steps in the multi-person three-dimensional ergonomic risk assessment method in the first embodiment described above; therefore, they will not be described again here.

[0056] Third Embodiment

[0057] This embodiment provides an electronic device, such as... Figure 5 As shown, the electronic device includes a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment described above. Furthermore, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0058] Below, in conjunction with Figure 5 A detailed introduction to each component of this electronic device is provided below: The processor is the control center of the electronic device. The electronic device may include multiple processors, each of which can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The term "processor" can refer to a single processor or a collective term for multiple processing elements. For example, a processor can be one or more central processing units (CPUs), other general-purpose processors, application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), one or more field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0059] In a specific implementation, as one example, the processor may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 shown are, of course, merely illustrative examples.

[0060] The memory is used to store the software program that executes the solution of the present invention, and the processor controls its execution. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0061] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may be integrated with the processor or exist independently, and may be accessed through the interface circuit of the electronic device ( Figure 5 (Not shown in the image) is coupled to the processor; however, this embodiment of the invention does not impose specific limitations on this.

[0062] The transceiver may include a receiver and a transmitter. Figure 5 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function. The transceiver can be integrated with the processor or exist independently, and can be connected through the interface circuit of the electronic device (…). Figure 5 (Not shown in the image) is coupled to the processor, and this embodiment of the invention does not specifically limit this.

[0063] In addition, it should be noted that, Figure 5 The structure of the electronic device shown is not intended to limit the device. Actual devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects achieved by this electronic device when performing the method of the first embodiment described above can be referenced to the technical effects described in the first embodiment; therefore, they will not be repeated here.

[0064] Fourth embodiment

[0065] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0066] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely or partially hardware embodiment, a completely or partially software embodiment, or an embodiment combining software and hardware aspects. Moreover, when implemented in software, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any usable medium accessible to a computer or a data storage device such as a server or data center containing one or more sets of usable media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive (SSD).

[0067] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.

[0069] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element. Furthermore, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Additionally, the character " / " in this text generally indicates an "or" relationship between the preceding and following objects, but it can also indicate an "AND / OR" relationship. Please refer to the context for specific interpretations. "At least one" refers to one or more items, while "more than" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be represented as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0070] Furthermore, it is understood that in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0071] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0072] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of functional modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Additionally, the functional units in the various embodiments of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0073] If the method is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments of the present invention have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make several improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A multi-person three-dimensional ergonomics risk assessment method, characterized by, The method comprises the following steps: acquiring work image frames of a construction site; wherein the work image frames are acquired by a monocular image acquisition device, and each frame of the work image frames contains a plurality of workers; inputting the work image frames into a pre-trained posture and depth evaluation model to obtain posture information and depth information of each worker in the work image frames; for each worker, integrating the posture information and the depth information to reconstruct an absolute three-dimensional human posture of the worker in a world coordinate system, thereby obtaining absolute three-dimensional human postures of all workers in the same world coordinate system; based on the absolute three-dimensional human postures of the workers in the world coordinate system, performing real-time ergonomic risk assessment on the workers to obtain ergonomic risk assessment results of each worker.

2. The multi-person three-dimensional ergonomic risk assessment method of claim 1, wherein, The posture and depth evaluation model comprises a detection module, a posture estimation module and a depth estimation module; wherein the detection module is used for identifying and locating each worker in the work image frames; the posture estimation module is used for identifying a relative three-dimensional human posture of each worker located by the detection module with the worker's own pelvis as the origin, denoted as posture information; and extracting posture features of the worker; the depth estimation module is used for fusing the posture features extracted by the posture estimation module to predict an absolute depth of each worker located by the detection module in a three-dimensional space, denoted as depth information.

3. The multi-person three-dimensional ergonomic risk assessment method of claim 2, wherein, The depth information is represented as: ； wherein, k is a dynamic adjustment factor learned and predicted by the depth estimation module in relation to the real-time pose of the worker; and is the focal length of the camera pixel; A is a pre-set human body standard space area; W and H are the pixel width and height of the image bounding box of the worker wrapped with the depth information to be predicted, respectively.

4. The multi-person three-dimensional ergonomic risk assessment method of claim 2, wherein, The detection module identifies and locates each worker in the work image frames in the following manner: for a first frame of the work image frames, a Faster R-CNN target detection model is used for worker identification and location; for subsequent frames after the first frame of the work image frames, a tracking method based on posture key points is used to directly locate the worker region of the current frame by using the posture information estimated in the previous frame.

5. The multi-person three-dimensional ergonomic risk assessment method of claim 2, wherein, The backbone network of the posture estimation module adopts a ResNet network architecture.

6. The multi-person three-dimensional ergonomic risk assessment method of claim 2, wherein, The backbone network of the depth estimation module is composed of a plurality of sequentially connected blocks, and a depth separable convolution is used in each block to replace a standard convolution operation.

7. The multi-person three-dimensional ergonomic risk assessment method of claim 2, wherein, The training of the posture and depth evaluation model comprises the following two stages: in the first stage, the posture estimation module is trained; in the second stage, the posture estimation module is frozen and the depth estimation module is trained.

8. The multi-person three-dimensional ergonomic risk assessment method of claim 7, wherein, The training set of the posture estimation module is an enhanced training set that fuses a two-dimensional and three-dimensional human posture public data set; during training, a two-dimensional-three-dimensional collaborative learning technology is used, the depth dimension from the two-dimensional data set sample is marked as invisible so as to not participate in loss calculation; and a residual log-likelihood estimation method is used in the training process to calibrate the probability distribution of the predicted human key point coordinates.

9. The multi-person three-dimensional ergonomic risk assessment method of claim 1, wherein, The real-time ergonomic risk assessment based on the absolute three-dimensional human postures of the workers in the world coordinate system to obtain the ergonomic risk assessment results of each worker comprises: For each worker, based on the absolute three-dimensional body posture of the worker in the world coordinate, the ergonomics parameters of the worker are calculated, including the bending and twisting angles of the torso, the forward leaning and lateral bending angles of the neck, the support mode of the leg, the lifting and abduction angles of the upper arm, the flexion and extension angles of the forearm, and the flexion and extension and deflection angles of the wrist; The calculated ergonomics parameters are input into a preset rapid whole-body assessment system to obtain a quantitative posture fatigue risk score, and the fatigue risk of the worker is classified according to the obtained posture fatigue risk score; Wherein, the level of fatigue risk of the worker includes five levels of negligible risk, low risk, medium risk, high risk and extremely high risk; after classifying the fatigue risk of the worker according to the obtained posture fatigue risk score, the multi-person three-dimensional ergonomics risk assessment method further comprises: When the fatigue risk level of any worker reaches high risk or extremely high risk, a preset warning signal is triggered.

10. A multi-person three-dimensional ergonomics risk assessment device, characterized by, Comprise: Data acquisition module, for acquiring work image frames of construction site; Wherein, the work image frames are acquired by monocular image acquisition device, and each frame of image in the work image frames contains multiple workers; Data processing module, for: Input the work image frames into the pre-trained posture and depth assessment model to obtain the posture information and depth information of each worker in the work image frames; For each worker, integrate the posture information and depth information of the worker to reconstruct the absolute three-dimensional body posture of the worker in the world coordinate system, so as to obtain the absolute three-dimensional body posture of all workers in the same world coordinate system; Based on the absolute three-dimensional body posture of the worker in the world coordinate, the ergonomics risk of the worker is evaluated in real time to obtain the ergonomics risk evaluation result of each worker.

Images