Methods, devices, electronic equipment and storage media for determining worker behavior

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for determining worker behavior. The method includes: acquiring images of workers performing their tasks at a power construction site; determining key human body points and their relationships within the images based on a pre-established human posture recognition model; determining the worker's behavior corresponding to the key human body points and their relationships; comparing the behavior with pre-defined rules of conduct to determine if any violations have occurred. This invention's solution can determine violations using a human posture recognition model, eliminating the need for safety supervisors to review the images, reducing resource waste, and improving the accuracy and effectiveness of violation determination.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, apparatus, electronic device and storage medium for determining worker behavior. Background Technology

[0002] Currently, during power grid inspections in my country, a large number of personnel are still required to conduct equipment maintenance and hazard elimination at power equipment sites. Power work sites are high-risk environments with numerous personnel and equipment. To improve the safety standards of power grid operation and maintenance, it is necessary to monitor personnel behavior in power construction scenarios in real time, promptly and accurately detect violations such as not wearing safety helmets or work clothes correctly, and issue timely warnings to ensure the safety of operations within the monitored area.

[0003] In existing technologies, it is usually necessary to collect images of workers at the work site. Safety supervisors then review these images to determine if workers are violating regulations, such as not wearing safety helmets or work clothes correctly. However, this method consumes a significant amount of human resources and relies on subjective judgment to confirm violations, which can lead to errors and reduces the accuracy and effectiveness of the determination process. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for determining worker behavior, enabling the identification of violations through a human posture recognition model. This eliminates the need for safety supervisors to review work images, reducing resource waste and improving the accuracy and effectiveness of identifying violations.

[0005] In a first aspect, embodiments of the present invention provide a method for determining worker behavior, including:

[0006] Collect images of workers during their operations at power construction sites;

[0007] Based on a pre-established human posture recognition model, the key points of the human body of the worker in the work image and the correlation between each key point are determined.

[0008] The operator's work behavior corresponding to the key points of the human body and the associated relationship is determined, and the work behavior is compared with the pre-set violation behavior to determine whether the operator has violated the rules.

[0009] Secondly, embodiments of the present invention also provide a device for determining worker behavior, the device comprising:

[0010] The image acquisition module is used to acquire images of workers during their work at power construction sites.

[0011] The human body key point determination module is used to determine the human body key points of the worker in the work image and the correlation between each human body key point based on a pre-established human posture recognition model.

[0012] The task behavior determination module is used to determine the task behavior of the operator corresponding to the key points of the human body and the association relationship, compare the task behavior with the pre-set violation behavior, and determine whether the operator has violated the rules.

[0013] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the operator behavior determination method provided in any embodiment of the present invention.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the operator behavior determination method provided in any embodiment of the present invention.

[0018] The present invention provides a method for determining worker behavior, which constructs a human posture recognition model to automatically identify worker behavior without the need for manual monitoring, saving a significant amount of human resources, improving the accuracy of identifying violations, eliminating potential safety hazards, and enhancing operational safety. Furthermore, by analyzing worker behavior and issuing warning messages when violations are detected, the method can promptly identify safety issues and resolve potential safety hazards during power grid operations.

[0019] Furthermore, the worker behavior determination device, electronic device, and storage medium provided by this invention correspond to the above-described method and have the same beneficial effects. Attached Figure Description

[0020] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A flowchart illustrating a method for determining worker behavior according to an embodiment of the present invention;

[0022] Figure 2 A flowchart of another method for determining worker behavior provided in an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of key points and their relationships in the human body provided in an embodiment of the present invention;

[0024] Figure 4 A structural diagram of a worker behavior determination device provided in an embodiment of the present invention;

[0025] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0027] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0028] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Example 1

[0030] Figure 1 This is a flowchart illustrating a method for determining worker behavior according to an embodiment of the present invention. The method can be executed by a worker behavior determination device, which can be implemented through software and / or hardware, and can be configured in a terminal and / or server to implement the worker behavior determination method in this embodiment of the invention.

[0031] like Figure 1 As shown, the method in this embodiment may specifically include:

[0032] S101. Collect images of workers' work processes at power construction sites.

[0033] In practice, image acquisition devices installed at power construction sites can capture images of workers during their operations. By analyzing the workers' actions in these images, it can be determined whether their behavior is standardized. For example, two or more images can be captured simultaneously from the same work scene.

[0034] Specifically, to improve the pixel quality of the work images and enhance the features of the target to be detected, optimize detection performance, and avoid overfitting, preprocessing operations such as convolution calculation, linear rectified function calculation, and pooling operation can be performed on the work images.

[0035] Optionally, after acquiring images of workers performing their tasks at the power construction site, the process may also include: performing convolution calculations, linear rectification function calculations, and pooling operations on the work images, and updating the work images to the images obtained after the operations.

[0036] For example, in the preprocessing operation, the task image is input into a convolutional layer with a kernel size of 3*3. Through convolution calculation, the pixel values ​​of the task image are adjusted. The first image obtained after convolution calculation is then processed by a linear rectified function to generate a second image. The linear rectified function sets the pixel values ​​at positions in the image with pixel values ​​less than a preset threshold to 0, improving the network sparsity of the image and avoiding overfitting while improving detection performance. The second image is then input into a pooling layer of size 2*2, and the preprocessed feature map is output, thereby achieving image filtering and highlighting pixels with strong expressive power. Furthermore, to enhance the feature map's effect, the output feature map can be input again into a convolutional layer, a linear rectified function, and a pooling layer for processing; this embodiment of the invention does not limit this. For example, the preprocessing operation can involve a total of 12 convolutional layers, 12 linear rectified functions, and 3 pooling layers.

[0037] S102. Based on the pre-established human posture recognition model, determine the key points of the human body of the worker in the work image and the correlation between the key points of each human body.

[0038] In practical implementation, key human body points of the worker in the work image can be identified based on a pre-established human posture recognition model. For example, key human body points may include the nose, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right ankle, left hip, left knee, left ankle, right eye, left eye, right ear, and left ear. Furthermore, the relationships between these key human body points in the work image can be determined. It should be noted that these relationships refer to the positional and connectivity relationships of each key human body point in the work image. Based on these key human body points and their relationships, the worker's current posture can be determined.

[0039] Optionally, based on a pre-established human posture recognition model, the key points of the workers in the work image and the correlation between each key point are determined, including: inputting the work image into the human posture recognition model to generate a first number of confidence images and a second number of affinity field images corresponding to the work image; determining the key points of the workers in the work image based on the first number of confidence images; and determining the correlation between each key point based on the second number of affinity field images.

[0040] Specifically, the first number can be consistent with the number of human key points that need to be determined. For example, the first number can be 18, corresponding to 18 human key points. Each human key point can correspond to a confidence image, and the confidence value represented by each pixel position in the confidence image can reflect the degree of probability that each pixel position is the corresponding key point.

[0041] For example, in the confidence image corresponding to the nose, each pixel position corresponds to a confidence value. This confidence value reflects the likelihood that the information reflected at that pixel position in the work image is the worker's nose; the higher the confidence value, the greater the likelihood. Therefore, based on each confidence image, the key human body points of the worker in the work image and the pixel positions of each key human body point can be determined. Furthermore, in addition to including the images corresponding to each key human body point, the confidence images can also include the work background image to reflect the background information of the power site.

[0042] S103. Determine the work behavior of the operators corresponding to the key points and relationships of the human body, compare the work behavior with the pre-set violations, and determine whether the operators have violated regulations.

[0043] In practice, the behavioral characteristics of the worker can be determined by combining key points on the human body and the relationships between them. Based on these behavioral characteristics, the worker's posture in the given work image can be determined. For example, based on the relationships between key points on the human body, if the worker's behavioral characteristics are found to be that the left elbow, left hand, and left shoulder are at the same height, then the worker's posture can be determined to be raising the left arm.

[0044] Optionally, the work behavior of the workers corresponding to key human body points and relationships can be determined, including: determining the background information of the power construction site corresponding to the work image, and determining the work behavior of the workers based on the background information, key human body points and relationships.

[0045] Specifically, based on key points and relationships within the human body, the current posture of the worker can be determined, such as raising a leg, squatting, standing, lying down, and raising an arm. However, posture alone cannot accurately determine whether the worker is performing normal tasks or engaging in violations. To improve the accuracy of the determination process, the background information of the power construction site corresponding to the work image can be determined using information output by the human posture recognition model. Based on the background information and the worker's current posture, the worker's current work behavior is determined. It is then determined whether the worker's work behavior falls within the scope of violations defined in the behavioral guidelines. If so, it indicates that the worker is currently violating regulations; if not, it indicates that the worker is currently performing normal work without violating regulations.

[0046] For example, when it is determined that the worker's current action is a leg-raising action, the leg-raising action is not a violation; however, when the background information of the operation is combined to detect that the worker's leg is directly above the safety fence when raising it, it can be determined that the worker is crossing the fence, and the worker's current behavior can be determined as a violation.

[0047] Optionally, the operator's work behavior is determined based on the work background information, key points of the human body, and related relationships, including: determining whether the work background information includes work-related information; if so, determining the operator's work behavior based on the work-related information, key points, and related relationships.

[0048] The work-related information includes equipment information and / or work identification information. Equipment information includes information about electrical equipment, cables, and fences in the substation, while work identification information includes work safety notices and signs. When the work background information includes equipment information and / or work identification information, it is necessary to combine the equipment information and / or work identification information to accurately determine whether the worker's behavior violates the rules. When the work background information does not include work-related information, it means that the specific content of the work background information does not affect the judgment of the worker's behavior. For example, if the work background information only includes environmental information such as the sky and ground, it can be determined that the work background information does not include work-related information.

[0049] Furthermore, when the background information of the task does not include task-related information, the task behavior of the operator can be determined directly based on the key points of the human body and their relationships, which helps to improve the efficiency of determining the task behavior of the operator.

[0050] This invention provides a method for determining worker behavior, which involves acquiring images of workers performing their tasks at a power construction site; using a pre-established human posture recognition model to determine key human body points and their relationships within the images; identifying the worker's behavior corresponding to these key points and relationships; and comparing this behavior with pre-defined rules of conduct to determine if any violations have occurred. This invention enables the determination of violations through a human posture recognition model, eliminating the need for safety supervisors to review the images, reducing resource waste, and improving the accuracy and effectiveness of violation identification.

[0051] Example 2

[0052] Figure 2 This is a flowchart of another method for determining worker behavior provided by an embodiment of the present invention. This embodiment is based on and optimized from the above-described technical solutions. Optionally, before determining the key points of the worker's body and the correlation between these key points in the work image, the method further includes: acquiring at least one sample image of the worker's work process to form a sample dataset; pre-labeling the workers in each sample image of the sample dataset; training a convolutional neural network based on the labeled sample images; and establishing a human pose recognition model based on the training results. Optionally, after comparing the work behavior with pre-defined violations to determine whether the worker has violated regulations, the method further includes: when it is determined that the worker has violated regulations, generating a warning message containing the violation information and sending it to the safety personnel's terminal for notification. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0053] like Figure 2 As shown, the method in this embodiment may specifically include:

[0054] S201. Obtain at least one sample image of the operator's work process to form a sample dataset, and pre-label the operators in each sample image of the sample dataset.

[0055] Optionally, at least one sample image of the operator's work process can be obtained, including: acquiring historical monitoring video, performing frame extraction on the historical monitoring video at preset time intervals, and determining each frame image obtained from the frame extraction as a sample image.

[0056] In practice, at least one sample image of a worker's work process can be obtained by extracting frames from historical surveillance videos at preset time intervals. Specifically, the preset time interval can be set to one hour or one day, and during frame extraction, two or more video frames can be extracted from the surveillance video at the same time. Furthermore, the extracted video frames can be compared, and based on the similarity between the frames, the frames with the largest similarity differences are selected as sample images, forming a sample dataset. Further, key human body points and their relationships can be labeled in each sample image within the sample dataset.

[0057] For example, when acquiring sample images, to improve the diversity of sample images, video frames containing workers of different genders, ages, occupations, and body types can be extracted from historical surveillance videos as sample images. Furthermore, considering the background diversity of the sample images, video frames with different weather conditions and scenes can be extracted as sample images to enrich the obtained sample images and better train the convolutional neural network.

[0058] Optionally, assembling a sample dataset includes: performing transformation processing on at least one acquired sample image; and constructing a sample dataset based on the transformed images and sample images obtained after the transformation processing. The transformation processing includes at least one of scaling, contrast adjustment, color dithering, and noise addition. Specifically, to increase the diversity of the sample dataset, each sample image can be transformed, and the transformed images and sample images can be combined to form the sample dataset, thereby expanding the sample dataset.

[0059] S202. Based on the labeled sample images, train the convolutional neural network and establish a human pose recognition model based on the training results.

[0060] Specifically, labeled sample images are input into a convolutional neural network (CNN). The CNN is trained based on the labeled information of artificial keypoints and their relationships in the sample images, as well as the output of the CNN. A human pose recognition model is then constructed based on the generated parameters in the trained CNN.

[0061] Based on the above embodiments, the specific steps for training a convolutional neural network based on the labeled sample images and establishing a human pose recognition model based on the training results may include: inputting sample images into the convolutional neural network and outputting the recognition results of the convolutional neural network on the poses of workers in the sample images; comparing the recognition results with the labels in the sample images, adjusting the network parameters of the convolutional neural network based on the comparison results, and updating the convolutional neural network; repeatedly performing the operations of inputting sample images into the convolutional neural network, comparing the obtained recognition results with the labels, and adjusting the grid parameters of the neural network until the current convolutional neural network meets the preset conditions, ending the training process of the convolutional neural network, and determining the current convolutional neural network as the human pose recognition model.

[0062] Specifically, for each sample image, the labeled sample image can be input into a convolutional neural network (CNN). The CNN then identifies the posture of the workers in the sample images, obtaining the recognition result. For example, the recognition result includes the key points of the workers' bodies and the relationships between these key points. Based on the comparison between the recognition result and the labels in the sample images, the network parameters of the CNN are adjusted so that the adjusted CNN's recognition results for the sample images more closely approximate the labels, thereby improving the accuracy of the trained CNN.

[0063] Furthermore, the above operations can be repeated to continuously adjust the network parameters of the convolutional neural network until the current training process reaches the preset conditions. At this point, the training of the convolutional neural network can be stopped, and the current convolutional neural network can be identified as a human pose recognition model.

[0064] For example, the preset condition may be that the number of repeated executions reaches a preset threshold. Those skilled in the art can determine the preset threshold according to the actual application situation, and this embodiment of the invention does not limit it. Furthermore, the preset condition may also include that the number of markers inconsistent with the current recognition result is less than a preset threshold, that is, it indicates that the recognition effect of the current convolutional neural network is closer to the real situation of the workers in the sample image, and then training can be stopped.

[0065] S203. Collect images of workers' work processes at power construction sites.

[0066] S204. Based on a pre-established human posture recognition model, determine the key points of the human body of the worker in the work image and the correlation between the key points of each human body.

[0067] S205. Determine the work behavior of the operators corresponding to the key points and relationships of the human body, compare the work behavior with the pre-set violations, and determine whether the operators have violated any regulations.

[0068] This invention proposes a method for determining worker behavior. It constructs a human posture recognition model, which automatically identifies worker behavior without requiring manual monitoring, saving significant manpower, improving the accuracy of identifying violations, eliminating potential safety hazards, and enhancing operational safety. The human posture recognition model is trained using a sample dataset, enabling the identification of violations.

[0069] S206. When it is determined that a worker has violated regulations, a warning message containing the violation information is generated and sent to the safety personnel's terminal as a reminder.

[0070] Optionally, when a violation by a worker is identified, it is necessary to promptly alert both the worker and safety personnel to prevent operational hazards caused by the violation. Specifically, a warning message containing the violation information can be generated and sent to the safety personnel's terminal and / or the worker's terminal to alert the worker and enable the safety personnel to promptly identify the hazard. Furthermore, the warning message can be sent to the worker's terminal or the safety personnel's terminal based on pre-stored contact information such as phone number or email address.

[0071] The worker behavior analysis method proposed in this invention can promptly identify safety issues and resolve potential safety hazards during power grid operations by providing warning messages when workers are found to have violated regulations.

[0072] Example 3

[0073] The embodiments of the method for determining worker behavior have been described in detail above. To enable those skilled in the art to further understand the technical solution of this method, specific application scenarios are given below.

[0074] In practice, after inputting the image of the task to be detected into the human pose recognition model, the process can be divided into a first branch for determining human key points and a second branch for determining correlation relationships. The first branch outputs feature maps with 19 channels, each containing the confidence level of each key point, consisting of 18 human key point feature maps and one background feature map. The second branch outputs feature maps with 38 channels to represent the affinity field, i.e., the correlation relationships between the various human key points.

[0075] Specifically, the two branches each perform a combination of convolution calculations and linear rectified function operations four times, followed by a convolution operation on each branch. This results in the first branch outputting a 19-channel feature map and the second branch outputting a 38-channel feature map. The feature maps from the two branches are then concatenated to generate a unique target feature map. To improve the accuracy of the output feature map, the target feature map can be used as input to repeat the above steps; the number of repetitions can be determined by those skilled in the art based on actual needs. Based on the final concatenated feature map, the key points and relationships of the worker's body are determined.

[0076] Figure 3 A schematic diagram of key points on the human body and their relationships is provided as an embodiment of the present invention; for example... Figure 3 As shown, connecting key points of the human body based on correlation can reveal the worker's movements and postures, and determine the worker's work behavior.

[0077] For example, the process of determining a violation can be described using the violation of climbing over a guardrail as an example. Based on the human body key points and their relationships determined by the human posture recognition model, the key point coordinates of each key point are determined. When the height coordinate of the worker's right ankle is greater than that of the left knee, or vice versa, it can be determined that the right ankle or left ankle is above the guardrail based on the background information of the operation. If so, it is determined that the worker is currently climbing over the guardrail, which is a violation.

[0078] When a violation by a worker is identified, it is necessary to promptly alert both the worker and safety personnel to prevent operational hazards arising from this violation. Specifically, a warning message containing the violation information can be generated and sent to the safety personnel's terminal and / or the worker's terminal to alert the worker and enable safety personnel to promptly identify the hazard. Furthermore, the warning message can be sent to the worker's or safety personnel's terminal based on pre-stored contact information such as phone numbers and email addresses.

[0079] This embodiment proposes a method for determining worker behavior. It constructs a human posture recognition model to determine the key points of the worker's body in the work image and the correlations between these key points. By analyzing these key points and their correlations, the method determines the worker's work behavior and whether any violations exist. This overcomes the problem of accurately identifying violations during power grid production and inspection, improving the safety standards of power grid operation and maintenance, and enabling safe supervision of personnel behavior in power construction scenarios. Simultaneously, a worker behavior analysis method is proposed. Combining the work background and the worker's actions, it analyzes whether the worker's behavior violates regulations. If a violation is found, a warning message is generated, overcoming the problem of not being able to detect violations in a timely manner. This eliminates the need for safety supervisors to review work images, reducing resource waste and improving the accuracy and effectiveness of determining violations.

[0080] Example 4

[0081] Figure 4 This is a structural diagram of a worker behavior determination device provided in an embodiment of the present invention. This device is used to execute the worker behavior determination method provided in any of the above embodiments. This device and the worker behavior determination method in the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the worker behavior determination device can be found in the embodiments of the worker behavior determination method described above. Specifically, the device may include:

[0082] The image acquisition module 10 is used to acquire images of workers during their work at the power construction site.

[0083] The human body key point determination module 11 is used to determine the human body key points of the workers in the work image and the correlation between each human body key point based on the pre-established human posture recognition model.

[0084] The task behavior determination module 12 is used to determine the task behavior of the operator corresponding to the key points and relationships of the human body, compare the task behavior with the pre-set violation behavior, and determine whether the operator has violated the rules.

[0085] Based on any optional technical solution in the embodiments of the present invention, the device may optionally further include:

[0086] The human pose recognition model training module is used to acquire at least one sample image of the worker's work process before determining the key points of the worker's body and the correlation between the key points in the work image, forming a sample dataset; the workers in each sample image of the sample dataset are labeled in advance; based on the labeled sample images, a convolutional neural network is trained, and a human pose recognition model is established based on the training results.

[0087] Based on any optional technical solution in the embodiments of the present invention, the module for training the human pose recognition model may optionally include:

[0088] The transformation processing unit is used to perform transformation processing on at least one acquired sample image; based on the transformed images and sample images obtained after transformation processing, a sample dataset is formed; wherein, the transformation processing includes at least one of scaling processing, contrast adjustment processing, color dithering processing, and noise addition processing.

[0089] Based on any optional technical solution in the embodiments of the present invention, the module for training the human pose recognition model may optionally include:

[0090] The input sample image unit is used to input the sample image into the convolutional neural network and output the recognition result of the convolutional neural network on the posture of the worker in the sample image;

[0091] The convolutional neural network unit is updated to compare the recognition result with the label in the sample image, adjust the network parameters of the convolutional neural network based on the comparison result, and update the convolutional neural network.

[0092] The training process termination unit is used to repeatedly perform the operations of inputting the sample image into the convolutional neural network, comparing the obtained recognition result with the label, and adjusting the grid parameters of the neural network until the current training process meets the preset conditions, at which point the training process of the convolutional neural network is terminated, and the current convolutional neural network is determined as the human pose recognition model.

[0093] Based on any optional technical solution in the embodiments of the present invention, the preset condition may include the number of repeated executions reaching a preset threshold number.

[0094] Based on any optional technical solution in the embodiments of the present invention, the module for training the human pose recognition model may optionally include:

[0095] A historical surveillance video acquisition unit is used to acquire historical surveillance videos and perform frame extraction operations on the historical surveillance videos at preset time intervals; each frame image obtained by the frame extraction operation is determined as the sample image.

[0096] Based on any optional technical solution in the embodiments of the present invention, optionally, the human body key point determination module 11 includes:

[0097] Affinity field images are generated to input the work image into the human posture recognition model, generating a first number of confidence images and a second number of affinity field images corresponding to the work image; based on the first number of confidence images, the human body key points of the worker in the work image are determined; based on the second number of affinity field images, the correlation between each human body key point is determined.

[0098] Based on any optional technical solution in the embodiments of the present invention, the device may optionally further include:

[0099] The update module is used to perform convolution calculations, linear rectification function calculations, and pooling operations on the images of workers during the power construction process after they are acquired, and then update the images to the images obtained after the operations.

[0100] Based on any optional technical solution in the embodiments of the present invention, the device may optionally further include:

[0101] The alert module is used to compare the work behavior with the pre-set violation behavior to determine whether the worker has violated the rules. When a violation is found, the module generates a warning message containing the violation and sends it to the safety personnel's terminal as a reminder.

[0102] Based on any optional technical solution in the embodiments of the present invention, optionally, the job behavior determination module 12 includes:

[0103] The task background information determination module is used to determine the task background information of the corresponding power construction site in the task image. Based on the task background information, key human body points and correlations, the task behavior of the workers is determined.

[0104] Based on any optional technical solution in the embodiments of the present invention, optionally, the job behavior determination module 12 includes:

[0105] A "determine whether to include job association information" unit is used to determine whether the job background information includes job association information; wherein, the job association information includes equipment information and / or job identification information; if included, the job behavior of the operator is determined based on the job association information, the key points, and the association relationship.

[0106] Based on any optional technical solution in the embodiments of the present invention, optionally, the job behavior determination module 12 further includes:

[0107] A task behavior determination unit is used to determine the task behavior of the worker based on the human body key points and the correlation relationship when the task background information does not include task association information.

[0108] The present invention provides a worker behavior determination device that can perform the following method: acquiring images of workers performing their work at a power construction site; determining key human body points and the relationships between these key points in the images based on a pre-established human posture recognition model; determining the worker's work behavior corresponding to the key human body points and their relationships; comparing the work behavior with pre-set violation behaviors to determine whether the worker has committed any violations. This invention enables the determination of violations through a human posture recognition model, eliminating the need for safety supervisors to review the work images, reducing resource waste, and improving the accuracy and effectiveness of violation determination.

[0109] It is worth noting that in the embodiments of the above-mentioned operator behavior determination device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0110] Example 5

[0111] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. Figure 5 A block diagram of an exemplary electronic device 20 suitable for implementing embodiments of the present invention is shown. The illustrated electronic device 20 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0112] like Figure 5 As shown, the electronic device 20 is presented in the form of a general-purpose computing device. The components of the electronic device 20 may include, but are not limited to: one or more processors or processing units 201, system memory 202, and bus 203 connecting different system components (including system memory 202 and processing unit 201).

[0113] Bus 203 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0114] Electronic device 20 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 20, including volatile and non-volatile media, removable and non-removable media.

[0115] System memory 202 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 204 and / or cache memory 205. Electronic device 20 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 206 may be used to read and write non-removable, non-volatile magnetic media. Disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disk drives for reading and writing to removable non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 203 via one or more data media interfaces. Memory 202 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0116] A program / utility 208 having a set (at least one) of program modules 207 may be stored, for example, in memory 202. Such program modules 207 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 207 typically perform the functions and / or methods described in the embodiments of the present invention.

[0117] Electronic device 20 can also communicate with one or more external devices 209 (e.g., keyboard, pointing device, display 210, etc.), and with one or more devices that enable a user to interact with electronic device 20, and / or with any device that enables electronic device 20 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 211. Furthermore, electronic device 20 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 212. As shown, network adapter 212 communicates with other modules of electronic device 20 via bus 203. It should be understood that other hardware and / or software modules can be used in conjunction with electronic device 20, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] The processing unit 201 executes various functional applications and data processing by running programs stored in the system memory 202.

[0119] The electronic device provided by this invention can perform the following method: acquiring images of workers performing their tasks at a power construction site; determining key human body points and their relationships based on a pre-established human posture recognition model; determining the worker's work behavior corresponding to the key human body points and their relationships; comparing the work behavior with pre-set violation behaviors to determine whether the worker has committed any violations. This invention enables the determination of violations through a human posture recognition model, eliminating the need for safety supervisors to review the work images, reducing resource waste, and improving the accuracy and effectiveness of violation determination.

[0120] Example 6

[0121] This invention provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a method for determining worker behavior, the method comprising:

[0122] This invention collects images of workers performing their tasks at power construction sites. Based on a pre-established human posture recognition model, it determines key human body points and the relationships between these points. It then identifies the worker's corresponding actions based on these key points and relationships, comparing these actions with pre-defined rules of conduct to determine if any violations have occurred. This embodiment of the invention enables the identification of violations using a human posture recognition model, eliminating the need for safety supervisors to review the images, reducing resource waste, and improving the accuracy and effectiveness of violation identification.

[0123] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the operator behavior determination method provided in any embodiment of the present invention.

[0124] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0125] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0126] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0127] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for determining worker behavior, characterized in that, include: Collect images of workers during their operations at power construction sites; Based on a pre-established human posture recognition model, the key points of the human body of the worker in the work image and the correlation between each key point are determined. The operator's work behavior corresponding to the key points of the human body and the correlation is determined, and the work behavior is compared with the pre-set violation behavior to determine whether the operator has violated the rules. The determination of the operator's work behavior corresponding to the key points of the human body and the correlation includes: Determine the background information of the power construction site corresponding to the operation image; Determine whether the work background information includes work-related information; wherein, the work-related information includes equipment information and work identification information, the equipment information includes information on power equipment, cables and fences in the power construction site, and the work identification information includes work safety reminders and safety signs in the power construction site; If included, the work behavior of the operator is determined based on the work association information, the key points, and the association relationship.

2. The method according to claim 1, characterized in that, Before determining the key points of the worker's body in the work image and the correlation between the key points, the method further includes: Obtain at least one sample image of the operator's work process to form a sample dataset; The workers in each sample image of the sample dataset are pre-labeled; Based on the labeled sample images, a convolutional neural network is trained, and the human pose recognition model is established based on the training results.

3. The method according to claim 2, characterized in that, The constituent sample dataset includes: Transformation processing is performed on at least one of the acquired sample images; The sample dataset is composed of the transformed images obtained after transformation processing and the sample images. The transformation process includes at least one of scaling, contrast adjustment, color dithering, and noise addition.

4. The method according to claim 2, characterized in that, The process of training a convolutional neural network based on the labeled sample images and establishing the human pose recognition model based on the training results includes: The sample image is input into the convolutional neural network, and the convolutional neural network outputs the recognition result of the posture of the worker in the sample image; The recognition result is compared with the marker in the sample image, and the network parameters of the convolutional neural network are adjusted and updated based on the comparison result. The process of inputting the sample image into the convolutional neural network, comparing the obtained recognition result with the label, and adjusting the grid parameters of the neural network is repeated until the current training process meets the preset conditions. Then, the training process of the convolutional neural network ends, and the current convolutional neural network is determined as the human pose recognition model.

5. The method according to claim 4, characterized in that, The preset conditions include the number of times the execution is repeated reaching a preset threshold.

6. The method according to claim 2, characterized in that, The acquisition of at least one sample image of the operator's work process includes: Acquire historical surveillance video and perform frame extraction on the historical surveillance video at preset time intervals; Each frame image obtained from the frame extraction operation is determined as the sample image.

7. The method according to claim 1, characterized in that, The method of determining the key human body points of the worker in the work image and the correlation between these key human body points, based on a pre-established human posture recognition model, includes: The task image is input into the human posture recognition model to generate a first number of confidence images and a second number of affinity field images corresponding to the task image; Based on the first number of confidence images, determine the key human body points of the worker in the work image; Based on the second number of affinity field images, the correlation between each of the key points of the human body is determined.

8. The method according to claim 1, characterized in that, After acquiring images of workers performing their tasks at the power construction site, the method further includes: The operation graph is subjected to convolution calculation, linear rectified function calculation and pooling operation, and then the operation graph is updated to the graph obtained after the operation.

9. The method according to claim 1, characterized in that, After comparing the work behavior with pre-defined violations to determine whether the worker has committed a violation, the process further includes: When it is determined that the operator has violated regulations, a warning message containing the violation information is generated and sent to the safety personnel's terminal as a notification.

10. The method according to claim 1, characterized in that, Also includes: If not included, the operator's work behavior is determined based on the key points of the human body and the correlation.

11. A device for determining worker behavior, characterized in that, include: The image acquisition module is used to acquire images of workers during their work at power construction sites. The human body key point determination module is used to determine the human body key points of the worker in the work image and the correlation between each human body key point based on a pre-established human posture recognition model. The work behavior determination module is used to determine the work behavior of the worker corresponding to the human body key points and the association relationship, compare the work behavior with the pre-set violation behavior, and determine whether the worker has violated the rules. The module for determining work behavior includes The task background information determination module is used to determine the task background information of the power construction site corresponding to the task image, and to determine the task behavior of the workers based on the task background information, the human body key points and the correlation. The module for determining the background information of the operation includes: The "Determine whether to include work-related information" unit is used to determine whether the work background information includes work-related information; wherein, the work-related information includes equipment information and work identification information, the equipment information includes information on power equipment, cables, and fences at the power construction site, and the work identification information includes work safety prompts and safety signs at the power construction site; if included, the work behavior of the operator is determined based on the work-related information, the key points, and the association relationship.

12. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the operator behavior determination method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the operator behavior determination method as described in any one of claims 1-10.

Images