Operation monitoring method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the field of visual inspection, and discloses an operation monitoring method, device and equipment and a storage medium, and the method comprises the steps: obtaining a video stream of a monitoring region which comprises an operation region and a non-operation region; performing target detection on the video stream of the monitoring area according to the target characteristics of the operating personnel; under the condition that the operator is detected, obtaining an operator area image containing the operator, and carrying out identity identification on the operator in the obtained operator area image to obtain an identity of the operator; performing target tracking on the identity label to obtain a movement track of the operator; based on the motion trail of the operator, judging the area where the operator is located, and when it is judged that the operator is in the non-operation area, determining that the operation process of the operator does not conform to the operation specification, and deleting the motion trail and the identity label of the operator. According to the invention, the problem that an operator is misjudged to be not on duty when the operator is shielded by a shielding object in an operation area can be solved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of visual inspection, and in particular to an operation monitoring method, device, equipment and storage medium. Background Art

[0002] Operation monitoring usually includes on-the-job monitoring of operators and monitoring of operators' operation behaviors. The current on-the-job monitoring method mainly detects whether operators are in the operation area through target detection in the operation area, so as to judge whether operators are on the job. However, in the case where operators are blocked by obstructions in the operation area, it will be misjudged that the operators are not on the job because the operators are not detected in the operation area. In addition, the current operator behavior monitoring method mainly relies on the behavior recognition of operators to confirm the operation status of operators based on the behavior recognition results. However, the current behavior recognition algorithm has limited accuracy and it is difficult to distinguish similar actions, such as "waving" and "wiping". Therefore, it is unreliable to rely on behavior recognition results to determine the operation status of operators. Summary of the invention

[0003] The purpose of the present invention is to at least provide a work monitoring method, which can at least solve the problem that when the work area is blocked by obstructions, the work personnel may be misjudged as not being on duty.

[0004] To solve the above technical problems, at least one embodiment of the present application provides a method for monitoring an operation, including:

[0005] Acquire a video stream of a monitoring area, wherein the monitoring area includes an operating area and a non-operating area;

[0006] Performing target detection on the video stream of the monitoring area according to the target characteristics of the operator;

[0007] When the operator is detected, a personnel area image including the operator is acquired, and the identity of the operator in the acquired personnel area image is identified to obtain the identity of the operator;

[0008] Tracking the identity marker to obtain the movement trajectory of the operator;

[0009] Based on the movement trajectory of the operator, the area where the operator is located is determined. When it is determined that the operator is in the non-operating area, it is determined that the operating process of the operator does not comply with the operating specifications, and the movement trajectory and identity identification of the operator are deleted.

[0010] In some optional embodiments, after determining the area where the operator is located, the method further includes:

[0011] When it is determined that the operator is in the operating area, the operator is determined to be on duty, and the movement trajectory and identity of the operator are stored.

[0012] In some optional embodiments, after determining that the operator is on duty and storing the movement trajectory and identity of the operator, the method further includes:

[0013] Detecting the equipment status of the operating equipment and the action behavior of the operating personnel;

[0014] Comparing the operator's action behavior with a preset standardized operation behavior to determine whether the operator's action behavior is an operation behavior;

[0015] When the equipment state of the operating equipment is an operating state, and the action behavior of the operator is an operating behavior, determining that the operating process of the operator complies with the operating specification;

[0016] When the equipment state of the operating equipment is a non-operating state, or when the action of the operator is a non-operating action, it is determined that the operating process of the operator does not meet the operating specifications.

[0017] In some optional embodiments, the process of detecting the equipment status of the operating equipment includes:

[0018] Get the video stream of the work area;

[0019] Performing target detection on the video stream of the operation area according to the target features of the operation equipment;

[0020] When the working equipment is detected, the working equipment area image is acquired, and the equipment status of the acquired working equipment area image is identified to obtain the equipment status of the working equipment.

[0021] In some optional embodiments, the process of detecting the operator's action behavior includes:

[0022] Get the video stream of the work area;

[0023] Performing target detection on the video stream of the operation area according to the target features of the operator;

[0024] When the operator is detected, the operator area image is acquired, and behavior recognition is performed on the acquired operator area image to obtain the action behavior of the operator.

[0025] In some optional embodiments, the identity identification includes the identity ID of the operator.

[0026] In some optional embodiments, after determining that the operating process of the operating personnel does not comply with the operating specifications, it also includes: generating warning information, and the warning information includes the identity of the operating personnel whose operating process does not comply with the operating specifications.

[0027] At least one embodiment of the present application further provides a work monitoring device, including:

[0028] An acquisition module, used to acquire a video stream of a monitoring area, wherein the monitoring area includes an operation area and a non-operation area;

[0029] A target detection module, used to perform target detection on the video stream of the monitoring area according to the target characteristics of the operator;

[0030] An identity recognition module is used to obtain a personnel area image containing the operator when the operator is detected, and to perform identity recognition on the operator in the obtained personnel area image to obtain the identity of the operator;

[0031] A target tracking module is used to track the identity identifier to obtain the movement trajectory of the operator;

[0032] The operation detection module is used to determine the area where the operator is located based on the movement trajectory of the operator. When it is determined that the operator is in a non-operating area, it is determined that the operation process of the operator does not comply with the operation specifications, and the movement trajectory and identity identification of the operator are deleted.

[0033] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned job monitoring method.

[0034] At least one embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned job monitoring method when executed by a processor.

[0035] The embodiment of the present application provides a method, device, equipment and storage medium for operation monitoring, which obtains a video stream of a monitoring area, wherein the monitoring area includes an operation area and a non-operation area; then, according to the target characteristics of the operator, the video stream of the monitoring area is subjected to target detection to identify the operator in the monitoring area; when the operator is detected, a personnel area image containing the operator is obtained, and the operator in the obtained personnel area image is subjected to identity identification to obtain the identity of the operator to lock the identity of the operator; the identity identification is subjected to target tracking to accurately obtain the movement trajectory of the operator; then, based on the movement trajectory of the operator, the area where the operator is located is judged, and when it is judged that the operator is in the non-operation area, it is determined that the operation process of the operator does not comply with the operation specification, and the movement trajectory and identity of the operator are deleted. In this way, even if the operator is blocked by an obstruction in the operation area, since the operator is not in the non-operation area, there will be no misjudgment that the operator is not at work. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0037] Figure 1 is a flow chart of a method for monitoring an operation provided by an embodiment of the present application;

[0038] Figure 2 is a flow chart of a method for monitoring an operation provided by another embodiment of the present application;

[0039] Figure 3 is a flow chart of a method for monitoring an operation provided by another embodiment of the present application;

[0040] Figure 4 is a structural schematic diagram of an operation monitoring device provided by another embodiment of the present application;

[0041] Figure 5 It is a structural schematic diagram of an operation monitoring device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0042] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0043] In order to facilitate understanding of the embodiments of the present application, relevant content about operation monitoring is first introduced here.

[0044] Operation monitoring usually includes on-the-job monitoring of operators and monitoring of operators' operation behaviors. The current on-the-job monitoring method mainly detects whether operators are in the operation area through target detection in the operation area, so as to judge whether operators are on the job. However, in the case where operators are blocked by obstructions in the operation area, it will be misjudged that the operators are not on the job because the operators are not detected in the operation area. In addition, the current behavior monitoring method for operators mainly relies on the behavior recognition of operators to confirm the operation status of operators based on the behavior recognition results. However, the current behavior recognition algorithm has limited accuracy and it is difficult to distinguish similar actions, such as "waving" and "wiping". Therefore, it is unreliable to rely on behavior recognition results to determine the operation status of operators.

[0045] In order to solve the above-mentioned technical problem that the workers may be misjudged as not being on duty when the work area is blocked by obstructions, the present invention proposes a work monitoring method. The implementation details of the work monitoring method of this embodiment are described in detail below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0046] Embodiment 1:

[0047] The operation monitoring method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. The specific process can be as follows: Figure 1 As shown, including:

[0048] Step S11, obtaining a video stream of a monitoring area, wherein the monitoring area includes an operation area and a non-operation area.

[0049] Specifically, the operation area refers to the area that the operator needs to pass through during the operation process, and the non-operation area refers to the area that the operator does not need to pass through during the operation process.

[0050] Step S12: performing target detection on the video stream of the monitoring area according to the target features of the operator.

[0051] Specifically, target detection can be achieved through a target detection model, such as a YOLOv8 target detection model, a Faster R-CNN target detection model, a DETR target detection model, and the like.

[0052] In this embodiment, the YOLOv8 target detection model is selected for target detection because the YOLOv8 target detection model consumes less video memory resources, runs fast, and has high detection accuracy when running.

[0053] Step S13, when the operator is detected, a personnel area image including the operator is acquired, and the identity of the operator in the acquired personnel area image is identified to obtain the identity of the operator.

[0054] Specifically, the identification mark of the operator refers to an identification mark that can identify or distinguish the identity of the operator, such as an identity ID, and is usually set on items that the operator must wear during the operation, such as a safety helmet, a work badge, and work clothes.

[0055] Step S14, tracking the identity marker to obtain the movement trajectory of the operator.

[0056] Specifically, in the case where there are two or more operators in the work area, if the target is tracked based on the characteristics of the person, it is impossible to distinguish which motion trajectory belongs to which operator, and thus it is impossible to accurately determine which operator is off duty or on duty. The present application can distinguish different operators by tracking the identity identifier to accurately obtain the motion trajectory of the operator. Target tracking can be achieved through a target tracking model. The target tracking model includes a deepsort target tracking model, a KCF target tracking model, a bytetrack tracking model, and the like.

[0057] Step S15: determining the area where the operator is located based on the movement trajectory of the operator.

[0058] Specifically, when the movement track of the operator covers the non-operation area, it is determined that the operator is in the non-operation area. When the movement track of the operator only covers the operation area, it is determined that the operator is in the operation area.

[0059] Step S16, when it is determined that the operator is in the non-operating area, it is determined that the operating process of the operator does not comply with the operating specifications, and the movement trajectory and identity identification of the operator are deleted.

[0060] Specifically, when it is determined that the operator is in the non-operating area, it can be considered that the operator has come to the non-operating area, that is, the operator has left his post, and it can be determined that the operator's operation process does not meet the operation specifications. By deleting the operator's movement trajectory and identity identification, the subsequent on-the-job status of the operator can be judged.

[0061] In some embodiments, Figure 2 As shown, after determining the area where the operator is located, the method further includes:

[0062] Step S17, when it is determined that the operator is in the operating area, the operator is determined to be on duty, and the movement trajectory and identity of the operator are stored.

[0063] This embodiment obtains a video stream of a monitoring area, wherein the monitoring area includes an operation area and a non-operation area; then, according to the target features of the operator, target detection is performed on the video stream of the monitoring area to identify the operator in the monitoring area; when the operator is detected, a personnel area image containing the operator is obtained, and the operator in the obtained personnel area image is identified to obtain the identity of the operator to lock the identity of the operator; the identity is tracked to accurately obtain the movement trajectory of the operator; then, based on the movement trajectory of the operator, the area where the operator is located is judged, and when it is judged that the operator is in the non-operation area, it is determined that the operation process of the operator does not comply with the operation specification, and the movement trajectory and identity of the operator are deleted. In this way, even if the operator is blocked by an obstruction in the operation area, since the operator is not in the non-operation area, there will be no misjudgment that the operator is not at work.

[0064] Embodiment 2:

[0065] On the basis of the first embodiment, the operation monitoring method provided in this embodiment adds a state detection process for the operator after step S17 in the first embodiment. The specific process is as follows: Figure 3 As shown, including:

[0066] Step S21, detecting the equipment status of the operating equipment and the action behavior of the operator.

[0067] Specifically, the device status includes an operating status (also called an on status) and a closed status.

[0068] In one example, the process of detecting the equipment status of the operating equipment includes: acquiring a video stream of the operating area, and then performing target detection on the video stream of the operating area based on the target characteristics of the operating equipment; acquiring an image of the operating equipment area when the operating equipment is detected, and performing equipment status recognition on the acquired image of the operating equipment area to obtain the equipment status of the operating equipment.

[0069] Among them, the method adopted for equipment status identification includes: (1) identifying the status of the operating equipment by detecting the color of the indicator light of the operating equipment. For example, red indicates on and green indicates off. (2) identifying the status of the operating equipment by detecting the position and status of the switch button of the operating equipment. For example, pressing the switch button indicates on and popping up the switch button indicates off. (3) judging the status of the operating equipment by detecting the motion status of the operating equipment. For example, the operating equipment is in motion when it is on and the operating equipment is stationary when it is off.

[0070] In some embodiments, the process of detecting the action behavior of the operator includes: acquiring a video stream of the work area; then performing target detection on the video stream of the work area based on the target characteristics of the operator; acquiring an image of the operator area when the operator is detected, and performing behavior recognition on the acquired image of the operator area to obtain the action behavior of the operator.

[0071] Specifically, the method for behavior recognition includes inputting the operator area image into a behavior recognition model for behavior recognition. The behavior recognition model includes a slowfast behavior recognition model, a HOG behavior recognition model, a C3D behavior recognition model, and the like.

[0072] Step S22, comparing the operator's action behavior with the preset standardized operation behavior to determine whether the operator's action behavior is an operation behavior.

[0073] Specifically, by comparing the similarity between the operator's action behavior and the preset standardized work behavior, when the similarity between the operator's action behavior and the preset standardized work behavior is higher than the preset similarity threshold, the operator's action behavior is considered to be work behavior; when the similarity between the operator's action behavior and the preset standardized work behavior is lower than the similarity threshold, the operator's action behavior is considered not to be work behavior.

[0074] Step S23, when the equipment state of the operating equipment is the operating state, and the action behavior of the operator is the operating behavior, it is determined that the operating process of the operator complies with the operating specification.

[0075] Step S24, when the equipment state of the operating equipment is a non-operating state, or when the action of the operator is a non-operating action, it is determined that the operating process of the operator does not comply with the operating specification.

[0076] This embodiment comprehensively judges whether the operator's operating process complies with operating specifications by combining the equipment status of the operating equipment and whether the operator's action behavior is an operating behavior, so as to make up for the deficiency of low accuracy in identifying the operator's behavior alone, thereby improving the accuracy of detecting whether the operator's operating process complies with operating specifications.

[0077] Based on the first embodiment, this embodiment combines the equipment status of the operating equipment and whether the operator's action behavior is an operating behavior to judge whether the operating process of the operator in the operating area complies with the operating specifications, so as to monitor the operating status of the on-duty operator.

[0078] Embodiment three:

[0079] Another embodiment of the present application relates to an operation monitoring device. The implementation details of the operation monitoring device of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details, and is not necessary for the implementation of this solution. The schematic diagram of the operation monitoring device of this embodiment can be as follows Figure 4 As shown, it includes an acquisition module 401, a target detection module 402, an identity recognition module 403, a target tracking model 404 and a job detection module 405.

[0080] The acquisition module 401 is used to acquire a video stream of a monitoring area, where the monitoring area includes an operation area and a non-operation area.

[0081] Specifically, the operation area refers to the area that the operator needs to pass through during the operation process, and the non-operation area refers to the area that the operator does not need to pass through during the operation process.

[0082] The target detection module 402 is used to perform target detection on the video stream of the monitoring area according to the target features of the operator.

[0083] Specifically, target detection can be achieved through a target detection model, such as a YOLOv8 target detection model, a Faster R-CNN target detection model, a DETR target detection model, and the like.

[0084] In this embodiment, the YOLOv8 target detection model is selected for target detection because the YOLOv8 target detection model consumes less video memory resources, runs fast, and has high detection accuracy when running.

[0085] The identity recognition module 403 is used to obtain a personnel area image containing the operator when the operator is detected, and to perform identity recognition on the operator in the obtained personnel area image to obtain the identity of the operator.

[0086] Specifically, the identification mark of the operator refers to an identification mark that can identify or distinguish the identity of the operator, such as an identity ID, and is usually set on items that the operator must wear during the operation, such as a safety helmet, a work badge, and work clothes.

[0087] The target tracking module 404 is used to track the identity mark to obtain the movement trajectory of the operator.

[0088] Specifically, in the case where there are two or more operators in the work area, if the target is tracked based on the characteristics of the person, it is impossible to distinguish which motion trajectory belongs to which operator, and thus it is impossible to accurately determine which operator is off duty or on duty. The present application can distinguish different operators by tracking the identity identifier to accurately obtain the motion trajectory of the operator. Target tracking can be achieved through a target tracking model. The target tracking model includes a deepsort target tracking model, a KCF target tracking model, a bytetrack tracking model, and the like.

[0089] The operation detection module 405 is used to determine the area where the operator is located based on the movement trajectory of the operator; it is also used to determine that the operation process of the operator does not comply with the operation specifications when it is determined that the operator is in a non-operating area, and delete the movement trajectory and identity identification of the operator.

[0090] Specifically, when it is determined that the operator is in the non-operating area, it can be considered that the operator has come to the non-operating area, that is, the operator has left his post, and it can be determined that the operator's operation process does not meet the operation specifications. By deleting the operator's movement trajectory and identity identification, the subsequent on-the-job status of the operator can be judged.

[0091] In some embodiments, the operation detection model 405 is also used to determine that the operator is on duty when it is determined that the operator is in the operation area, and store the movement trajectory and identity of the operator.

[0092] In this embodiment, the acquisition module 401 acquires the video stream of the monitoring area, and the monitoring area includes the operation area and the non-operation area; then the target detection module 402 performs target detection on the video stream of the monitoring area according to the target features of the operator to identify the operator in the monitoring area; then the identity recognition module 403 acquires the personnel area image containing the operator when the operator is detected, and performs identity identification on the operator in the acquired personnel area image to obtain the identity of the operator to lock the identity of the operator; then the target tracking module 404 performs target tracking on the identity to accurately obtain the motion trajectory of the operator; then the operation detection module 405 determines the area where the operator is located based on the motion trajectory of the operator, and when it is determined that the operator is in the non-operation area, it is determined that the operation process of the operator does not meet the operation specifications, and the motion trajectory and identity of the operator are deleted. In this way, even if the operator is blocked by an obstruction in the operation area, because the operator is not in the non-operation area, there will be no misjudgment that the operator is not at work.

[0093] In one embodiment, the operation monitoring device further includes an equipment status detection module 406 and a behavior recognition module 407 .

[0094] The device status detection module 406 is used to detect the device status of the operating equipment.

[0095] Specifically, the device status includes an operating status (also called an on status) and a closed status.

[0096] In one example, the device state detection module 406 includes a first acquisition unit, a first target detection unit, and a device state identification unit.

[0097] The first acquisition unit is used to acquire a video stream of the working area.

[0098] The first target detection unit is used to perform target detection on the video stream of the working area according to the target characteristics of the working equipment.

[0099] The equipment status recognition unit is used to obtain the working equipment area image when the working equipment is detected, and perform equipment status recognition on the obtained working equipment area image to obtain the equipment status of the working equipment.

[0100] Among them, the method adopted for equipment status identification includes: (1) identifying the status of the operating equipment by detecting the color of the indicator light of the operating equipment. For example, red indicates on and green indicates off. (2) identifying the status of the operating equipment by detecting the position and status of the switch button of the operating equipment. For example, pressing the switch button indicates on and popping up the switch button indicates off. (3) judging the status of the operating equipment by detecting the motion status of the operating equipment. For example, the operating equipment is in motion when it is on and the operating equipment is stationary when it is off.

[0101] The behavior recognition module 407 is used to detect the action behavior of the operator.

[0102] In one example, the behavior recognition module 407 includes

[0103] The second acquisition unit is used to acquire the video stream of the working area.

[0104] The second target detection unit is used to perform target detection on the video stream of the working area according to the target characteristics of the operating personnel.

[0105] The behavior recognition unit is used to obtain the operator area image when the operator is detected, and perform behavior recognition on the obtained operator area image to obtain the action behavior of the operator.

[0106] Specifically, the method for behavior recognition includes inputting the operator area image into a behavior recognition model for behavior recognition. The behavior recognition model includes a slowfast behavior recognition model, a HOG behavior recognition model, a C3D behavior recognition model, and the like.

[0107] The operation detection module 405 is also used to compare the operator's action behavior with the preset standardized operation behavior to determine whether the operator's action behavior is an operation behavior; when the equipment state of the operation equipment is an operation state, and the operator's action behavior is an operation behavior, it is determined that the operator's operation process complies with the operation specifications; when the equipment state of the operation equipment is a non-operation state, or when the operator's action behavior is a non-operation behavior, it is determined that the operator's operation process does not comply with the operation specifications.

[0108] Specifically, by comparing the similarity between the operator's action behavior and the preset standardized work behavior, when the similarity between the operator's action behavior and the preset standardized work behavior is higher than the preset similarity threshold, the operator's action behavior is considered to be work behavior; when the similarity between the operator's action behavior and the preset standardized work behavior is lower than the similarity threshold, the operator's action behavior is considered not to be work behavior.

[0109] This embodiment comprehensively judges whether the operator's operating process complies with operating specifications by combining the equipment status of the operating equipment and whether the operator's action behavior is an operating behavior, so as to make up for the deficiency of low accuracy in identifying the operator's behavior alone, thereby improving the accuracy of detecting whether the operator's operating process complies with operating specifications.

[0110] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0111] Embodiment 4:

[0112] Another embodiment of the present application relates to an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the job monitoring method in the above-mentioned embodiments.

[0113] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0114] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0115] Embodiment five:

[0116] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0117] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0118] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for monitoring an operation, characterized in that: include: Acquire a video stream of a monitoring area, where the monitoring area includes an operating area and a non-operating area; Performing target detection on the video stream of the monitoring area according to the target characteristics of the operator; When the operator is detected, a personnel area image including the operator is acquired, and the identity of the operator in the acquired personnel area image is identified to obtain the identity of the operator; Tracking the identity marker to obtain the movement trajectory of the operator; Based on the movement trajectory of the operator, the area where the operator is located is determined. When it is determined that the operator is in the non-operating area, it is determined that the operating process of the operator does not comply with the operating specifications, and the movement trajectory and identity identification of the operator are deleted.

2. The operation monitoring method according to claim 1, characterized in that: After determining the area where the operator is located, the method further includes: When it is determined that the operator is in the operating area, the operator is determined to be on duty, and the movement trajectory and identity of the operator are stored.

3. The operation monitoring method according to claim 2, characterized in that: After determining that the operator is on duty and storing the movement trajectory and identity of the operator, the method further includes: Detecting the equipment status of the operating equipment and the action behavior of the operating personnel; Comparing the operator's action behavior with a preset standardized operation behavior to determine whether the operator's action behavior is an operation behavior; When the equipment state of the operating equipment is an operating state, and the action behavior of the operator is an operating behavior, determining that the operating process of the operator complies with the operating specification; When the equipment state of the operating equipment is a non-operating state, or when the action of the operator is a non-operating action, it is determined that the operating process of the operator does not meet the operating specifications.

4. The operation monitoring method according to claim 3, characterized in that: The process of detecting the equipment status of the operating equipment includes: Get the video stream of the work area; Performing target detection on the video stream of the operation area according to the target features of the operation equipment; When the working equipment is detected, the working equipment area image is acquired, and the equipment status of the acquired working equipment area image is identified to obtain the equipment status of the working equipment.

5. The operation monitoring method according to claim 3, characterized in that: The detection process of the operator's action behavior includes: Get the video stream of the work area; Performing target detection on the video stream of the operation area according to the target features of the operator; When the operator is detected, the operator area image is acquired, and behavior recognition is performed on the acquired operator area image to obtain the action behavior of the operator.

6. The operation monitoring method according to claim 1, characterized in that: The identity identification includes the identity ID of the operator.

7. The operation monitoring method according to any one of claims 1 to 6, characterized in that: After determining that the operation process of the operator does not meet the operation specification, the method further includes: Generate warning information, the warning information including the identity of the operator whose operation process does not comply with the operation specification.

8. An operation monitoring device, characterized in that: include: An acquisition module, used to acquire a video stream of a monitoring area, wherein the monitoring area includes an operation area and a non-operation area; A target detection module, used to perform target detection on the video stream of the monitoring area according to the target characteristics of the operator; An identity recognition module is used to obtain a personnel area image containing the operator when the operator is detected, and to perform identity recognition on the operator in the obtained personnel area image to obtain the identity of the operator; A target tracking module is used to track the identity identifier to obtain the movement trajectory of the operator; The operation detection module is used to determine the area where the operator is located based on the movement trajectory of the operator. When it is determined that the operator is in a non-operating area, it is determined that the operation process of the operator does not comply with the operation specifications, and the movement trajectory and identity identification of the operator are deleted.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the operation monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the operation monitoring method according to any one of claims 1 to 7 is implemented.

Images