Metrological verification operation normalization automatic monitoring method and system based on behavior recognition

Abstract

The invention provides a metrological verification operation normalization automatic monitoring method and system based on behavior recognition, and belongs to the technical field of image processing, and the method comprises the steps: obtaining video stream data of a target verification operator in an operation process in real time; identifying the video stream data through the trained behavior identification model to obtain a behavior identification result; determining an operation stage of the target verification operator, and obtaining behavior rule information corresponding to the operation stage; determining whether the behavior recognition result conforms to behavior rule information corresponding to the operation stage or not; and if not, outputting reminding information to remind an operator that behavior information which is not consistent with the behavior rule information exists in the current operation behavior. According to the metrological verification operation normalization automatic monitoring method and system based on behavior recognition, the efficiency and accuracy of operation behavior recognition can be improved, nonstandard operation behaviors in the operation process can be found in time, and an operator can correct errors in time.

Description

A Method and System for Automatic Monitoring of Standardized Metrological Verification Operations Based on Behavior Recognition Technical Field

[0001] This application belongs to the field of image processing technology, and more specifically, relates to an automatic monitoring method and system for the standardization of metrological verification operations based on behavior recognition. Background Technology

[0002] Ensuring the standardization and traceability of the verification / calibration process is a core requirement of quality management in statutory metrological verification institutions, third-party calibration laboratories, and internal metrology departments within enterprises. Both domestic and international standards emphasize the importance of "process control."

[0003] Currently, the supervision of the standardization of verification operations is mainly carried out in two ways. One way relies mainly on manual supervision, that is, quality supervisors conduct on-site inspections to check whether workers have any non-compliance with the standards during the operation. The other way is to conduct random checks after the operation by video surveillance, that is, recording operation videos and then reviewing them manually afterwards.

[0004] The first method, which involves manual inspection, is difficult to achieve full coverage, inefficient, subjective, and unsustainable. The second method, which involves post-event spot checks of video surveillance, results in a huge workload for the inspectors and is very time-consuming. Furthermore, since it is a post-event check, problems may be discovered too late to be corrected in a timely manner. Summary of the Invention

[0005] The purpose of this application is to provide an automatic monitoring method and system for the standardization of metrological verification operations based on behavior recognition, so as to improve the efficiency and accuracy of identifying operational behaviors, and to promptly detect non-standard operational behaviors during the operation process, and enable operators to correct errors in a timely manner.

[0006] A first aspect of this application provides an automatic monitoring method for the standardization of metrological verification operations based on behavior recognition, comprising:

[0007] Real-time acquisition of video stream data of the target verification operator during the operation process;

[0008] The video stream data is used to perform behavior recognition through a trained behavior recognition model to obtain behavior recognition results, which include: real-time behavior sequences and human posture data.

[0009] The operation stage of the target inspection operator is determined, and the behavioral rule information corresponding to the operation stage is obtained. The behavioral rule information includes: standard behavioral information and standard human posture data corresponding to the operation stage.

[0010] Determine whether the behavior recognition result matches the behavior rule information corresponding to the operation stage;

[0011] If the two do not match, a reminder message will be output. The reminder message is used to remind the operator that there is behavioral information in the current operation that does not match the behavior rule information.

[0012] A second aspect of this application provides an automatic monitoring system for the standardization of metrological verification operations based on behavior recognition, comprising:

[0013] The image acquisition module is used to acquire video stream data of the target inspection operator in real time during the operation process;

[0014] The behavior recognition and analysis module is used to perform behavior recognition on the video stream data through a trained behavior recognition model to obtain behavior recognition results, which include: real-time behavior sequences and human posture data.

[0015] The procedure rule acquisition module is used to determine the operation stage of the target inspection operator and acquire the behavior rule information corresponding to the operation stage. The behavior rule information includes: standard behavior information and standard human posture data corresponding to the operation stage.

[0016] The main control and decision-making module is used to determine whether the behavior recognition result matches the behavior rule information corresponding to the operation stage;

[0017] The output module is used to output a reminder message if there is a discrepancy. The reminder message is used to remind the operator that there is behavioral information in the current operation that does not conform to the behavior rule information.

[0018] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described automatic monitoring method for the standardization of metrological verification operations based on behavior recognition.

[0019] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described automatic monitoring method for the standardization of metrological verification operations based on behavior recognition.

[0020] A fifth aspect of this application provides a computer program product, including a computer program or computer executable instructions, wherein when the computer program or computer executable instructions are executed by a processor, the steps of the above-described automatic monitoring method for the standardization of metrological verification operations based on behavior recognition are implemented.

[0021] The beneficial effects of the automatic monitoring method and system for the standardization of metrological verification operations based on behavior recognition provided in this application are as follows:

[0022] In this embodiment, video stream data of the target verification operator during the operation is acquired in real time, and the video stream data is used to perform behavior recognition through a trained behavior recognition model to obtain behavior recognition results. The behavior recognition results include real-time behavior sequences and human posture data. That is, the operation behavior and posture of the target verification operator can be detected in real time. After identifying the operation behavior and posture, the identified operation behavior and posture are matched with the standard operation behavior and standard human posture corresponding to the current operation stage to determine whether the current operation behavior and posture conform to the standard operation behavior and standard human posture. If they do not conform, a reminder message can be output in a timely manner to remind the operator that there is an operation that does not conform to the standard operation behavior. Compared with the prior art, the automatic monitoring method for metrological verification operation standardization based on behavior recognition provided in this embodiment can improve the efficiency and accuracy of monitoring. Furthermore, this embodiment identifies real-time acquired video and outputs reminder messages in real time when non-compliance with standard behavior is detected, so as to remind the operator that there is non-compliant behavior information in the current operation, thereby enabling the operator to correct it in a timely manner. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 is a flowchart illustrating an automatic monitoring method for the standardization of metrological verification operations based on behavior recognition, provided in an embodiment of this application.

[0025] Figure 2 is a flowchart illustrating another automatic monitoring method for the standardization of metrological verification operations based on behavior recognition provided in an embodiment of this application.

[0026] Figure 3 is a structural block diagram of an automatic monitoring system for the standardization of metrological verification operations based on behavior recognition, provided in an embodiment of this application.

[0027] Figure 4 is a schematic block diagram of the electronic device provided in an embodiment of this application. Detailed Implementation

[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0030] This application provides an embodiment of an automatic monitoring method for the standardization of metrological verification operations based on behavior recognition. This method automatically identifies whether the behavior of verification operators conforms to predetermined procedural requirements, enabling real-time monitoring, violation warnings, and objective evaluation of the operation process, thereby effectively ensuring the quality and reliability of the verification results. As shown in Figure 1, this method can be executed by electronic equipment and may include:

[0031] S101: Real-time acquisition of video stream data of the target verification operator during the operation process.

[0032] In this embodiment, when the target inspection operator enters the workstation, the operator can trigger an inspection start command to begin the inspection, which means activating the image acquisition device. This allows the image acquisition device to begin real-time acquisition of video stream data of the target inspection operator during the operation, and to perform behavior recognition and detection. Alternatively, the image acquisition device can acquire data in real-time, and if it detects the target inspection operator entering the workstation, it can trigger the inspection, thus beginning behavior recognition and detection based on the real-time acquired video stream data of the target inspection operator during the operation.

[0033] Specifically, one or more image acquisition devices are deployed at different angles of the inspection station to acquire video stream data containing the target inspection operator.

[0034] Specifically, real-time acquisition of video stream data of the target inspection operator during the operation process can include: acquiring video stream data collected by multiple video acquisition devices as the video stream data of the target inspection operator during the operation process, with each video acquisition device used to acquire video stream data of the target inspection operator from different perspectives; or, acquiring video stream data collected by multiple video acquisition devices, fusing the various video stream data to obtain fused video stream data as the video stream data of the target inspection operator during the operation process.

[0035] Specifically, the image acquisition device can be a high-definition camera. Multiple cameras are deployed at different angles at the calibration station to simultaneously capture the frontal and side views of the operator. For example, two 1080P high-definition cameras are deployed on the front and side of the pressure gauge calibration table, respectively, and equipped with ring light sources to ensure uniform and stable illumination. The front camera is mainly used to capture instrument panel readings and the operator's upper body posture, while the side camera is mainly used to monitor hand operations and equipment connection status. The cameras are connected to a main control computer with built-in behavior recognition and analysis software (such as the electronic device shown in the above embodiment), and a display screen is set up next to the calibration table for real-time feedback.

[0036] By acquiring video stream data from multiple cameras, all of them can be used as video stream data of the target inspection operator during the operation, so as to perform real-time behavior recognition based on these video stream data; or the video stream data acquired by each camera can be stitched together to form a full-view video stream for subsequent real-time behavior recognition.

[0037] S102. Perform behavior recognition on the video stream data using the trained behavior recognition model to obtain the behavior recognition result.

[0038] Furthermore, before performing behavior recognition on video stream data using a trained behavior recognition model, the process may include: collecting standard and regulated verification operation videos, training the behavior recognition model, and obtaining the trained behavior recognition model.

[0039] In this embodiment, the real-time acquired video stream is processed by the trained behavior recognition model to obtain real-time behavior recognition results, which include real-time behavior sequences and human posture data.

[0040] The real-time behavior sequence includes the current operational behavior and the order of these behaviors. Human posture data may include the relationship between the target inspection operator and the equipment, such as the angle between the operator's line of sight and the display panel.

[0041] Specifically, the video stream data is used to perform behavior recognition through a trained behavior recognition model to obtain behavior recognition results. Specifically, this may include: identifying and locating the target inspection operator from the video stream data, and identifying the operator's key skeletal point information to obtain a key skeletal point information sequence; based on the key skeletal point information sequence, identifying the real-time behavior sequence and human posture data, wherein the real-time behavior sequence includes the current operation behavior and the order of the operation behavior.

[0042] In this embodiment, the operator in the video frame is detected in real time and continuously tracked. Then, key point detection is performed to identify key skeletal points of the operator's body (such as elbows, wrists, heads, spine, etc.) to obtain a skeletal point information sequence. Based on the skeletal point sequence data, specific operating behaviors are identified. The current operating behavior includes at least one of the following: picking up the standard, picking up the instrument under test, reading the reading, tapping the instrument, adjusting the knob, wearing gloves, and installing equipment, etc., as well as the order of the operating behaviors, such as the order of the operating behaviors from first to last as "picking up the standard", "picking up the instrument under test", "reading the reading", "tapping the instrument", "adjusting the knob", "wearing gloves", etc.

[0043] As seen in the above embodiments, the video stream data can be video stream data from different perspectives. The following embodiments use video streams from two perspectives as an example to introduce the method of acquiring skeletal point information sequences from behavior: A collaborative acquisition scheme of "main view + auxiliary view" is adopted. The stereoscopic information of the dual-view video stream is supplemented to replace the spatial information missing of the single video stream, providing data support for 3D positioning of skeletal points, so as to identify the key skeletal point information of the operator and obtain the key skeletal point information sequence (by fusing the video streams from the two perspectives to generate the three-dimensional skeletal point sequence information of the operator). Among them, the main view provides the two-dimensional pixel coordinates and main pose features of the skeletal points, and the auxiliary view video stream supplements the skeletal point information of the occluded area, providing a basis for spatial depth calculation; specifically, the intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (relative position, rotation matrix) of the dual-view camera are calibrated by a checkerboard calibration board to generate a stereo matching mapping table; the dual-view frames are accurately synchronized based on hardware trigger signals to ensure that each pair of video frames corresponds to the same operation time.

[0044] Furthermore, features are extracted from the main view video stream and the auxiliary view video stream to obtain their respective feature maps; the feature map corresponding to the main view is used to predict the visibility of each bone point to obtain the visibility score (0-1).

[0045] Based on the visibility scores corresponding to each skeletal point, the feature map of the main view is supplemented by the feature map of the auxiliary view. This mainly involves supplementing and adjusting the skeletal point regions where the visibility scores are less than a preset threshold. Based on the adjusted feature map, skeletal point recognition is performed to obtain a sequence of key skeletal point information.

[0046] S103. Determine the operation stage of the target verification operator and obtain the behavioral rule information corresponding to the operation stage. The behavioral rule information includes: standard behavioral information and standard human posture data corresponding to the operation stage.

[0047] In this embodiment, the operation stage where the target verification operator is located can be a pre-set operation stage, or the operation stage input by the target verification operator can be determined as the operation stage where the target verification operator is located, or the operation stage input by other monitoring personnel can be determined as the operation stage where the target verification operator is located.

[0048] Specifically, obtaining the behavioral rule information corresponding to the operation stage can include: obtaining the behavioral information corresponding to the operation stage from the procedure rule base. The procedure rule base stores the behavioral information corresponding to each operation stage. The behavioral information is a conditional judgment statement. The condition part of the conditional judgment statement is constructed based on at least one of the operation behavior category, human posture information and operation stage information.

[0049] It should be noted that the procedure rule base stores verification procedure knowledge in a computable logical representation, transforming textual procedure clauses into a machine-understandable rule sequence. In this embodiment, the behavioral information corresponding to each operation stage can also be stored in a standard source device, meaning that the behavioral rule information corresponding to that operation stage can also be obtained from the standard source device.

[0050] The textual requirements of the verification procedures are transformed into logical rules that are understandable and judgeable by machines; for example, the rule base contains the following rules:

[0051] Rule ID-P01 (Safety and Preparedness): IF Operation Phase == "Install Pressure Gauge" THEEN Must Occur Action == "Wear Gloves";

[0052] Rule ID-P02 (Equipment Connection): IF Operation Phase == "Connect to Pressure System" THEN Must Meet Condition == "Hand Action == Rotary Wrench" AND "Connection Object == Pressure Gauge Connector" (to prevent misoperation);

[0053] Rule ID-P03 (Pre-compression operation): IF Standard source reading == 0 THEN Next action must occur == "Perform pre-compression (increase pressure to the upper limit of the range and stabilize the pressure)" (ensure the instrument cavity is fully filled);

[0054] Rule ID-P04 (Verification Procedure): IF Current Verification Point == Xi THEN Must Occur Behavior Sequence == ["Smoothly increase voltage to Xi", "Tap the casing", "Read the value directly"];

[0055] Rule ID-P05 (Reading Norms): IF Behavior == "Reading" THEN Must Meet Conditions == "Head Posture == Facing the dial" AND "Angle between line of sight and dial plane normal < 15 degrees" AND "Duration after stabilization > 3 seconds";

[0056] Rule ID-P06 (Device Operation): IF Behavior == "Operation Standard Source" THEN Must Meet Condition == "Hand Movement == Slow Rotation" (Determine if the boost / deboost rate is stable);

[0057] After determining the operational stage of the target verification operator, the standard behavior information (i.e., the standard operational behavior to be performed in this stage, and the standard execution order of the operational behavior) and standard human posture data are obtained from the procedure rule base. For example, the angle between the line of sight and the normal of the dial plane is less than 15 degrees.

[0058] S104. Determine whether the behavior recognition result matches the behavior rule information corresponding to the operation stage.

[0059] In this embodiment, after identifying the target verification operator's operational behavior, operational execution sequence, and human posture information during the verification process, it is matched with the standard behavioral rule information corresponding to the current operational stage (standard operational behavior, standard operational execution sequence, and standard human posture information of the current operational stage) to determine whether they match.

[0060] If they match, proceed to steps S101-S104 for behavior recognition.

[0061] S105. If the match is not found, output a reminder message.

[0062] The reminder information is used to alert operators that their current actions do not conform to the behavior rules.

[0063] Specifically, outputting reminder information may include: outputting reminder information in a target format. In this embodiment, the target format includes at least one of: screen display prompts, flashing lights, or voice broadcasts. That is, when a violation is detected, a real-time alarm is issued via display screen, voice, or light to remind the target verification operator to correct it immediately.

[0064] Furthermore, the alert message may also include inconsistent behavioral information, the standard behavioral information corresponding to the inconsistent behavioral information, etc.

[0065] In this embodiment, a reminder message is output to alert the operator that their current operation behavior and / or posture do not conform to the standard behavior rules, prompting the target verification operator to make timely corrections. After outputting the reminder message, the acquired video stream data continues to be acquired for behavior recognition to determine whether the target verification operator has corrected their operation behavior and / or posture. Furthermore, after detecting that the target verification operator has corrected their operation behavior and / or posture, the acquired video stream continues to be acquired for behavior detection.

[0066] Furthermore, if the operation of the target verification operator is detected to have ended, the method further includes: obtaining the target operation time, target operation type, and target operation segment corresponding to each inconsistent operation behavior of the target verification operator during the operation process, wherein each target operation time is the operation time of each inconsistent operation behavior, each target operation type is the operation type corresponding to each inconsistent operation behavior, and each target operation segment is a video segment containing each inconsistent operation behavior; and generating a standardization audit report based on each target operation time, each target operation type, and each target operation segment.

[0067] The system automatically generates a compliance audit report, recording the operation process, the time points and types of violations, and evidence fragments (images or video clips) to facilitate post-event traceability and review. Specifically, the compliance audit report may include: inspector ID, inspected form number, inspection time, a list of violations throughout the process (such as "not wearing gloves", "not performing pre-compression", "reading line tilting"), a timestamp for each violation, and a snapshot of the evidence.

[0068] In this embodiment, the automatically generated report includes a video evidence chain, which greatly facilitates internal quality audits and external reviews. Reviewers can quickly locate problems without having to watch the entire recording.

[0069] Based on the above embodiments, this embodiment introduces an automatic monitoring method for the standardization of metrological verification operations based on behavior recognition through a specific example. As shown in Figure 2, the verification process of the target verification operator is initiated, real-time video stream is collected, and then behavior recognition is performed through a trained behavior recognition model to determine whether the identified behavior conforms to the procedural rules. If it does not conform to the rules, a real-time alarm is triggered to remind the target verification operator to correct the erroneous behavior and / or posture, and the current video stream is collected again, and the behavior recognition and behavior judgment process continues to be executed. If it conforms to the rules, the conformity result is silently recorded. Furthermore, when it is detected whether the verification has ended, if it has ended, an audit report is automatically generated and archived; otherwise, the real-time video stream collection, behavior recognition, and behavior judgment processes continue until the verification is detected to be over.

[0070] Based on the above embodiments, the following embodiments illustrate an example of an automatic monitoring method for the standardization of metrological verification operations based on behavior recognition, which may specifically include:

[0071] Phase 1: Installation Preparation

[0072] (1) The operator enters the workstation and begins the inspection. The front camera captures the operator picking up the pressure gauge being inspected without wearing gloves.

[0073] (2) The behavior recognition and analysis module identified the behavior: "picking up the instrument", but did not identify the behavior: "wearing gloves".

[0074] (3) The main control and decision-making module calls rule P01 to make a judgment, and the conclusion is a violation. Immediately issue a voice warning through the speaker: "Please wear gloves before operating", and at the same time flash a prompt icon on the screen.

[0075] (4) After hearing the prompt, the operator puts on gloves and picks up the pressure gauge again. The system recognizes the "wearing gloves" behavior, determines that it complies with the procedure, and records it silently.

[0076] Phase Two: Verification Process

[0077] (1) The operator completes the installation and begins the pressure test. The system obtains the current output value of the standard source as 0MPa through the data interface, then activates rule P03 and waits for the "pre-pressure" behavior;

[0078] (2) The operator did not perform pre-pressure and started the calibration directly from 0 point. The system did not recognize the pre-pressure behavior and triggered the warning again: "Please perform the pre-pressure operation to increase the pressure to the upper limit of the range first";

[0079] (3) After the operator completes the pre-pressure test as required, the formal verification begins. The system controls the standard source to increase the pressure to the first verification point (e.g., 2.0 MPa) and stabilizes the pressure;

[0080] (4) The behavior recognition and analysis module identified that the operator had “tapped the watch case”, but then his head shifted and his line of sight formed a large angle with the watch face.

[0081] (5) The main control module determines that the reading posture is in violation according to rule P05 and issues a warning: "Please keep the reading facing forward." At the same time, the system will wait for a sufficiently long voltage stabilization time before acquiring the pointer image at this time for identification to ensure the accuracy of the reading.

[0082] Furthermore, after the entire gauge calibration is completed, the output module automatically generates a "Pressure Gauge Calibration Process Standardization Audit Report". The report includes: the calibrator ID, the gauge number being calibrated, the calibration time, a list of violations throughout the process (such as "not wearing gloves", "not performing pre-pressure", "reading line tilting"), and a timestamp and evidence snapshot for each violation. In this embodiment, the report is archived together with the original data records generated by the automatic calibration system, providing an objective and quantitative basis for quality traceability and personnel assessment.

[0083] Building upon the above embodiments, this application transforms passive post-event sampling into proactive real-time monitoring, enabling immediate intervention when errors occur and preventing unqualified results. It eliminates the subjective bias of manual supervision, ensuring consistent evaluation standards for the same operational behavior, thus achieving greater fairness and impartiality. Furthermore, the automatically generated reports, accompanied by video evidence chains, greatly facilitate internal quality audits and external reviews, allowing reviewers to quickly pinpoint problems without needing to view the entire recording.

[0084] Corresponding to the behavior recognition-based automatic monitoring system for standardized metrological verification operations described in the above embodiments, Figure 3 is a structural block diagram of a behavior recognition-based automatic monitoring system for standardized metrological verification operations provided in an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this application are shown. Referring to Figure 3, the behavior recognition-based automatic monitoring system 20 for standardized metrological verification operations includes: an image acquisition module 21, a behavior recognition analysis module 22, a procedure rule acquisition module 23, a main control and decision-making module 24, and an output module 25, wherein,

[0085] Image acquisition module 21 is used to acquire video stream data of the target inspection operator in real time during the operation process;

[0086] The behavior recognition and analysis module 22 is used to perform behavior recognition on video stream data through a trained behavior recognition model to obtain behavior recognition results, which include: real-time behavior sequences and human posture data.

[0087] The procedure rule acquisition module 23 is used to determine the operation stage of the target inspection operator and acquire the behavioral rule information corresponding to the operation stage. The behavioral rule information includes: standard behavioral information and standard human posture data corresponding to the operation stage.

[0088] The main control and decision-making module 24 is used to determine whether the behavior recognition result matches the behavior rule information corresponding to the operation stage;

[0089] Output module 25 is used to output a reminder message if there is a discrepancy. The reminder message is used to remind the operator that there is behavior information in the current operation that does not conform to the behavior rule information.

[0090] In one possible implementation of this application embodiment, when the procedure rule acquisition module 23 acquires the behavior rule information corresponding to the operation phase, it is specifically used for:

[0091] The behavioral information corresponding to each operation stage is obtained from the procedure rule base. The procedure rule base stores the behavioral information corresponding to each operation stage. The behavioral information is a conditional judgment statement. The condition part of the conditional judgment statement is constructed based on at least one of the operation behavior category, human posture information and operation stage information.

[0092] In another possible implementation of this application embodiment, if the system 20 detects that the target verification operator has finished their operation, it further includes:

[0093] The audit report generation module is used to obtain the target operation time, target operation type and target operation segment corresponding to each inconsistent operation behavior of the target verification operator during the operation process, and generate a standard audit report based on each target operation time, target operation type and target operation segment;

[0094] Each target operation time is the operation time of each inconsistent operation behavior, each target operation type is the operation type corresponding to each inconsistent operation behavior, and each target operation segment is a video segment containing each inconsistent operation behavior.

[0095] In another possible implementation of this application embodiment, when the image acquisition module 21 acquires video stream data of the target inspection operator during the operation process in real time, it is specifically used for:

[0096] Video stream data collected by multiple video acquisition devices is used as the video stream data of the target inspection operator during the operation. Each video acquisition device is used to collect video stream data of the target inspection operator from different perspectives during the operation; or...

[0097] The system acquires video stream data collected by multiple video acquisition devices, merges the individual video stream data to obtain the merged video stream data, which serves as the video stream data for the target verification operator during the operation.

[0098] In another possible implementation of this application embodiment, when the behavior recognition analysis module 22 performs behavior recognition on the video stream data through the trained behavior recognition model to obtain the behavior recognition result, it is specifically used for:

[0099] Identify and locate the target inspection operator from the video stream data, and identify the operator's key skeletal point information to obtain the key skeletal point information sequence;

[0100] Based on the key skeletal point information sequence, real-time behavior sequence and human posture data are identified. The real-time behavior sequence includes the current operation and the order of the operation.

[0101] Another possible implementation of the embodiments of this application includes at least one of the following operations: picking up the standard, picking up the instrument under test, reading the reading, tapping the instrument, adjusting the knob, wearing gloves, and installing the equipment.

[0102] In another possible implementation of this application embodiment, when outputting reminder information, the output module 25 is specifically used for:

[0103] Output reminder information in target format;

[0104] The target form includes at least one of the following: screen display prompts, flashing lights, or voice announcements.

[0105] Referring to Figure 4, which is a schematic block diagram of an electronic device provided in an embodiment of this application, the electronic device 300 in this embodiment, as shown in Figure 4, may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 are used to store computer programs, which include program instructions. The processors 301 are used to execute the program instructions stored in the memories 304. The processors 301 are configured to invoke the program instructions to execute the functions of each module / unit in the above-described device embodiments, such as the functions of the image acquisition module 21, behavior recognition and analysis module 22, procedure rule acquisition module 23, main control and decision module 24, and output module 25 shown in Figure 3.

[0106] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0107] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0108] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store behavioral information corresponding to each operation stage.

[0109] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the automatic monitoring method for the standardization of metrological verification operations based on behavior recognition provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0110] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0111] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0112] This application provides a computer program product, which includes computer-executable instructions or a computer program. The computer-executable instructions or the computer program are stored in a computer-readable storage medium. The processor of the electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the automatic monitoring method for metrological verification operation standardization based on behavior recognition described in this application.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0116] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0117] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0118] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatic monitoring of the standardization of metrological verification operations based on behavior recognition, characterized in that, include: Real-time acquisition of video stream data of the target verification operator during the operation process; The video stream data is processed by a trained behavior recognition model to obtain behavior recognition results, which include real-time behavior sequences and human posture data. The operation stage of the target detection operator is determined, and the behavior rule information corresponding to the operation stage is obtained. The behavior rule information includes standard behavior information and standard human posture data corresponding to the operation stage. It is determined whether the behavior recognition results match the behavior rule information corresponding to the operation stage. If they do not match, a reminder message is output to remind the operator that there is behavior information in the current operation that does not match the behavior rule information.

2. The method according to claim 1, characterized in that, The step of obtaining the behavior rule information corresponding to the operation stage includes: obtaining the behavior information corresponding to the operation stage from the procedure rule base, wherein the procedure rule base stores the behavior information corresponding to each operation stage, and the behavior information is a condition judgment statement, wherein the condition part of the condition judgment statement is constructed based on at least one of the operation behavior category, human posture information and operation stage information.

3. The method according to claim 1, characterized in that, If the operation of the target verification operator is detected to have ended, the method further includes: acquiring the target operation time, target operation type, and target operation segment corresponding to each inconsistent operation behavior of the target verification operator during the operation process, wherein each target operation time is the operation time of each inconsistent operation behavior, each target operation type is the operation type corresponding to each inconsistent operation behavior, and each target operation segment is a video segment containing each inconsistent operation behavior; and generating a standardization audit report based on each target operation time, each target operation type, and each target operation segment.

4. The method according to claim 1, characterized in that, The real-time acquisition of video stream data of the target inspection operator during the operation includes: acquiring video stream data collected by multiple video acquisition devices as the video stream data of the target inspection operator during the operation, wherein each video acquisition device is used to acquire the video stream data of the target inspection operator during the operation from different perspectives; or, acquiring video stream data collected by multiple video acquisition devices, fusing the video stream data to obtain fused video stream data as the video stream data of the target inspection operator during the operation.

5. The method according to claim 1, characterized in that, The step of performing behavior recognition on the video stream data through a trained behavior recognition model to obtain behavior recognition results includes: identifying and locating the target inspection operator from the video stream data, and identifying the key skeletal point information of the operator to obtain a key skeletal point information sequence; based on the key skeletal point information sequence, identifying a real-time behavior sequence and human posture data, wherein the real-time behavior sequence includes the current operation behavior and the order of the operation behavior.

6. The method according to claim 5, characterized in that, The current operating action includes at least one of the following: picking up the standard, picking up the instrument under test, reading the reading, tapping the instrument, adjusting the knob, wearing gloves, and installing the equipment.

7. The method according to claim 1, characterized in that, The output reminder information includes: outputting the reminder information in a target format; wherein, the target format includes at least one of: screen display prompt, light flashing, or voice broadcast.

8. An automatic monitoring system for the standardization of metrological verification operations based on behavior recognition, characterized in that, include: The image acquisition module is used to acquire video stream data of the target inspection operator in real time during the operation process; The behavior recognition and analysis module is used to perform behavior recognition on the video stream data through a trained behavior recognition model to obtain behavior recognition results, which include real-time behavior sequences and human posture data. The procedure rule acquisition module is used to determine the operation stage of the target verification operator and acquire the behavior rule information corresponding to the operation stage, which includes standard behavior information and standard human posture data corresponding to the operation stage. The master control and decision-making module is used to determine whether the behavior recognition results match the behavior rule information corresponding to the operation stage. The output module is used to output a reminder message if there is a discrepancy, which reminds the operator that there is behavior information in the current operation that does not match the behavior rule information.

9. The system according to claim 8, characterized in that, When the procedure rule acquisition module acquires the behavior rule information corresponding to the operation stage, it is specifically used to: acquire the behavior information corresponding to the operation stage from the procedure rule library, the procedure rule library stores the behavior information corresponding to each operation stage, the behavior information is a condition judgment statement, and the condition part of the condition judgment statement is constructed based on at least one of the operation behavior category, human posture information and operation stage information.

10. The system according to claim 8, characterized in that, If the system detects that the target verification operator has finished operating, it further includes: an audit report generation module, used to obtain the target operation time, target operation type, and target operation segment corresponding to each inconsistent operation behavior of the target verification operator during the operation, and to generate a standard audit report based on each target operation time, each target operation type, and each target operation segment; wherein, each target operation time is the operation time of each inconsistent operation behavior, each target operation type is the operation type corresponding to each inconsistent operation behavior, and each target operation segment is a video segment containing each inconsistent operation behavior.

Images