Abnormal behavior early warning method, device and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an abnormal behavior early warning method, device and system, electronic equipment and a storage medium, and belongs to the technical field of nuclear power safety detection. The method comprises the following steps: acquiring a real-time front-end operation picture and a real-time camera acquisition picture transmitted by a nuclear power engineering terminal; and performing front-end operation identification on the real-time front-end operation picture to obtain current operation information corresponding to the nuclear power engineering terminal. And performing facial feature recognition on the real-time camera-shooting acquired picture to obtain facial feature data of the operator. And determining the identity information of the operator according to the facial feature data of the operator. And based on a preset database and the current operation information, performing operation authority judgment on the identity information of the operator to obtain an authority judgment result. And executing an early warning operation in response to the condition that the permission judgment result and the current operation information meet a preset non-security condition. According to the embodiment of the invention, the operation reliability of the engineer station can be improved.

Description

Technical Field

[0001] The present application relates to the field of nuclear power safety detection technology, and in particular to an abnormal behavior early warning method, device and system, electronic equipment and storage medium. Background Art

[0002] In the field of nuclear power, the function of the engineer station is to configure and configure the non-safety-level digital control system (DCS) to achieve full-process monitoring of the production process. In the engineer station, a large number of high-risk operations such as design changes, temporary control changes, temporary work instruction implementation, and characteristic parameter settings need to be performed. In this process, there is a risk of abnormal operation behaviors such as missed operations, erroneous operations, non-standard operations, and unauthorized operations, which may lead to unplanned shutdowns of nuclear power plants and affect nuclear safety. Therefore, how to improve the reliability of the engineer station operation has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose an abnormal behavior warning method, device and system, electronic equipment and storage medium, aiming to improve the reliability of engineering station operations.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes an abnormal behavior early warning method, which is applied to a nuclear power prevention and control server, comprising: Obtain real-time front-end operation images and real-time camera acquisition images transmitted from nuclear power engineering terminals; Performing front-end operation identification on the real-time front-end operation picture to obtain current operation information corresponding to the nuclear power engineering terminal; Performing facial feature recognition on the real-time camera capture image to obtain facial feature data of the operator; Determining the identity information of the operator according to the facial feature data of the operator; Based on the preset database and the current operation information, the operator identity information is subjected to operation authority judgment to obtain an authority judgment result; In response to the authority determination result and the current operation information satisfying a preset non-safety condition, a warning operation is performed.

[0005] In some embodiments, the step of performing facial feature recognition on the real-time camera capture image to obtain facial feature data of the operator includes: Acquire an initial facial region image of the operator in the real-time camera capture picture; Extracting facial feature points from the facial area to obtain the facial feature point coordinates of the operator; Calculating an affine transformation matrix between the facial feature point coordinates and preset reference facial feature point coordinates; Aligning feature points according to the affine transformation matrix to obtain a middle facial area image; Facial feature extraction is performed on the middle facial area image to obtain facial feature data of the operator.

[0006] In some embodiments, determining the operator identity information according to the operator facial feature data includes: Calculating similarity between the operator's facial feature data and the preset facial information in the preset database to obtain similarity data; Filtering the preset facial information according to the similarity data to obtain target facial information; The operator identity information is determined according to the target facial information.

[0007] In some embodiments, the performing front-end operation identification on the real-time front-end operation picture to obtain current operation information corresponding to the nuclear power engineering terminal includes: Identify the operation type of the real-time front-end operation screen according to a preset operation identification model to obtain current operation type information; Performing operation action recognition on the real-time front-end operation screen according to the operation recognition model to obtain current action information; Performing a monitoring requirement query on a preset operation mapping table according to the current operation type information to obtain monitoring requirement information; The current operation information is determined according to the current action information, the current operation type information and the monitoring requirement information.

[0008] In some embodiments, before identifying the operation type of the real-time front-end operation screen according to the preset operation identification model to obtain the current operation type information, the abnormal behavior early warning method further includes: Extracting video frames from the real-time front-end operation screen to obtain multiple continuous operation video frames; The operation window data is filtered out from the operation video frame, and the real-time front-end operation screen is replaced by the operation window data.

[0009] In some embodiments, the operation authority determination is performed on the operator identity information based on the preset database and the current operation information to obtain the authority determination result, including: When the monitoring requirement information indicates that the current operation needs to be monitored, first identity information of a first target object and second identity information of a second target object are obtained; wherein the operator identity information includes the first identity information and the second identity information, the first target object is the operator performing the current operation at the nuclear power engineering terminal, and the second target object is the operator monitoring the current operation; Extracting first authority information from the first identity information; extracting second authority information from the second identity information; An operation permission judgment is performed on the first permission information and the second permission information based on the preset database to obtain the permission judgment result.

[0010] In some embodiments, obtaining the first identity information of the first target object and the second identity information of the second target object includes: Acquire target pixel coordinate data of the operator in the real-time camera acquisition picture; Determine the identity type of the operator based on the first preset pixel coordinate data of the real-time camera acquisition picture, the second preset pixel coordinate data of the real-time camera acquisition picture, and the target pixel coordinates, wherein the identity type is used to characterize that the operator is the first target object or the second target object; The operator identity information is determined to be the first identity information or the second identity information according to the identity type.

[0011] In some embodiments, in response to the permission determination result and the current operation information satisfying a preset non-safety condition, performing an early warning operation includes: Performing data query on a preset action mapping table according to the current action information to obtain an action query result; In response to the permission judgment result and the action query result satisfying a preset non-safety condition, a warning operation is performed, wherein the preset non-safety condition includes: the permission judgment result is characterized as a permission abnormality, and the action query result is characterized as a current action being an abnormal operation.

[0012] In some embodiments, before executing the warning operation in response to the permission determination result and the current operation information satisfying a preset non-safety condition, the abnormal behavior warning method further includes: Get the identification code value and application status of the work order; Determine the duration of the operation according to the identification code value and the application status; Generate a candidate operation log according to the operator identity information, the current operation information and the authority judgment result; According to the execution time and the operation duration in the current operation information, the candidate operation logs are screened to obtain a target operation log; The work order is updated according to the target operation log and the identification code value.

[0013] In some embodiments, determining the duration of the operation according to the identification code value and the application status includes: If the application status is the application execution status, a data query is performed on the identification code value to obtain a number query result; If the order number query result indicates that the identification code value is obtained for the first time, then start timing to obtain the initial running time; If the application status is an application cancellation status, the cancellation item information of the work order is obtained, and the end timing is determined according to the application cancellation status and the cancellation item information to obtain the target running time; The duration between the initial running time and the target running time is taken as the job duration.

[0014] In some embodiments, after the warning operation is performed in response to the permission judgment result and the current operation information satisfying the preset non-safety condition, the abnormal behavior warning method further includes: Obtaining the instruction type of the work instruction sheet, and determining a preset duration according to the instruction type; In response to the operation duration satisfying the preset non-safety condition, the early warning operation is performed; wherein the preset non-safety condition also includes that the operation duration is greater than the preset duration.

[0015] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes an abnormal behavior warning device, comprising: An acquisition module, which is used to acquire real-time front-end operation images and real-time camera acquisition images transmitted from a nuclear power engineering terminal; An operation identification module, the operation identification module is used to perform front-end operation identification on the real-time front-end operation screen to obtain current operation information corresponding to the nuclear power engineering terminal; A facial feature recognition module, which is used to perform facial feature recognition on the real-time camera collection picture to obtain facial feature data of the operator; An identity information determination module, the identity information determination module is used to determine the identity information of the operator according to the facial feature data of the operator; An authority judgment module, the authority judgment module is used to judge the operation authority of the operator identity information based on a preset database and the current operation information to obtain an authority judgment result; The early warning operation execution module is used to execute an early warning operation in response to the authority judgment result and the current operation information satisfying a preset non-safe condition.

[0016] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the abnormal behavior warning method described in the technical solution of the first aspect.

[0017] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an abnormal behavior warning system, the system comprising a nuclear power engineering terminal, a camera device and a nuclear power control server, the nuclear power control server comprising the electronic device described in the technical solution of the third aspect; the nuclear power control server is respectively connected to the nuclear power engineering terminal and the camera device, and the camera device is used to collect real-time camera collection images.

[0018] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the abnormal behavior warning method described in the technical solution of the first aspect.

[0019] The abnormal behavior early warning method, device and system, electronic device and storage medium proposed in the present application first obtain the real-time front-end operation screen and real-time camera acquisition screen transmitted from the nuclear power engineering terminal. Then, the front-end operation recognition is performed on the real-time front-end operation screen to obtain the current operation information corresponding to the nuclear power engineering terminal. Then, the facial feature data of the operator is obtained by performing facial feature recognition on the real-time camera acquisition screen. Then, the identity information of the operator is determined according to the facial feature data of the operator. Then, based on the preset database and the current operation information, the operation authority is judged on the identity information of the operator to obtain the authority judgment result. Finally, in response to the authority judgment result and the current operation information meeting the preset non-safety condition, the early warning operation is performed. Through these steps, the embodiment of the present application can realize the comprehensive monitoring and management of the nuclear power engineering operation process, ensuring that only operators with appropriate authority can perform the corresponding operation. At the same time, it can issue an early warning in time when an operation that does not meet the safety conditions is detected to prevent the occurrence of potential risks, thereby improving the safety and reliability of the operation of the nuclear power engineer station.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of the abnormal behavior warning method provided in an embodiment of the present application; Figure 2 This is a structural diagram of an abnormal behavior early warning system provided in an embodiment of the present application; Figure 3 is another flow chart of the abnormal behavior warning method provided in an embodiment of the present application; Figure 4 yes Figure 1 Flow chart of step S102 in FIG. Figure 5 yes Figure 1 Flow chart of step S103 in FIG. Figure 6 yes Figure 1 Flow chart of step S104 in FIG. Figure 7 yes Figure 1 Flow chart of step S105 in FIG. Figure 8 yes Figure 7 Flow chart of step S701 in FIG. Fig. 9 is another flow chart of the abnormal behavior warning method provided in an embodiment of the present application; Fig.10 yes Fig. 9 Flowchart of step S902 in FIG. Fig.11 yes Figure 1 Flow chart of step S106 in FIG. Fig.12 is another flow chart of the abnormal behavior warning method provided in an embodiment of the present application; Fig.13 is a schematic diagram of the structure of an abnormal behavior warning device provided in an embodiment of the present application; Fig.14 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0025] The function of the engineer station is to configure and configure the DCS, so as to realize the whole process monitoring of the production process. In the engineer station, a large number of high-risk operations such as design changes, temporary control changes, implementation of temporary work instructions and characteristic parameter settings need to be performed, involving a wide range of systems and a large amount of manual modifications. In this process, error events caused by abnormal operation behaviors such as missed operations, erroneous operations, non-standard operations, and unauthorized operations often occur, which may cause unplanned shutdowns of nuclear power plants and affect nuclear safety in serious cases.

[0026] After obtaining the operation authorization, the user logs in to the engineer station to perform operations according to the account and password assigned to different permissions (operator, control engineer, system administrator). However, the password can be easily lent or cracked and misused by a third party, that is, the operation is not performed by the password holder, thus performing unauthorized operations.

[0027] In addition, temporary work orders with uncertain requirements are usually managed in paper work orders. When operators use the engineer station, they must register their name / account number, move-in time, station number, move-out time, work content, etc. When it is necessary to trace back the operation content of the engineer station, they rely on manual review, and log review requires high skills of personnel, making it difficult to quickly locate the expected information. In addition, paper temporary work order sheets are prone to loss, incomplete records, or errors, resulting in the inability to trace the source of DCS forced signals or configuration changes. In addition, it takes a long time and is prone to errors to locate the required information in a large amount of registered information.

[0028] Based on this, the embodiments of the present application provide an abnormal behavior warning method, device and system, electronic device and storage medium, aiming to improve the reliability of engineering station operations.

[0029] The abnormal behavior warning method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the abnormal behavior warning method in the embodiments of the present application is described.

[0030] The abnormal behavior warning method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the abnormal behavior warning method, etc., but is not limited to the above forms.

[0031] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0032] The abnormal behavior warning method provided in the embodiment of the present application is applied to a nuclear power prevention and control server. Please refer to Figure 1 , Figure 1 This is an optional flowchart of the abnormal behavior warning method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0033] Step S101, obtaining the real-time front-end operation picture and the real-time camera acquisition picture transmitted from the nuclear power engineering terminal.

[0034] Step S102, performing front-end operation recognition on the real-time front-end operation screen to obtain current operation information corresponding to the nuclear power engineering terminal.

[0035] Step S103, performing facial feature recognition on the real-time camera capture image to obtain facial feature data of the operator.

[0036] Step S104, determining the operator's identity information based on the operator's facial feature data.

[0037] Step S105, based on the preset database and the current operation information, the operation authority of the operator identity information is judged to obtain the authority judgment result.

[0038] Step S106, in response to the authority determination result and the current operation information satisfying a preset non-safety condition, executing a warning operation.

[0039] In the steps S101 to S106 shown in the embodiment of the present application, the real-time front-end operation screen and the real-time camera acquisition screen transmitted from the nuclear power engineering terminal are first obtained. Then, the front-end operation recognition is performed on the real-time front-end operation screen to obtain the current operation information corresponding to the nuclear power engineering terminal. Then, the facial feature data of the operator is obtained by performing facial feature recognition on the real-time camera acquisition screen. Then, the identity information of the operator is determined according to the facial feature data of the operator. Subsequently, based on the preset database and the current operation information, the operation authority is judged on the identity information of the operator to obtain the authority judgment result. Finally, in response to the authority judgment result and the current operation information satisfying the preset non-safety condition, the early warning operation is performed. Through these steps, the embodiment of the present application can realize the comprehensive monitoring and management of the nuclear power engineering operation process, ensuring that only operators with appropriate authority can perform the corresponding operation. At the same time, when an operation that does not meet the safety conditions is detected, an early warning can be issued in time to prevent the occurrence of potential risks, thereby improving the safety and reliability of the operation of the nuclear power engineer station.

[0040] In step S101 of some embodiments, in the application scenario of nuclear power plant control, the nuclear power prevention and control server is used to execute the abnormal behavior warning method provided by the embodiment of the present application, and can be physically stored in the cabinet of the nuclear power plant. The nuclear power engineering terminal is an engineer station, that is, an operating terminal for operators to perform operations, which can be a computer device. The operator can perform variable debugging, configuration modification, screen modification and other operations on the nuclear power engineering terminal. The real-time front-end operation screen is the screen for implementing operations on the nuclear power engineering terminal, which can be a real-time screen on the interactive interface of the nuclear power engineering terminal. The real-time camera acquisition screen can be a real-time screen around the nuclear power engineering terminal, or a real-time screen in front of the display screen of the nuclear power engineering terminal, which can be acquired by a camera.

[0041] It should be noted that the abnormal behavior warning method provided in the embodiment of the present application needs to be implemented based on an abnormal behavior warning system in hardware. Therefore, the abnormal behavior warning system provided in the embodiment of the present application is first introduced. Figure 2 , Figure 2It is a structural schematic diagram of the abnormal behavior warning system provided by the embodiment of the present application. The abnormal behavior warning system provided by the embodiment of the present application includes a nuclear power engineering terminal, a camera device and a nuclear power prevention and control server. The nuclear power prevention and control server is connected to the nuclear power engineering terminal and the camera device respectively. The camera device is used to collect real-time images around the interactive interface of the nuclear power engineering terminal, that is, the real-time camera acquisition picture in the above embodiment. Specifically, in some actual application scenarios in the field of nuclear power, the nuclear power prevention and control server will be placed in a cabinet, and the nuclear power engineering terminal will be connected to the cabinet through a screen splitter, a capture card and a USB hub respectively. The screen splitter distributes the signal output by the nuclear power engineering terminal connected thereto to other displays. Due to the information security requirements of the nuclear power plant, the nuclear power engineering terminal does not have the authority to install third-party screen recording software, and uses a capture card to collect real-time front-end operation pictures. The camera device collects real-time pictures of the area in front of the display of the nuclear power engineering terminal, transmits these pictures to the nuclear power prevention and control server, and obtains real-time camera acquisition pictures. It should be noted that this method will perform regional segmentation on the real-time video acquisition screen, determine part of the screen area in the screen as the operation area, and determine part of the screen area as the monitoring area, and determine the personnel in the operation area as the executor who performs the current operation, and the personnel in the monitoring area as the monitoring personnel. The specific process of determining the identity type of the operator will be described in detail in the following embodiments of steps S601 to S603. It can be understood that multiple nuclear power engineering terminals are connected to the cabinet through a USB hub so that the cabinet can simultaneously receive real-time front-end operation pictures and real-time video acquisition pictures transmitted by multiple nuclear power engineering terminals. In some embodiments, the abnormal behavior warning system will also include a temporary work instruction terminal, which is connected to the cabinet through a work intranet and is used to apply for and cancel information entry of temporary work instructions.

[0042] See also Figure 3 In some embodiments, before step S102, the abnormal behavior warning method provided in the embodiment of the present application further includes steps S301 to S302: Step S301, extracting video frames from the real-time front-end operation screen to obtain a plurality of continuous operation video frames.

[0043] Step S302, filter out operation window data from the operation video frame, and replace the real-time front-end operation screen with the operation window data.

[0044] In step S301 of some embodiments, the operation video frame is obtained by intercepting the real-time front-end operation screen at a preset frequency, wherein the frequency of screen interception may be 5 Hz, which is not strictly limited in the embodiments of the present application.

[0045] In step S302 of some embodiments, the operation window data is the image data of the operation window displayed in the operation video frame. The operation window data can be obtained by automatically detecting and identifying the operation window from the video frame through a computer vision algorithm (including CNN, YOLO or FasterR-CNN, etc.), or using a template matching technology (such as a template matching algorithm of an image processing library such as OpenCV), or matching a known operation window template with the operation video frame to identify an area similar to the template, or performing specific window recognition through a region of interest (ROI) setting, but is not limited thereto. After obtaining the operation window data, the operation window data is used as a processing object for subsequent recognition operations.

[0046] It is understandable that in the subsequent process of determining the current operation, not all operation video frames need to be processed, and it can be determined based on the pop-up window information in the current interface. Steps S301 to S302 shown in the embodiment of the present application avoid unnecessary processing by filtering out the operation window data in the operation video frame, thereby saving the computing power resources of the server.

[0047] See also Figure 4 In some embodiments, step S102 may include but is not limited to steps S401 to S404: Step S401, identifying the operation type of the real-time front-end operation screen according to a preset operation identification model to obtain current operation type information.

[0048] Step S402: performing operation action recognition on the real-time front-end operation screen according to the operation recognition model to obtain current action information.

[0049] Step S403: query the preset operation mapping table for monitoring requirements according to the current operation type information to obtain monitoring requirement information.

[0050] Step S404, determining current operation information according to current action information, current operation type information and monitoring requirement information.

[0051] In step S401 of some embodiments, the preset operation recognition model may be a pre-trained deep learning network. The current operation type information is the type information corresponding to the current operation. It should be noted that in the application scenario of the nuclear power engineer station, the operations performed at the nuclear power engineering terminal can be classified, and the current operation type information may be: version control software (Visual Source Safe, VSS) use operation, engineering master control software use operation, signal force operation, configuration modification operation, screen modification operation, alarm card / procedure modification operation and prohibited operation. It can be understood that the type of the current operation can be determined by only determining the static elements of the current operation window, wherein the static elements include but are not limited to text, symbols, numbers, check boxes and menu bars. Therefore, in addition to determining the operation type by identifying features through a deep learning network, the static elements corresponding to the current window can also be obtained through optical character recognition technology (Optical Character Recognition, OCR), and these static elements are compared with the preset typical operation semantic library in real time to finally obtain the current operation type information.

[0052] In step S402 of some embodiments, the current action information is the action information triggered by the input device when performing the current operation. Specifically, the current action information can be the path of mouse movement, the order of mouse clicks, or the keyboard input of values ​​for a specific window or the use of shortcut keys to call up a menu. Taking the acquisition of mouse actions as the current action information as an example, the current action information can be determined in combination with different areas of the current operation window and the mouse click action. Specifically, it is detected that the mouse clicks the "OK" key of the current window, clicks the "Force" key in the window of the next frame, and then clicks the "Write" key in the window of the next frame, and so on, to obtain a continuous mouse click action and finally determine the current action information.

[0053] In step S403 of some embodiments, the operation mapping table includes a mapping relationship between the current operation type information and the monitoring requirement information. The monitoring requirement information is used to reflect whether the current operation needs to be monitored, and the monitoring requirement information may include "needs monitoring" and "does not need monitoring". Exemplarily, if the current operation type information is an operation using the engineering master control software, the monitoring requirement information corresponding to such operation is queried in the operation mapping table as not requiring monitoring. If the current operation type information is a configuration modification operation, the corresponding monitoring requirement information can be determined as requiring monitoring according to the operation mapping table.

[0054] In step S404 of some embodiments, for the same type of operation, the corresponding current action information, current operation type information and monitoring requirement information are integrated to obtain current operation information.

[0055] Steps S401 to S404 shown in the embodiment of the present application automatically identify the operation type and operation action of the real-time front-end operation screen, and perform monitoring demand query in combination with the operation type information and the preset operation mapping table, thereby realizing automated and intelligent monitoring demand analysis, and further realizing efficient processing of the real-time front-end operation screen and accurate acquisition of the current operation.

[0056] In step S103 of some embodiments, please refer to Figure 5 In some embodiments, step S103 may include but is not limited to steps S501 to S505: Step S501, obtaining an initial facial region image of the operator in the real-time camera capture image.

[0057] Step S502: extract facial feature points from the facial area to obtain the coordinates of the facial feature points of the operator.

[0058] Step S503, calculating an affine transformation matrix between the facial feature point coordinates and the preset reference facial feature point coordinates.

[0059] Step S504, aligning feature points according to the affine transformation matrix to obtain a middle facial region image.

[0060] Step S505, extracting facial features from the middle facial region image to obtain facial feature data of the operator.

[0061] In step S501 of some embodiments, the real-time video capture screen can be preprocessed to decompose the real-time video capture screen into multiple continuous video frames, and face detection and recognition are performed on each video frame. This step usually uses a face detection algorithm or model to identify the face position, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) and R-CNN (Region-based Convolutional Neural Networks). The algorithm locates the bounding box of the face in each video frame and extracts the facial region image of the operator, that is, the initial facial region image.

[0062] In an embodiment of the present application, the operator may include an executive who performs operations at a nuclear power engineering terminal, and may also include a supervisor who monitors the current operation, and there will be a pair of supervisors and executives in front of each nuclear power engineering terminal. The facial features of the operator include the facial features of the executive and the facial features of the supervisor. In some embodiments, after recognizing that the current operation screen displays a window with a specific meaning, the camera can be called for acquisition to obtain the facial features of the operator.

[0063] It should be noted that, in some embodiments, it is not necessary to obtain the facial features of the supervisor and the operator at the same time throughout the entire process, and the facial features of the operator that need to be obtained are determined from the supervisory requirement information in the current operation information. Specifically, when the supervisory requirement information indicates that the current operation does not need to be supervised, only the facial features of the executor are obtained. When the supervisory requirement information indicates that the current operation needs to be supervised, the facial features of the executor and the supervisor need to be obtained at the same time. This scenario will be described in detail in the following step S105 related embodiments.

[0064] In step S502 of some embodiments, facial feature points include the positions of eyes, nose, mouth, chin, etc. In order to obtain the precise positions of these feature points, facial feature point detection algorithms such as Dlib or MediaPipe are usually used. These algorithms can mark up to 68 or more key points on the facial area image and accurately describe the geometric structure of the face. Facial feature point coordinates refer to the coordinate data corresponding to each feature point, reflecting the position of each feature point in the picture.

[0065] In step S503 of some embodiments, the preset reference facial feature points may be a predefined standard facial model, which specifies the ideal position of each feature point, and is usually set according to the frontal facial features of the face. Alternatively, a reference frame may be selected from the above continuous video frames, and the facial feature points in the reference frame may be used as reference facial feature points. The reference facial feature point coordinates are the coordinate data of the reference facial feature points. To calculate the affine transformation matrix between the facial feature point coordinates and the preset reference facial feature point coordinates, it is necessary to select three pairs of corresponding feature points from the facial feature point coordinates obtained through the real-time acquisition screen and the reference facial feature point coordinates, thereby calculating the transformation matrix of the facial feature points converted to the reference facial feature points.

[0066] In step S504 of some embodiments, the initial facial region image is geometrically transformed according to the affine transformation matrix calculated in step S503, and the facial feature points are aligned to the coordinates of the reference facial feature points, thereby generating an intermediate facial region image, whose feature points are aligned with the preset reference facial model. This process can ensure that the operator's facial posture can be standardized to a unified reference system no matter how it changes.

[0067] In step S505 of some embodiments, after achieving face alignment, facial feature extraction can be performed using feature extraction methods such as local binary patterns or convolutional neural network deep learning models to obtain more accurate facial feature data. The operator's facial feature data can be a high-dimensional vector that describes the geometry, texture, color, and other information of the operator's face.

[0068] Steps S501 to S505 shown in the embodiment of the present application implement the process of accurately acquiring and extracting the facial feature data of the operator from the real-time camera image. First, the system can effectively identify and extract the facial image of the operator from the video frame. Then, through facial feature point detection, the key points of the face are accurately located to ensure that the facial feature points can be aligned with the preset reference model. Then, the operator's face is standardized through affine transformation to eliminate the influence of facial posture changes. Finally, the system extracts features from the aligned facial image to generate high-dimensional facial feature data, which provides a basis for subsequent identity recognition and verification. This embodiment improves the accuracy of facial recognition.

[0069] In step S104 of some embodiments, please refer to Figure 6 In some embodiments, step S104 may include but is not limited to steps S601 to S603: Step S601, performing similarity calculation between the operator's facial feature data and preset facial information in a preset database to obtain similarity data.

[0070] Step S602, screening the preset facial information according to the similarity data to obtain the target facial information.

[0071] Step S603, determining the operator identity information according to the target facial information.

[0072] In step S601 of some embodiments, the preset database includes operator identity information, wherein the operator identity information includes preset facial information, department, employee number and name, and the database primary key may be the employee number. In some embodiments, the Euclidean distance or cosine similarity between the preset facial information and the operator's facial feature data may be calculated, and the final calculation result, namely the similarity data, may be in the form of a percentage, with a value range of 0 to 100%.

[0073] In step S602 of some embodiments, a threshold value (such as 60%) may be preset. If the similarity data between the facial feature data of the operator and a preset facial information is greater than the threshold value, the preset facial information is determined to be the target facial information. If multiple preset facial information are greater than the threshold value, the preset facial information corresponding to the maximum similarity data is selected as the target facial information.

[0074] In step S603 of some embodiments, the target facial information is queried in a preset database to obtain operator identity information corresponding to the target facial information.

[0075] Steps S601 to S603 shown in the embodiment of the present application achieve accurate identification of the operator's identity through similarity calculation and screening processes, improve the efficiency and accuracy of identity identification, and ensure that the system can reliably verify and identify the operator's identity in various application scenarios, thereby enhancing the security of the system and the effectiveness of management.

[0076] See also Figure 7 In some embodiments, step S105 includes but is not limited to steps S701 to S707: Step S701, when the monitoring requirement information indicates that the current operation needs to be monitored, the first identity information of the first target object and the second identity information of the second target object are obtained. The operator identity information includes the first identity information and the second identity information, the first target object is the operator performing the current operation at the nuclear power engineering terminal, and the second target object is the operator monitoring the current operation.

[0077] Step S702: extract first authority information from the first identity information.

[0078] Step S703: extract second authority information from the second identity information.

[0079] Step S704: Performing an operation permission judgment on the first permission information and the second permission information based on a preset database to obtain a permission judgment result.

[0080] In step S701 of some embodiments, it has been explained in the above embodiments that the operators may include executors who perform operations at the nuclear power engineering terminal and supervisors who monitor the current operations. The first target object is the executor, and the first identity information is the operator identity information of the executor. The second target object is the supervisor, and the second identity information is the operator identity information of the supervisor.

[0081] See also Figure 8 In some embodiments, step S701 may include but is not limited to steps S801 to S803: Step S801, obtaining target pixel coordinate data of the operator in the real-time camera capture image.

[0082] Step S802, determining the identity type of the operator based on the first preset pixel coordinate data of the real-time camera acquisition screen, the second preset pixel coordinate data of the real-time camera acquisition screen and the target pixel coordinates, the identity type being used to characterize whether the operator is the first target object or the second target object.

[0083] Step S803: determining, according to the identity type, whether the operator identity information is the first identity information or the second identity information.

[0084] In step S801 of some embodiments, a target detection algorithm may be used to identify the operator, which locates the operator's bounding box in each video frame of the real-time camera capture image, and the pixel coordinate set corresponding to the pixels constituting the bounding box is the target pixel coordinate data.

[0085] In step S802 of some embodiments, the control Figure 2 , the shooting screen of the real-time video acquisition screen is divided into regions to obtain first preset pixel coordinate data and second preset pixel coordinate data, the first preset pixel coordinate data is the pixel coordinates representing the operation area in the real-time video acquisition screen, and the second preset pixel coordinate data is the pixel coordinates representing the monitoring area in the real-time video acquisition screen. If the target pixel coordinates fall into the first preset pixel coordinate data, it indicates that the operator is located in the operation area, and the corresponding identity type is the executive. If the target pixel coordinates fall into the second preset pixel coordinate data, it indicates that the operator is located in the monitoring area, and the corresponding identity type is the monitoring personnel.

[0086] In step S803 of some embodiments, if the identity type is an executor, the obtained operator identity information is the first identity information; if the identity type is a guardian, the obtained operator identity information is the second identity information.

[0087] In addition to detecting the operator's position, the operator's type can also be determined by sitting or standing posture and clothing color, and facial features that do not meet the sitting or standing characteristics and specific clothing colors will be eliminated. For example, if the operator is detected sitting, the operator is determined to be an executive, and if the operator is detected standing, the operator is determined to be a guardian.

[0088] Steps S801 to S803 shown in the embodiment of the present application can accurately identify the identity type of the operator through real-time detection. First, the system obtains the location data of the operator from the camera image, and then determines whether the operator is an executor or a guardian based on the area or posture of the operator.

[0089] In step S702 and step S703 of some embodiments, the first permission information and the second permission information may both be Assistant Test Supervisor (ATS) and Test Supervisor (TS).

[0090] In step S704 of some embodiments, the preset database includes a mapping relationship between the combination of the first permission information and the second permission information and the permission judgment result. In some embodiments, the level of the second permission information is required to be higher than the level of the first permission information, and the first identity information is required to be a control engineer or a system administrator. When the first permission information and the second permission information meet the above conditions, the permission judgment result is characterized as normal permission, and when the first permission information and the second permission information do not meet the above conditions, the permission judgment result is characterized as abnormal permission.

[0091] Steps S701 to S704 shown in the embodiment of the present application improve the safety and reliability of nuclear power project operations and avoid the risk of unauthorized operation and monitoring. This multi-level authority verification mechanism can effectively prevent misoperation and unauthorized operation, thereby maintaining the safe operation of nuclear power projects.

[0092] See also Fig. 9 In some application scenarios where work is performed at an engineer station, before step S106, it is necessary to track the temporary work order. Specifically, the abnormal behavior warning method provided in the embodiment of the present application may also include but is not limited to steps S901 to S905: Step S901, obtaining the identification code value and application status of the work order.

[0093] Step S902, determining the operation duration according to the identification code value and the application status.

[0094] Step S903: Generate a candidate operation log according to the operator identity information, current operation information and authority determination result.

[0095] Step S904: Screen the candidate operation logs according to the execution time and operation duration in the current operation information to obtain the target operation log.

[0096] Step S905: Update the work order according to the target operation log and the identification code value.

[0097] In step S901 of some embodiments, in the application scenario of performing work at the engineer station, the work order is a temporary work order, the identification code value may be the order number of the temporary work order, and the application status may include the application execution status and the application cancellation status. When the content of the work order has gone through the review process and passed the approval, an identification code corresponding to the work order will be generated, wherein the identification code value and the application status can be obtained by scanning the identification code with a code scanning device, or can be obtained by the operator inputting it through the temporary work order terminal.

[0098] See also Fig.10In some embodiments, step S902 may also include but is not limited to steps S1001 to S1004: Step S1001, if the application status is the application execution status, perform a data query on the identification code value to obtain the order number query result.

[0099] Step S1002: If the identification code value represented by the order number query result is obtained for the first time, then start timing to obtain the initial running time.

[0100] Step S1003, if the application status is the application cancellation status, the cancellation item information of the work order is obtained, and the end timing is determined according to the application cancellation status and the cancellation item information to obtain the target running time.

[0101] Step S1004: The duration between the initial running time and the target running time is used as the job duration.

[0102] In step S1001 of some embodiments, the order number query result is used to indicate whether there is a record for the same temporary work order in the system. The order number query result may be that the same order number is found or not.

[0103] In step S1002 of some embodiments, if the query result of the order number is that the same order number is not found, it can be determined that the identification code value is obtained for the first time, and the temporary work order corresponding to the identification code value is a newly applied work order, and the time when the timing starts is the initial operation time of the temporary work order. If the query result of the order number is that the same order number is found, it means that the identification code value has been obtained before, and the temporary work order corresponding to the identification code value has been timed before. In order to avoid repeated timing, after determining that the identification code value is not obtained for the first time, no timing operation is performed.

[0104] In step S1003 of some embodiments, in the field of nuclear power safety, a cancellation application form is provided for the work instruction form. In the cancellation application form, it is necessary to check the type conversion operation of the existing temporary work instruction. The information about these operations is the cancellation matter information. The instruction type is introduced in detail in step S1201, and will not be repeated here. Exemplarily, the cancellation matter information includes: 1. Cancel all temporary work instructions; 2. Convert all temporary work instructions into temporary control change instructions; 3. Cancel some temporary work instructions and convert the remaining temporary work instructions into temporary control change instructions. When the cancellation matter information is the first case, after determining that all the work contents associated with all temporary work instructions are cancelled, the work instruction form is set to end, and the end time is used as the target running time. When the cancellation matter information is the second case, the work instruction form is directly determined to end, and the end time is used as the target running time. When the cancellation matter information is the third case, the associated work content is determined according to the temporary work instruction confirmed to be cancelled, and these work contents are tracked. After confirming the cancellation, the work instruction form is determined to end, and the end time is used as the target running time.

[0105] In step S1004 of some embodiments, corresponding to the same temporary work order, the duration between the initial running time and the target running time is used as the operation duration of the temporary work order.

[0106] Steps S1001 to S1004 shown in the embodiment of the present application achieve improved accuracy and efficiency of job management through accurate timing of each temporary work order.

[0107] In step S903 of some embodiments, in the actual application scenario of the embodiments of the present application, a corresponding operation log file, i.e., a candidate operation log, is generated for each operation of the executor. Exemplarily, the specific content of the candidate operation log may be: the name of the executor, the execution time of the current operation, the current operation action, and the corresponding operation video frame. Or: the name of the executor, the name of the guardian, the execution time of the current operation, the current operation action, and the corresponding operation video frame.

[0108] In step S904 of some embodiments, when the execution time is between the initial running time and the target running time of the job duration, the operation log corresponding to the execution time is the target operation log.

[0109] In step S905 of some embodiments, for the same identification code value, the information required in the target operation log (such as operation type information, operator identity information, specific data before and after the target variable is forced, execution time, etc.) is automatically filled in the temporary work instruction table. When the corresponding engineer station needs to execute other temporary work instructions, it automatically switches to the corresponding temporary work instruction sheet according to the new identification code for recording.

[0110] Steps S901 to S905 shown in the embodiment of the present application filter out the target operation log by operation duration, and update the content of the temporary work instruction sheet in real time, thereby realizing the electronicization of the temporary work instruction sheet and ensuring the accuracy of the content of the temporary work instruction sheet.

[0111] See also Fig.11 In some embodiments, step S106 may include but is not limited to steps S1101 to S1102: Step S1101, querying the preset action mapping table according to the current action information to obtain the action query result.

[0112] Step S1102, in response to the permission judgment result and the action query result satisfying the preset non-safety condition, performing a warning operation.

[0113] In step S1101 of some embodiments, the action mapping table includes action information of correct operation. If the action information consistent with the current action information is obtained by querying the mapping table, the action query result can be characterized as the current action is a safe operation. If the action information consistent with the current action information is not found in the mapping table, the action query result can be characterized as the current action is an abnormal operation.

[0114] In step S1102 of some embodiments, the preset non-safety condition includes: the permission judgment result is characterized as a permission abnormality, and the action query result is characterized as the current action is an abnormal operation.

[0115] It is understandable that the warning operation may be to broadcast a warning voice through a voice broadcast device, or to display the warning information on the interactive interface of the nuclear power engineering terminal.

[0116] Steps S1101 to S1102 shown in the embodiment of the present application can issue a warning in time when an abnormal authority or abnormal operation is detected, thereby preventing the occurrence of potential risks and ensuring the safe operation of the nuclear power project.

[0117] See also Fig.12 In some embodiments, after step S106, the abnormal behavior warning method may also include but is not limited to steps S1201 to S1202: Step S1201, obtaining the instruction type of the work instruction sheet, and determining the preset duration according to the instruction type.

[0118] Step S1202, in response to the operation duration satisfying the preset non-safety condition, executing a warning operation.

[0119] In step S1201 of some embodiments, for non-safety-level DCS change instructions, instruction types include: temporary work instructions, temporary control change instructions, design change instructions, and characterization parameter instructions. If it is a temporary work instruction, the instruction is required to be completed within 24 hours. If it is a temporary control change instruction, it is required to be completed within 180 days. If it is a design change instruction and a characterization parameter instruction, it needs to be permanently retained without time limit. The time limit corresponding to the instruction type is determined as a preset duration. In the embodiment of the present application, the instruction types corresponding to the temporary work instruction sheet are all temporary work instructions, so the corresponding preset duration is 24 hours.

[0120] In step S1202 of some embodiments, the preset non-safety condition also includes that the operation duration is greater than the preset duration. When the operation duration is greater than the preset duration, it is determined that the operation duration meets the preset non-safety condition, and an early warning operation is performed. The specific content of the early warning operation has been introduced in the above embodiments and will not be repeated here.

[0121] Steps S1201 to S1202 shown in the embodiment of the present application can issue an early warning in a timely manner when it is detected that the operation time of the temporary work order is too long, thereby preventing the occurrence of potential risks and ultimately ensuring the safe operation of the nuclear power project.

[0122] See also Fig.13 The embodiment of the present application also provides an abnormal behavior warning device, which can implement the above abnormal behavior warning method, and the device includes: The acquisition module is used to obtain the real-time front-end operation screen and real-time camera acquisition screen transmitted from the nuclear power engineering terminal.

[0123] The operation identification module is used to identify the front-end operation of the real-time front-end operation screen and obtain the current operation information corresponding to the nuclear power engineering terminal.

[0124] The facial feature recognition module is used to perform facial feature recognition on the real-time camera collection screen to obtain the facial feature data of the operator.

[0125] The identity information determination module is used to determine the identity information of the operator based on the facial feature data of the operator.

[0126] The authority judgment module is used to judge the operation authority of the operator's identity information based on the preset database and current operation information to obtain the authority judgment result.

[0127] The early warning operation execution module is used to execute the early warning operation in response to the authority judgment result and the current operation information satisfying the preset non-safe condition.

[0128] The specific implementation of the abnormal behavior warning device is basically the same as the specific implementation of the abnormal behavior warning method described above, and will not be repeated here.

[0129] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above abnormal behavior warning method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0130] See also Fig.14 , Fig.14 The hardware structure of an electronic device of another embodiment is illustrated. In the embodiment of the present application, the electronic device is a part of the nuclear power prevention and control server of the present embodiment. The electronic device includes: The processor 1401 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 1402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1402 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1402, and the processor 1401 calls and executes the abnormal behavior warning method of the embodiment of this application; Input / output interface 1403, used to implement information input and output; The communication interface 1404 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.); A bus 1405 that transmits information between various components of the device (e.g., the processor 1401, the memory 1402, the input / output interface 1403, and the communication interface 1404); The processor 1401 , the memory 1402 , the input / output interface 1403 and the communication interface 1404 are connected to each other in communication within the device via the bus 1405 .

[0131] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned abnormal behavior warning method is implemented.

[0132] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The abnormal behavior warning method, abnormal behavior warning device, electronic device, abnormal behavior warning system and storage medium provided in the embodiments of the present application can realize comprehensive monitoring and management of the nuclear power engineering operation process, ensuring that only operators with appropriate authority can perform corresponding operations. At the same time, when an operation that does not meet safety conditions is detected or the operation time exceeds the limit, an early warning can be issued in time to prevent the occurrence of potential risks, thereby improving the safety and reliability of the operation of the nuclear power engineer station.

[0134] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0135] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0136] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0138] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0139] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0140] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0144] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An abnormal behavior early warning method, characterized in that: Applied to nuclear power prevention and control servers, including: Obtain real-time front-end operation images and real-time camera acquisition images transmitted from nuclear power engineering terminals; Performing front-end operation identification on the real-time front-end operation picture to obtain current operation information corresponding to the nuclear power engineering terminal; Performing facial feature recognition on the real-time camera capture image to obtain facial feature data of the operator; Determining the identity information of the operator according to the facial feature data of the operator; Based on the preset database and the current operation information, the operator identity information is subjected to operation authority judgment to obtain an authority judgment result; In response to the authority determination result and the current operation information satisfying a preset non-safety condition, a warning operation is performed.

2. The abnormal behavior early warning method according to claim 1, characterized in that: The step of performing facial feature recognition on the real-time camera capture image to obtain facial feature data of the operator includes: Acquire an initial facial region image of the operator in the real-time camera capture picture; Extracting facial feature points from the facial area to obtain the facial feature point coordinates of the operator; Calculating an affine transformation matrix between the facial feature point coordinates and preset reference facial feature point coordinates; Aligning feature points according to the affine transformation matrix to obtain a middle facial area image; Facial features are extracted from the middle facial area image to obtain facial feature data of the operator.

3. The abnormal behavior early warning method according to claim 1, characterized in that: The determining of the operator identity information according to the operator facial feature data includes: Calculating similarity between the operator's facial feature data and the preset facial information in the preset database to obtain similarity data; Filtering the preset facial information according to the similarity data to obtain target facial information; The operator identity information is determined according to the target facial information.

4. The abnormal behavior early warning method according to claim 1, characterized in that: The performing front-end operation identification on the real-time front-end operation picture to obtain current operation information corresponding to the nuclear power engineering terminal includes: Identify the operation type of the real-time front-end operation screen according to a preset operation identification model to obtain current operation type information; Performing operation action recognition on the real-time front-end operation screen according to the operation recognition model to obtain current action information; Performing a monitoring requirement query on a preset operation mapping table according to the current operation type information to obtain monitoring requirement information; The current operation information is determined according to the current action information, the current operation type information and the monitoring requirement information.

5. The abnormal behavior early warning method according to claim 4, characterized in that: Before identifying the operation type of the real-time front-end operation screen according to the preset operation identification model to obtain the current operation type information, the abnormal behavior early warning method further includes: Extracting video frames from the real-time front-end operation screen to obtain multiple continuous operation video frames; The operation window data is filtered out from the operation video frame, and the real-time front-end operation screen is replaced by the operation window data.

6. The abnormal behavior early warning method according to claim 4, characterized in that: The operation authority judgment is performed on the operator identity information based on the preset database and the current operation information to obtain the authority judgment result, including: When the monitoring requirement information indicates that the current operation needs to be monitored, first identity information of a first target object and second identity information of a second target object are obtained; wherein the operator identity information includes the first identity information and the second identity information, the first target object is the operator performing the current operation at the nuclear power engineering terminal, and the second target object is the operator monitoring the current operation; Extracting first authority information from the first identity information; extracting second authority information from the second identity information; An operation permission judgment is performed on the first permission information and the second permission information based on the preset database to obtain the permission judgment result.

7. The abnormal behavior early warning method according to claim 6, characterized in that: The obtaining of the first identity information of the first target object and the second identity information of the second target object includes: Acquire target pixel coordinate data of the operator in the real-time camera acquisition picture; Determine the identity type of the operator based on the first preset pixel coordinate data of the real-time camera acquisition picture, the second preset pixel coordinate data of the real-time camera acquisition picture, and the target pixel coordinates, wherein the identity type is used to characterize that the operator is the first target object or the second target object; The operator identity information is determined to be the first identity information or the second identity information according to the identity type.

8. The abnormal behavior early warning method according to claim 4, characterized in that: In response to the authority determination result and the current operation information satisfying a preset non-safety condition, executing an early warning operation includes: Performing data query on a preset action mapping table according to the current action information to obtain an action query result; In response to the permission judgment result and the action query result satisfying a preset non-safety condition, a warning operation is performed, wherein the preset non-safety condition includes: the permission judgment result is characterized as a permission abnormality, and the action query result is characterized as a current action being an abnormal operation.

9. The abnormal behavior early warning method according to any one of claims 1 to 8, characterized in that: Before executing the warning operation in response to the authority determination result and the current operation information satisfying the preset non-safety condition, the abnormal behavior warning method further includes: Get the identification code value and application status of the work order; Determine the duration of the operation according to the identification code value and the application status; Generate a candidate operation log according to the operator identity information, the current operation information and the authority judgment result; According to the execution time and the operation duration in the current operation information, the candidate operation logs are screened to obtain a target operation log; The work order is updated according to the target operation log and the identification code value.

10. The abnormal behavior early warning method according to claim 9, characterized in that: The determining of the operation duration according to the identification code value and the application status includes: If the application status is the application execution status, a data query is performed on the identification code value to obtain a number query result; If the order number query result indicates that the identification code value is obtained for the first time, then start timing to obtain the initial running time; If the application status is an application cancellation status, the cancellation item information of the work order is obtained, and the end timing is determined according to the application cancellation status and the cancellation item information to obtain the target running time; The duration between the initial running time and the target running time is taken as the job duration.

11. The abnormal behavior early warning method according to claim 10, characterized in that: After the warning operation is performed in response to the authority judgment result and the current operation information satisfying the preset non-safety condition, the abnormal behavior warning method further includes: Obtaining the instruction type of the work instruction sheet, and determining a preset duration according to the instruction type; In response to the operation duration satisfying the preset non-safety condition, the early warning operation is performed; wherein the preset non-safety condition also includes that the operation duration is greater than the preset duration.

12. An abnormal behavior warning device, characterized in that: include: An acquisition module, which is used to acquire real-time front-end operation images and real-time camera acquisition images transmitted from a nuclear power engineering terminal; An operation identification module, the operation identification module is used to perform front-end operation identification on the real-time front-end operation screen to obtain current operation information corresponding to the nuclear power engineering terminal; A facial feature recognition module, which is used to perform facial feature recognition on the real-time camera collection picture to obtain facial feature data of the operator; An identity information determination module, the identity information determination module is used to determine the identity information of the operator according to the facial feature data of the operator; An authority judgment module, the authority judgment module is used to judge the operation authority of the operator identity information based on a preset database and the current operation information to obtain an authority judgment result; The early warning operation execution module is used to execute an early warning operation in response to the authority judgment result and the current operation information satisfying a preset non-safe condition.

13. An electronic device, characterized in that The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the abnormal behavior warning method according to any one of claims 1 to 11 when executing the computer program.

14. Abnormal behavior early warning system, characterized in that: The system includes a nuclear power engineering terminal, a camera device and a nuclear power control server, and the nuclear power control server includes the electronic device as described in claim 13; the nuclear power control server is connected to the nuclear power engineering terminal and the camera device respectively, and the camera device is used to collect real-time camera collection images.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the abnormal behavior warning method according to any one of claims 1 to 11 is implemented.

Images