Off-line non-sign-off detection method and related device

Abstract

The invention discloses an off-line non-sign-off detection method and a related device, which can be applied to the field of artificial intelligence or finance, and can be used for identifying whether a person exists in a monitoring image under each monitoring view angle by calling a target detection model after obtaining the monitoring images under a plurality of monitoring view angles in an office area. According to the method, on-duty detection of workers under multiple view angles is realized, a monitoring view angle corresponding to a monitoring image in which people are identified is determined as a target monitoring view angle, a target tracking model which is lighter than a target detection model is adopted, and target tracking is performed on target personnel in the monitoring image under the target monitoring view angle. Therefore, whether the staff is off-line is determined. And if the worker is off-line, the screen state target detection model is called to identify whether the screen state in the monitoring image under the screen monitoring visual angle is system non-sign-off, so that automatic off-line non-sign-off detection is realized, the off-line non-sign-off detection efficiency and accuracy are improved, and the missing report rate of an off-line non-sign-off behavior is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for detecting leaving a machine without signing out and a related device. Background Art

[0002] The work computers of banking staff involve the bank's core systems and a large amount of sensitive customer information. When the staff leave the computer but fail to sign out of the work computer system, information leakage may easily occur, leading to production safety accidents.

[0003] At present, the main method of checking whether personnel leave the aircraft without signing out is manual inspection, random inspection of monitoring images, etc., which is inefficient, slow to respond, and prone to underreporting. Summary of the invention

[0004] In view of the above problems, the present application provides a method and related device for detecting the behavior of leaving the machine without signing out, which improves the efficiency and accuracy of the detection of the behavior of leaving the machine without signing out, and reduces the underreporting rate of the behavior of leaving the machine without signing out. The specific scheme is as follows:

[0005] A first aspect of the present application provides a method for detecting a non-sign-out condition when leaving the machine, comprising:

[0006] Acquire surveillance images from multiple surveillance perspectives in an office area, wherein the multiple surveillance perspectives include a screen surveillance perspective;

[0007] Calling the target detection model to respectively identify whether there is a person in the surveillance image under each of the surveillance perspectives;

[0008] Determine the monitoring angle corresponding to the monitoring image in which a person is identified as a target monitoring angle;

[0009] Tracking the target person in the monitoring image under the target monitoring perspective;

[0010] If the duration for which the target person is not detected in the monitoring image under the target monitoring perspective reaches a first preset value, the screen status target detection model is called to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not signed out.

[0011] In a possible implementation, after determining the surveillance angle corresponding to the surveillance image in which a person is identified as the target surveillance angle, the off-board non-sign-out detection method further includes:

[0012] Stop calling the target detection model to respectively identify whether there is a person in the surveillance image under each of the surveillance perspectives;

[0013] When it is determined that the target person has left the computer and the system has been signed out, or when it is determined that the target person has left the computer and the system has not been signed out, the process returns to the step of acquiring surveillance images from multiple surveillance perspectives in the office area.

[0014] In a possible implementation, if the duration for which no person is detected in the surveillance image under each of the surveillance perspectives reaches a first preset value, the step of calling the screen status target detection model to identify whether the screen status in the surveillance image under the screen monitoring perspective is that the system has not signed out is executed.

[0015] In a possible implementation, the training method of the target detection model includes:

[0016] Acquire multiple frames of historical monitoring images under each monitoring perspective, wherein the historical monitoring images include a first image and a second image, wherein the first image contains a person and the second image contains no person;

[0017] Partially cutting out the person in the first image to obtain a cutout image;

[0018] Performing data enhancement processing on the cut image, and randomly pasting it into the scene image to obtain a third image, wherein the first image, the second image and the third image constitute a training sample set;

[0019] Performing category labeling and identification box labeling on the training sample set;

[0020] The target detection model is trained using the training sample set.

[0021] In a possible implementation, after calling the screen status target detection model to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not been signed out, the off-device non-sign-out detection method further includes:

[0022] When it is determined that the target person has left the aircraft and the system has not signed out, an early warning is triggered;

[0023] Wherein, if the screen status in the monitoring image under the screen monitoring angle is that the system has not been signed out, it is determined that the target person has left the machine and the system has not been signed out;

[0024] or

[0025] If the number of times that the screen status in the monitoring image under the screen monitoring angle of view indicates that the system has not been signed out reaches a second preset value within the preset time, it is determined that the target person has left the machine and has not signed out of the system.

[0026] A second aspect of the present application provides a device for detecting a non-sign-out condition after leaving the machine, comprising:

[0027] An image acquisition unit, used to acquire monitoring images from multiple monitoring perspectives in the office area, wherein the multiple monitoring perspectives include a screen monitoring perspective;

[0028] A target detection unit, used to call a target detection model to respectively identify whether there is a person in the surveillance image under each surveillance perspective;

[0029] A viewing angle determination unit, configured to determine the monitoring viewing angle corresponding to the monitoring image in which a person is identified as a target monitoring viewing angle;

[0030] A target tracking unit, used to track a target person in the monitoring image under the target monitoring viewing angle;

[0031] The sign-out identification unit is used to call the screen status target detection model to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not signed out if the duration for which the target person is not detected in the monitoring image under the target monitoring perspective reaches a first preset value.

[0032] A third aspect of the present application provides an edge computing device, comprising at least one processor and a memory connected to the processor, wherein:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program so that the edge computing device can implement the off-device non-sign-out detection method of the above-mentioned first aspect or any implementation manner of the first aspect.

[0035] A fourth aspect of the present application provides a system for detecting leaving the machine without signing out, including: a cloud platform and at least one edge computing device;

[0036] The cloud platform remotely communicates with at least one edge computing device respectively;

[0037] The edge computing device is communicatively connected to surveillance cameras corresponding to multiple surveillance viewing angles in the office area, and is used to implement the method for detecting leaving the machine without signing out of the machine according to the first aspect or any implementation method of the first aspect.

[0038] In a fifth aspect, the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an edge computing device, the edge computing device implements the method for detecting off-device non-sign-out according to the first aspect or any implementation of the first aspect.

[0039] The sixth aspect of the present application provides a computer storage medium, characterized in that the storage medium carries one or more computer programs. When the one or more computer programs are executed by an edge computing device, the edge computing device can implement the off-device non-sign-out detection method of the above-mentioned first aspect or any implementation method of the first aspect.

[0040] By means of the above technical solution, the present application provides a method and related device for detecting leaving the machine without signing out. After acquiring surveillance images from multiple surveillance perspectives in the office area, the target detection model is called to identify whether there are people in the surveillance images from each surveillance perspective, so as to realize the detection of staff on duty from multiple perspectives, and the surveillance perspective corresponding to the surveillance image in which people are identified is determined as the target surveillance perspective, and a target tracking model that is more lightweight than the target detection model is used to track the target personnel in the surveillance image from the target surveillance perspective to determine whether the staff is leaving the machine. If the staff is leaving the machine, the screen status in the surveillance image from the screen surveillance perspective is identified as not signing out of the system by calling the screen status target detection model, so as to realize automatic detection of leaving the machine without signing out, improve the efficiency and accuracy of detection of leaving the machine without signing out, and reduce the underreporting rate of leaving the machine without signing out. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0042] Figure 1 A system architecture diagram provided for an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the structure of an edge computing device provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a cloud platform provided in an embodiment of the present application;

[0045] Figure 4 A flowchart of a method for detecting a non-sign-out condition when leaving the machine is provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of a target detection model provided in an embodiment of the present application;

[0047] Figure 6 A flowchart of another method for detecting a non-sign-out condition when leaving the machine provided in an embodiment of the present application;

[0048] Figure 7A schematic diagram of the structure of another device for detecting leaving the machine without signing out provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0050] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0051] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0052] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include at least one edge computing device 100, a cloud platform 200, and a surveillance camera 300. The cloud platform 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0053] The cloud platform 200 remotely communicates with at least one edge computing device 100 respectively, and the edge computing device 100 is communicatively connected with the monitoring cameras 300 under multiple monitoring perspectives in the office area.

[0054] Among them, an application can be installed on the edge computing device 100. The edge computing device 100 can trigger a first-level warning and send a warning message to the cloud platform 200 when it detects that the staff in the office area leave the machine without signing out. The cloud platform 200 can issue a configuration file or software upgrade instruction to the edge computing device 100. The edge computing device 100 can update the detection frequency, etc. according to the configuration file, and can also respond to the software upgrade instruction and perform software upgrades based on the upgrade package issued by the cloud platform 200.

[0055] It should be understood that in some optional implementations, the edge computing device 100 can also complete by itself the action of obtaining the processing result based on the received parameters (configuration information, upgrade instructions, etc.), or directly send warning information to the terminal held by the target management personnel and / or the personnel who leave the machine but do not sign out, without the need for the cooperation of the cloud platform 200, and the embodiments of the present application are not limited to this.

[0056] Next describe Figure 1 The product form of the edge computing device 100;

[0057] The edge computing device 100 in the embodiment of the present application can be a network device with edge computing function (such as a smart router, a smart switch, a smart gateway, etc.), a smart camera, an edge server, a PLC (programmable logic controller), a microcontroller, a mobile device (such as a smart phone, a tablet computer, etc.), etc., and the embodiment of the present application does not impose any restrictions on this.

[0058] like Figure 2 As shown, the edge computing device 100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 to a random access memory (RAM) 103. When the edge computing device 100 is powered on, the RAM 103 also stores various programs and data required for the operation of the edge computing device 100. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0059] Typically, the following devices may be connected to the I / O interface 105: an input device 106, an output device 107, a storage device 108 including, for example, a memory card, a hard disk, etc., and a communication device 109. The communication device 109 may allow the edge computing device 100 to communicate with other devices wirelessly or by wire to exchange data. Figure 2 The edge computing device 100 having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0060] Next describe Figure 1 The product form of Zhongyun Platform 200;

[0061] Figure 3 A schematic diagram of the structure of a cloud platform 200 is provided, such as Figure 3As shown, the cloud platform 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0062] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0063] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0064] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0065] The memory 204 may be used to store software codes related to the off-line without signing out detection method, and the processor 202 may execute the related steps of the off-line without signing out detection method, and may also schedule other units to implement corresponding functions.

[0066] It should be understood that the above-mentioned edge computing device 100 and cloud platform 200 can be centralized or distributed devices, and the processors in the above-mentioned edge computing device 100 and cloud platform 200 can be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (digital signal processing, DSP), microprocessors or microcontrollers, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0067] The embodiment of the present application provides a method for detecting a user leaving the machine without signing out, which is applied to an edge computing device. The method for detecting a user leaving the machine without signing out according to the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0068] Reference Figure 4 , Figure 4 A flowchart of a method for detecting a non-sign-out condition when leaving the machine is provided in an embodiment of the present application, such as Figure 4 As shown, a method for detecting an off-machine non-sign-out provided in an embodiment of the present application may include steps 401 to 405, and these steps are described in detail below.

[0069] 401: Acquire surveillance images from multiple surveillance perspectives in an office area, where the multiple surveillance perspectives include a screen surveillance perspective;

[0070] Since there are often workstation baffles, seat backs, objects on the desk, etc. that block the staff in the office area, a single monitoring angle cannot accurately monitor whether the staff is on duty. In this embodiment, monitoring cameras are installed at multiple monitoring angles in the office area to achieve multi-angle monitoring. The multiple monitoring angles include a screen monitoring angle, which is a monitoring angle facing the office computer screen.

[0071] The edge computing device communicates with the surveillance cameras in multiple surveillance perspectives in the office area to obtain the surveillance video data of the office area collected by the surveillance cameras in multiple surveillance perspectives. The surveillance video data is then pre-processed by extracting frames at a preset frame rate to obtain surveillance images in multiple surveillance perspectives.

[0072] 402: Calling the target detection model to identify whether there is a person in the surveillance image under each surveillance perspective;

[0073] Since this embodiment identifies whether there is a person in the surveillance image under each surveillance perspective respectively, joint reasoning and analysis based on the surveillance images under multiple surveillance perspectives is achieved.

[0074] Exemplarily, a target detection model is called to identify whether there is a person in the surveillance image under each surveillance perspective, that is, a target detection model is called serially to identify whether there is a person in the surveillance image under each surveillance perspective, first identifying whether there is a person in the surveillance image under surveillance perspective 1, then identifying whether there is a person in the surveillance image under surveillance perspective 2, and so on, until the identification of the surveillance image under each surveillance perspective is completed, and then the next cycle is started to call the target detection model to identify whether there is a person in the surveillance image under each surveillance perspective.

[0075] Exemplarily, each monitoring perspective corresponds to a target detection model, and multiple target detection models identify in parallel whether there is a person in the monitoring image under each monitoring perspective.

[0076] The target detection model is pre-trained. The input data of the target detection model is the surveillance image, and the output data is the target detection confidence. The target detection confidence indicates the credibility of the presence of a person in the surveillance image.

[0077] If multiple monitoring perspectives correspond to one target detection model, the training sample set of the target detection model includes training samples under multiple monitoring perspectives, thereby realizing joint training of training samples under multiple monitoring perspectives.

[0078] If each monitoring perspective corresponds to a target detection model, in one possible implementation, the network structure of each target detection model is the same but the training sample set is different. For example, the training sample set of the target detection model corresponding to monitoring perspective A is the training sample under monitoring perspective A. In another possible implementation, the network structure of each target detection model is the same and the training sample set is also the same, including training samples under multiple monitoring perspectives.

[0079] If the target detection confidence is greater than the confidence threshold, it is determined that there is a person in the surveillance image.

[0080] 403: Determine the monitoring angle corresponding to the monitoring image in which a person is identified as a target monitoring angle;

[0081] If there is a person in the surveillance image under only one surveillance perspective, the surveillance perspective is determined as the target surveillance perspective.

[0082] If there is a person in the surveillance images under more than one surveillance angle, there are multiple ways to determine the target surveillance angle.

[0083] Exemplarily, a monitoring angle corresponding to any monitoring image in which a person is identified is determined as a target monitoring angle.

[0084] Exemplarily, the target detection confidences of people in the surveillance images under each surveillance perspective are sorted, and the surveillance perspective corresponding to the highest target detection confidence is determined as the target surveillance perspective to improve the reliability of subsequent target tracking results under this surveillance perspective.

[0085] 404: Tracking a target person in a surveillance image under a target surveillance perspective;

[0086] A target tracking model that is more lightweight than the target detection model is used to track the target person, for example, the DeepSort multi-target tracking model is used for target tracking.

[0087] 405: If the duration of not detecting the target person in the monitoring image under the target monitoring perspective reaches a first preset value, calling a screen status target detection model to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not been signed out.

[0088] The duration during which the target person is not detected in the monitoring image under the target monitoring perspective reaches a first preset value, indicating that the staff in the office area has left the machine.

[0089] It should also be noted that if in step 402, the duration for which no person is detected in the surveillance image under each surveillance perspective reaches a first preset value, it also indicates that the staff in the office area has left the machine, and it is also necessary to jump to the step of calling the screen status target detection model to identify whether the screen status in the surveillance image under the screen monitoring perspective is that the system has not signed out.

[0090] At this time, it is necessary to call the pre-trained screen status target detection model to detect whether the office computer used by the staff has been signed out of the system. Specifically, the monitoring image under the screen monitoring perspective is input into the screen status target detection model to obtain the target detection confidence output by the screen status target detection model. The target detection confidence indicates the degree of confidence that the screen status is that the system has not been signed out. If the target detection confidence is greater than the confidence threshold, it is determined that the screen status is that the system has not been signed out.

[0091] The present embodiment discloses a method for detecting off-machine without signing out. After acquiring surveillance images from multiple surveillance perspectives in an office area, the target detection model is called to identify whether there are people in the surveillance images from each surveillance perspective, thereby realizing on-the-job detection of staff from multiple perspectives. The surveillance perspective corresponding to the surveillance image in which people are identified is determined as the target surveillance perspective, and a target tracking model that is more lightweight than the target detection model is used to track the target person in the surveillance image from the target surveillance perspective to determine whether the staff is off-machine. If the staff is off-machine, the screen status in the surveillance image from the screen surveillance perspective is identified as not signing out of the system by calling the screen status target detection model, thereby realizing automated off-machine without signing out detection, improving the efficiency and accuracy of off-machine without signing out detection, and reducing the underreporting rate of off-machine without signing out behavior.

[0092] Considering the limited number of surveillance cameras in the office area and the incompleteness of the workers in the surveillance image due to the occlusion of the workstations, the success rate of the target detection model in identifying the workers is affected. In order to solve this technical problem, a training method for a target detection model provided in this embodiment includes the following steps A1-A5:

[0093] A1: Acquire multiple frames of historical surveillance images at each surveillance perspective, where the historical surveillance images include a first image and a second image, where there is a person in the first image and no one in the second image;

[0094] The historical monitoring video data of people and people in each monitoring angle of the office area are collected, and the collected historical monitoring video data are de-framed and de-duplicated to obtain historical monitoring images. Therefore, the historical monitoring images include a first image and a second image.

[0095] In order to improve the robustness of the target detection model for occluded person recognition, a large number of historical surveillance images that only show the face, head or upper body of a person can be collected. In addition, the sample data can be expanded, as shown in steps A2-A3.

[0096] A2: Partially cut out the person in the first image to obtain a cutout image;

[0097] For example, cutting out a character's face, head, upper body, etc.

[0098] A3: Perform data augmentation processing on the cut image and randomly paste it into the scene image to obtain the third image. The first image, the second image and the third image constitute the training sample set;

[0099] The data enhancement processing includes: at least one of random rotation, random scaling, and random color perturbation.

[0100] There are no people in the scene image, so the cut image is processed for data augmentation and then randomly pasted into the scene image to achieve sample data expansion.

[0101] A4: Label the training sample set by category and identification box;

[0102] The categories include people and no people. For training samples with the category of people, identification boxes need to be marked. The identification boxes represent the part of the training samples with people.

[0103] A5: Use the training sample set to train the target detection model.

[0104] For example, the YOLOv5 target detection model is used for training. Figure 5 The YOLOv5 network structure shown in the figure, YOLOv5 uses the backbone network to extract multi-scale features, performs multi-scale feature fusion in the bottleneck part, and finally performs target detection at multiple scales, outputting information such as the target category, target recognition box, and confidence of each identified target. This embodiment uses the Adam optimizer for optimization training, and the learning rate can be set to 0.001, the batch size to 64, and a total of 500 rounds of training convergence.

[0105] The screen status detection model can also be trained using the YOLOv5 target detection model. The training samples of the screen status detection model include the working computer screen images corresponding to the system signing out and the working computer screen images corresponding to the system not signing out, and they are labeled with categories and recognition boxes. The categories include system signing out and system not signing out. The recognition box corresponding to the system signing out represents the system sign-out marker, such as the login prompt box, and the recognition box corresponding to the system not signing out represents the system not signing out marker, such as the identity information box.

[0106] Considering the limited computing resources and performance of edge computing devices, in a possible implementation, after determining the monitoring angle corresponding to the monitoring image in which a person is identified as the target monitoring angle, this embodiment stops calling the target detection model to respectively identify whether there is a person in the monitoring image under each monitoring angle, and uses a target tracking model that is more lightweight than the target detection model to track the target person, thereby releasing part of the computing power of the edge computing device and ensuring the real-time performance of target tracking. Until it is determined that the target person has left the machine and the system has signed out, or it is determined that the target person has left the machine and the system has not signed out, the step of obtaining monitoring images under multiple monitoring angles in the office area is triggered, so that the off-machine and non-sign-out detection method is executed in a loop.

[0107] There are multiple ways to determine that the target person has left the machine and has not signed out of the system.

[0108] Exemplarily, since in the above embodiment step 405 is a screen status detection triggered after the duration of not detecting the target person in the monitoring image under the target monitoring perspective reaches a first preset value, that is, the screen status detection is triggered after it is determined that the target person has left the machine, at this time, if the screen status in the monitoring image under the screen monitoring perspective is that the system has not signed out, it can be determined to a certain extent that the target person has left the machine and the system has not signed out.

[0109] Exemplarily, in order to improve the reliability of determining that the target person has left the machine and the system has not signed out and to avoid misjudgment, after calling the screen status target detection model to identify whether the screen status in the surveillance image under the screen monitoring perspective is that the system has not signed out, if the number of times the screen status in the surveillance image under the screen monitoring perspective is that the system has not signed out within a preset time reaches a second preset value, it is determined that the target person has left the machine and the system has not signed out.

[0110] See also Figure 6 The flowchart of the off-device non-sign-out detection method shown in the figure shows that after the object detection model and the screen status object detection model are trained, model verification and format conversion are performed, for example, conversion to the cross-platform universal ONNX format, and finally deployed on the edge computing device.

[0111] After the model is deployed, it can be put into use. The edge computing device obtains surveillance images from multiple surveillance perspectives in the office area, and calls the target detection model to identify whether there are people in the surveillance images from each surveillance perspective. If a person is identified, the surveillance perspective corresponding to the surveillance image in which a person is identified is determined as the target surveillance perspective, and then the target person in the surveillance image is tracked from the target surveillance perspective. If the duration for which the target person is not detected in the surveillance image from the target surveillance perspective reaches a first preset value, it indicates that the staff in the office area has left the machine, and the screen status target detection model is called to identify whether the screen status in the surveillance image from the screen surveillance perspective is that the system has not been signed out. It should also be noted that in the process of calling the target detection model to identify whether there are people in the surveillance images from each surveillance perspective, if the duration for which no person is detected in the surveillance images from each surveillance perspective reaches a first preset value, it also indicates that the staff in the office area has left the machine, and it is also necessary to jump to the step of calling the screen status target detection model to identify whether the screen status in the surveillance image from the screen surveillance perspective is that the system has not been signed out. If the screen status is that the system has been signed out, then exit normally and return to execute multi-view real-time monitoring image data input; if the screen status is that the system has not been signed out, an alarm is triggered and an alarm message is sent to the cloud platform, so that the cloud platform forwards the alarm message to the terminal held by the target manager or the person who has left the machine without signing out, and at the same time returns to execute multi-view real-time monitoring image data input, so that the method for detecting leaving the machine without signing out is executed in a loop.

[0112] Furthermore, in order to avoid misjudgment, if the number of times that the screen status in the monitoring image under the screen monitoring angle of view indicates that the system has not been signed out reaches a second preset value (eg, 2 times) within a preset time, an early warning is triggered.

[0113] The above-mentioned warning information may include monitoring images, the acquisition time of the monitoring images and office area identification, so that the target management personnel can promptly understand which office area has safety hazards and deal with them in a timely manner.

[0114] A method for detecting an off-machine failure to sign out provided in an embodiment of the present application is described above. A device for executing the method for detecting an off-machine failure to sign out will be described below.

[0115] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a device for detecting a machine without signing out provided in an embodiment of the present application. Figure 7 As shown, the device for detecting the non-sign-out after leaving the aircraft includes:

[0116] An image acquisition unit 701 is used to acquire surveillance images from multiple surveillance perspectives in an office area, wherein the multiple surveillance perspectives include a screen surveillance perspective;

[0117] The target detection unit 702 is used to call the target detection model to respectively identify whether there is a person in the surveillance image under each surveillance perspective;

[0118] A viewing angle determining unit 703, configured to determine the monitoring viewing angle corresponding to the monitoring image in which a person is identified as a target monitoring viewing angle;

[0119] A target tracking unit 704 is used to track the target person in the monitoring image under the target monitoring perspective;

[0120] The sign-out identification unit 705 is used to call the screen status target detection model to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not signed out if the duration for which the target person is not detected in the monitoring image under the target monitoring perspective reaches a first preset value.

[0121] In a possible implementation, the device for detecting off-board failure to sign out further includes:

[0122] A stop detection unit, configured to stop calling the target detection model to respectively identify whether there is a person in the surveillance image under each surveillance perspective after determining the surveillance perspective corresponding to the surveillance image in which a person is identified as a target surveillance perspective;

[0123] The trigger detection unit is used to trigger the image acquisition unit 701 when it is determined that the target person has left the aircraft and the system has been signed out, or when it is determined that the target person has left the aircraft and the system has not been signed out.

[0124] In a possible implementation, the target detection unit 702 is further configured to trigger the sign-out recognition unit 705 if a duration in which no person is detected in the surveillance images under each surveillance viewing angle reaches a first preset value.

[0125] In a possible implementation, the device for detecting off-board failure to sign out further includes:

[0126] A model training unit is used to obtain multiple frames of historical monitoring images under each monitoring perspective, the historical monitoring images include a first image and a second image, the first image contains a person, and the second image contains no person; partially cut the person in the first image to obtain a cut image; perform data enhancement processing on the cut image, and randomly paste it into a scene image to obtain a third image, the first image, the second image and the third image constitute a training sample set; perform category labeling and recognition box labeling on the training sample set; and use the training sample set to train the target detection model.

[0127] In a possible implementation, the device for detecting off-board failure to sign out further includes:

[0128] An early warning unit, configured to trigger an early warning when determining that the target person has left the aircraft and the system has not signed out;

[0129] Wherein, if the screen status in the monitoring image under the screen monitoring angle is that the system has not been signed out, it is determined that the target person has left the machine and the system has not been signed out;

[0130] or

[0131] If the number of times that the screen status in the monitoring image under the screen monitoring angle of view indicates that the system has not been signed out reaches a second preset value within the preset time, it is determined that the target person has left the machine and has not signed out of the system.

[0132] The present embodiment discloses a device for detecting staff leaving the machine without signing out. After acquiring surveillance images from multiple surveillance perspectives in an office area, the device calls a target detection model to identify whether there are people in the surveillance images from each surveillance perspective, thereby realizing on-the-job detection of staff from multiple perspectives. The surveillance perspective corresponding to the surveillance image in which people are identified is determined as the target surveillance perspective, and a target tracking model that is more lightweight than the target detection model is used to track the target person in the surveillance image from the target surveillance perspective to determine whether the staff has left the machine. If the staff has left the machine, the screen status in the surveillance image from the screen surveillance perspective is identified as not signing out of the system by calling the screen status target detection model, thereby realizing automated detection of staff leaving the machine without signing out, improving the efficiency and accuracy of detection of staff leaving the machine without signing out, and reducing the underreporting rate of staff leaving the machine without signing out.

[0133] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an edge computing device, the edge computing device implements any of the methods for detecting leaving the device without signing out provided in the embodiments of the present application.

[0134] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an edge computing device, the edge computing device can implement any of the methods for detecting off-device non-sign-out provided in the embodiments of the present application.

[0135] The present application provides a method for detecting a user leaving the machine without signing out and a related device that can be used in the field of artificial intelligence or finance. The above is only an example and does not limit the application field of the method for detecting a user leaving the machine without signing out and a related device that can be used in the field of artificial intelligence or finance.

[0136] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0137] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0138] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0139] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for detecting a user leaving the machine without signing out, characterized in that: include: Acquire surveillance images from multiple surveillance perspectives in an office area, wherein the multiple surveillance perspectives include a screen surveillance perspective; Calling the target detection model to respectively identify whether there is a person in the surveillance image under each of the surveillance perspectives; Determine the monitoring angle corresponding to the monitoring image in which a person is identified as a target monitoring angle; Tracking the target person in the monitoring image under the target monitoring perspective; If the duration for which the target person is not detected in the monitoring image under the target monitoring perspective reaches a first preset value, the screen status target detection model is called to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not signed out.

2. The method for detecting leaving the machine without signing out according to claim 1, characterized in that: After determining the monitoring angle corresponding to the monitoring image in which a person is identified as the target monitoring angle, the method for detecting leaving the aircraft without signing out further includes: Stop calling the target detection model to respectively identify whether there is a person in the surveillance image under each of the surveillance perspectives; When it is determined that the target person has left the computer and the system has been signed out, or when it is determined that the target person has left the computer and the system has not been signed out, the process returns to the step of acquiring surveillance images from multiple surveillance perspectives in the office area.

3. The method for detecting leaving the machine without signing out according to claim 1, characterized in that: After calling the target detection model to respectively identify whether there is a person in the surveillance image under each surveillance perspective, the off-board non-sign-out detection method further includes: If the duration in which no person is detected in the surveillance image under each of the surveillance perspectives reaches a first preset value, the step of calling the screen status target detection model to identify whether the screen status in the surveillance image under the screen monitoring perspective is that the system has not signed out is executed.

4. The method for detecting leaving the machine without signing out according to claim 1, characterized in that: The training method of the target detection model includes: Acquire multiple frames of historical monitoring images under each monitoring perspective, wherein the historical monitoring images include a first image and a second image, wherein the first image contains a person and the second image contains no person; Partially cutting out the person in the first image to obtain a cutout image; Performing data enhancement processing on the cut image, and randomly pasting it into the scene image to obtain a third image, wherein the first image, the second image and the third image constitute a training sample set; Performing category labeling and identification box labeling on the training sample set; The target detection model is trained using the training sample set.

5. The method for detecting leaving the machine without signing out according to claim 1, characterized in that: After calling the screen status target detection model to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not been signed out, the off-device non-sign-out detection method further includes: When it is determined that the target person has left the aircraft and the system has not signed out, an early warning is triggered; Wherein, if the screen status in the monitoring image under the screen monitoring angle is that the system has not been signed out, it is determined that the target person has left the machine and the system has not been signed out; or If the number of times that the screen status in the monitoring image under the screen monitoring angle of view indicates that the system has not been signed out reaches a second preset value within the preset time, it is determined that the target person has left the machine and has not signed out of the system.

6. A device for detecting a person leaving the machine without signing out, characterized in that: include: An image acquisition unit, used to acquire monitoring images from multiple monitoring perspectives in the office area, wherein the multiple monitoring perspectives include a screen monitoring perspective; A target detection unit, used to call a target detection model to respectively identify whether there is a person in the surveillance image under each surveillance perspective; A viewing angle determination unit, configured to determine the monitoring viewing angle corresponding to the monitoring image in which a person is identified as a target monitoring viewing angle; A target tracking unit, used to track a target person in the monitoring image under the target monitoring viewing angle; The sign-out identification unit is used to call the screen status target detection model to identify whether the screen status in the monitoring image under the screen monitoring perspective is that the system has not signed out if the duration for which the target person is not detected in the monitoring image under the target monitoring perspective reaches a first preset value.

7. An edge computing device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the edge computing device can implement the off-device non-sign-out detection method as described in any one of claims 1 to 5.

8. A system for detecting a non-sign-out after leaving the aircraft, characterized in that: include: A cloud platform and at least one edge computing device; The cloud platform remotely communicates with at least one of the edge computing devices respectively; The edge computing device is communicatively connected to surveillance cameras corresponding to multiple surveillance viewing angles in the office area, and is used to implement the method for detecting leaving the machine without signing out as described in any one of claims 1 to 5.

9. A computer program product, characterized in that The method comprises computer-readable instructions, and when the computer-readable instructions are executed on an edge computing device, the edge computing device implements the method for detecting off-board non-sign-out as claimed in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs. When the one or more computer programs are executed by the edge computing device, the edge computing device can implement the off-device non-sign-out detection method as described in any one of claims 1 to 5.

Images