A method for monitoring access to a vault and related apparatus

Abstract

The application discloses a kind of monitoring methods and related devices of entering and exiting vault behavior, which can be applied to artificial intelligence field or financial field. The pre-deployed vault door identification model and personnel identification model are used to identify the vault door and personnel in the monitoring image. Based on the target monitoring image, a two-dimensional coordinate system in the vault door plane is constructed, and a two-dimensional mapped vault door two-dimensional coordinate set and two-dimensional personnel position information are obtained. Based on the vault door two-dimensional coordinate set and two-dimensional personnel position information, an action judgment result is generated. Based on the personnel identification result of each personnel monitoring image and the action judgment result sequence, it is determined whether it conforms to the pre-configured vault entry and exit rule. When it is not in compliance, an early warning is issued. The present scheme improves the accuracy of action judgment through two-dimensional mapping, and realizes the real-time and accuracy of the entry and exit vault behavior monitoring through the edge side terminal.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a method for monitoring access to a vault and related apparatus. BACKGROUND

[0002] A vault is a very important security location, which is distributed throughout the country and involves many high-standard compliance requirements, regulatory requirements and internal control requirements. The vault, as a key security area in the financial scene, is equipped with multiple cameras. Taking a bank vault as an example, the bank security system performs all-weather monitoring on the environment of each area in the vault through multiple monitors. For different vault areas, the monitoring objects of the bank security system are different. The monitoring focus of the vault door area is whether the behavior of personnel accessing the vault meets the access specification. For example, the type of personnel entering the vault, the number of personnel, and whether the time of each personnel entering the vault meets the access specification.

[0003] Currently, personnel access behavior is checked in real time by manually observing monitoring images, or checked in a sampling inspection manner. Both of the two ways are manually observed monitoring images, and after discovering that there may be risks, manually triggering the alarm device of the security system. Manual observation of monitoring images may miss or misjudge, so as to ensure the real-time performance and accuracy of the bank security system for personnel access behavior risk alarm. SUMMARY

[0004] In view of the above problems, the present application provides a method for monitoring access to a vault and related apparatus to achieve the purpose of improving the real-time performance and accuracy of access to the vault behavior monitoring. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for monitoring access to a vault, applied to a terminal, comprising:

[0006] sequentially inputting monitoring images in a real-time monitoring image sequence into a vault door recognition model to obtain a vault door recognition result output by the vault door recognition model; the vault door recognition result comprises a vault door opening and closing state and a vault door key point coordinate set;

[0007] If a change in the vault door opening and closing state is detected, sequentially input monitoring images in a real-time monitoring image sequence into a personnel recognition model starting from a target monitoring image to obtain a personnel recognition result output by the personnel recognition model, the personnel recognition result comprising personnel type and personnel position information, the target monitoring image being a monitoring image in which the vault door opening and closing state changes; wherein the vault door recognition model and the personnel recognition model are obtained by training a machine learning model based on a training data set by a server;

[0008] constructing a two-dimensional coordinate system in a vault door plane as a two-dimensional vault door coordinate system based on the target monitoring image;

[0009] mapping the vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set;

[0010] For each personnel monitoring image, mapping the personnel position information of the personnel monitoring image to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, the personnel monitoring image being a monitoring image input into a personnel recognition model;

[0011] For each of the personnel monitoring images, generating an action judgment result based on the vault door two-dimensional coordinate set and the two-dimensional personnel position information, the action judgment result indicating whether a target personnel in the personnel monitoring image triggers an access vault door action;

[0012] Based on the personnel recognition result of each of the personnel monitoring images and the action determination result sequence, it is determined whether the preconfigured vault access rule is met, the action determination result sequence including the action judgment results of the personnel monitoring images arranged in time sequence;

[0013] If the vault access rule is not met, an early warning information is issued.

[0014] Optionally, the monitoring method of access to the vault further comprises:

[0015] After obtaining each vault door recognition result output by the vault door recognition model, updating a vault door opening and closing state sequence, the vault door opening and closing state sequence including the vault door opening and closing states of each monitoring image arranged in time sequence;

[0016] Based on the vault door opening and closing state sequence, it is determined whether the vault door opening and closing state changes.

[0017] Optionally, based on the target monitoring image, a two-dimensional coordinate system in a vault door plane is constructed as a two-dimensional vault door coordinate system, comprising:

[0018] Based on the vault door key point coordinate set, a vault door vertex coordinate set is generated;

[0019] A two-dimensional coordinate system in a vault door plane is established as the two-dimensional vault door coordinate system with a horizontal line passing through the lowermost vertex in the vault door vertex coordinate set and a vertical line passing through the leftmost vertex in the vault door vertex coordinate set as coordinate axes;

[0020] Mapping the vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set, comprising:

[0021] mapping each vertex coordinate in the vault door vertex coordinate set to the two-dimensional vault door coordinate system to obtain a two-dimensional vault door coordinate set.

[0022] Optionally, the personnel position information of the personnel monitoring image is mapped to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, including:

[0023] Based on the personnel position information, an edge coordinate set of a personnel rectangular frame is obtained, the edge coordinate set including the longitudinal coordinates of each horizontal edge and the horizontal coordinates of each vertical edge of the personnel rectangular frame.

[0024] The edge coordinate set of the personnel rectangular frame is mapped to the two-dimensional vault door coordinate system to obtain the two-dimensional personnel position information.

[0025] Optionally, based on the two-dimensional vault door coordinate set and the two-dimensional personnel position information, an action judgment result is generated, including:

[0026] It is judged that each coordinate in the two-dimensional personnel position information encloses a rectangle within the outline enclosed by each vertex in the two-dimensional vault door coordinate set.

[0027] If yes, it is determined that the personnel triggers an action of entering or exiting the vault door.

[0028] Optionally, the monitoring method of the behavior of entering or exiting the vault also includes:

[0029] The vault door recognition model and the personnel recognition model are loaded, and the initial state of the vault door recognition model and the personnel recognition model is set to be closed.

[0030] If a change in the vault door opening and closing state is detected, the vault door recognition model is closed, and the personnel recognition model is opened.

[0031] When a preset time threshold is reached or the rule judgment result is not met, the personnel recognition model is closed, and the vault door recognition model is opened.

[0032] The fourth aspect of the present application provides a monitoring device for the behavior of entering or exiting a vault, including:

[0033] A vault door recognition unit is configured to sequentially input monitoring images in a real-time monitoring image sequence to a vault door recognition model to obtain a vault door recognition result output by the vault door recognition model; the vault door recognition result includes a vault door opening and closing state and a vault door key point coordinate set.

[0034] The personnel recognition unit is configured to input, in sequence, monitoring images in a real-time monitoring image sequence to a personnel recognition model starting from a target monitoring image to obtain a personnel recognition result output by the personnel recognition model, the personnel recognition result including a personnel type and personnel position information, and the target monitoring image being a monitoring image in which a vault door switching state changes.

[0035] The two-dimensional coordinate system construction unit is configured to construct a two-dimensional coordinate system in a vault door plane as a two-dimensional vault door coordinate system based on the target monitoring image.

[0036] The two-dimensional coordinate mapping unit is configured to map a vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set, and map, for each personnel monitoring image, personnel position information of the personnel monitoring image to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, the personnel monitoring image being a monitoring image input to the personnel recognition model.

[0037] The action judgment unit is configured to generate, for each personnel monitoring image, an action judgment result based on the vault door two-dimensional coordinate set and the two-dimensional personnel position information, the action judgment result indicating whether a target person in the personnel monitoring image triggers an in-out vault door action.

[0038] The rule judgment unit is configured to judge whether a preconfigured in-out vault rule is met based on a personnel recognition result of each personnel monitoring image and a sequence of action judgment results, the sequence of action judgment results including action judgment results of the personnel monitoring images arranged in time sequence.

[0039] The early warning unit is configured to issue early warning information if the in-out vault rule is not met.

[0040] The third aspect of the present application provides a computer program product, including computer readable instructions, when the computer readable instructions run on an electronic device, the electronic device implements the in-out vault behavior monitoring method of the first aspect or any implementation manner of the first aspect.

[0041] The fourth aspect of the present application provides an electronic device, including at least one processor and a memory connected with the processor, wherein:

[0042] The memory is configured to store a computer program.

[0043] The processor is configured to execute the computer program, so that the electronic device can implement the in-out vault behavior monitoring method of the first aspect or any implementation manner of the first aspect.

[0044] The fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs, when the one or more computer programs are executed by an electronic device, the electronic device can implement the monitoring method of the cash-in and cash-out behavior of the first aspect or any implementation manner of the first aspect.

[0045] By the above technical solution, the present application provides a monitoring method of cash-in and cash-out behavior and related devices, the monitoring images in the real-time monitoring image sequence are sequentially input into the vault door recognition model to obtain the vault door recognition result output by the vault door recognition model. If it is detected that the vault door opening and closing state changes, the monitoring images in the real-time monitoring image sequence are sequentially input into the personnel recognition model starting from the target monitoring image to obtain the personnel recognition result output by the personnel recognition model, and the target monitoring image is the monitoring image in which the vault door opening and closing state changes. Based on the target monitoring image, a two-dimensional coordinate system in the vault door plane is constructed as a two-dimensional vault door coordinate system. The vault door key point coordinate set of the target monitoring image is mapped to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set. For each personnel monitoring image, the personnel position information of the personnel monitoring image is mapped to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, and the personnel monitoring image is the monitoring image input into the personnel recognition model. For each personnel monitoring image, based on the vault door two-dimensional coordinate set and the two-dimensional personnel position information, an action judgment result is generated. Based on the personnel recognition result of each personnel monitoring image and the action judgment result sequence, it is judged whether it conforms to the preconfigured cash-in and cash-out rule of the vault, and the action judgment result sequence includes the action judgment results of the personnel monitoring images arranged in time sequence. If it does not conform to the cash-in and cash-out rule of the vault, a warning information is sent. It can be seen that the present application improves the real-time performance and accuracy of the monitoring of the cash-in and cash-out behavior of the vault by using the pre-deployed vault door recognition model and personnel recognition model by the terminal, and by mapping the personnel position information and the vault door key point coordinates to a two-dimensional plane through the two-dimensional vault door coordinate system, the difficulty of action judgment is reduced, the accuracy of the action judgment result is improved, and the accuracy of the monitoring of the cash-in and cash-out behavior of the vault is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] The above and other features, advantages, and aspects of the embodiments of the present application will become more apparent by referring to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals indicate the same or similar elements. It should be understood that the drawings are schematic, and the sizes and elements are not necessarily drawn to scale.

[0047] Figure 1 An architecture diagram of a distributed monitoring system of cash-in and cash-out behavior provided by the present application;

[0048] Figure 2A hardware structure schematic diagram of a terminal provided for the present application;

[0049] Figure 3 A hardware structure schematic diagram of a server provided for the present application;

[0050] Figure 4 A specific structure schematic diagram of a monitoring system of a deposit and withdrawal behavior provided for the present application;

[0051] Figure 5 A specific implementation flowchart of a monitoring method of a deposit and withdrawal behavior provided for an embodiment of the present application;

[0052] Figure 6 A flowchart of a monitoring method of a deposit and withdrawal behavior provided for an embodiment of the present application;

[0053] Figure 7 A structure schematic diagram of a monitoring device of a deposit and withdrawal behavior provided for an embodiment of the present application;

[0054] Figure 8 A structure schematic diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0055] The embodiments of the present application are described below in conjunction with the accompanying drawings. The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0056] The embodiments of the present application are described below in conjunction with the accompanying drawings. It is known to those skilled in the art that as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0057] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a distinguishing way used in the description of the embodiments of the present application to describe the objects with the same attributes. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not necessarily limit to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0058] The present application can be applied to the technical field of risk monitoring of bank vaults, and specifically can be applied to the monitoring scenario of deposit and withdrawal behavior in the vault.

[0059] The bank has a set of specifications for the personnel category, number of people entering the vault, and door opening and closing action process, including daily vault operations by personnel of a specific type according to specified requirements. The traditional inspection method is to check by manually observing the monitoring image, find irregular behavior, and manually trigger an alarm. This method has the disadvantages of heavy workload, high labor cost, easy omission, and limited coverage, so in actual operation, a sampling inspection method is often used to find irregular behavior and manually trigger an alarm. Due to the subjectivity and ability difference of manual observation, the existing bank security system has the technical shortcomings of low real-time, low accuracy, and high lag in risk monitoring of the in-and-out behavior of the vault.

[0060] To solve the above technical shortcomings, the present scheme proposes a bank security system based on machine vision technology, which can collect monitoring images of the area inside the vault door in real time, and use a pre-trained vault door recognition model and a personnel recognition model to obtain vault door state information and personnel state information, respectively. Based on the vault door state information and the personnel state information, the in-and-out behavior information corresponding to the monitoring image is determined, and based on the in-and-out behavior information of multiple monitoring images within a preset time period, it is determined whether it conforms to the in-and-out behavior specification. If it does not conform, it is determined that there is a risk of in-and-out behavior of the vault, and the alarm device of the bank security system for the risk of in-and-out behavior of the vault is automatically triggered. The vault door recognition model and the personnel recognition model are based on a deep learning model and are optimized, and the in-and-out behavior specification is pre-configured, including: personnel type rules, personnel quantity rules, and in-and-out time rules.

[0061] Thus, the recognition model (including the vault door recognition model and the personnel recognition model) monitors the state of the vault door opening and closing and the behavior state of the personnel based on the monitoring image, and analyzes the in-and-out behavior information based on the recognized state of the vault door opening and closing and the behavior state of the personnel. Thus, the risk monitoring of the in-and-out behavior of the vault is realized. The present scheme can significantly reduce the risk of manual supervision and labor cost, and improve the real-time performance of the in-and-out behavior monitoring of the vault. Further, the present scheme can solve the problem of heavy workload and inspection lag, and can cover all vault environments and all time periods, reducing the omission and subjective errors of manual inspection.

[0062] Through research, it is found that the bank security system is deployed in the form of a central server, and the deep learning model is deployed on the server (such as a cloud server) in the bank security system. The image sensitive information involved in the in-and-out behavior of the vault involves security and privacy protection problems in the process of transmission to the server, which may lead to leakage or malicious use of image sensitive information. To this end, the present scheme further optimizes the in-and-out behavior monitoring scheme of the vault by combining machine vision technology and edge computing technology, and proposes a distributed in-and-out behavior monitoring system of the vault.

[0063] The embodiment of the present application provides a kind of monitoring system of gold in and out behavior, refer to Figure 1 , Figure 1 A system architecture schematic diagram is shown.The system can include terminal 100, and server 200.Wherein, server 200 can include one or more servers( Figure 1 It is described with one server as an example in the middle), server 200 can provide the method provided by the embodiment of the present application for one or more terminals.

[0064] Wherein, terminal 100 can be installed with the monitoring application of gold in and out behavior, above-mentioned application and webpage can provide an interface, terminal 100 can receive the relevant parameters that user inputs on the monitoring interface of gold in and out behavior, and above-mentioned parameters are sent to server 200, server 200 can obtain processing result based on received parameter, and processing result is returned to terminal 100.

[0065] It should be understood that, in some optional implementation, terminal 100 can also be based on received parameter by itself, obtain processing result action, without the cooperation of server to realize, the embodiment of the present application does not limit.

[0066] Next describe Figure 1 The product form of terminal 100 in the middle;

[0067] The terminal 100 in the embodiment of the present application can be mobile phone, tablet computer, wearable device, vehicle-mounted device, augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, notebook computer, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA) and the like, and the embodiment of the present application does not make any limitation.

[0068] Figure 2 An optional hardware structure schematic diagram of terminal 100 is shown.

[0069] Reference Figure 2 As shown in the figure, terminal 100 can include radio frequency unit 110, memory 120, input unit 130, display unit 140, camera 150 (optional), audio circuit 160 (optional), speaker 161 (optional), microphone 162 (optional), earphone jack 163 (optional), processor 170, external interface 180, power supply 190 and the like components.The person skilled in the art can understand, Figure 2The terminal or multifunctional device is merely an example and does not limit the terminal or multifunctional device, which can include more or fewer components than shown, or combine certain components, or have different components.

[0070] The input unit 130 can be used to receive inputted digital or character information, and to generate key signal input related to user settings and function control of the portable multifunctional device. Specifically, the input unit 130 can include a touch screen 131 (optional) and / or other input device 132. The touch screen 131 can collect touch operations of a user thereon or thereabout (such as operations of the user using a finger, a knuckle, a stylus, or any suitable object on or near the touch screen), and drive corresponding connected devices according to a pre-set program. The touch screen can detect touch actions of the user on the touch screen, convert the touch actions into touch signals and send the touch signals to the processor 170, and can receive commands from the processor 170 and execute the commands; the touch signals at least include touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, etc.

[0071] The input device 132 can receive inputted data, etc.

[0072] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playing of any kind of multimedia file. In the embodiments of the present application, the display unit 140 can be used to display interfaces for monitoring cash deposit and withdrawal behaviors, processing results, etc.

[0073] The storage 120 can be used to store instructions and data. The storage 120 can mainly include a storage instruction area and a storage data area. The storage data area can store various data such as multimedia files, texts, etc.; the storage instruction area can store software units required by the operating system, applications, at least one function, etc., or their subsets, expanded sets. It can also include a non-volatile random access memory; provide the processor 170 with software and applications that include management of hardware, software, and data resources in the computing processing device, support control. It is also used for storage of multimedia files, and storage of running programs and applications.

[0074] The processor 170 is the control center of the terminal 100, connects each part of the whole terminal 100 by various interfaces and lines, executes various functions of the terminal 100 and processes data by running or executing the instructions stored in the memory 120 and calling the data stored in the memory 120, thereby performing overall control on the terminal device. Optionally, the processor 170 can include one or more processing units; preferably, the processor 170 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 170. In some embodiments, the processor, the memory, can be realized on a single chip, and in some embodiments, they can also be realized on independent chips respectively. The processor 170 can also be used to generate corresponding operation control signals to send to corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein executes corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0075] The memory 120 can be used to store the software code related to the monitoring method of the in-and-out behavior of the cash box, and the processor 170 can execute the steps of the monitoring method of the in-and-out behavior of the cash box, and can also dispatch other units (such as the above-mentioned input unit 130 and display unit 140) to realize corresponding functions.

[0076] The RF unit 110 (optional) can be used to receive and send signals in the process of information or communication, for example, receiving the downlink information of the base station, and processing by the processor 170. In addition, the uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0077] In the embodiments of the present application, the RF unit 110 can send data to the server 200 and receive the processing result sent by the server 200.

[0078] It should be understood that the RF unit 110 is optional, which can be replaced by other communication interfaces, for example, a network interface.

[0079] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so as to realize the functions of managing charging, discharging, and power consumption management through the power management system.

[0080] The terminal 100 also includes an external interface 180, which can be a standard Micro USB interface, or a multi-pin connector, and can be used to connect the terminal 100 and other devices for communication, or can be used to connect a charger to charge the terminal 100.

[0081] Although not shown, the terminal 100 can also include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, different function sensors, etc., which will not be described here. Some or all of the methods described below can be applied in the terminal 100 as shown. Figure 2

[0082] ​Next, the product form of the server 200 is described. Figure 1 The product form of the server 200 is described.

[0083] Figure 3 A structural diagram of the server 200 is provided, as shown in the figure. Figure 3 The server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate through the bus 201.

[0084] The bus 201 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 In the figure, only one thick line is used, but it does not mean that there is only one bus or one type of bus.

[0085] The processor 202 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0086] The memory 204 can include a volatile memory, such as a random access memory (RAM). The memory 204 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a mechanical hard drive (HDD), or a solid state drive (SSD).

[0087] The memory 204 can be used to store software code related to the monitoring method of the in-and-out cash behavior, and the processor 202 can execute the steps of the monitoring method of the in-and-out cash behavior of the chip, or can schedule other units to realize the corresponding functions.

[0088] It should be understood that the terminal 100 and the server 200 described above can be centralized or distributed devices, and the processors (for example, the processor 170 and the processor 202) in the terminal 100 and the server 200 can be hardware circuits (for example, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processing (DSP), a microprocessor, a microcontroller, or the like) or a combination of the hardware circuits, for example, the processor can be a hardware system with an instruction execution function, such as a CPU, a DSP, or the like, or a hardware system without an instruction execution function, such as an ASIC, an FPGA, or the like, or a combination of the hardware system without the instruction execution function and the hardware system with the instruction execution function.

[0089] Figure 4 A specific structure schematic diagram of a cash-in and cash-out behavior monitoring system provided by an embodiment of the present application is shown in the figure. The monitoring system includes a model training platform, an algorithm development module, a normal operation edge box, a cashbox monitoring camera, and a cloud platform. The normal operation edge box is deployed on a terminal device at the edge side to realize deployment and operation. The terminal device is connected to the cashbox monitoring camera pre-deployed in the cashbox door operation area of the cashbox. The cloud platform is deployed on a cloud server. The model training platform and the algorithm development module are deployed on a server (including but not limited to a cloud server) to realize a development method.

[0090] The model training platform is used to realize a data collection and processing module, a data labeling module, a model training module, and a model testing and format conversion module.

[0091] The data collection and processing module is used to collect a cashbox door image dataset and a personnel image dataset. The cashbox door image dataset includes cashbox door images in different states, and the personnel image dataset includes personnel images in different states.

[0092] The data labeling module is used to set a cashbox door state label set and a personnel label set, label the collected cashbox door state data based on the cashbox door state label set to obtain a cashbox door recognition training set, and label the personnel image dataset based on the personnel state label set to obtain a personnel recognition training set.

[0093] The model training module is used to train a machine learning model based on the cashbox door recognition training set to obtain a cashbox door recognition model, and train a machine learning model based on the personnel recognition training set to obtain a personnel recognition model.

[0094] The model testing and format conversion module is configured to: obtain a model testing result for the trained vault door recognition model and the personnel recognition model, and optimize the vault door recognition model and the personnel recognition model based on the model testing result until the model testing condition is met. The recognition model is converted into an onnx format that is common across platforms and is deployed on an edge box, so that the deployed recognition model can be converted and pruned according to the architecture platform of the terminal device on the edge side, and the computing performance of different hardware platforms is fully utilized.

[0095] The algorithm development module includes an access behavior analysis algorithm development module and a pre-warning rule judgment algorithm development module.

[0096] The access behavior analysis algorithm is configured to:

[0097] (1) Define the vault door area: divide the monitoring image of each vault door into regions, and map the vault door to a two-dimensional quadrilateral in the monitoring image, that is, a vault door planar quadrilateral.

[0098] (2) Establish a two-dimensional coordinate: establish a two-dimensional vault door coordinate system with the lowermost vertex of the vault door planar quadrilateral as the horizontal line and the leftmost vertex as the vertical line of the monitoring image, and obtain the two-dimensional vault door coordinate.

[0099] (3) Frame the personnel: obtain the rectangular frame position information of the target personnel output by the personnel recognition model, and determine whether the target personnel performs an access vault door action based on the rectangular frame position information and the vault door planar quadrilateral vertex coordinates.

[0100] Specifically, the rectangular frame position information of the target personnel is denoted as (x, y, w, h), where x and y are the coordinates of the top-left vertex, and w and h are the width and height of the rectangular frame, respectively.

[0101] Based on the rectangular frame position information and the vault door planar quadrilateral vertex coordinates, the method for determining whether the target personnel triggers an access vault door action specifically includes:

[0102] 1. Calculate the center point coordinates of the rectangular frame of the target personnel, map the center point coordinates to the two-dimensional vault door coordinate system, and determine whether the center point of the rectangular frame is within the vault door planar quadrilateral. If so, it is determined that the target personnel triggers an access vault door action.

[0103] 2. In order to avoid recognition errors caused by misplacement in the monitoring image or the irregular quadrilateral of the vault door in the monitoring image, the rectangular frame position information (x, y, w, h) of the target personnel is converted into a side coordinate set (x, x1+w, x+(w / 2), x+h+(w / 2)), as follows:

[0104] (x, y, w, h) -> x, x+w, x+(w / 2), x+h+(w / 2)

[0105] wherein x in the edge coordinate set is the horizontal coordinate of the left edge line of the rectangular frame, x+w is the horizontal coordinate of the right edge line of the rectangular frame, x+(w / 2) is the vertical coordinate of the upper edge line of the rectangular frame, and x+h+(w / 2) is the vertical coordinate of the lower edge line of the rectangular frame, each coordinate in the edge coordinate set is converted into a coordinate in the two-dimensional vault door coordinate system, and an edge coordinate set is obtained.

[0106] It is determined whether the horizontal coordinate of the left edge line of the rectangular frame, the horizontal coordinate of the right edge line of the rectangular frame, the vertical coordinate of the upper edge line of the rectangular frame, and the vertical coordinate of the lower edge line of the rectangular frame in the edge coordinate set are all within the quadrilateral of the vault door plane. If yes, it is considered that the target personnel triggers the two-dimensional plane of the vault door, that is, the target personnel triggers the action of entering or exiting the vault door.

[0107] The early warning rule determination algorithm is used to determine whether the gold warehouse access rules are met based on the vault door state information, personnel state information, and action determination results of each monitoring image in the continuous time image sequence. If not, an early warning is triggered, and early warning information is generated and sent to the cloud platform.

[0108] The normal operation edge box includes a stream data processing layer, an inference layer, a logic processing layer, and an early warning pushing layer.

[0109] The stream data processing layer is configured to receive the monitoring video stream of the vault door in real time through the channel of the vault monitoring camera, decode, frame, and pre-process the monitoring video stream, and convert the video stream into a video image meeting the model input requirements.

[0110] The inference layer is configured to simultaneously load the vault door recognition model and the personnel recognition model, trigger the vault door model to perform vault door recognition on the monitoring image to obtain a vault door recognition result, trigger the personnel recognition model to perform personnel recognition on the monitoring image when the vault door recognition result indicates that the opening and closing state of the vault door changes, and output a personnel recognition result. The vault door recognition result and the personnel recognition result are output to the logic processing layer.

[0111] The inference layer realizes the inference operation of multiple models through a time division multiplexing mechanism, divides the continuous time into time slices of a fixed length, and alternately runs the vault door recognition model and the personnel recognition model in adjacent time slices to achieve the purpose of reasonably utilizing the computing power. Simultaneously loading the two models into the inference layer of the box is beneficial to shorten the model switching time and reduce the performance loss of model switching.

[0112] The logic processing layer is configured to execute the deployed algorithm, perform behavior analysis and pre-warning rule judgment based on the vault door recognition result and the personnel recognition result output by the inference layer, obtain a vault access rule judgment result, and trigger a pre-warning, generate pre-warning information and send the pre-warning information to the cloud platform when the vault access rule is not met. The pre-warning information includes a pre-warning image, a vault access rule, a vault door recognition result, a personnel recognition result and a rule violation time, wherein the pre-warning image is obtained by drawing a vault door and a target personnel rectangular frame on a monitoring image.

[0113] The pre-warning pushing layer is configured to package and push the pre-warning information to the cloud platform, so that the cloud platform receives the pre-warning information sent by each edge box and processes and stores the pre-warning information, and sends the pre-warning information to a management personnel client through a short message, a telephone, an enterprise WeChat and the like.

[0114] The application embodiment provides a kind of access behavior of vault monitoring system, and the behavior of access vault door is monitored in real time by the edge box in terminal equipment deployed in edge side, and whether the behavior of access vault door is identified in real time whether it meets the rule of vault access, and the illegal behavior is alarmed in time. The training and optimization of identification model are realized by model training platform deployed in service end, and the model is issued to each terminal equipment, and the development of behavior analysis algorithm and pre-warning rule judgment algorithm is realized by algorithm development module deployed in service end, and the algorithm is deployed to each terminal equipment.

[0115] Therefore, the pre-deployed vault door recognition model and personnel recognition model are used to perform vault door recognition and personnel recognition on the monitoring image, improve the real-time performance of vault door recognition and personnel recognition, reduce the computing pressure of server by distributed deployment of edge side terminal, avoid leakage or malicious use of image sensitive information, and improve data security.

[0116] Figure 5 The specific implementation flowchart of the access behavior of vault monitoring method provided by the application embodiment is shown in Figure 5 The method specifically includes:

[0117] S501, obtain the original data set.

[0118] In this embodiment, the original data set includes a vault door image data set and a personnel image data set, wherein the vault door image data set includes vault door images in different states, and the personnel image data set includes personnel images in different states.

[0119] S502, data processing is performed on the original data set to obtain a training data set.

[0120] In this embodiment, the process of data processing on the original data set includes:

[0121] 1. Preprocess the data, wherein the data preprocessing includes preset preprocessing operations such as deduplication, standardization, and screening.

[0122] 2. Data labeling is performed on the preprocessed data to obtain a training data set.

[0123] In this embodiment, the training data set includes a vault door training data set and a personnel training data set.

[0124] The vault door image in the vault door image data set is labeled to obtain the vault door training data set, and the vault door labeling includes switch state labeling and vault door contour labeling, and the vault door contour labeling is the coordinates of each key point of the vault door. The personnel image in the personnel image data set is labeled to obtain the personnel training data set, and the personnel labeling includes personnel type and personnel position labeling, and the personnel position labeling is the vertex set of the personnel image bounding rectangle.

[0125] S503, train the machine learning model based on the training data set through the pre-built model training platform to obtain a vault door recognition model and a personnel recognition model, and deploy the vault door recognition model and the personnel recognition model to the edge box.

[0126] In this embodiment, the vault door recognition model is used to output a vault door recognition result, and the vault door recognition result includes a vault door opening and closing state and a vault door key point coordinate set. The personnel recognition model outputs a personnel recognition result, and the personnel recognition result includes a personnel type and personnel position information, and the personnel position information includes a personnel rectangle vertex coordinate set, or a target vertex coordinate and a length-width value.

[0127] In this embodiment, the vault door recognition model is trained based on the vault door training data set until the training completion condition is reached, and the model parameters of the vault door recognition model are obtained. The vault door recognition model is deployed on the edge box, the personnel recognition model is trained based on the personnel training data set until the training completion condition is reached, and the model parameters of the personnel recognition model are obtained. The personnel recognition model is deployed on the edge box.

[0128] In this embodiment, the edge box is configured on the terminal device on the edge side.

[0129] It should be noted that S501-S503 are executed by a server, the server can connect multiple bank systems, collect as much raw data as possible, rely on powerful computing power to improve the quality of raw data, obtain training data with large data volume and high quality, and improve the accuracy of the risk identification model.

[0130] S504, sequentially input the monitoring images in the monitoring image sequence to the vault door recognition model in the edge box to obtain the vault door recognition result of the monitoring image output by the vault door recognition model, and generate a vault door opening and closing state sequence.

[0131] In this embodiment, the monitoring images within a fixed time window range are frame extracted in real time to form a monitoring image sequence, that is, a monitoring image sequence, which includes monitoring images to be identified arranged in time sequence, and the monitoring image sequence is updated based on time and identification progress, and the monitoring images with unchanged door state are deleted. The monitoring images in the monitoring image sequence are sequentially input into the vault door identification model in the edge box to obtain the vault door identification result of the monitoring image output by the vault door identification model, and a vault door opening and closing state sequence is generated. The vault door opening and closing state sequence includes the vault door opening and closing state of the monitoring images arranged in sequence.

[0132] S505, based on the vault door opening and closing state sequence, determining that the vault door opening and closing state changes, and inputting the monitoring images in the monitoring image sequence into the personnel identification model in the edge box starting from the target monitoring image to obtain the personnel identification result output by the personnel identification model.

[0133] The target monitoring image is a monitoring image in which the vault door opening and closing state is determined to change. In this step, the triggering logic of the vault door identification model and the personnel identification model is configured to close the vault door identification result and start the personnel identification model after the vault door identification result indicates that the vault door opening and closing state changes, thereby improving the model detection efficiency, reducing the consumption of model calculation resources, and alternating the operation of the vault door identification model and the personnel identification model to achieve the purpose of reasonably utilizing the computing power.

[0134] S506, generating a vault door quadrilateral vertex coordinate set based on the vault door key point coordinate set.

[0135] In this embodiment, the vault door quadrilateral vertex coordinate set includes the vertex coordinates of the vault door planar contour. Optionally, the monitoring image is regionally divided, and the vault door is mapped as a two-dimensional quadrilateral in the monitoring image, that is, a vault door planar quadrilateral contour. The vault door quadrilateral vertex coordinate set is generated based on the vault door key point coordinate set.

[0136] S507, establishing a two-dimensional vault door coordinate system with a horizontal line passing through the lowermost vertex of the vault door planar quadrilateral and a vertical line passing through the leftmost vertex of the vault door planar quadrilateral as coordinate axes, and mapping the vertex coordinate set of the vault door planar quadrilateral to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set.

[0137] In this embodiment, the vault door two-dimensional coordinate set includes the coordinates of each vertex of the vault door planar quadrilateral in the two-dimensional vault door coordinate system.

[0138] S508, obtain personnel position information of the personnel monitoring image output by the personnel recognition model, obtain a side coordinate set of a personnel rectangle frame based on the personnel position information, map the side coordinate set to a two-dimensional vault door coordinate system, and obtain two-dimensional personnel position information.

[0139] In this embodiment, the personnel position information is denoted as (x, y, w, h), where x and y are the coordinates of the top left corner of the personnel rectangle frame, and w and h are the width and height of the rectangle frame, respectively. The horizontal side of the personnel rectangle frame is horizontal to the horizontal side of the monitoring image.

[0140] The personnel position information (x, y, w, h) is converted into a side coordinate set (x, x1+w, x+(w / 2), x+h+(w / 2)), as follows:

[0141] (x, y, w, h)→x, x+w, x+(w / 2), x+h+(w / 2)

[0142] where x in the side coordinate set is the horizontal coordinate of the left side of the rectangle frame, x+w is the horizontal coordinate of the right side of the rectangle frame, x+(w / 2) is the vertical coordinate of the top side of the rectangle frame, and x+h+(w / 2) is the vertical coordinate of the bottom side of the rectangle frame. Each coordinate in the side coordinate set is converted into a coordinate in the two-dimensional vault door coordinate system to obtain a two-dimensional side coordinate set, i.e., two-dimensional personnel position information.

[0143] S509, based on the side coordinate set and the two-dimensional coordinate set of the vault door, determine whether the target personnel triggers the access action of the vault door, and generate an action determination result.

[0144] In this embodiment, it is determined whether the horizontal coordinate of the left side of the rectangle frame, the horizontal coordinate of the right side of the rectangle frame, the vertical coordinate of the top side of the rectangle frame, and the vertical coordinate of the bottom side of the rectangle frame in the side coordinate set are all within the quadrilateral of the vault door plane. If yes, it is considered that the target personnel triggers the two-dimensional plane of the vault door, i.e., the target personnel triggers the access action of the vault door.

[0145] S510, based on the personnel recognition results of each personnel monitoring image in the sequence of continuous time images, and the sequence of action determination results, determine whether the pre-configured vault access rule is met, and obtain a rule determination result.

[0146] In this embodiment, the personnel monitoring image is the monitoring image after the target monitoring image, i.e., the monitoring image input to the personnel recognition result. The vault access rule at least includes a specified number of personnel, a specified type of personnel, and a specified time length. The specified time length refers to the maximum interval time length between the first personnel access door time and the last personnel access door time.

[0147] In this embodiment, the sequence of action determination results includes the action determination results and occurrence times of each monitoring image in sequence.

[0148] In this embodiment, if the type of the person who triggers the action of entering or exiting the vault door does not belong to the specified personnel type, it is determined that the rule of entering or exiting the vault is not met, and if it belongs, the person who triggers the action of entering or exiting the vault door is marked as a compliant person.

[0149] If the type of the person who triggers the action of entering or exiting the vault door belongs to the specified personnel type, it is determined whether the number of compliant personnel meets the specified number of personnel, and all the compliant personnel trigger the action of entering or exiting the vault door within the specified time length.

[0150] Optionally, whether the interval time from the first compliant person to the last compliant person is within the specified time length, if yes, it is determined that all the compliant personnel trigger the action of entering or exiting the vault door within the specified time length.

[0151] S511, if not, triggering an early warning, generating an early warning information and sending it to the cloud platform.

[0152] In this embodiment, the early warning information includes an early warning image, a vault entry and exit rule, a vault door identification result, a personnel identification result, and a rule violation time, wherein the early warning image is obtained by drawing a vault door and a target personnel rectangular frame on a monitoring image.

[0153] S512, when reaching a preset personnel identification time length threshold or the rule determination result is not meeting the vault entry and exit rule, closing the personnel identification model and starting the vault door identification model.

[0154] As can be seen from the above technical solution, the application provides a kind of monitoring method of entering and exiting vault behavior, which utilizes pre-deployed vault door identification model and personnel identification model to identify vault door and personnel in monitoring image, and carries out rule determination based on vault door identification result and personnel identification result, and when not compliant, early warning is issued, real-time monitoring and real-time alarm are achieved, which can significantly reduce the risk control risk and labor cost of manual supervision in this scene, realize to reduce time loss and solve the problem of hysteresis. Through distributed deployment of edge side terminal, the computing pressure of server is reduced, and the leakage or malicious use of image sensitive information is avoided, and the data security is improved.

[0155] Further, this multi-model switching method ensures that only one identification model runs at each time, reduces the computing load of the device, and improves the operation efficiency, while loading the vault door identification model and the personnel identification model into the edge box, which is beneficial to shorten the model switching time and reduce the performance loss of model switching.

[0156] Further, a two-dimensional model of the vault door is established, the three-dimensional contact surface of the vault door is mapped in two dimensions, and the detection of the three-dimensional opening, entering and closing actions of the vault door is realized in two dimensions. The action of personnel entering and exiting the vault door is recognized and marked to detect the number and time of personnel entering and exiting the vault door. A two-dimensional coordinate is established based on the contact surface of the vault door, and the target detection frame of the personnel is converted to the two-dimensional coordinate for calculation to improve the recognition accuracy of the action of personnel entering and exiting the vault door and reduce the running load.

[0157] It should be noted that the above is only an optional specific implementation process of the monitoring method of the behavior of entering and exiting the vault provided by the embodiment, and the application can also be implemented by other specific implementations, for example, the monitoring image is simultaneously input into the vault door recognition model and the personnel recognition model in the edge box to obtain the vault door recognition result output by the vault door recognition model and the personnel recognition result output by the personnel recognition model. For another example, the model training platform can be built on a server or a terminal device.

[0158] In summary, Figure 6 A flowchart of a monitoring method of a behavior of entering and exiting a vault provided by an embodiment of the application is shown in Figure 6 As shown, the method is applied to a terminal, and the method comprises the following steps:

[0159] S601, sequentially input monitoring images in a real-time monitoring image sequence into a vault door recognition model to obtain a vault door recognition result output by the vault door recognition model.

[0160] In the embodiment, the vault door recognition result comprises a vault door opening and closing state and a vault door key point coordinate set.

[0161] In the embodiment, the monitoring images within a fixed time window range are frame-extracted in real time to form a monitoring image sequence, that is, a monitoring image sequence, which comprises monitoring images to be recognized arranged in time sequence, and the monitoring image sequence is updated based on time and recognition progress. The vault door opening and closing state sequence comprises the vault door opening and closing states of the monitoring images arranged in sequence.

[0162] It should be noted that the vault door opening and closing state in the vault door recognition result specifically comprises a vault door closing state or an opening degree.

[0163] S602, if a change in the vault door opening and closing state is detected, sequentially input monitoring images in a real-time monitoring image sequence into a personnel recognition model starting from a target monitoring image to obtain a personnel recognition result output by the personnel recognition model.

[0164] In the embodiment, the personnel recognition result comprises a personnel type and personnel position information, and the target monitoring image is the monitoring image in which the vault door opening and closing state changes.

[0165] In this embodiment, the vault door recognition model and the personnel recognition model are obtained by training a machine learning model based on a training data set by the server.

[0166] S603, based on the target monitoring image, a two-dimensional coordinate system in the vault door plane is constructed as a two-dimensional vault door coordinate system.

[0167] S604, the vault door key point coordinate set of the target monitoring image is mapped to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set.

[0168] S605, for each personnel monitoring image, the personnel position information of the personnel monitoring image is mapped to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information.

[0169] In this embodiment, the personnel monitoring image is the monitoring image input into the personnel recognition model.

[0170] S606, for each personnel monitoring image, based on the vault door two-dimensional coordinate set and the two-dimensional personnel position information, an action judgment result is generated.

[0171] In this embodiment, the action judgment result indicates whether the target personnel in the personnel monitoring image triggers the access vault door action.

[0172] S607, based on the personnel recognition result of each personnel monitoring image and the action determination result sequence, it is determined whether it conforms to the preconfigured vault access rule.

[0173] In this embodiment, the action determination result sequence includes the action judgment results of the personnel monitoring images arranged in time sequence.

[0174] In this embodiment, the vault access rule at least includes a specified number of personnel, a specified type of personnel, and a specified time length. The specified time length refers to the maximum interval time length between the first personnel access door time and the last personnel access door time.

[0175] In an optional embodiment, if the personnel type that triggers the access vault door action does not belong to the specified personnel type, it is determined that the vault access rule is not met, and if it belongs, the personnel that triggers the access vault door action is marked as a compliant personnel. If the personnel type that triggers the access vault door action belongs to the specified personnel type, it is determined whether the number of compliant personnel meets the specified number of personnel, and the compliant personnel all trigger the access vault door action within the specified time length. Optionally, the interval time length from the first compliant personnel to the last compliant personnel is within the specified time length, and if so, it is determined that the compliant personnel all trigger the access vault door action within the specified time length.

[0176] S608, if the vault access rule is not met, an early warning information is sent out.

[0177] As can be seen from the above technical solutions, the method for monitoring vault entry and exit behavior provided in this application embodiment utilizes a pre-deployed vault door recognition model and personnel recognition model to perform vault door recognition and personnel recognition on the monitoring images. Based on the target monitoring image, a two-dimensional coordinate system in the plane of the vault door is constructed as the two-dimensional vault door coordinate system. Based on the set of two-dimensional coordinates of the vault door and the two-dimensional personnel position information, an action judgment result is generated. Based on the personnel recognition result and action judgment result sequence of each of the personnel monitoring images, it is determined whether it conforms to the pre-configured vault entry and exit rules. If it does not conform, an early warning is issued. This solution improves the real-time performance of vault entry and exit behavior monitoring and improves the accuracy of action judgment through two-dimensional mapping.

[0178] It should be noted that the method and related apparatus for monitoring vault entry and exit behavior provided by this invention can be used in the fields of artificial intelligence or finance. The above are merely examples and do not limit the application areas of the method and related apparatus for monitoring vault entry and exit behavior provided by this invention.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0180] The above describes a method for monitoring entry and exit behavior of a vault provided by an embodiment of this application. The following will describe the apparatus for performing the above-described method for monitoring entry and exit behavior of a vault. Figure 7 A schematic diagram of a monitoring device for monitoring entry and exit from a vault, provided in an embodiment of this application, is shown below. Figure 7 As shown, this device includes:

[0181] The vault door recognition unit 701 is used to sequentially input the monitoring images in the real-time monitoring image sequence into the vault door recognition model to obtain the vault door recognition result output by the vault door recognition model; the vault door recognition result includes the vault door opening / closing status and the set of key point coordinates of the vault door;

[0182] The personnel identification unit 702 is used to input monitoring images from the real-time monitoring image sequence sequentially into the personnel identification model, starting from the target monitoring image, to obtain the personnel identification result output by the personnel identification model. The personnel identification result includes personnel type and personnel location information. The target monitoring image is a monitoring image showing a change in the opening and closing state of the vault door. Both the vault door identification model and the personnel identification model are obtained by the server training a machine learning model based on a training dataset.

[0183] a two-dimensional coordinate system construction unit 703, configured to construct a two-dimensional coordinate system in a vault door plane as a two-dimensional vault door coordinate system based on the target monitoring image;

[0184] a two-dimensional coordinate mapping unit 704, configured to map a vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set; for each personnel monitoring image, map personnel position information of the personnel monitoring image to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, the personnel monitoring image being a monitoring image input into a personnel recognition model;

[0185] an action judgment unit 705, configured to, for each personnel monitoring image, generate an action judgment result based on the vault door two-dimensional coordinate set and the two-dimensional personnel position information, the action judgment result indicating whether a target personnel in the personnel monitoring image triggers an access vault door action;

[0186] a rule judgment unit 706, configured to judge whether a preconfigured vault access rule is met based on a personnel recognition result of each personnel monitoring image and a sequence of action judgment results, the sequence of action judgment results including the action judgment results of the personnel monitoring images arranged in time sequence;

[0187] a warning unit 707, configured to issue a warning information if the vault access rule is not met.

[0188] Optionally, the monitoring device of the access vault behavior further includes a switch state judgment unit, configured to update a vault door switch state sequence after obtaining each vault door recognition result output by the vault door recognition model, the vault door switch state sequence including vault door switch states of each monitoring image arranged in time sequence; and determine whether a vault door switch state changes based on the vault door switch state sequence.

[0189] Optionally, the two-dimensional coordinate system construction unit is specifically configured to, when constructing a two-dimensional coordinate system in a vault door plane as a two-dimensional vault door coordinate system based on the target monitoring image:

[0190] generate a vault door vertex coordinate set based on the vault door key point coordinate set;

[0191] establish a two-dimensional coordinate system in the vault door plane as the two-dimensional vault door coordinate system with a horizontal line passing through a lowermost vertex in the vault door vertex coordinate set and a vertical line passing through a leftmost vertex in the vault door vertex coordinate set as coordinate axes;

[0192] Optionally, the two-dimensional coordinate mapping unit is used to map the set of key point coordinates of the vault door in the target monitoring image to the two-dimensional vault door coordinate system. Specifically, when obtaining the two-dimensional coordinate set of the vault door, it is used for:

[0193] Map the coordinates of each vertex in the set of vault door vertex coordinates to the two-dimensional vault door coordinate system to obtain the two-dimensional coordinate set of the vault door.

[0194] Optionally, the two-dimensional coordinate mapping unit is used to map the personnel location information of the personnel monitoring image to the two-dimensional vault door coordinate system. Specifically, when obtaining the two-dimensional personnel location information, it is used for:

[0195] Based on the personnel location information, a set of side coordinates of the personnel rectangle is obtained, the set of side coordinates including the ordinate of each horizontal side and the abscissa of each vertical side of the personnel rectangle;

[0196] The coordinates of the personnel rectangle are mapped to the coordinate system of the two-dimensional vault door to obtain the two-dimensional personnel location information.

[0197] Optionally, when the action judgment unit generates the action judgment result based on the two-dimensional coordinate set of the vault door and the two-dimensional personnel position information, it specifically uses the following methods:

[0198] It is determined that the rectangle formed by the coordinates of the two-dimensional personnel location information is within the outline formed by the vertices of the two-dimensional coordinate set of the vault door;

[0199] If so, then it is confirmed that the person triggered the action of entering or leaving the vault door.

[0200] Optionally, the monitoring device for entering and exiting the vault also includes a model switching unit for:

[0201] Load the vault door recognition model and the personnel recognition model, and set the initial state of the vault door recognition model and the personnel recognition model to off;

[0202] If a change in the vault door's open / closed state is detected, the vault door recognition model is turned off, and the personnel recognition model is turned on.

[0203] When the preset time threshold is reached or the rule judgment result is not met, the personnel recognition model is turned off and the vault door recognition model is turned on.

[0204] This application also provides an electronic device in its embodiments. (See reference...) Figure 8 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc.Figure 8 The electronic device shown is merely one example, and should not be taken as limiting the functionality or use of embodiments of the application.

[0205] As shown in Figure 8 The electronic device can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded into a random access memory (RAM) 803 from a storage device 808. In a state where the electronic device is powered on, the RAM 803 also stores various programs and data required for operation of the electronic device. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0206] Generally, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a memory card, a hard disk, etc.; and communication devices 809. The communication devices 809 can allow the electronic device to communicate wirelessly or wired with other devices to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that all of the shown devices are not required to implement or have the electronic device. More or less devices can alternatively be implemented or have.

[0207] The embodiments of the present application also provide a computer program product, including computer readable instructions, which, when executed on an electronic device, cause the electronic device to implement any one of the monitoring methods of cash deposit and withdrawal behaviors provided by the embodiments of the present application.

[0208] The embodiments of the present application also provide a computer readable storage medium, which carries one or more computer programs, which, when executed by an electronic device, can cause the electronic device to implement any one of the monitoring methods of cash deposit and withdrawal behaviors provided by the embodiments of the present application.

[0209] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separate, and the units displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0210] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and the necessary general hardware, and of course can also be realized by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily realized by corresponding hardware, and the specific hardware structure for realizing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products, which are stored in readable storage media, such as computer floppy disks, U disks, mobile hard disks, ROM, RAM, magnetic or optical disks, etc., including a plurality of instructions for making a computer device (which can be a personal computer, a training device, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0211] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of a computer program product in whole or in part.

[0212] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

Claims

1. A method of monitoring access to a vault, comprising: The application is applied to a terminal and comprises: sequentially inputting monitoring images in a real-time monitoring image sequence into a vault door recognition model to obtain a vault door recognition result output by the vault door recognition model; the vault door recognition result comprises a vault door opening and closing state and a vault door key point coordinate set; if a change in the vault door opening and closing state is detected, sequentially inputting monitoring images in a real-time monitoring image sequence into a personnel recognition model starting from a target monitoring image to obtain a personnel recognition result output by the personnel recognition model, the personnel recognition result comprising a personnel type and personnel position information, the target monitoring image being a monitoring image in which the vault door opening and closing state changes; wherein the vault door recognition model and the personnel recognition model are obtained by training a machine learning model based on a training data set by a server; based on the target monitoring image, constructing a two-dimensional coordinate system in a vault door plane as a two-dimensional vault door coordinate system; mapping the vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set; for each personnel monitoring image, mapping the personnel position information of the personnel monitoring image to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, the personnel monitoring image being a monitoring image input into the personnel recognition model; for each personnel monitoring image, based on the vault door two-dimensional coordinate set and the two-dimensional personnel position information, generating an action judgment result, the action judgment result indicating whether a target person in the personnel monitoring image triggers an access vault door action; based on the personnel recognition result of each personnel monitoring image and a sequence of action judgment results, determining whether a preconfigured vault access rule is met, the sequence of action judgment results comprising action judgment results of the personnel monitoring images arranged in time sequence; if the vault access rule is not met, issuing a warning information.

2. The method of claim 1, wherein The monitoring method of the access vault behavior further comprises: after obtaining each vault door recognition result output by the vault door recognition model, updating a vault door opening and closing state sequence, the vault door opening and closing state sequence comprising vault door opening and closing states of each monitoring image arranged in time sequence; based on the vault door opening and closing state sequence, determining whether a change in the vault door opening and closing state occurs.

3. The method of claim 1, wherein the method further comprises: The method of constructing a two-dimensional coordinate system in a vault door plane based on the target monitoring image as a two-dimensional vault door coordinate system comprises: based on the vault door key point coordinate set, generating a vault door vertex coordinate set; establishing a two-dimensional coordinate system in the vault door plane as the two-dimensional vault door coordinate system with a horizontal line passing through the lowermost vertex in the vault door vertex coordinate set and a vertical line passing through the leftmost vertex in the vault door vertex coordinate set as coordinate axes; mapping the vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain a vault door two-dimensional coordinate set, comprising: mapping each vertex coordinate in the vault door vertex coordinate set to the two-dimensional vault door coordinate system to obtain the vault door two-dimensional coordinate set.

4. The method of claim 3, wherein the step of monitoring the access to the vault is performed by a computer system. The personnel position information of the personnel monitoring image is mapped to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, including: Based on the personnel position information, a side coordinate set of a personnel rectangular frame is obtained, the side coordinate set including the ordinate of each horizontal side of the personnel rectangular frame and the abscissa of each vertical side; The side coordinate set of the personnel rectangular frame is mapped to the two-dimensional vault door coordinate system to obtain the two-dimensional personnel position information.

5. The method of claim 4, wherein the step of monitoring the access to the vault is performed by a computer system. The action judgment result is generated based on the two-dimensional vault door coordinate set and the two-dimensional personnel position information, including: It is judged that the rectangle surrounded by each coordinate in the two-dimensional personnel position information is within the contour surrounded by each vertex in the two-dimensional vault door coordinate set. If yes, it is determined that the personnel triggers the action of entering or exiting the vault door.

6. The method of claim 1, wherein The monitoring method of the entering and exiting vault behavior further includes: The initial state of the vault door recognition model and the personnel recognition model is set to be closed by loading the vault door recognition model and the personnel recognition model; If it is detected that the vault door switching state changes, the personnel recognition model is closed and the vault door recognition model is opened. When a preset time threshold is reached or the rule judgment result is not in conformity, the personnel recognition model is closed and the vault door recognition model is opened.

7. A monitoring device for entering and exiting a vault, characterized in that, It includes: The vault door recognition unit is used for sequentially inputting the monitoring image in the real-time monitoring image sequence into the vault door recognition model to obtain the vault door recognition result output by the vault door recognition model; the vault door recognition result includes the vault door opening and closing state and the vault door key point coordinate set; The personnel recognition unit is used for sequentially inputting the monitoring image in the real-time monitoring image sequence into the personnel recognition model from the target monitoring image to obtain the personnel recognition result output by the personnel recognition model, the personnel recognition result including the personnel type and the personnel position information, and the target monitoring image being the monitoring image in which the vault door opening and closing state changes; wherein the vault door recognition model and the personnel recognition model are obtained by training a machine learning model based on a training data set by a server; The two-dimensional coordinate system construction unit is used for constructing a two-dimensional coordinate system in the vault door plane as a two-dimensional vault door coordinate system based on the target monitoring image; The two-dimensional coordinate mapping unit is used for mapping the vault door key point coordinate set of the target monitoring image to the two-dimensional vault door coordinate system to obtain the two-dimensional vault door coordinate set; for each personnel monitoring image, the personnel position information of the personnel monitoring image is mapped to the two-dimensional vault door coordinate system to obtain two-dimensional personnel position information, and the personnel monitoring image is the monitoring image input into the personnel recognition model; The action judgment unit is used for generating an action judgment result based on the two-dimensional vault door coordinate set and the two-dimensional personnel position information for each personnel monitoring image, the action judgment result indicating whether the target personnel in the personnel monitoring image triggers the action of entering or exiting the vault door. A rule judging unit is configured to judge whether the preconfigured in-and-out vault rule is met based on the personnel recognition result of each personnel monitoring image and a sequence of action determination results of the personnel monitoring image, the sequence of action determination results including the action determination results of the personnel monitoring image arranged in time sequence; An early warning unit is configured to issue a pre-warning information if the in-and-out vault rule is not met.

8. A computer program product, characterised in that, The computer readable instructions, when executed on an electronic device, cause the electronic device to implement the in-and-out vault behavior monitoring method according to any one of claims 1 to 6.

9. An electronic device, comprising: The memory is configured to store a computer program; The processor is configured to execute the computer program to enable the electronic device to implement the in-and-out vault behavior monitoring method according to any one of claims 1 to 6. The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the in-and-out vault behavior monitoring method according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that ​

Images