Secret-related area photographing behavior detection method and related device

Abstract

The invention discloses a secret-related area photographing behavior detection method and a related device, which can be applied to the field of artificial intelligence or finance, after a monitoring image of a secret-related area is acquired, whether a person uses a mobile phone in the monitoring image is identified by calling a target detection model, and when the person uses the mobile phone in the monitoring image, the photographing behavior of the secret-related area is detected. A target tracking model which is lighter than a target detection model is adopted to carry out target tracking on a target person using a mobile phone, a video clip of the target person is obtained, and then a video behavior recognition model related to a time sequence is called to accurately recognize whether the target person in the video clip has a mobile phone photographing behavior or not. The detection efficiency and accuracy of the photographing behavior in the secret-related area are improved, and the missing report rate of the photographing behavior in the secret-related area is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for detecting photo-taking behavior in confidential areas and related devices. Background Art

[0002] Banks contain many confidential areas such as safes, confidential meeting rooms, and back-end transaction areas. If office staff use mobile phones in these confidential areas or even take photos, it may cause the bank's important internal privacy information and user sensitive information to be leaked, leading to production safety problems.

[0003] Currently, violations such as using mobile phones and taking photos in confidential areas of banks are mainly detected through personnel inspections or surveillance spot checks. This method is inefficient and has a high rate of missed reports. Summary of the invention

[0004] In view of the above problems, this application provides a method and related device for detecting photo-taking behavior in confidential areas, which improves the efficiency and accuracy of detecting photo-taking behavior in confidential areas. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for detecting photo-taking behavior in a confidential area, comprising:

[0006] Obtain surveillance images of confidential areas;

[0007] Calling a target detection model to identify whether a person is using a mobile phone in the surveillance image, wherein the target detection model is trained using image training samples annotated with target categories and target recognition frames, wherein the target categories include person targets and mobile phone targets;

[0008] If a person is using a mobile phone in the surveillance image, target tracking is performed on the target person using the mobile phone to obtain a video clip of the target person;

[0009] A video behavior recognition model is called to identify whether the target person in the video clip has taken photos with a mobile phone. The video behavior recognition model is trained using video training samples labeled with behavior categories, and the behavior categories include using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

[0010] In a possible implementation, tracking a target person using a mobile phone to obtain a video clip of the target person includes:

[0011] Tracking the target person to obtain an image including a bounding box corresponding to the target person;

[0012] In the image, the bounding box is enlarged by a preset multiple, and the enlarged bounding box is cut to obtain a sub-image of the target person;

[0013] Saving the sub-image to a buffer;

[0014] When the number of the sub-images in the buffer reaches a threshold, the sub-images in the buffer are integrated into a video clip of the target person.

[0015] In a possible implementation, after the sub-image is saved in a buffer, the method for detecting the behavior of taking photos in a confidential area further includes:

[0016] Performing flash recognition on each of the sub-images in the buffer;

[0017] If the flash is identified, it is determined that the target person is taking photos with a mobile phone.

[0018] In a possible implementation, performing flash recognition on each sub-image in the buffer includes:

[0019] For each of the sub-images, convert the sub-image into a grayscale image;

[0020] Performing gradient calculation on the grayscale image to obtain a gradient image corresponding to the grayscale image;

[0021] Binarizing the gradient image to obtain a binary image;

[0022] If there is a white area in the binary image that meets the preset range, it is determined that a flash is recognized in the sub-image.

[0023] In a possible implementation, the method for detecting the behavior of taking photos in a confidential area further includes:

[0024] If anyone is using a mobile phone in the surveillance image, a first-level warning is triggered.

[0025] In a possible implementation, the method for detecting the behavior of taking photos in a confidential area further includes:

[0026] If the target person takes photos with a mobile phone, a second-level warning will be triggered.

[0027] The second aspect of the present application provides a device for detecting photo-taking behavior in a confidential area, comprising:

[0028] An image acquisition unit, used to acquire surveillance images of confidential areas;

[0029] A target detection unit, used to call a target detection model to identify whether there is a person using a mobile phone in the monitoring image, wherein the target detection model is trained using image training samples marked with target categories and target recognition frames, wherein the target categories include person targets and mobile phone targets;

[0030] A target tracking unit, configured to track a target person using a mobile phone if a person is using a mobile phone in the surveillance image, and obtain a video clip of the target person;

[0031] A behavior recognition unit is used to call a video behavior recognition model to identify whether the target person in the video clip has taken photos with a mobile phone. The video behavior recognition model is trained using video training samples marked with behavior categories, and the behavior categories include using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

[0032] A third aspect of the present application provides an edge computing device, comprising at least one processor and a memory connected to the processor, wherein:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program so that the electronic device can implement the method for detecting photo-taking behavior in a confidential area according to the first aspect or any implementation method of the first aspect.

[0035] The fourth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an edge computing device, the edge computing device implements the method for detecting photographing behavior in a confidential area according to the first aspect or any implementation method of the first aspect.

[0036] The fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an edge computing device, the edge computing device can implement the method for detecting photo-taking behavior in confidential areas according to the first aspect or any implementation method of the first aspect.

[0037] Through the above-mentioned technical scheme, the present application provides a method and related devices for detecting photographing behavior in confidential areas. After obtaining the surveillance image of the confidential area, the target detection model is called to identify whether there is a person using a mobile phone in the surveillance image. When there is a person using a mobile phone in the surveillance image, a target tracking model that is more lightweight than the target detection model is used to track the target person using the mobile phone to obtain a video clip of the target person, and then the video behavior recognition model related to the time series is called to accurately identify whether the target person in the video clip has a mobile phone photographing behavior, thereby improving the efficiency and accuracy of photographing behavior detection in confidential areas and reducing the underreporting rate of photographing behavior in confidential areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0039] Figure 1 A system architecture diagram provided for an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the structure of an edge computing device provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of a cloud platform provided in an embodiment of the present application;

[0042] Figure 4 A flowchart of a method for detecting photo-taking behavior in a confidential area provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of a target detection model structure provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of a video behavior recognition model structure provided in an embodiment of the present application;

[0045] Figure 7 A flowchart of another method for detecting photo-taking behavior in a confidential area provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the structure of another device for detecting photo-taking behavior in confidential areas provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following description, in order to ensure the accuracy of the citation and the fluency of reading, the key technical terms, abbreviations or abbreviations involved in the text are summarized and explained as follows:

[0048] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0049] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include at least one edge computing device 100, a cloud platform 200, and a surveillance camera 300. The cloud platform 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0052] The cloud platform 200 remotely communicates with at least one edge computing device 100, and the edge computing device 100 is communicatively connected with at least one surveillance camera 300 in the confidential area.

[0053] Among them, an application can be installed on the edge computing device 100. The edge computing device 100 can trigger a first-level warning when it detects that someone is using a mobile phone in a confidential area and send a first-level warning information to the cloud platform 200. The edge computing device 100 can also trigger a second-level warning when it detects that someone is taking photos with a mobile phone in a confidential area and send a second-level warning information to the cloud platform 200. The cloud platform 200 can issue a configuration file or software upgrade instruction to the edge computing device 100. The edge computing device 100 can update the detection frequency, etc. according to the configuration file, and can also respond to the software upgrade instruction and perform software upgrades based on the upgrade package issued by the cloud platform 200.

[0054] It should be understood that in some optional implementations, the edge computing device 100 can also complete the action of obtaining the processing result based on the received parameters (configuration information, upgrade instructions, etc.) by itself, or directly send warning information to the terminal held by the target manager without the need for the cooperation of the cloud platform 200, and the embodiments of the present application are not limited.

[0055] Next describe Figure 1 The product form of the edge computing device 100;

[0056] The edge computing device 100 in the embodiment of the present application can be a network device with edge computing function (such as a smart router, a smart switch, a smart gateway, etc.), a smart camera, an edge server, a PLC (programmable logic controller), a microcontroller, a mobile device (such as a smart phone, a tablet computer, etc.), etc., and the embodiment of the present application does not impose any restrictions on this.

[0057] like Figure 2 As shown, the edge computing device 100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 to a random access memory (RAM) 103. When the edge computing device 100 is powered on, the RAM 103 also stores various programs and data required for the operation of the edge computing device 100. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0058] Typically, the following devices may be connected to the I / O interface 105: an input device 106, an output device 107, a storage device 108 including, for example, a memory card, a hard disk, etc., and a communication device 109. The communication device 109 may allow the edge computing device 100 to communicate with other devices wirelessly or by wire to exchange data. Figure 2 The edge computing device 100 having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0059] Next describe Figure 1 The product form of Zhongyun Platform 200;

[0060] Figure 3 A schematic diagram of the structure of a cloud platform 200 is provided, such as Figure 3 As shown, the cloud platform 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0061] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0062] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0063] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a nonvolatile memory, such as a read only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0064] Among them, the memory 204 can be used to store software codes related to the method for detecting photographing behavior in confidential areas, the processor 202 can execute relevant steps of the method for detecting photographing behavior in confidential areas, and can also schedule other units to implement corresponding functions.

[0065] It should be understood that the above-mentioned edge computing device 100 and cloud platform 200 can be centralized or distributed devices, and the processors in the above-mentioned edge computing device 100 and cloud platform 200 can be hardware circuits (such as application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), general-purpose processors, digital signal processors (digital signal processing, DSP), microprocessors or microcontrollers, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0066] The embodiment of the present application provides a method for detecting the behavior of taking photos in a confidential area, which is applied to an edge computing device. The method for detecting the behavior of taking photos in a confidential area in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0067] Reference Figure 4 , Figure 4 The process diagram of a method for detecting photo-taking behavior in a confidential area provided in an embodiment of the present application is as follows: Figure 4 As shown, a method for detecting photo-taking behavior in a confidential area provided in an embodiment of the present application may include steps 401 to 404, and these steps are described in detail below.

[0068] 401: Obtain surveillance images of confidential areas;

[0069] Confidential areas may include safes, confidential meeting rooms, back-end transaction areas, etc.

[0070] The edge computing device is connected to at least one surveillance camera in the confidential area to obtain surveillance video data of the confidential area collected by the surveillance camera. The surveillance video data is pre-processed by extracting frames according to a preset frame rate to obtain a surveillance image.

[0071] 402: Calling the target detection model to identify whether there is a person using a mobile phone in the surveillance image. The target detection model is trained using image training samples labeled with target categories and target recognition frames. The target categories include human targets and mobile phone targets.

[0072] Exemplarily, the training method of the target detection model includes the following steps A1-A4:

[0073] A1: Collect surveillance video data of confidential areas, including video data of people using mobile phones and taking photos with mobile phones.

[0074] A2: De-frame and deduplicate the collected video data to obtain image data.

[0075] A3: Label the image data to obtain image training samples.

[0076] Specifically, the target recognition frame and target category of the image data are annotated, the target category includes a person target and a mobile phone target, and the target recognition frame includes a human body detection frame and a mobile phone detection frame.

[0077] A4: Use image training samples to train the model.

[0078] For example, the YOLOv5 target detection model is used for training. Figure 5The YOLOv5 network structure shown in the figure, YOLOv5 uses the backbone network to extract multi-scale features, performs multi-scale feature fusion in the bottleneck part, and finally performs target detection at multiple scales, outputting information such as the target category, target recognition box, and confidence of each identified target. This embodiment uses the Adam optimizer for optimization training, and the learning rate can be set to 0.001, the batch size to 64, and a total of 500 rounds of training convergence.

[0079] The input data of the target detection model is the surveillance image, and the output data is the target category, target recognition box, confidence and other information of the identified target. According to the output data, it is determined whether there is a person using a mobile phone in the surveillance image. For example, if the confidence of the person target is greater than the threshold and the confidence of the mobile phone target is greater than the threshold, it is determined that there is a person using a mobile phone in the surveillance image.

[0080] 403: If there is a person using a mobile phone in the surveillance image, target tracking is performed on the target person using the mobile phone to obtain a video clip of the target person;

[0081] If no person is using a mobile phone in the surveillance image, then continue to execute step 401 .

[0082] If there is a person using a mobile phone in the surveillance image, a target tracking model that is more lightweight than the target detection model is used to track the target person using the mobile phone, such as using the DeepSort multi-target tracking model for target tracking.

[0083] Furthermore, if there is a person using a mobile phone in the surveillance image, stop calling the target detection model to identify whether there is a person using a mobile phone in the surveillance image, thereby reducing the computational workload of the edge computing device until it is determined that the target person has taken a photo with a mobile phone or the target person cannot be tracked, and then start calling the target detection model to identify whether there is a person using a mobile phone in the surveillance image.

[0084] 404: Call the video behavior recognition model to identify whether the target person in the video clip is taking photos with a mobile phone. The video behavior recognition model is trained using video training samples labeled with behavior categories. The behavior categories include using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

[0085] Exemplarily, the training method of the video behavior recognition model includes the following steps B1-B4:

[0086] B1: Collect surveillance video data of confidential areas, including video data of people using mobile phones and taking photos with mobile phones.

[0087] B2: Cut the video data to obtain video clip data taken with a mobile phone.

[0088] B3: For each video clip data, extract the video clip data at a sampling rate of 30 frames per second to obtain the video non-key frames, and extract the video key frames at a sampling rate of 1 frame per second, and annotate each key frame with the behavior category, which includes using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior, to obtain video training samples. Specifically, it is necessary to use a pre-trained target detection model to pre-annotate these key frames for personnel detection, and then manually annotate all detected personnel identification frames to see whether they are one of the three behavior categories: using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

[0089] B4: Use video training samples to train the model.

[0090] For example, the SlowFast behavior recognition network model is used for training. Figure 6 The SlowFast network model structure shown in the figure uses a dual-stream structure to extract video information. Figure 6 The slow branch above inputs low frame rate video frames to capture the video spatial semantic information; Figure 6 The fast branch below inputs high frame rate video frames to capture motion information in the video. The two branches will fuse feature information at different feature extraction stages, and finally output the behavior recognition prediction results contained in the video. This embodiment uses the stochastic gradient descent algorithm to optimize the training of the SlowFast model, and the learning rate can be set to 0.01. For the slow branch, one frame is drawn as input for every 15 frames of the video training sample, all of which are key frames; for the fast branch, one frame is drawn as input for every 2 frames of the video training sample, including both key frames and non-key frames, and a total of 100 rounds of training are required to complete convergence.

[0091] The input data of the video behavior recognition model are low-frame rate video frames and high-frame rate video frames of the video clips of the target person. The frame extraction method is the same as the frame extraction method of the video training samples. The output data of the video behavior recognition model is the behavior prediction result, that is, one of the following: using a mobile phone but not taking pictures with a mobile phone, taking pictures with a mobile phone, and normal behavior.

[0092] Since the video behavior recognition model can capture both the spatial semantic information of the video and the motion information in the video, it is more accurate than recognizing the photo-taking behavior through a single frame image.

[0093] The present embodiment discloses a method for detecting photographing behavior in confidential areas. After acquiring a surveillance image of the confidential area, a target detection model is called to identify whether there is a person using a mobile phone in the surveillance image. When there is a person using a mobile phone in the surveillance image, a target tracking model that is more lightweight than the target detection model is used to track the target person using the mobile phone to obtain a video clip of the target person. Then, a video behavior recognition model related to a time series is called to accurately identify whether the target person in the video clip has a mobile phone photographing behavior, thereby improving the efficiency and accuracy of photographing behavior detection in confidential areas and reducing the underreporting rate of photographing behavior in confidential areas.

[0094] In a possible implementation, in step 103 of the above embodiment, tracking the target person using the mobile phone to obtain the video clip of the target person includes the following steps 1031-1034:

[0095] 1031: Track the target person to obtain an image including a bounding box corresponding to the target person;

[0096] Specifically, the target tracking model is called to track the target person, the input data of the target tracking model is the monitoring image, and the output data is an image including a bounding box corresponding to the target person.

[0097] 1032: enlarging the bounding box in the image by a preset multiple, and cutting the enlarged bounding box to obtain a sub-image of the target person;

[0098] Since the bounding box may not cover the entire target person, such as only framing part of the target person, in order to improve the accuracy of subsequent video behavior recognition, the bounding box is enlarged by a preset multiple in the image, such as enlarging the bounding box by 1.5 times to ensure that the target person is fully framed.

[0099] 1033: Save the sub-image obtained by cutting into a buffer;

[0100] The buffer continuously stores sub-images of the target person using a mobile phone, and if the buffer is full, the oldest sub-image is replaced by the latest sub-image.

[0101] 1034: When the number of sub-images in the buffer reaches a threshold, the sub-images in the buffer are integrated into a video clip of the target person. Considering that the flash of the mobile phone may be turned on when taking pictures, the method for detecting the photo-taking behavior in the confidential area provided in this embodiment assists in determining whether the target person has taken photos with a mobile phone by performing flash recognition on each sub-image in the buffer.

[0102] In a possible implementation, performing flash recognition on each sub-image in the buffer specifically includes the following steps C1-C4:

[0103] C1: For each sub-image, convert the sub-image into a grayscale image;

[0104] C2: Calculate the gradient of the grayscale image to obtain the gradient image corresponding to the grayscale image;

[0105] Exemplarily, the gradient of the grayscale image is calculated based on the Sobel operator, and the gradient image reflects the rate of change of pixel brightness in the grayscale image.

[0106] C3: Binarize the gradient image to obtain a binary image;

[0107] The binarization threshold in the process of binarizing the gradient image is dynamically adjusted according to the specific application scenario. In the binarized image, the area with a large brightness change will be marked as white, otherwise it will be marked as black.

[0108] C4: If there is a white area in the binary image that meets the preset range, it is determined that the flash is recognized in the sub-image.

[0109] The preset range is the range of the area affected by the flash brightness.

[0110] Furthermore, contour extraction can be performed on the white areas in the binary image, and all extracted contour areas can be size filtered to eliminate the influence of other noises. Finally, the white areas that meet the preset range are determined as the location where the flash appears, and it is determined that the flash is recognized in the sub-image. Conversely, if there is no white area that meets the preset range in the binary image, it is determined that there is no flash in the sub-image.

[0111] See also Figure 7 The flowchart of the method for detecting photo-taking behavior in confidential areas is shown in the figure. After the training of the target detection model and the video behavior recognition model is completed, the model is verified and format converted, for example, into the cross-platform universal ONNX format, and finally deployed on the edge computing device.

[0112] After the model is deployed, it can be put into use. The edge computing device obtains the surveillance image of the confidential area, calls the target detection model to identify whether there is a person using a mobile phone in the surveillance image. If there is a person using a mobile phone in the surveillance image, a first-level warning is triggered and a first-level warning information is sent to the cloud platform. At the same time, the target person using the mobile phone is tracked to obtain an image including the bounding box corresponding to the target person. Then the image including the bounding box is pre-processed and stored in the buffer.

[0113] When a sub-image is stored in the buffer, flash recognition is triggered. If the flash is recognized, it is determined that the target person is taking a photo with a mobile phone, and a secondary warning is triggered, and the secondary warning information is sent to the cloud platform. If the flash is not recognized, the target person continues to be tracked.

[0114] When the number of sub-images in the buffer reaches the threshold of X frames, the sub-images in the buffer are integrated into a video clip of the target person, and the video behavior recognition model is called to identify whether the target person in the video clip has taken photos with a mobile phone. If so, a secondary warning is triggered and a secondary warning message is sent to the cloud platform. If not, the target person continues to be tracked.

[0115] The above-mentioned first-level warning information and second-level warning information can both include monitoring images, the acquisition time of the monitoring images and the identification of confidential areas, so that the target management personnel can promptly understand which confidential areas have security risks and deal with them in a timely manner.

[0116] The above introduces a method for detecting photographing behavior in a confidential area provided in an embodiment of the present application. The following will introduce a device for executing the above method for detecting photographing behavior in a confidential area.

[0117] See also Figure 8 , Figure 8 This is a schematic diagram of a device for detecting the behavior of taking photos in a confidential area provided in an embodiment of the present application. Figure 8 As shown, the device for detecting the behavior of taking photos in a confidential area includes:

[0118] An image acquisition unit 801 is used to acquire surveillance images of confidential areas;

[0119] A target detection unit 802 is used to call a target detection model to identify whether there is a person using a mobile phone in the monitoring image, wherein the target detection model is trained using image training samples marked with target categories and target recognition frames, and the target categories include person targets and mobile phone targets;

[0120] A target tracking unit 803 is used to track a target person using a mobile phone if there is a person using a mobile phone in the monitoring image, and obtain a video clip of the target person;

[0121] The behavior recognition unit 804 is used to call the video behavior recognition model to identify whether the target person in the video clip has taken photos with a mobile phone. The video behavior recognition model is trained using video training samples marked with behavior categories, and the behavior categories include using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

[0122] In a possible implementation, the target tracking unit 803 is specifically used to track the target person to obtain an image including a bounding box corresponding to the target person; enlarge the bounding box in the image by a preset multiple, and cut the enlarged bounding box to obtain a sub-image of the target person; save the sub-image to a buffer; when the number of sub-images in the buffer reaches a threshold, integrate the sub-images in the buffer into a video clip of the target person.

[0123] In a possible implementation, the device for detecting the behavior of taking photos in a confidential area further includes:

[0124] A flash recognition unit is used to perform flash recognition on each of the sub-images in the buffer; if a flash is recognized, it is determined that the target person has taken a photo with a mobile phone.

[0125] In a possible implementation, the flash recognition unit is specifically used to convert the sub-image into a grayscale image for each of the sub-images; perform gradient calculation on the grayscale image to obtain a gradient image corresponding to the grayscale image; perform binarization processing on the gradient image to obtain a binary image; if there is a white area within a preset range in the binary image, it is determined that a flash is recognized in the sub-image.

[0126] In a possible implementation, the device for detecting the behavior of taking photos in a confidential area further includes:

[0127] The first-level warning unit is used to trigger a first-level warning if a person uses a mobile phone in the monitoring image.

[0128] In a possible implementation, the device for detecting the behavior of taking photos in a confidential area further includes:

[0129] The secondary warning unit is used to trigger a secondary warning if the target person takes photos with a mobile phone.

[0130] The present embodiment discloses a device for detecting photographing behavior in confidential areas. After acquiring a surveillance image of the confidential area, a target detection model is called to identify whether there is a person using a mobile phone in the surveillance image. When there is a person using a mobile phone in the surveillance image, a target tracking model that is more lightweight than the target detection model is used to track the target person using the mobile phone to obtain a video clip of the target person. Then, a video behavior recognition model related to a time series is called to accurately identify whether the target person in the video clip has a mobile phone photographing behavior, thereby improving the efficiency and accuracy of photographing behavior detection in confidential areas and reducing the underreporting rate of photographing behavior in confidential areas.

[0131] Also provided in an embodiment of the present application is a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an edge computing device, the edge computing device implements any of the methods for detecting photographing behavior in confidential areas provided in the embodiments of the present application.

[0132] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an edge computing device, the edge computing device can implement any of the methods for detecting photo-taking behavior in confidential areas provided in an embodiment of the present application.

[0133] The method and related device for detecting the behavior of taking photos in a confidential area provided by the present application can be used in the field of artificial intelligence or finance. The above is only an example and does not limit the application field of the method and related device for detecting the behavior of taking photos in a confidential area provided by the present invention.

[0134] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0135] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0136] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0137] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for detecting photography behavior in confidential areas, characterized in that: include: Obtain surveillance images of confidential areas; Calling a target detection model to identify whether a person is using a mobile phone in the surveillance image, wherein the target detection model is trained using image training samples annotated with target categories and target recognition frames, wherein the target categories include person targets and mobile phone targets; If a person is using a mobile phone in the surveillance image, target tracking is performed on the target person using the mobile phone to obtain a video clip of the target person; A video behavior recognition model is called to identify whether the target person in the video clip has taken photos with a mobile phone. The video behavior recognition model is trained using video training samples labeled with behavior categories, and the behavior categories include using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

2. The method for detecting the behavior of taking photos in confidential areas according to claim 1, characterized in that: The target person using the mobile phone is tracked to obtain a video clip of the target person, including: Tracking the target person to obtain an image including a bounding box corresponding to the target person; In the image, the bounding box is enlarged by a preset multiple, and the enlarged bounding box is cut to obtain a sub-image of the target person; Saving the sub-image to a buffer; When the number of the sub-images in the buffer reaches a threshold, the sub-images in the buffer are integrated into a video clip of the target person.

3. The method for detecting the behavior of taking photos in confidential areas according to claim 2, characterized in that: After the sub-image is saved in the buffer, the method for detecting the behavior of taking photos in a confidential area further includes: Performing flash recognition on each of the sub-images in the buffer; If the flash is identified, it is determined that the target person is taking photos with a mobile phone.

4. The method for detecting the behavior of taking photos in confidential areas according to claim 3, characterized in that: The step of performing flash recognition on each sub-image in the buffer includes: For each of the sub-images, convert the sub-image into a grayscale image; Performing gradient calculation on the grayscale image to obtain a gradient image corresponding to the grayscale image; Binarizing the gradient image to obtain a binary image; If there is a white area in the binary image that meets the preset range, it is determined that a flash is recognized in the sub-image.

5. The method for detecting the behavior of taking photos in confidential areas according to claim 1, characterized in that: The method for detecting the behavior of taking photos in confidential areas also includes: If anyone is using a mobile phone in the surveillance image, a first-level warning is triggered.

6. The method for detecting the behavior of taking photos in confidential areas according to claim 1 or 3, characterized in that: The method for detecting the behavior of taking photos in confidential areas also includes: If the target person takes photos with a mobile phone, a second-level warning will be triggered.

7. A device for detecting the behavior of taking photos in confidential areas, characterized in that: include: An image acquisition unit, used to acquire surveillance images of confidential areas; A target detection unit, used to call a target detection model to identify whether there is a person using a mobile phone in the monitoring image, wherein the target detection model is trained using image training samples marked with target categories and target recognition frames, wherein the target categories include person targets and mobile phone targets; A target tracking unit, configured to track a target person using a mobile phone if a person is using a mobile phone in the surveillance image, and obtain a video clip of the target person; A behavior recognition unit is used to call a video behavior recognition model to identify whether the target person in the video clip has taken photos with a mobile phone. The video behavior recognition model is trained using video training samples marked with behavior categories, and the behavior categories include using a mobile phone but not taking photos with a mobile phone, taking photos with a mobile phone, and normal behavior.

8. An edge computing device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the edge computing device can implement the method for detecting photo-taking behavior in confidential areas as described in any one of claims 1 to 6.

9. A computer program product, characterized in that It includes computer-readable instructions. When the computer-readable instructions are executed on an edge computing device, the edge computing device implements the method for detecting photographing behavior in a confidential area as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs. When the one or more computer programs are executed by the edge computing device, the edge computing device can implement the method for detecting photographing behavior in confidential areas as described in any one of claims 1 to 6.

Images