Monitoring method, device, monitoring equipment and computer readable storage medium

Abstract

The present application provides a monitoring method, device, monitoring equipment and computer-readable storage medium. The method includes: obtaining a current image, which is an image obtained from a video stream obtained by a camera shooting the area where the door is located; inputting the current image into a trained state detection model; when the state detection model outputs information indicating that the door in the current image is in an open state, and before the current image is acquired, information that the door is in a closed state is pre-stored, the current image is input into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model indicating whether passage is allowed. In this way, there is no need to continuously use the pedestrian detection model to detect pedestrians, which can improve detection efficiency while avoiding missed detections.

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to a monitoring method, apparatus, monitoring device and computer-readable storage medium. Background Art

[0002] Double-person double-lock means installing two locks on the warehouse door, and then two managers each keep a key. Only when both are present can the door be opened to take out items, thus playing a role of mutual supervision. The current management of special items, for example, in the management of hazardous chemicals, generally requires managers to implement the double-person double-lock management system. If managers do not strictly follow the requirements to implement the double-person double-lock, it may lead to the loss of dangerous goods, safety hazards and even accidents.

[0003] At present, manual visual inspection is usually used to detect double-locking by two people. Supervisors mainly use video surveillance systems to monitor the surveillance images in real time or regularly screen historical videos to check whether the staff has implemented double-locking by two people. However, due to the limitations of human vision, manual visual inspection is prone to missed detections, and has the disadvantages of low detection efficiency and high labor costs. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a monitoring method, an apparatus, a monitoring device and a computer-readable storage medium, which can improve the problem of easy missed detection and low detection efficiency.

[0005] In order to achieve the above purpose, the embodiments of the present application are implemented in the following manner:

[0006] In a first aspect, an embodiment of the present application provides a monitoring method, the method comprising: acquiring a current image, the current image being an image obtained from a video stream obtained by a camera shooting an area where a door is located; inputting the current image into a trained state detection model; when the state detection model outputs information representing that the door in the current image is in an open state, and before acquiring the current image, information that the door is in a closed state is pre-stored, inputting the current image into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model representing whether passage is allowed.

[0007] In the above implementation, the state detection model and the pedestrian detection model are combined. That is, the state detection model is used to detect the switch state of the door first, and when it is detected that the door changes from the closed state to the open state, the pedestrian detection model is triggered to detect pedestrians. In this way, there is no need to continuously use the pedestrian detection model to detect pedestrians, which can improve the detection efficiency while avoiding missed detection.

[0008] In combination with the first aspect, in some optional embodiments, before acquiring the current image, the method includes: acquiring a first training image set, each image in the first training image set being provided with a mark representing that the door is in an open state or a closed state; using each image in the first training image set, training a preset state detection model to obtain the trained state detection model.

[0009] In combination with the first aspect, in some optional embodiments, before obtaining the current image, the method includes: obtaining a second training image set, wherein pedestrian marks are set in images in the pedestrian area in the second training image set; using each image in the second training image set, training a preset pedestrian detection model to obtain the trained pedestrian detection model.

[0010] In combination with the first aspect, in some optional implementations, obtaining the current image includes: obtaining image frames at a specified frequency from the video stream captured by the camera; and cropping the image area corresponding to the door from the image frames as the current image.

[0011] In the above implementation, the switch state of the door is detected only for the image area corresponding to the door, which is helpful to filter out the image area irrelevant to the door, reduce noise data, and improve the accuracy of detection.

[0012] In combination with the first aspect, in some optional embodiments, the current image is input into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model indicating whether passage is allowed, including: inputting the current image into a trained pedestrian detection model, and having the pedestrian detection model perform pedestrian detection on the current image; when there are pedestrians in a designated area of ​​the current image and the number of pedestrians is within a set range, the pedestrian detection model outputs a first detection result indicating that passage is allowed; when the number of pedestrians in the designated area is not within the set range, the pedestrian detection model outputs a second detection result indicating that passage is not allowed.

[0013] In the above-mentioned implementation, when the number of pedestrians in the designated area is within a set range, it indicates that passage is permitted. In this way, the judgment on whether passage is permitted can be realized quickly and automatically.

[0014] In combination with the first aspect, in some optional implementations, after obtaining the detection result, the method further includes:

[0015] The pre-stored information that the door is in a closed state is replaced by the information in the current image that the door is in an open state.

[0016] In combination with the first aspect, in some optional implementations, the method further includes: when the detection result indicates that passage is not permitted, a motor controlling a door lock of the door drives the door to be in a closed state.

[0017] In a second aspect, the present application further provides a monitoring device, the device comprising:

[0018] An acquisition unit, used for acquiring a current image, wherein the current image is an image obtained from a video stream obtained by a camera shooting an area where the door is located;

[0019] A first input detection unit, used for inputting the current image into a trained state detection model;

[0020] The second input detection unit is used to input the current image into a trained pedestrian detection model when the state detection model outputs information indicating that the door in the current image is in an open state, and before acquiring the current image, information that the door is in a closed state is pre-stored, to obtain a detection result output by the pedestrian detection model indicating whether passage is allowed.

[0021] In a third aspect, the present application further provides a monitoring device, comprising a processor and a memory coupled to each other, wherein the memory stores a computer program, and when the computer program is executed by the processor, the monitoring device executes the above method.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed on a computer, the computer executes the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of the structure of the monitoring device provided in an embodiment of the present application.

[0025] Figure 2 A flowchart of a monitoring method provided in an embodiment of the present application.

[0026] Figure 3 A schematic diagram of the network structure of the pedestrian detection model provided in an embodiment of the present application.

[0027] Figure 4 A block diagram of a monitoring device provided in an embodiment of the present application.

[0028] Icons: 10 - monitoring device; 11 - processing module; 12 - storage module; 200 - monitoring device; 210 - acquisition unit; 220 - first input detection unit; 230 - second input detection unit. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. It should be noted that the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0030] In the current management of special items, or in other door monitoring management, for example, in the management of hazardous chemicals, it is generally necessary to monitor whether two or more people enter the warehouse door at the same time to ensure management safety and avoid the loss of hazardous items or other accidents caused by a single person entering the warehouse.

[0031] Please refer to Figure 1 The present application provides a monitoring device 10, which can be used to automatically monitor the opening and closing of a door and the process of pedestrians entering the door, and can replace the monitoring method of manual visual inspection to avoid missed inspections and reduce labor costs. The door can be a door of a building such as a warehouse or a factory, and can be flexibly determined according to actual conditions.

[0032] In this embodiment, the monitoring device 10 may include a processing module 11 and a storage module 12. The storage module 12 stores a computer program, and when the computer program is executed by the processing module 11, the monitoring device 10 can perform each step in the following method.

[0033] Understandably, Figure 1 The structure shown is only a schematic diagram of a structure of the monitoring device 10. The monitoring device 10 may also include Figure 1 More components are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0034] For example, the monitoring device 10 may also include a display screen, a camera, etc. Of course, in other embodiments, the camera may also be an external module independent of the monitoring device 10. The camera may be used to monitor and shoot the gate of a building such as a warehouse or a factory, and the shooting picture usually includes the door and the ground in front of the door.

[0035] Please refer to Figure 2The present application also provides a monitoring method, which can be applied to the above-mentioned monitoring device 10. The monitoring device 10 executes or implements each step of the method. The method may include the following steps:

[0036] Step S110, obtaining a current image, wherein the current image is an image obtained from a video stream obtained by a camera shooting an area where the door is located;

[0037] Step S120, inputting the current image into a trained state detection model;

[0038] Step S130, when the state detection model outputs information indicating that the door in the current image is in an open state, and before acquiring the current image, information indicating that the door is in a closed state is pre-stored, the current image is input into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model indicating whether passage is allowed.

[0039] In the above implementation, the state detection model and the pedestrian detection model are combined. That is, the state detection model is used to detect the switch state of the door first, and when it is detected that the door changes from the closed state to the open state, the pedestrian detection model is triggered to detect pedestrians. In this way, there is no need to continuously use the pedestrian detection model to detect pedestrians, which can improve the detection efficiency while avoiding missed detection.

[0040] The following is a detailed description of each step of the method:

[0041] Before step S110, the state detection model and the pedestrian detection model may be trained models or untrained models. If the model is trained, the state detection model can automatically detect the open or closed state of the door in the picture according to the picture taken of the door. In addition, the pedestrian detection model can detect whether there are pedestrians in the specified image area of ​​the image and the number of pedestrians.

[0042] If both the state detection model and the pedestrian detection model have not been trained, the method may further include a step of training the model before step S110. For example, before acquiring the current image, the method may include:

[0043] Acquire a first training image set, wherein each image in the first training image set is provided with a mark indicating that the door is in an open state or in a closed state;

[0044] The preset state detection model is trained using each image in the first training image set to obtain the trained state detection model.

[0045] Understandably, the first training image set may include a large number of images as training images and test images. Each image is an image obtained by photographing a door and is marked with a mark indicating that the door is actually in an open or closed state. In the first training image set, the number of images may be thousands or tens of thousands, etc., and may be flexibly determined according to actual conditions.

[0046] The preset state detection model can be flexibly determined according to the actual situation. For example, the preset state detection model is a ResNet18 network model, or other models based on deep learning algorithms. The ResNet18 network model includes 18 convolutional layers, 1 global average pooling layer, and 1 Softmax function layer. When using the ResNet18 network model for image training or detection, the input image can be recorded as:

[0047] I input ∈R H×W×3 , where I input refers to the input image; R refers to a real number, which is a parameter of the expression well known to those skilled in the art; W refers to the width of the image; H refers to the length of the image, and the number 3 is the number of image channels, namely red, green and blue channels. input ∈R H×W×3 , refers to the format content of the input image, which is a real number matrix of size H×W×3.

[0048] During training, the labeled training images are input into the ResNet18 network model, and after a series of neural network operations, the ResNet18 network model can output the probability vector representing the category to which the image belongs: ρ∈R 1×K , where ρ is the probability of the category to which the image belongs, and K represents the total number of categories. In the embodiment of the present application, the total number of switch state categories of the door is 2, namely, open state and closed state. In addition, during the training process, the operation and processing process of the ResNet18 network model is well known to those skilled in the art and will not be repeated here.

[0049] After the ResNet18 network model is trained, the probability of the door being in the open state and the closed state can be obtained. For example, the probability of the door in the image being in the open state is ρ when the ResNet18 network model is used to detect the door. dk and the probability of being in the closed state ρ gb If ρ dk >ρ gb , it means the door is open; if ρ dk ≤ρ gb , it means the door is closed.

[0050] In this embodiment, before acquiring the current image, the method includes:

[0051] Acquire a second training image set, wherein pedestrian marks are set in the images in the pedestrian image area in the second training image set;

[0052] The preset pedestrian detection model is trained using each image in the second training image set to obtain the trained pedestrian detection model.

[0053] Understandably, the second training image set may include a large number of images with pedestrians, and the pedestrian image area is marked. Then, each image in the second training image set is used to train the preset pedestrian detection model, so that the trained preset pedestrian model has the function of pedestrian detection.

[0054] In this embodiment, the preset pedestrian detection model can be flexibly determined according to actual conditions. Figure 3 The preset pedestrian detection model may be a CenterNet network model, which includes a backbone network and a target detection module. The backbone network may be a ResNet18-FPN network model, which is used to extract deep features of images.

[0055] For example, input image I to the CenterNet network model input ∈R H×W×3 ,First, after the calculation of the backbone network, three feature maps of different scales are output, namely C2, C3 and C4, where, Then, through the processing of the FPN (Feature Pyramid Networks) structure, the three feature map information of C2, C3, and C4 are fused to output the deep feature maps P2, P3, and P4 of the image, respectively.

[0056] The target detection module includes two branches. The first branch is used to predict the center point position of the target object (in this embodiment, the target object or the target is a pedestrian), including two convolutional layers (for example, the convolution kernel size of the convolution layer Conv1_1 is 3×3, and the output channel is 256; the convolution kernel size of the convolution layer Conv1_2 is 1×1, and the output channel is K, K represents the total number of target categories) and 1 Sigmoid function layer (to ensure that the output value range is 0 to 1); the second branch is used to predict the size of the target, including two convolutional layers (Conv2_1 convolution kernel size is 3×3, and the output channel is 256; Conv2_2 convolution kernel size is 1×1, and the output channel is 2, corresponding to the width and height of the target, respectively) and 1 Exp function layer (to ensure that the output value is positive).

[0057] The depth feature map P4 is input to the target detection module, and after calculation by the target detection module, the target center point feature map is output. and target size feature map Remember F hm Each element in is ρ x，y，k , F size Each element in is S x，y，i ,in k={0, 1,..., K-1}, i={0, 1}. Then ρ x，y，k It represents the probability that the position point (x, y) is predicted to be the center point of the k-th target, S x，y，0 and S x，y，1 They represent the length and width of the target corresponding to the position point (x, y) respectively.

[0058] In this embodiment, by using each image in the second training image set to train the preset pedestrian detection model, the pedestrian detection model obtained after training can have a pedestrian detection function and can count the number of pedestrians in the specified area of ​​the image.

[0059] In step S110, the monitoring device 10 can directly obtain the current image from the video stream obtained by the camera shooting the area where the door is located. The current image can be an image frame in the video stream, or a region in the image frame that is specific to the area where the door is located.

[0060] For example, step S110 may include:

[0061] Acquire image frames at a specified frequency from the video stream captured by the camera;

[0062] From the image frame, a region corresponding to the door is cropped as the current image.

[0063] The designated frequency may be determined according to actual conditions. For example, in a video stream, the monitoring device 10 may select an image frame from every five frames, and then cut out the image area corresponding to the door from the image frame as the current image. The image area corresponding to the door may be marked by the management personnel according to actual conditions, for example, including the area where the door is located and part of the image area in front of the door, which is not specifically limited here.

[0064] By cropping the image frame, only the image area corresponding to the door is used as the current image. In this way, the interference of other information on the recognition of the door switch state can be reduced, the accuracy of detection can be improved, the whole image does not need to be detected, and the amount of calculation can be reduced.

[0065] In step S120, after the current image is input into the state detection model, the state detection model can detect the switch state information of the door in the current image to determine whether the door in the current image is in an open state or a closed state, and record the state information.

[0066] In step S130, the pre-stored door state information refers to information on whether the door is in a closed state or an open state in an image before the current image.

[0067] When the pre-stored state information indicates that the door is in a closed state, and the door in the current image is recognized to be in an open state, it means that the door has changed from closed to open during the period from the last image to the current image captured by the camera. At this time, the pedestrian detection model can be triggered to perform pedestrian detection on the current image. That is, the current image is input into the pedestrian detection model to perform pedestrian detection and determine whether there are pedestrians and the number of pedestrians in the specified image area of ​​the current image.

[0068] The designated area can be set according to actual conditions, and is usually the area in the monitoring screen where the camera captures the person when the person is standing in front of the door while the door is open.

[0069] In this embodiment, step S130 may include:

[0070] Inputting the current image into a trained pedestrian detection model, and allowing the pedestrian detection model to perform pedestrian detection on the current image;

[0071] When there are pedestrians in the designated area of ​​the current image and the number of pedestrians is within a set range, outputting a first detection result indicating that passage is permitted through the pedestrian detection model;

[0072] When the number of pedestrians in the designated area is not within the set range, the pedestrian detection model outputs a second detection result indicating that passage is not permitted.

[0073] In the pedestrian detection model, the target detection module can calculate the width and height of the pedestrian, and determine the coordinates of the center point of the pedestrian based on the position information of the width and height in the image. The coordinates of the center point can be the center point of the rectangle circumscribing the pedestrian's outline. If the coordinates of the center point are within the specified area, it means that the pedestrian is in front of the door. In this case, the number of pedestrians can be counted. If the coordinates of the center point are not within the specified area, it means that the pedestrian is not in front of the door. In this case, there is no need to count the number of pedestrians.

[0074] In this embodiment, the number of pedestrians in the designated area is the number of pedestrian center points in the designated area. The setting range can be flexibly determined according to actual conditions. For example, the setting range can be a range greater than or equal to 2. When the number of pedestrians in the designated area is greater than or equal to 2, passage is allowed to improve the safety of entering the door. If the number of pedestrians in the designated area is 1, the pedestrian is prompted that passage is not allowed.

[0075] In this embodiment, a speaker may be provided near the door, which is electrically connected to the monitoring device 10 and can, under the control of the monitoring device 10, issue a voice prompt indicating that passage is prohibited or permitted, so as to conveniently prompt pedestrians whether passage is permitted.

[0076] In this embodiment, after obtaining the detection result, the method further includes:

[0077] The pre-stored information that the door is in a closed state is replaced by the information in the current image that the door is in an open state.

[0078] Understandably, by timely updating the pre-stored door status information, it is convenient to subsequently use the updated door status information to determine whether to trigger the pedestrian detection model to perform pedestrian detection, thereby improving the timeliness of the status information.

[0079] As an optional implementation, the method may further include:

[0080] When the detection result indicates that passage is not permitted, the motor controlling the door lock of the door drives the door to be in a closed state.

[0081] In this embodiment, the door may also be provided with a first motor for driving the door switch and a second motor for obtaining the door lock switch. The first motor is used to drive the door to be in an open or closed state, and the second motor can close or open the door lock when the door is in a closed state. Both the first motor and the second motor can be electrically connected to the monitoring device 10.

[0082] When the detection result indicates that passage is not allowed, the monitoring device 10 can control the first motor to drive the door to a closed state, and then control the second electrode to put the door lock in a locked state, thereby preventing pedestrians from entering the door to improve entry and exit safety.

[0083] Please refer to Figure 4The embodiment of the present application also provides a monitoring device 200, which can be applied to the above-mentioned monitoring device 10 to execute each step in the method. The monitoring device 200 includes at least one software function module that can be stored in the storage module 12 in the form of software or firmware or solidified in the operating system (OS) of the monitoring device 10. The processing module 11 is used to execute the executable module stored in the storage module 12, such as the software function module and computer program included in the monitoring device 200.

[0084] The monitoring device 200 may include an acquisition unit 210, a first input detection unit 220, and a second input detection unit 230. The functions of each unit may be as follows:

[0085] An acquisition unit 210 is used to acquire a current image, where the current image is an image obtained from a video stream obtained by a camera shooting an area where the door is located;

[0086] A first input detection unit 220, configured to input the current image into a trained state detection model;

[0087] The second input detection unit 230 is used to input the current image into a trained pedestrian detection model when the state detection model outputs information indicating that the door in the current image is in an open state, and before acquiring the current image, information that the door is in a closed state is pre-stored, to obtain a detection result output by the pedestrian detection model indicating whether passage is allowed.

[0088] Optionally, the monitoring device 200 may further include a first training unit. Before acquiring the current image, the acquisition unit 210 is further configured to acquire a first training image set, wherein each image in the first training image set is provided with a mark indicating that the door is in an open state or in a closed state. The first training unit is configured to train a preset state detection model using each image in the first training image set to obtain the trained state detection model.

[0089] Optionally, the monitoring device 200 may further include a second training unit. Before acquiring the current image, the acquisition unit 210 is further configured to acquire a second training image set, wherein pedestrian marks are set in the images in the pedestrian image area in the second training image set; and the second training unit is configured to train a preset pedestrian detection model using each image in the second training image set to obtain the trained pedestrian detection model.

[0090] Optionally, the acquisition unit 210 is further configured to: acquire image frames at a specified frequency from the video stream captured by the camera; and crop an image region corresponding to the door from the image frames as the current image.

[0091] Optionally, the second detection unit is further used for:

[0092] Inputting the current image into a trained pedestrian detection model, and allowing the pedestrian detection model to perform pedestrian detection on the current image;

[0093] When there are pedestrians in the designated area of ​​the current image and the number of pedestrians is within a set range, outputting a first detection result indicating that passage is permitted through the pedestrian detection model;

[0094] When the number of pedestrians in the designated area is not within the set range, the pedestrian detection model outputs a second detection result indicating that passage is not permitted.

[0095] Optionally, the monitoring device 200 may further include an updating unit. After obtaining the detection result, the updating unit is configured to replace the pre-stored information that the door is in a closed state with the information that the door is in an open state in the current image.

[0096] Optionally, the monitoring device 200 may further include a control unit, configured to control a motor of a door lock of the door to drive the door to a closed state when the detection result indicates that passage is not permitted.

[0097] In this embodiment, the processing module 11 can be an integrated circuit chip with signal processing capabilities. The above-mentioned processing module 11 can be a general-purpose processor. For example, the processor can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of this application.

[0098] The storage module 12 may be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc. In this embodiment, the storage module 12 may be used to store video streams, current images, state detection models, pedestrian detection models, etc. Of course, the storage module 12 may also be used to store programs, and the processing module 11 executes the program after receiving the execution instruction.

[0099] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the monitoring device 200 and the monitoring equipment 10 described above can refer to the corresponding process of each step in the aforementioned method, and will not be elaborated here.

[0100] The embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer executes the monitoring method described in the above embodiment.

[0101] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by hardware, and can also be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0102] In summary, in this solution, the state detection model and the pedestrian detection model are combined. That is, the state detection model is used to detect the switch state of the door first, and when it is detected that the door changes from the closed state to the open state, the pedestrian detection model is triggered to detect pedestrians. In this way, there is no need to continuously use the pedestrian detection model to detect pedestrians, which can improve the detection efficiency while avoiding missed detection.

[0103] In the embodiments provided in the present application, it should be understood that the disclosed device, system and method can also be implemented in other ways. The above-described device, system and method embodiments are merely schematic, for example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0104] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A monitoring method, characterized in that: The method comprises: Acquire a current image, where the current image is an image obtained from a video stream obtained by a camera shooting an area where the door is located; Inputting the current image into a trained state detection model; When the state detection model outputs information indicating that the door in the current image is in an open state, and before the current image is acquired, information indicating that the door is in a closed state is pre-stored, the current image is input into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model indicating whether passage is permitted; The pedestrian detection model includes a backbone network and a target detection module; the target detection module includes a first branch and a second branch, the first branch is used to predict the center point position of the pedestrian, and the second branch is used to predict the size of the pedestrian; The current image is calculated by the backbone network, and three feature maps of different scales are output, namely C2, C3 and C4, where C2∈R^(H / 4×W / 4×64), C3∈R^(H / 8×W / 8×128), C4∈R^(H / 16×W / 16×256), W refers to the width of the image; H refers to the length of the image, and R refers to a real number; After being processed by the feature pyramid network structure, the three feature map information of C2, C3, and C4 are fused to obtain the deep feature maps P2, P3, and P4 of the image, where P2∈R^(H / 16×W / 16×256), P3∈R^(H / 8×W / 8×128), and P4∈R^(H / 4×W / 4×64); The depth feature map P4 is input to the target detection module, and after calculation by the target detection module, a pedestrian center point feature map and a pedestrian size feature map are output.

2. The method according to claim 1, characterized in that Before acquiring the current image, the method includes: Acquire a first training image set, wherein each image in the first training image set is provided with a mark indicating that the door is in an open state or in a closed state; The preset state detection model is trained using each image in the first training image set to obtain the trained state detection model.

3. The method according to claim 1, characterized in that Before acquiring the current image, the method includes: Acquire a second training image set, wherein pedestrian marks are set in the images in the pedestrian image area in the second training image set; The preset pedestrian detection model is trained using each image in the second training image set to obtain the trained pedestrian detection model.

4. The method according to claim 1, characterized in that: Get the current image, including: Acquire image frames at a specified frequency from the video stream captured by the camera; From the image frame, a region corresponding to the door is cropped as the current image.

5. The method according to claim 1, characterized in that Inputting the current image into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model indicating whether passage is permitted, including: Inputting the current image into a trained pedestrian detection model, and allowing the pedestrian detection model to perform pedestrian detection on the current image; When there are pedestrians in the designated area of ​​the current image and the number of pedestrians is within a set range, outputting a first detection result indicating that passage is permitted through the pedestrian detection model; When the number of pedestrians in the designated area is not within the set range, the pedestrian detection model outputs a second detection result indicating that passage is not permitted.

6. The method according to claim 1, characterized in that After obtaining the detection result, the method further includes: The pre-stored information that the door is in a closed state is replaced by the information in the current image that the door is in an open state.

7. The method according to claim 1, characterized in that The method further comprises: When the detection result indicates that passage is not permitted, the motor controlling the door lock of the door drives the door to be in a closed state.

8. A monitoring device, characterized in that: The device comprises: An acquisition unit, used for acquiring a current image, wherein the current image is an image obtained from a video stream obtained by a camera shooting an area where the door is located; A first input detection unit, used for inputting the current image into a trained state detection model; a second input detection unit, configured to input the current image into a trained pedestrian detection model to obtain a detection result output by the pedestrian detection model indicating whether passage is permitted, when the state detection model outputs information indicating that the door in the current image is in an open state and information that the door is in a closed state is pre-stored before the current image is acquired; The pedestrian detection model includes a backbone network and a target detection module; the target detection module includes a first branch and a second branch, the first branch is used to predict the center point position of the pedestrian, and the second branch is used to predict the size of the pedestrian; The current image is calculated by the backbone network, and three feature maps of different scales are output, namely C2, C3 and C4, where C2∈R^(H / 4×W / 4×64), C3∈R^(H / 8×W / 8×128), C4∈R^(H / 16×W / 16×256), W refers to the width of the image; H refers to the length of the image, and R refers to a real number; After being processed by the feature pyramid network structure, the three feature map information of C2, C3, and C4 are fused to obtain the deep feature maps P2, P3, and P4 of the image, where P2∈R^(H / 16×W / 16×256), P3∈R^(H / 8×W / 8×128), and P4∈R^(H / 4×W / 4×64); The depth feature map P4 is input to the target detection module, and after calculation by the target detection module, a pedestrian center point feature map and a pedestrian size feature map are output.

9. A monitoring device, characterized in that: The monitoring device comprises a processor and a memory coupled to each other, wherein the memory stores a computer program. When the computer program is executed by the processor, the monitoring device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images