A camera occlusion detection method in a box, an elevator operation control method and device

Through the camera occlusion detection method in the box, the malicious occlusion behavior of the battery car when entering the elevator is identified. Combined with the neural network and texture brightness comparison, the malicious occlusion problem in the detection of the battery car entering the elevator is solved, and the effective blockade of the battery car entering the room illegally is achieved.

CN114140746BActive Publication Date: 2024-11-08CHENGDU XINCHAO MEDIA GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111450216.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-11-08
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

The existing battery car access inspection plan cannot identify pedestrians' malicious obstruction of cameras, resulting in the inability to detect someone evade detection in time, and thus the inability to accurately block the elevator, effectively blocking the passage of electric vehicles entering the room illegally.

Method used

Using the camera occlusion detection method in the box, the surveillance images in the box are obtained, the spatial feature vectors of the personal joint nodes are extracted, the skeleton sequence spatiotemporal feature map is constructed, and the neural network model is input for identification to determine the confidence of the camera occlusion behavior. , and compare texture and brightness to determine whether the target area is blocked, distinguishing malicious occlusion behavior.

Benefits of technology

It realizes the timely detection of malicious occlusion behavior, ensures accurate escalation behavior, effectively blocks the passage of electric vehicles entering the room illegally. It is different from the situation where ordinary cameras are blocked, and is suitable for complex scenarios. It comprehensively uses object detection, behavior recognition and ladder control status logic. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114140746B_ABST
    Figure CN114140746B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and discloses an in-box camera occlusion detection method, an elevator operation control method and a device, wherein, through the elevator operation control method, after the elevator car door is opened, it can be first identified whether there is a target prohibited item entering the elevator car, if so, an elevator blocking behavior is performed, and it is determined whether a malicious camera occlusion event occurs, if it is determined to have occurred, a deep elevator blocking behavior is performed, and it is continued to be determined whether the target prohibited item is still in the elevator car and whether the occlusion effect disappears, and only when it is determined that the target prohibited item is not in the elevator car and the occlusion effect disappears, the elevator blocking behavior is cancelled, and the elevator car is restored to the normal operation mechanism of the elevator, thereby preventing pedestrians from maliciously occluding the target prohibited item and mistakenly believing that the target prohibited item has left the elevator car, thereby avoiding the scene where the target prohibited item such as a battery car successfully enters the room by elevator due to escape detection, and effectively blocking the channel for the battery car to enter the room illegally.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to a method for detecting camera occlusion in a box, and an elevator operation control method and device. Background Art

[0002] With the development of lithium battery technology, electric bicycles have become a daily means of transportation for people due to their convenience. However, in recent years, electric bicycle safety accidents have occurred frequently, especially fires caused by electric bicycle battery problems, resulting in casualties and property losses. Relevant government departments have introduced policies to strengthen the supervision and management of electric bicycles entering indoor areas illegally. Among them, implementing detection when electric bicycles enter elevator cabins and issuing elevator blocking actions is a relatively feasible solution. It can block the illegal entry of electric bicycles into the room from the source, which has important practical value.

[0003] With the development of artificial intelligence technology, especially deep learning technology, the mainstream solution is to detect the battery car entering the elevator by using the image obtained by the rear surveillance camera in the elevator and trigger the elevator blocking behavior. This solution can achieve relatively satisfactory results under ideal conditions. However, as people explore in practice, many users maliciously cover the camera with their hands or other objects after pushing the car into the elevator, so as to escape detection and allow the battery car to enter the room smoothly. Figure 1 As shown. For this scenario, the existing methods are not directly applicable due to certain deficiencies, which makes it difficult to find effective solutions. Therefore, how to identify pedestrians maliciously blocking the camera so as to timely find people who are evading the detection of the target area of ​​interest, and then facilitate accurate blocking of the elevator and effectively block the passage of battery vehicles illegally entering the room is a new topic that technicians in this field need to study urgently. Summary of the invention

[0004] In order to solve the problem that the existing electric vehicle entering the elevator detection scheme cannot identify the malicious behavior of pedestrians blocking the camera, resulting in the failure to timely detect people who are evading the detection of the target area of ​​interest and accurately blocking the elevator, the purpose of the present invention is to provide an in-box camera occlusion detection method, elevator operation control method, device, computer equipment and computer-readable storage medium, which can timely detect people who are evading the detection of the target area of ​​interest, facilitate accurate actions such as blocking the elevator, and effectively block the passage of electric vehicles illegally entering the room.

[0005] In a first aspect, the present invention provides a method for detecting camera occlusion in a box, comprising:

[0006] After obtaining a continuous multi-frame monitoring image captured by a camera in the box, extracting spatial feature vectors of multiple human joints at the time of each frame image capture from the continuous multi-frame monitoring image, wherein the camera in the box is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body;

[0007] Constructing a skeleton sequence spatiotemporal feature graph for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected;

[0008] Input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and having been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior;

[0009] Determining whether the first confidence prediction value is greater than a first preset threshold;

[0010] If it is determined that the first confidence prediction value is greater than the first preset threshold, the last frame image in the continuous multiple frames of in-box monitoring images is used as the current monitoring image, and according to the first focus area picture corresponding to the target focus area in the current monitoring image and the second focus area picture corresponding to the target focus area in the in-box background image, it is determined whether the target focus area is blocked by comparing the texture and brightness, wherein the in-box background image is collected in advance by the in-box camera when the box is empty;

[0011] If it is determined that the target focus area is blocked, it is determined that a malicious camera blocking event has occurred.

[0012] Based on the above invention, a scheme for implementing the malicious camera blocking behavior detection in combination with the scene is provided, that is, firstly, based on the continuous multi-frame monitoring images in the box collected by the camera in the box, a skeleton sequence spatiotemporal feature map for identifying the camera blocking behavior is constructed, and then the skeleton sequence spatiotemporal feature map is imported into the camera blocking behavior recognition model based on the neural network and has been trained, and the corresponding confidence prediction value for identifying the camera blocking behavior is output, and then, through the comparison result of the confidence prediction value and the preset threshold, when it is determined that there is a camera blocking behavior, it is further determined whether the blocking effect is generated, and if so, it is determined that a malicious camera blocking event has occurred, and then it can be found in time that someone is evading the detection of the target focus area, which is convenient and accurate to make actions such as blocking the elevator, which is conducive to effectively blocking the passage of the battery vehicle illegally entering the room. At the same time, because the camera blocking problem is divided into malicious blocking behavior according to the actual scene, the detection method can be distinguished from the situation where the ordinary camera is blocked, and the situation that the traditional method cannot judge whether the camera is maliciously blocked at the semantic level is solved, which is more targeted. In addition, this detection method can be applied to complex scenarios that may be encountered after electric bicycles enter the elevator. It can comprehensively use object detection technology, behavior recognition technology, the camera occlusion detection method inside the box and the elevator control status logic to provide a feasible elevator blocking solution, so as to effectively block the electric bicycles from illegally entering the room.

[0013] In a possible design, the spatial feature vectors of multiple human joints at the time of each frame image acquisition are extracted from the continuous multi-frame in-box monitoring images, including:

[0014] For each frame of the continuous multi-frame in-box monitoring image, the human posture recognition project OpenPose software is used to identify the corresponding 18 human joints marked according to the COCO18 model joint point labels from the corresponding picture, wherein the 18 human joints belong to the same human body;

[0015] For each frame of image, a plurality of human joint points that are preset and can represent the camera blocking behavior of the human body are selected from the corresponding eighteen human joint points;

[0016] For each frame of image, all spatial coordinates of the corresponding multiple human joint points in the camera coordinate system of the camera in the box are used as spatial feature vectors of the multiple human joint points during corresponding acquisition.

[0017] In one possible design, the multiple human body joints include a nose node, a head node, a right shoulder node, a right elbow node, a right hand head node, a left shoulder node, a left elbow node, a left hand head node, a right eye node, a right ear node, a left eye node and a left ear node.

[0018] In a possible design, the skeleton sequence spatiotemporal feature map is represented by G=(V, E), where G represents the skeleton sequence spatiotemporal feature map, V represents a set of spatial feature vectors, and V={v ti |t=0,1,2,…,T;i=0,1,2,…,N}, t represents the frame number in the continuous multi-frame in-box monitoring image, T represents the maximum frame number in the continuous multi-frame in-box monitoring image, i represents the joint point number in the multiple human joint points, N represents the maximum joint point number in the multiple human joint points, v ti represents the spatial feature vector corresponding to the frame number t and the joint point number i, E represents the graph structure and contains the edge structure ES={v ti v tj |(i,j)∈H} and the edge structure EF in the time domain = {v ti v (t+1)i}, H represents the set of natural connection pairs of human joints, v ti v tj Represents the spatial feature vector v ti To the spatial eigenvector v tj The edge vector, v ti v (t+1)i Represents the spatial feature vector v ti To the spatial eigenvector v (t+1)i The edge vector of .

[0019] In a possible design, judging whether the target focus area is blocked by comparing the texture and brightness of a first focus area picture corresponding to the target focus area in the current monitoring image and a second focus area picture corresponding to the target focus area in the background image in the box, includes:

[0020] According to the predetermined target focus area, a corresponding first focus area picture is intercepted from the current monitoring image, and a corresponding second focus area picture is intercepted from the background image in the box, wherein the background image in the box is previously acquired by the camera in the box when the box is empty;

[0021] Dividing the target area of ​​interest into a plurality of sub-areas of the same size;

[0022] For each of the plurality of sub-regions, according to the corresponding picture information in the first region of interest picture, a corresponding first directional gradient histogram eigenvalue and a first average grayscale value are calculated, and according to the corresponding picture information in the second region of interest picture, a corresponding second directional gradient histogram eigenvalue and a second average grayscale value are calculated;

[0023] For each of the sub-regions, if it is determined that the similarity between the corresponding first directional gradient histogram feature value and the corresponding second directional gradient histogram feature value is less than the second preset threshold, it is determined that the corresponding first average grayscale value is less than the corresponding second average grayscale value, and it is determined that the corresponding first average grayscale value is less than the third preset threshold, it is determined that the corresponding sub-region is currently in an obscured state;

[0024] Obtaining by counting a first ratio of the number of sub-regions currently in an obstructed state to the total number of sub-regions of the plurality of sub-regions;

[0025] Determining whether the first ratio is greater than a fourth preset threshold;

[0026] If so, it is determined that the target focus area is blocked.

[0027] In a second aspect, the present invention provides an in-box camera occlusion detection device, comprising a feature vector extraction unit, a feature map construction unit, an occlusion behavior recognition unit, a first judgment unit, a second judgment unit, and an event occurrence determination unit, which are sequentially communicatively connected;

[0028] The feature vector extraction unit is used to extract spatial feature vectors of multiple human joints at the time of each frame image acquisition from the continuous multiple frames of the in-box monitoring images acquired by the in-box camera, wherein the in-box camera is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body;

[0029] The feature map construction unit is used to construct a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected;

[0030] The occlusion behavior recognition unit is used to input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and has been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior;

[0031] The first judgment unit is used to judge whether the first confidence prediction value is greater than a first preset threshold;

[0032] The second judgment unit is used to, when determining that the first confidence prediction value is greater than the first preset threshold, use the last frame image in the continuous multiple frames of in-box monitoring images as the current monitoring image, and judge whether the target focus area is blocked based on the first focus area picture in the current monitoring image and corresponding to the target focus area and the second focus area picture in the in-box background image and corresponding to the target focus area, by comparing the texture and brightness, wherein the in-box background image is pre-acquired by the in-box camera when the box is empty;

[0033] The event occurrence determination unit is used to determine that a malicious camera blocking event occurs when it is determined that the target focus area is blocked.

[0034] In a third aspect, the present invention provides an elevator operation control method, comprising:

[0035] After the elevator car door is opened, a monitoring image inside the elevator collected in real time by a camera inside the elevator is obtained, wherein the camera inside the elevator is installed at the top of the elevator car and faces the elevator car door, and the camera field of view is fixed to cover the ground area inside the elevator car;

[0036] Input the surveillance image in the elevator as an image sample to be tested into a trained target contraband recognition model based on a target detection algorithm to obtain a first recognition result reflecting whether there is a target contraband;

[0037] If the first recognition result indicates that there is a target contraband, determining a contraband focus area according to the location of the target contraband in the elevator monitoring image, and controlling the elevator car to switch from a first state for allowing the elevator to operate normally to a second state for maintaining the elevator car door open;

[0038] After switching to the second state, the in-elevator surveillance image captured by the in-elevator camera in real time continues to be acquired, and each time a new frame of in-elevator surveillance image is acquired, the in-elevator camera is used as the in-box camera, the contraband focus area is used as the target focus area, the in-elevator surveillance image captured by the in-elevator camera when the elevator car is empty is used as the in-box background image, and the continuous multiple frames of in-elevator surveillance images newly captured by the in-elevator camera are used as the continuous multiple frames of in-box surveillance images, and the in-box camera occlusion detection method as described in the first aspect or any possible design of the first aspect is started to be executed, and a determination result reflecting whether a malicious camera occlusion event has occurred is obtained;

[0039] If the determination result indicates that a malicious camera blocking event has occurred, controlling the elevator car to switch from the second state to a third state for continuing to maintain the elevator car door open;

[0040] After switching to the third state, the elevator surveillance image collected by the elevator camera in real time continues to be acquired, and each time a new elevator surveillance image is acquired, the new elevator surveillance image is input into the target contraband recognition model as another image sample to be tested, to obtain a second recognition result reflecting whether the target contraband exists, and based on the third focus area picture corresponding to the contraband focus area in the new elevator surveillance image and the fourth focus area picture corresponding to the contraband focus area in the box background image, through comparison results in texture and brightness, it is determined whether the contraband focus area is not blocked;

[0041] If the second recognition result indicates that there is no target contraband and it is determined that the contraband focus area is not blocked, the elevator car is controlled to switch back to the first state.

[0042] Based on the above invention content, a solution for blocking elevators based on the detection results of malicious blocking behavior of the camera is also provided, that is, after the elevator car door is opened, it is first identified whether any target contraband enters the elevator car. If so, the elevator is blocked and it is determined whether a malicious camera blocking event occurs. If it is determined to have occurred, a deep elevator blocking is performed and it is continued to be determined whether the target contraband is still in the elevator car and whether the blocking effect disappears. Only when it is determined that the target contraband is not in the elevator car and the blocking effect disappears, the elevator blocking behavior is canceled, and the elevator car is restored to its normal operation mechanism, thereby preventing pedestrians from maliciously blocking the target contraband and mistakenly believing that the target contraband has left the elevator car, thereby avoiding the scenario where target contraband such as electric vehicles can successfully enter the room by elevator due to escape detection.

[0043] In a possible design, judging whether the contraband area of ​​interest is not blocked by comparing the texture and brightness of the third area of ​​interest picture corresponding to the contraband area of ​​interest in the surveillance image in the new elevator and the fourth area of ​​interest picture corresponding to the contraband area of ​​interest in the background image in the box, includes:

[0044] According to the contraband focus area, a corresponding third focus area image is intercepted from the new elevator monitoring image, and a corresponding fourth focus area image is intercepted from the box background image;

[0045] Dividing the contraband area of ​​concern into a plurality of sub-blocks of the same size;

[0046] For each of the plurality of sub-blocks, according to the corresponding picture information in the third region of interest picture, a corresponding third directional gradient histogram eigenvalue and a third average grayscale value are calculated, and according to the corresponding picture information in the fourth region of interest picture, a corresponding fourth directional gradient histogram eigenvalue, a fourth average grayscale value and a grayscale standard deviation are calculated;

[0047] For each of the sub-blocks, if it is determined that the similarity between the corresponding third-direction gradient histogram feature value and the corresponding fourth-direction gradient histogram feature value is greater than a fifth preset threshold and it is determined that the absolute difference between the corresponding first average grayscale value and the corresponding second average grayscale value is less than the corresponding grayscale standard deviation, it is determined that the corresponding sub-block is currently in an unobstructed state;

[0048] Obtaining by counting a second ratio of the number of sub-blocks currently in an unblocked state to the total number of sub-blocks of the plurality of sub-blocks;

[0049] Determining whether the second ratio is greater than a sixth preset threshold;

[0050] If so, it is determined that the contraband area of ​​interest is not blocked.

[0051] In a fourth aspect, the present invention provides an elevator operation control device, comprising a monitoring image acquisition module, a contraband identification module, a focus area determination module, an elevator car control module, a malicious occlusion detection module and a judgment module, wherein the malicious occlusion detection module comprises a feature vector extraction unit, a feature map construction unit, an occlusion behavior identification unit, a first judgment unit, a second judgment unit and an event occurrence determination unit which are sequentially communicatively connected;

[0052] The monitoring image acquisition module is used to acquire the monitoring image in the elevator collected by the elevator camera in real time after the elevator car door is opened, wherein the elevator camera is installed at the top of the elevator car and faces the elevator car door, and the camera field of view is fixed to cover the ground area in the elevator car;

[0053] The contraband identification module is communicatively connected to the monitoring image acquisition module, and is used to input the monitoring image in the elevator as an image sample to be tested into a target contraband identification model that has been trained based on a target detection algorithm, and obtain a first identification result that reflects whether there is a target contraband;

[0054] The focus area determination module is communicatively connected to the contraband identification module, and is used to determine the contraband focus area according to the location of the target contraband in the surveillance image in the ladder when the first identification result indicates the presence of the target contraband;

[0055] The elevator car control module is communicatively connected to the contraband identification module, and is used to control the elevator car to switch from a first state for allowing the elevator to operate normally to a second state for maintaining the elevator car door open when the first identification result indicates the presence of target contraband;

[0056] The malicious occlusion detection module is respectively communicatively connected to the focus area determination module, the elevator car control module and the monitoring image acquisition module, and is used to, after switching to the second state, continue to acquire the elevator monitoring image acquired in real time by the elevator camera, and each time a new elevator monitoring image is acquired, use the elevator camera as the box camera, use the prohibited items focus area as the target focus area, use the elevator monitoring image acquired by the elevator camera when the elevator car is empty as the box background image, and use the latest continuous multiple frames of elevator monitoring images acquired by the elevator camera as the continuous multiple frames of box monitoring images, start the feature vector extraction unit, the feature map construction unit, the occlusion behavior recognition unit, the first judgment unit, the second judgment unit and the event occurrence determination unit, and obtain a determination result reflecting whether a malicious camera occlusion event has occurred;

[0057] The feature vector extraction unit is used to extract spatial feature vectors of multiple human joints at the time of each frame image acquisition from the continuous multiple frames of the in-box monitoring images acquired by the in-box camera, wherein the in-box camera is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body;

[0058] The feature map construction unit is used to construct a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected;

[0059] The occlusion behavior recognition unit is used to input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and has been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior;

[0060] The first judgment unit is used to judge whether the first confidence prediction value is greater than a first preset threshold;

[0061] The second judgment unit is used to, when determining that the first confidence prediction value is greater than the first preset threshold, use the last frame image in the continuous multiple frames of in-box monitoring images as the current monitoring image, and judge whether the target focus area is blocked based on the first focus area picture in the current monitoring image and corresponding to the target focus area and the second focus area picture in the in-box background image and corresponding to the target focus area, by comparing the texture and brightness, wherein the in-box background image is pre-acquired by the in-box camera when the box is empty;

[0062] The event occurrence determination unit is used to determine that a malicious camera blocking event occurs when it is determined that the target focus area is blocked;

[0063] The elevator car control module is further configured to control the elevator car to switch from the second state to a third state for continuing to maintain the elevator car door open when the determination result indicates that a malicious camera blocking event has occurred;

[0064] The contraband identification module is further used to, after switching to the third state, continue to obtain the elevator surveillance image collected by the elevator camera in real time, and each time a new elevator surveillance image is obtained, input the new elevator surveillance image as another image sample to be tested into the target contraband identification model to obtain a second identification result reflecting whether the target contraband exists;

[0065] The judgment module is respectively connected to the focus area determination module, the elevator car control module and the monitoring image acquisition module, and is used to continue to acquire the elevator monitoring image collected by the elevator camera in real time after switching to the third state, and each time a new elevator monitoring image is acquired, it is judged whether the contraband focus area is not blocked according to the third focus area picture corresponding to the contraband focus area in the new elevator monitoring image and the fourth focus area picture corresponding to the contraband focus area in the box background image through comparison results in texture and brightness;

[0066] The elevator car control module is further configured to control the elevator car to switch back to the first state when the second recognition result indicates that there is no target prohibited item and it is determined that the prohibited item focus area is not blocked.

[0067] In a fifth aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive information, and the processor is used to read the computer program to execute the in-box camera occlusion detection method as described in the first aspect or any possible design of the first aspect / the elevator operation control method as described in the third aspect or any possible design of the third aspect.

[0068] In a sixth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the method for detecting camera occlusion in a box in the first aspect or any possible design of the first aspect / the method for controlling elevator operation in the third aspect or any possible design of the third aspect is executed.

[0069] In the seventh aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the in-box camera occlusion detection method as described in the first aspect or any possible design of the first aspect / the elevator operation control method as described in the third aspect or any possible design of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 This is an example of a scenario in which a user maliciously blocks the camera after pushing a cart into an elevator.

[0072] Figure 2 It is a flow chart of the method for detecting camera occlusion in a box provided by the present invention.

[0073] Figure 3 This is an example diagram of the correspondence between the joint point labels and the joint points of the COCO18 model provided by the present invention.

[0074] Figure 4 It is a structural schematic diagram of the camera occlusion detection device in a box provided by the present invention.

[0075] Figure 5 It is a flow chart of the elevator operation control method provided by the present invention.

[0076] Figure 6 It is a structural schematic diagram of the elevator operation control device provided by the present invention.

[0077] Figure 7 It is a structural schematic diagram of the computer device provided by the present invention.

[0078] In the above drawings: 1- elevator car; 11- elevator car door; 2- camera inside the elevator; 3- human body; 4- battery car; 5- obstruction. DETAILED DESCRIPTION

[0079] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that although the description of these embodiments is used to help understand the present invention, it does not constitute a limitation of the present invention. The specific structures and functional details disclosed herein are only used to describe the embodiments of the present invention. However, the present invention can be embodied in many alternative forms, and it should not be understood that the present invention is limited to the embodiments set forth herein.

[0080] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0081] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can represent three situations: A exists alone, B exists alone, or A and B exist at the same time; the term " / and" that may appear in this article describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can represent two situations: A exists alone or A and B exist at the same time; in addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0082] like Figures 2-3As shown, the in-box camera occlusion detection method provided in the first aspect of this embodiment can be executed by, but is not limited to, a computer device with certain computing resources, such as a personal computer (PC, referring to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PAD), wearable devices, elevator control devices or camera devices (which can obtain real-time monitoring video images and are connected to the elevator control device) and other electronic devices, so as to promptly detect someone evading detection of the target area of ​​interest, facilitate accurate actions such as blocking the elevator, and effectively block the passage of electric vehicles illegally entering the room. Figure 2 As shown, the method for detecting camera occlusion in a box may include, but is not limited to, the following steps S1 to S6.

[0083] S1. After acquiring a continuous multi-frame monitoring image captured by a camera inside the box, extract spatial feature vectors of multiple human joints at the time of capturing each frame of the image from the continuous multi-frame monitoring image, wherein the camera inside the box is installed on the top of the box, and the lens field of view is fixed to cover the ground area inside the box, and the multiple human joints belong to the same human body and can represent the human body's camera blocking behavior.

[0084] In step S1, the camera inside the box is used to collect real-time monitoring images inside the box, wherein the box can be, but is not limited to, an elevator car or a carriage, etc., where prohibited items such as battery vehicles are explicitly prohibited from entering. Figure 1As shown, an in-lift camera 2 serving as the in-carriage camera is installed in the elevator car 1. The in-lift camera 2 is located at the rear of the inner top of the elevator car 1 and faces the elevator car door 11, and the lens field of view is fixed to cover the entire floor area and the elevator car door area of ​​the elevator car 1, so that the in-lift camera 2 can capture the entire process of the human body 3 pushing the battery vehicle 4 into the elevator. The computer device can obtain the in-carriage monitoring image captured in real time by the in-carriage camera by connecting the camera via wired or wireless communication. The continuous multi-frame in-carriage monitoring image is a segment of in-carriage monitoring video. For example, 30 continuous frames of in-carriage monitoring images can constitute an in-carriage monitoring video with a duration of 1 second. In addition, considering that the box door is generally open when prohibited items such as electric bicycles are put into the box, the camera inside the box may capture images outside the box door. Therefore, after obtaining the monitoring image, the effective detection area in the monitoring image (for example, the internal area of ​​the elevator car 1, and the area outside the elevator car door 11 is not detected) can be delineated by a preset area (that is, because the lens of the camera inside the box is fixed), thereby capturing the image of the effective detection area as the monitoring image inside the box.

[0085] In the step S1, specifically, spatial feature vectors of multiple human joints at the time of each frame image acquisition are extracted from the continuous multi-frame in-box monitoring images, including but not limited to the following steps S11 to S13.

[0086] S11. For each frame of the continuous multi-frame in-box monitoring images, use the human posture recognition project OpenPose software to identify the corresponding eighteen human joints marked according to the COCO18 model joint point labels from the corresponding picture, wherein the eighteen human joints belong to the same human body.

[0087] In step S11, the human posture recognition project OpenPose software is an open source software developed by Carnegie Mellon University (CMU) in the United States based on convolutional neural networks and supervised learning and using Caffe as the framework. It can realize posture estimation such as human movements, facial expressions and finger movements, and obtain the spatial position information of each joint point of the human skeleton in the input image, etc., wherein the COCO18 mode joint point labeling is the existing node labeling model, and the order and corresponding positions of the labeled 18 joint points can be as shown in Figure 3As shown. Based on the existing human posture recognition project OpenPose software and the COCO18 model joint point labels, the following 18 corresponding joint points can be identified from each frame image: nose node (corresponding to label 0), head node (corresponding to label 1), right shoulder node (corresponding to label 2), right elbow node (corresponding to label 3), right hand head node (corresponding to label 4), left shoulder node (corresponding to label 5), left elbow node (corresponding to label 6), left hand head node (corresponding to label 7), right waist node (corresponding to label 8), right knee node (corresponding to label 9), right foot head node (corresponding to label 10), left waist node (corresponding to label 11), left knee node (corresponding to label 12), left foot head node (corresponding to label 13), right eye node (corresponding to label 14), left eye node (corresponding to label 15), right ear node (corresponding to label 16) and left ear node (corresponding to label 17), etc.

[0088] S12. For each frame of image, select a plurality of preset human joint points from the corresponding eighteen human joint points that can represent the human body's camera blocking behavior.

[0089] In the step S12, the multiple human joints are the joints that play a key role in identifying the camera blocking behavior of the human body. According to actual considerations and multiple tests by the inventor, 12 joints including the nose node, head node, right shoulder node, right elbow node, right right head node, left shoulder node, left elbow node, left hand head node, right eye node, right ear node, left eye node and left ear node are preferably selected to constitute the multiple human joints.

[0090] S13. For each frame of image, all spatial coordinates of the corresponding multiple human joints in the camera coordinate system of the camera in the box are used as spatial feature vectors of the multiple human joints at the time of corresponding acquisition.

[0091] In the step S13, the coordinates of the human joints in the camera coordinate system can be two-dimensional coordinates or three-dimensional coordinates. Therefore, the camera in the box is preferably a camera with a binocular lens, so as to directly obtain the three-dimensional coordinates of each pixel point (including the pixel points corresponding to the human joints) in the monitoring image in the box based on the binocular ranging principle, to ensure the accuracy of the construction of the subsequent skeleton sequence spatiotemporal feature map and the accuracy of the final camera occlusion detection result in the box. In addition, if the estimated confidence of each human joint point is also obtained through the human posture recognition project OpenPose software, the estimated confidence can also be included in the spatial feature vector of the corresponding human joint point.

[0092] S2. Constructing a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior based on the spatial feature vectors of the multiple human joints when each frame of image is collected.

[0093] In the step S2, specifically, the skeleton sequence spatiotemporal feature map is represented by G=(V, E), wherein G represents the skeleton sequence spatiotemporal feature map, V represents a set of spatial feature vectors, and V={v ti |t=0,1,2,…,T;i=0,1,2,…,N}, t represents the frame number in the continuous multi-frame in-box monitoring image, T represents the maximum frame number in the continuous multi-frame in-box monitoring image, i represents the joint point number in the multiple human joint points, N represents the maximum joint point number in the multiple human joint points, v ti represents the spatial feature vector corresponding to the frame number t and the joint point number i, E represents the graph structure and contains the edge structure ES={v ti v tj |(i,j)∈H} and the edge structure EF in the time domain = {v ti v (t+1)i}, H represents the set of natural connection pairs of human joints, v ti v tj Represents the spatial feature vector v ti To the spatial eigenvector v tj The edge vector, v ti v (t+1)i Represents the spatial feature vector v ti To the spatial eigenvector v (t+1)i 's edge vector. Based on the foregoing specific description, the skeleton sequence spatiotemporal feature graph can be constructed by existing mathematical knowledge according to the spatial feature vectors of the multiple human joints at the time of each frame image acquisition. In detail, when the multiple human joints include a nose node, a head node, a right shoulder node, a right elbow node, a right hand head node, a left shoulder node, a left elbow node, a left hand head node, a right eye node, a right ear node, a left eye node and a left ear node, the value of N is 17 (i.e., the joint point number of the left ear node), and the set H = {(4,3), (3,2), (2,1), (1,5), (5,6), (6,7), (0,1), (0,14), (0,15), (14,16), (15,17)}.

[0094] S3. Input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model that has been trained and based on a neural network, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior.

[0095] In the step S3, the neural network is a complex network system formed by a large number of simple processing units (called neurons) widely connected to each other, which reflects many basic characteristics of human brain functions and is a highly complex nonlinear dynamic learning system. Therefore, through a conventional sample training method, the camera occlusion behavior recognition model for identifying whether there is a camera occlusion behavior according to the input data can be obtained, so that after the test sample is input, the corresponding recognition result and the confidence prediction value and other information can be output. Specifically, the neural network can be but not limited to the back propagation BP (Back Propagation) network, Hopfield network, adaptive resonance theory ART (Adaptive Resonance Theory) network, Kohonen network, spatial temporal graph convolutional network SG-GCN (Spatial Temporal Graph Convolutional Networks, a model based on this network is proposed in the existing paper "Spatial Temporal Graph Convolutional Networks for Skeleton Based Action Recognition", which can solve the problem of human action recognition based on key points of the human skeleton), etc. In detail, the camera occlusion behavior recognition model is preferably trained using a model based on the spatiotemporal graph convolutional network SG-GCN: first, multiple in-box surveillance videos collected when a human body performs camera occlusion behavior are obtained, and then each in-box surveillance video is decoded into a continuous multi-frame in-box surveillance image, and the corresponding skeleton sequence spatiotemporal feature map is constructed through the aforementioned steps S1 to S2. Finally, these feature map sets and corresponding labeling information are used as multiple positive samples and sent to the SG-GCN-based model for camera occlusion behavior recognition training, so as to obtain a recognition result that can output a higher confidence level when identifying camera occlusion behavior.

[0096] S4. Determine whether the first confidence prediction value is greater than a first preset threshold.

[0097] In step S4, considering that a person blocking the camera is a prerequisite for determining whether a malicious camera blocking event has occurred, if the first preset threshold is set appropriately, it can be used as a key basis for determining whether a malicious camera blocking event has occurred. The value range of the first preset threshold is (0, 1). For example, the first preset threshold can be determined to be 0.6 based on experiments.

[0098] S5. If it is determined that the first confidence prediction value is greater than the first preset threshold, the last frame image in the continuous multiple frames of in-box monitoring images is used as the current monitoring image, and based on the first focus area image in the current monitoring image and corresponding to the target focus area and the second focus area image in the background image in the box and corresponding to the target focus area, it is determined whether the target focus area is blocked by comparing the texture and brightness, wherein the background image in the box is pre-captured by the camera in the box when the box is empty.

[0099] In step S5, if it is determined that the first confidence prediction value is greater than the first preset threshold, it is considered that a human body has blocked the camera. As to whether the behavior is intentional or unintentional, it needs to be determined based on whether a blocking effect is produced. Therefore, it can be judged based on the last frame image in the continuous multi-frame in-box monitoring image whether the camera blocking behavior produces a blocking effect. If so, it indicates that a malicious camera blocking incident has occurred. Otherwise, it can be considered that the camera blocking behavior was performed unconsciously. Therefore, further, according to the first focus area picture corresponding to the target focus area in the current monitoring image and the second focus area picture corresponding to the target focus area in the background image in the box, by comparing the results in texture and brightness, it is determined whether the target focus area is blocked, so as to achieve the purpose of determining whether the camera blocking behavior produces a blocking effect, wherein the target focus area is a certain area in the box, which can be predetermined, for example, an area where contraband such as electric vehicles have recently appeared in the box; the background image in the box also needs to be acquired in advance, and when the camera in the box is collected in advance when the box is empty to obtain multiple images in the box, the background image in the box can be obtained after the multiple images in the box are averaged (for example, the grayscale mean of each pixel on the multiple images in the box is calculated). In addition, if it is determined that the first confidence prediction value is less than or equal to the first preset threshold, it can be considered that there is no camera blocking behavior, and there is no malicious camera blocking event.

[0100] S6. If it is determined that the target focus area is blocked, it is determined that a malicious camera blocking event has occurred.

[0101] In step S6, considering the blocking effect is another necessary condition for determining the occurrence of a malicious camera blocking event. Therefore, if it is determined that the target area of ​​interest is blocked, it can be considered that this camera blocking behavior has triggered a malicious camera blocking event, and then it can be found in time that someone is evading the detection of the target area of ​​interest, which is convenient for accurate actions such as blocking the elevator, and is conducive to effectively blocking the passage of battery vehicles illegally entering the room. On the contrary, if it is determined that the target area of ​​interest is not blocked, it can be considered that this camera blocking behavior is unintentional, and no one is evading the detection of the target area of ​​interest.

[0102] Based on the in-box camera occlusion detection method described in the aforementioned steps S1 to S6, a solution for detecting malicious camera occlusion behavior in combination with scenes is provided, that is, first, based on the continuous multi-frame in-box monitoring images collected by the in-box camera, a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior is constructed, and then the skeleton sequence spatiotemporal feature map is imported into the camera occlusion behavior recognition model based on the neural network and has been trained, and the corresponding confidence prediction value for identifying camera occlusion behavior is output, and then, through the comparison result of the confidence prediction value and the preset threshold, when it is determined that there is camera occlusion behavior, it is further determined whether an occlusion effect has been generated, and if so, it is determined that a malicious camera occlusion event has occurred, and then it can be found in time that someone is evading the detection of the target focus area, which is convenient for accurate actions such as blocking the elevator, which is conducive to effectively blocking the passage of battery vehicles illegally entering the room. At the same time, because the camera occlusion problem is divided into malicious occlusion behavior according to the actual scene, the detection method can be distinguished from the situation where the ordinary camera is blocked, and the situation that the traditional method cannot judge whether the camera is maliciously blocked at the semantic level is solved, which is more targeted. In addition, this detection method can be applied to complex scenarios that may be encountered after electric bicycles enter the elevator. It can comprehensively use object detection technology, behavior recognition technology, the camera occlusion detection method inside the box and the elevator control status logic to provide a feasible elevator blocking solution, so as to effectively block the electric bicycles from illegally entering the room.

[0103] Based on the technical solution of the first aspect mentioned above, this embodiment further specifically provides a possible design for determining whether an occlusion effect occurs, namely, according to a first focus area picture in the current monitoring image and corresponding to the target focus area and a second focus area picture in the background image in the box and corresponding to the target focus area, by comparing the texture and brightness, determining whether the target focus area is occluded, including but not limited to the following steps S51 to S57.

[0104] S51. According to the predetermined target area of ​​interest, a corresponding first area of ​​interest screen is captured from the current monitoring image, and a corresponding second area of ​​interest screen is captured from the background image inside the box, wherein the background image inside the box is pre-captured by the camera inside the box when the box is empty.

[0105] In step S51, the target area of ​​interest may be, but is not limited to, a rectangular area, a circular area, a regular hexagonal area or the like. Since the lens field of view of the camera inside the box is fixed to cover the ground area inside the box, the first area of ​​interest image and the second area of ​​interest image are images of the same area and are imaged at different times, which can be used for subsequent comparisons in texture and brightness.

[0106] S52. Divide the target region of interest into a plurality of sub-regions of the same size.

[0107] In the step S52, for example, when the target area of ​​interest is a rectangular area, the sub-area is a smaller rectangle, and the multiple sub-areas including P*Q small rectangles are obtained, where P and Q are positive integers greater than 2 respectively.

[0108] S53. For each of the multiple sub-areas, based on the corresponding picture information in the first focus area picture, the corresponding first directional gradient histogram eigenvalue and the first average grayscale value are calculated, and based on the corresponding picture information in the second focus area picture, the corresponding second directional gradient histogram eigenvalue and the second average grayscale value are calculated.

[0109] In the step S53, the first directional gradient histogram eigenvalue and the second directional gradient histogram eigenvalue are respectively HOG (Histogram of Oriented Gradient) eigenvalues, which are feature descriptors used for object detection in computer vision and image processing, representing features on texture; the first average grayscale value and the second average grayscale value are respectively the average grayscale of all pixels on the corresponding screen, representing features on brightness, and therefore the aforementioned directional gradient histogram eigenvalue and the aforementioned average grayscale value of each sub-region can be calculated based on the screen information based on conventional calculation methods.

[0110] S54. For each of the sub-regions, if it is determined that the similarity between the corresponding first directional gradient histogram eigenvalue and the corresponding second directional gradient histogram eigenvalue is less than the second preset threshold, it is determined that the corresponding first average grayscale value is less than the corresponding second average grayscale value, and it is determined that the corresponding first average grayscale value is less than the third preset threshold, then it is determined that the corresponding sub-region is currently in an obscured state.

[0111] In the step S54, the second preset threshold and the third preset threshold can be obtained in advance based on experiments. When the three conditions of a sub-region (i.e., the similarity between the corresponding first directional gradient histogram eigenvalue and the corresponding second directional gradient histogram eigenvalue is less than the second preset threshold, the corresponding first average grayscale value is less than the corresponding second average grayscale value, and the corresponding first average grayscale value is less than the third preset threshold) are met at the same time, it can be considered that the sub-region has produced an occlusion effect in texture and brightness, otherwise it cannot be determined.

[0112] S55. Obtain a first ratio of the number of sub-regions currently in an obstructed state to the total number of sub-regions of the plurality of sub-regions.

[0113] In the step S55, for example, if the total number of the sub-regions is 100, and it is determined through the aforementioned step S54 that 67 sub-regions are currently in an obscured state, then the first ratio is 67%.

[0114] S56. Determine whether the first ratio is greater than a fourth preset threshold.

[0115] In step S56, the fourth preset threshold can also be obtained in advance based on experiments, for example, 68%, which means that if most sub-areas are currently in an occluded state, it is considered that the target focus area has an occlusion effect in texture and brightness, otherwise it cannot be determined.

[0116] S57. If yes, determine that the target area of ​​interest is blocked.

[0117] Therefore, based on the possible design one described in the aforementioned steps S51 to S57, it is possible to effectively determine whether the target area of ​​interest is obscured by comparing the texture and brightness of the current monitoring image and the background image in the box that corresponds to the target area of ​​interest, thereby achieving the purpose of judging whether an obstruction effect occurs and ensuring the accuracy of subsequent determination of a malicious camera obstruction event.

[0118] like Figure 4 As shown, the second aspect of this embodiment provides a virtual device for implementing the method for detecting camera occlusion in a box as described in the first aspect or any possible design of the first aspect, comprising a feature vector extraction unit, a feature map construction unit, an occlusion behavior recognition unit, a first judgment unit, a second judgment unit and an event occurrence determination unit which are sequentially connected in communication;

[0119] The feature vector extraction unit is used to extract spatial feature vectors of multiple human joints at the time of each frame image acquisition from the continuous multiple frames of the in-box monitoring images acquired by the in-box camera, wherein the in-box camera is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body;

[0120] The feature map construction unit is used to construct a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected;

[0121] The occlusion behavior recognition unit is used to input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and has been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior;

[0122] The first judgment unit is used to judge whether the first confidence prediction value is greater than a first preset threshold;

[0123] The second judgment unit is used to, when determining that the first confidence prediction value is greater than the first preset threshold, use the last frame image in the continuous multiple frames of in-box monitoring images as the current monitoring image, and judge whether the target focus area is blocked based on the first focus area picture in the current monitoring image and corresponding to the target focus area and the second focus area picture in the in-box background image and corresponding to the target focus area, by comparing the texture and brightness, wherein the in-box background image is pre-acquired by the in-box camera when the box is empty;

[0124] The event occurrence determination unit is used to determine that a malicious camera blocking event occurs when it is determined that the target focus area is blocked.

[0125] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be referred to the in-box camera occlusion detection method described in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0126] like Figure 5 As shown, the third aspect of this embodiment provides an elevator operation control method based on the in-box camera occlusion detection method described in the first aspect or any possible design of the first aspect, which can also be executed by, but is not limited to, a computer device with certain computing resources, such as a personal computer (PC, referring to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PAD), wearable devices, elevator control devices or camera devices (which can obtain real-time monitoring video images and are connected to the elevator control device) and other electronic devices. It can prevent pedestrians from maliciously occluding the target contraband and mistakenly believing that the target contraband has left the elevator car, thereby avoiding the scenario where the target contraband such as battery vehicles can successfully enter the room by elevator due to escaping detection. Figure 5 As shown, the elevator operation control method may include but is not limited to the following steps S100 to S700.

[0127] S100. After the elevator car door is opened, obtain the elevator monitoring image collected in real time by the elevator camera, wherein the elevator camera is installed on the top of the elevator car and faces the elevator car door, and the lens field of view is fixed to cover the ground area in the elevator car.

[0128] In the step S100, the camera in the elevator is the camera in the box in step S1, and the monitoring image in the elevator is the monitoring image in the box in step S1. Their specific details can be found in the aforementioned step S1 and will not be repeated here.

[0129] S200. Input the surveillance image inside the elevator as an image sample to be tested into a target contraband recognition model that has been trained based on a target detection algorithm to obtain a first recognition result reflecting whether there is a target contraband.

[0130] In the step S200, the target detection algorithm is an existing artificial intelligence recognition algorithm for identifying objects in a picture and marking the position of the objects. Specifically, it can be but not limited to Faster R-CNN (Faster Regions with Convolutional Neural Networks features, a target detection algorithm proposed by He Kaiming et al. in 2015, which won multiple first places in the 2015 ILSVRV and COCO competitions) target detection algorithm, SSD (Single Shot MultiBox Detector, a target detection algorithm proposed by Wei Liu at ECCV 2016, which is one of the currently popular main detection frameworks) target detection algorithm or YOLO (You only look once, currently the latest version has been developed to V4, and is also widely used in the industry. Its basic principle is: first, the input image is divided into 7x7 grids, 2 borders are predicted for each grid, and then the target windows with relatively low probability are removed according to the threshold, and finally the redundant windows are removed by border merging to obtain the detection result) target detection algorithm, etc. Therefore, through conventional sample training methods, the target contraband recognition model can be obtained for identifying whether there are target contraband according to the input image, so that after the test image is input, the corresponding recognition result and confidence prediction value and other information can be output. For example, the target detection algorithm preferably adopts the YOLO V4 target detection algorithm, and the target contraband can be but is not limited to a battery vehicle.

[0131] Before the step S200, specifically, the model training can be performed in the following manner but is not limited to: first, N ladder sample images collected when the target contraband (such as electric bicycles, etc.) are put into the box, where N represents a natural number greater than or equal to five; then, for each of the N ladder sample images, after obtaining the manually annotated rectangular frame where the target contraband is located, the rectangular frame where the target contraband is located and the frame position are loaded into the corresponding image to obtain the corresponding image feature positive sample; then, according to the N image feature positive samples corresponding to the N ladder sample images one by one, n image feature positive samples are extracted to form an image feature positive sample set, where n represents a natural number greater than one and less than N, and the larger the value of N, the better, for example, 1000; finally, the image feature positive sample set is input into the YOLO target detection model for training to obtain the target contraband recognition model for identifying whether there are target contraband according to the input image. Specifically, 80% of the image feature positive samples can be randomly selected from the N image feature positive samples to form the image feature positive sample set; as for the remaining image feature positive samples, they can also form a test set so as to input the trained target contraband recognition model to judge the recognition accuracy of the target contraband recognition model and then decide whether it is usable. The acquisition method of the sample image in the ladder can refer to the monitoring image in the box, and taking the target contraband as an electric vehicle as an example, it can be collected in the following scenarios (A), (B), (C) and / or (D), etc.: (A) different imaging angles of the camera and / or camera; (B) multiple types of electric vehicles; (C) multiple colors of electric vehicles; (D) the electric vehicle is slightly blocked by a raincoat or other covering; etc.

[0132] S300. If the first recognition result indicates the presence of target contraband, the contraband focus area is determined based on the location of the target contraband in the monitoring image inside the elevator, and the elevator car is controlled to switch from a first state for allowing normal operation of the elevator to a second state for maintaining the elevator car door open.

[0133] In the step S300, specifically, the specific manner in which the first recognition result indicates the existence of the target contraband may include, but is not limited to: after obtaining the second confidence prediction value output by the target contraband recognition model and indicating the existence of the target contraband, if the second confidence prediction value is greater than the seventh preset threshold value (which is obtained based on experiments, for example, 68%), it is determined that the target contraband exists. The specific manner in which the contraband focus area is determined based on the location of the target contraband in the elevator monitoring image may include, but is not limited to: drawing an image area that can cover the entire target contraband area with the location as the center; and directly marking the target contraband box output by the target contraband recognition model. The control of the elevator car is switched from the first state for allowing the elevator to operate normally to the second state for maintaining the elevator car door open, which is used as a blocking behavior control method, by maintaining the elevator car door open, the elevator car is prohibited from closing the door and lifting, so as to urge pedestrians to take the target contraband out of the car. In addition, the first state can be marked with a blocked elevator flag of 0, and the second state can be marked with a blocked elevator flag of 1, so as to realize the writing of the control program and the transmission of control instructions; and if the first recognition result indicates that the target contraband does not exist, the first state is maintained and the step S100 is returned to be executed.

[0134] S400. After switching to the second state, continue to obtain the in-elevator surveillance image collected by the in-elevator camera in real time, and each time a new frame of in-elevator surveillance image is obtained, use the in-elevator camera as the in-box camera, use the contraband focus area as the target focus area, use the in-elevator surveillance image collected by the in-elevator camera when the elevator car is empty as the in-box background image, and use the latest continuous multiple-frame in-elevator surveillance image collected by the in-elevator camera as the continuous multiple-frame in-box surveillance image, start executing the in-box camera occlusion detection method as described in the first aspect or any possible design of the first aspect, and obtain a determination result reflecting whether a malicious camera occlusion event has occurred.

[0135] S500. If the determination result indicates that a malicious camera blocking event has occurred, the elevator car is controlled to switch from the second state to a third state for continuing to maintain the elevator car door open.

[0136] In step S500, the elevator car is controlled to switch from the second state to the third state for continuing to maintain the elevator car door open, which is used as a deep elevator blocking behavior control method to prevent pedestrians from maliciously blocking the target contraband and mistakenly believing that the target contraband has left the elevator car, thereby avoiding the scenario where the target contraband such as electric vehicles can successfully enter the room by elevator due to escape detection. Also, in order to facilitate the writing of control programs and the transmission of control instructions, the third state can be marked with the elevator blocking flag 2. In addition, if the determination result indicates that no malicious camera blocking event has occurred, the second state is maintained and the next time a new frame of elevator surveillance image is obtained, the in-box camera blocking detection method as described in the first aspect or any possible design of the first aspect is started to continue the in-box camera blocking detection.

[0137] S600. After switching to the third state, continue to obtain the elevator surveillance image collected by the elevator camera in real time, and each time a new elevator surveillance image is obtained, the new elevator surveillance image is used as another image sample to be tested and input into the target contraband recognition model to obtain a second recognition result reflecting whether the target contraband exists, and based on the third focus area picture in the new elevator surveillance image and corresponding to the contraband focus area and the fourth focus area picture in the box background image and corresponding to the contraband focus area, judge whether the contraband focus area is not blocked by comparing the texture and brightness results.

[0138] In the step S600, the new elevator surveillance image is input as another image sample to be tested into the target contraband identification model to obtain a specific description of the second identification result for reflecting whether the target contraband exists. Please refer to the aforementioned step S200 for details, which will not be repeated here. According to the third focus area picture corresponding to the contraband focus area in the new elevator surveillance image and the fourth focus area picture corresponding to the contraband focus area in the box background image, by comparing the texture and brightness, it is determined whether the contraband focus area is not blocked, which can achieve the purpose of effectively judging whether the blocking effect disappears after determining that a malicious camera blocking event has occurred, wherein the box background image, i.e., the elevator background image, also needs to be obtained in advance, and when the elevator camera in advance collects multiple elevator images when the elevator car is empty, the elevator background image can be obtained after the multiple elevator images are averaged (for example, the grayscale mean of each pixel point on the multiple elevator images is calculated).

[0139] S700. If the second recognition result indicates that there is no target prohibited item and it is determined that the prohibited item focus area is not blocked, control the elevator car to switch back to the first state.

[0140] In step S700, if the second recognition result indicates that there is no target contraband and it is determined that the contraband focus area is not blocked, it means that the blocking effect disappears and there is no target contraband in the elevator, and the previously locked target contraband has left the elevator car. At this time, the elevator car can be restored to the normal operation mechanism of the elevator (for example, normal closing of the door and lifting), and the step S100 is returned to be executed. In addition, if the second recognition result indicates that there is no target contraband and it is determined that the contraband focus area is still blocked, it is possible that the target contraband has escaped detection. At this time, no state change is made, and the step S600 is returned to be executed; and if it is other cases (that is, the second recognition result indicates that there is a target contraband), it means that the target contraband is still in the elevator car, and no state change is made, so as to return to the step S600.

[0141] Preferably, after step S300, the method further includes but is not limited to the following steps S301 to S302: S301. After switching to the second state, continue to obtain the elevator surveillance image collected by the elevator camera in real time, and each time a new elevator surveillance image is obtained, the new elevator surveillance image is input into the target contraband recognition model as another image sample to be tested to obtain a third recognition result reflecting whether the target contraband exists; S302. If the third recognition result indicates that the target contraband does not exist and the determination result indicates that no malicious camera blocking event has occurred, control the elevator car to switch back to the first state. Since the third recognition result indicates that the target contraband does not exist and the determination result indicates that no malicious camera blocking event has occurred, it can be indicated that the target contraband has left the elevator car at this time, and the elevator car can be restored to the normal operation mechanism of the elevator, and return to execute step S100. In addition, if the third recognition result indicates that the target contraband does not exist and the determination result indicates that no malicious camera obstruction event has occurred, it means that the target contraband is still in the elevator car, and the state is still not changed, so as to return to execute the steps S301 and S400; and as for other situations (that is, the determination result indicates that a malicious camera obstruction event has occurred), the elevator car is directly controlled to switch from the second state to the third state for continuing to maintain the elevator car door open.

[0142] Therefore, based on the elevator operation control method described in the aforementioned steps S100 to S700, a solution for blocking elevators based on the detection results of malicious camera blocking behavior is provided, that is, after the elevator car door is opened, it is first identified whether any target contraband has entered the elevator car. If so, an elevator blocking behavior is performed, and it is determined whether a malicious camera blocking event has occurred. If it is determined that it has occurred, a deep elevator blocking behavior is performed, and it is continued to be determined whether the target contraband is still in the elevator car and whether the blocking effect has disappeared. Only when it is determined that the target contraband is not in the elevator car and the blocking effect has disappeared, the elevator blocking behavior is cancelled, and the elevator car is restored to its normal operating mechanism, thereby preventing pedestrians from maliciously blocking the target contraband and mistakenly believing that the target contraband has left the elevator car, thereby avoiding the scenario where target contraband such as electric vehicles can successfully enter the room by elevator due to escape detection.

[0143] Based on the technical solution of the third aspect mentioned above, this embodiment further specifically provides a possible design 2 for determining whether the occlusion effect disappears, that is, according to the third area of ​​interest picture in the monitoring image in the new elevator and corresponding to the contraband area of ​​interest and the fourth area of ​​interest picture in the background image in the box and corresponding to the contraband area of ​​interest, by comparing the results in texture and brightness, it is determined whether the contraband area of ​​interest is not occluded, including but not limited to the following steps S601 to S607.

[0144] S601. According to the contraband focus area, a corresponding third focus area image is captured from the new elevator monitoring image, and a corresponding fourth focus area image is captured from the box background image.

[0145] In step S601, the prohibited items area of ​​concern may be, but is not limited to, a rectangular area, a circular area or a regular hexagonal area. Since the lens field of view of the in-elevator camera is fixed to cover the ground area in the elevator car, the third area of ​​concern image and the fourth area of ​​concern image are images of the same area and are imaged at different times, which can be used for subsequent comparisons in texture and brightness.

[0146] S602: Divide the contraband area of ​​concern into multiple sub-blocks of the same size.

[0147] In step S602, for example, when the contraband concern area is a rectangular area, the sub-block is a smaller rectangle, and the multiple sub-blocks including S*T small rectangles are obtained, where S and T are positive integers greater than 2 respectively.

[0148] S603. For each sub-block among the multiple sub-blocks, based on the corresponding picture information in the third focus area picture, the corresponding third directional gradient histogram eigenvalue and the third average grayscale value are calculated, and based on the corresponding picture information in the fourth focus area picture, the corresponding fourth directional gradient histogram eigenvalue, the fourth average grayscale value and the grayscale standard deviation are calculated.

[0149] In the step S603, the third oriented gradient histogram eigenvalue and the fourth oriented gradient histogram eigenvalue are respectively HOG (Histogram of Oriented Gradient) eigenvalues, which are also feature descriptors used for object detection in computer vision and image processing, representing features on texture; the third average grayscale value and the fourth average grayscale value are respectively the average grayscale of all pixels on the corresponding picture, representing features on brightness, and therefore the oriented gradient histogram eigenvalue, the average grayscale value and the grayscale standard deviation of each sub-block can be calculated based on the picture information based on conventional calculation methods.

[0150] S604. For each of the sub-blocks, if it is determined that the similarity between the corresponding third-direction gradient histogram eigenvalue and the corresponding fourth-direction gradient histogram eigenvalue is greater than the fifth preset threshold and it is determined that the absolute difference between the corresponding first average grayscale value and the corresponding second average grayscale value is less than the corresponding grayscale standard deviation, it is determined that the corresponding sub-block is currently in an unobstructed state.

[0151] In the step S604, the fifth preset threshold can be obtained in advance based on experiments. When two conditions of a sub-block (i.e., the similarity between the corresponding third-direction gradient histogram eigenvalue and the corresponding fourth-direction gradient histogram eigenvalue is greater than the fifth preset threshold and the absolute difference between the corresponding first average grayscale value and the corresponding second average grayscale value is less than the corresponding grayscale standard deviation) are met at the same time, it can be considered that the occlusion effect of the sub-block has disappeared in terms of texture and brightness, otherwise it cannot be determined.

[0152] S605. Obtain a second ratio of the number of sub-blocks currently in an unblocked state to the total number of sub-blocks of the plurality of sub-blocks.

[0153] In the step S605, for example, if the total number of the sub-blocks is 100, and it is determined through the aforementioned step S604 that 67 sub-blocks are currently in an unblocked state, then the second ratio is 67%.

[0154] S606. Determine whether the second ratio is greater than a sixth preset threshold.

[0155] In step S606, the sixth preset threshold can also be obtained in advance based on experiments, for example, 68%, which means that if most of the sub-blocks are currently in an unobstructed state, it is considered that the contraband focus area has disappeared the occlusion effect in texture and brightness, otherwise it cannot be determined.

[0156] S607. If yes, determine that the contraband area of ​​concern is not blocked.

[0157] Therefore, based on the possible design 2 described in the aforementioned steps S601 to S607, the image corresponding to the prohibited items area of ​​concern can be compared in texture and brightness to effectively determine whether the prohibited items area of ​​concern is not blocked, thereby achieving the purpose of determining whether the blocking effect has disappeared and ensuring the accuracy of the subsequent control of the elevator car to restore to a normal state. In addition, different detection strategies are designed for the two situations of whether the camera area of ​​concern is blocked and whether it is blocked again after being maliciously blocked, depending on the different scene requirements, which can be more targeted and ensure the accuracy and timeliness of making / canceling the elevator blocking behavior.

[0158] like Figure 6 As shown, the fourth aspect of this embodiment provides a virtual device for implementing the elevator operation control method described in the third aspect or any possible design of the third aspect, including a monitoring image acquisition module, a contraband identification module, a focus area determination module, an elevator car control module, a malicious occlusion detection module and a judgment module, wherein the malicious occlusion detection module includes a feature vector extraction unit, a feature map construction unit, an occlusion behavior recognition unit, a first judgment unit, a second judgment unit and an event occurrence determination unit which are sequentially connected in communication;

[0159] The monitoring image acquisition module is used to acquire the monitoring image in the elevator collected by the elevator camera in real time after the elevator car door is opened, wherein the elevator camera is installed at the top of the elevator car and faces the elevator car door, and the camera field of view is fixed to cover the ground area in the elevator car;

[0160] The contraband identification module is communicatively connected to the monitoring image acquisition module, and is used to input the monitoring image in the elevator as an image sample to be tested into a target contraband identification model that has been trained based on a target detection algorithm, and obtain a first identification result that reflects whether there is a target contraband;

[0161] The focus area determination module is communicatively connected to the contraband identification module, and is used to determine the contraband focus area according to the location of the target contraband in the surveillance image in the ladder when the first identification result indicates the presence of the target contraband;

[0162] The elevator car control module is communicatively connected to the contraband identification module, and is used to control the elevator car to switch from a first state for allowing the elevator to operate normally to a second state for maintaining the elevator car door open when the first identification result indicates the presence of target contraband;

[0163] The malicious occlusion detection module is respectively communicatively connected to the focus area determination module, the elevator car control module and the monitoring image acquisition module, and is used to, after switching to the second state, continue to acquire the elevator monitoring image acquired in real time by the elevator camera, and each time a new elevator monitoring image is acquired, use the elevator camera as the box camera, use the prohibited items focus area as the target focus area, use the elevator monitoring image acquired by the elevator camera when the elevator car is empty as the box background image, and use the latest continuous multiple frames of elevator monitoring images acquired by the elevator camera as the continuous multiple frames of box monitoring images, start the feature vector extraction unit, the feature map construction unit, the occlusion behavior recognition unit, the first judgment unit, the second judgment unit and the event occurrence determination unit, and obtain a determination result reflecting whether a malicious camera occlusion event has occurred;

[0164] The feature vector extraction unit is used to extract spatial feature vectors of multiple human joints at the time of each frame image acquisition from the continuous multiple frames of the in-box monitoring images acquired by the in-box camera, wherein the in-box camera is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body;

[0165] The feature map construction unit is used to construct a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected;

[0166] The occlusion behavior recognition unit is used to input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and has been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior;

[0167] The first judgment unit is used to judge whether the first confidence prediction value is greater than a first preset threshold;

[0168] The second judgment unit is used to, when determining that the first confidence prediction value is greater than the first preset threshold, use the last frame image in the continuous multiple frames of in-box monitoring images as the current monitoring image, and judge whether the target focus area is blocked based on the first focus area picture in the current monitoring image and corresponding to the target focus area and the second focus area picture in the in-box background image and corresponding to the target focus area, by comparing the texture and brightness, wherein the in-box background image is pre-acquired by the in-box camera when the box is empty;

[0169] The event occurrence determination unit is used to determine that a malicious camera blocking event occurs when it is determined that the target focus area is blocked;

[0170] The elevator car control module is further configured to control the elevator car to switch from the second state to a third state for continuing to maintain the elevator car door open when the determination result indicates that a malicious camera blocking event has occurred;

[0171] The contraband identification module is further used to, after switching to the third state, continue to obtain the elevator surveillance image collected by the elevator camera in real time, and each time a new elevator surveillance image is obtained, input the new elevator surveillance image as another image sample to be tested into the target contraband identification model to obtain a second identification result reflecting whether the target contraband exists;

[0172] The judgment module is respectively connected to the focus area determination module, the elevator car control module and the monitoring image acquisition module, and is used to continue to acquire the elevator monitoring image collected by the elevator camera in real time after switching to the third state, and each time a new elevator monitoring image is acquired, it is judged whether the contraband focus area is not blocked according to the third focus area picture corresponding to the contraband focus area in the new elevator monitoring image and the fourth focus area picture corresponding to the contraband focus area in the box background image through comparison results in texture and brightness;

[0173] The elevator car control module is further configured to control the elevator car to switch back to the first state when the second recognition result indicates that there is no target prohibited item and it is determined that the prohibited item focus area is not blocked.

[0174] The working process, working details and technical effects of the aforementioned device provided in the fourth aspect of this embodiment can be referred to the elevator operation control method described in the third aspect or any possible design of the third aspect, and will not be repeated here.

[0175] like Figure 7As shown, the fifth aspect of this embodiment provides a computer device for executing the camera occlusion detection method in the box as in the first aspect or any of the first aspects / as in the third aspect or any of the third aspects, which may design the elevator operation control method, including a memory, a processor and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive information, and the processor is used to read the computer program, and execute the camera occlusion detection method in the box as in the first aspect or any of the first aspects / as in the third aspect or any of the third aspects. For example, the memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and / or a first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0176] The working process, working details and technical effects of the aforementioned computer equipment provided in the fifth aspect of this embodiment can be referred to as the in-box camera occlusion detection method in the first aspect or any possible design of the first aspect / the elevator operation control method in the third aspect or any possible design of the third aspect, and will not be repeated here.

[0177] The sixth aspect of this embodiment provides a computer-readable storage medium storing instructions including the method for detecting camera occlusion in a box as in the first aspect or any possible design of the first aspect / the method for controlling the operation of an elevator as in the third aspect or any possible design of the third aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for detecting camera occlusion in a box as in the first aspect or any possible design of the first aspect / the method for controlling the operation of an elevator as in the third aspect or any possible design of the third aspect. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0178] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the sixth aspect of this embodiment can be referred to as the in-box camera occlusion detection method in the first aspect or any possible design of the first aspect / the elevator operation control method in the third aspect or any possible design of the third aspect, and will not be repeated here.

[0179] In a seventh aspect, the present embodiment provides a computer program product including instructions, which, when executed on a computer, causes the computer to execute the method for detecting camera occlusion in a box as in the first aspect or any possible design of the first aspect / the method for controlling elevator operation as in the third aspect or any possible design of the third aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0180] Finally, it should be noted that the present invention is not limited to the above optional implementations, and anyone can derive other various forms of products under the enlightenment of the present invention. The above specific implementations should not be understood as limiting the scope of protection of the present invention. The scope of protection of the present invention should be based on the definition in the claims, and the description can be used to interpret the claims.

Claims

1. A method for detecting camera occlusion in a box, characterized in that: include: After obtaining a continuous multi-frame monitoring image captured by a camera in the box, extracting spatial feature vectors of multiple human joints at the time of each frame image capture from the continuous multi-frame monitoring image, wherein the camera in the box is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body; Constructing a skeleton sequence spatiotemporal feature graph for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected; Input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and having been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior; Determining whether the first confidence prediction value is greater than a first preset threshold; If it is determined that the first confidence prediction value is greater than the first preset threshold, the last frame image in the continuous multiple frames of in-box monitoring images is used as the current monitoring image, and according to the first focus area picture corresponding to the target focus area in the current monitoring image and the second focus area picture corresponding to the target focus area in the in-box background image, it is determined whether the target focus area is blocked by comparing the texture and brightness, wherein the in-box background image is collected in advance by the in-box camera when the box is empty; If it is determined that the target focus area is blocked, it is determined that a malicious camera blocking event has occurred.

2. The method for detecting camera occlusion in a box according to claim 1, characterized in that: Extracting spatial feature vectors of multiple human joints at the time of collecting each frame of image from the continuous multi-frame in-box monitoring images includes: For each frame of the continuous multi-frame in-box monitoring image, the human posture recognition project OpenPose software is used to identify the corresponding 18 human joints marked according to the COCO18 model joint point labels from the corresponding picture, wherein the 18 human joints belong to the same human body; For each frame of image, a plurality of human joint points that are preset and can represent the camera blocking behavior of the human body are selected from the corresponding eighteen human joint points; For each frame of image, all spatial coordinates of the corresponding multiple human joint points in the camera coordinate system of the camera in the box are used as spatial feature vectors of the multiple human joint points during corresponding acquisition.

3. The method for detecting camera occlusion in a box according to claim 1, characterized in that: The multiple human joints include a nose node, a head node, a right shoulder node, a right elbow node, a right hand head node, a left shoulder node, a left elbow node, a left hand head node, a right eye node, a right ear node, a left eye node and a left ear node; And / or, the skeleton sequence spatiotemporal feature map is represented by G=(V, E), wherein G represents the skeleton sequence spatiotemporal feature map, V represents a set of spatial feature vectors and V={v ti |t=0,1,2,…,T;i=0,1,2,…,N}, t represents the frame number in the continuous multi-frame in-box monitoring image, T represents the maximum frame number in the continuous multi-frame in-box monitoring image, i represents the joint point number in the multiple human joint points, N represents the maximum joint point number in the multiple human joint points, v ti represents the spatial feature vector corresponding to the frame number t and the joint point number i, E represents the graph structure and contains the edge structure ES={v ti v tj |(i,j)∈H} and the edge structure EF in the time domain = {v ti v (t+1)i }, H represents the set of natural connection pairs of human joints, v ti v tj Represents the spatial feature vector v ti To the spatial eigenvector v tj The edge vector, v ti v (t+1)i Represents the spatial feature vector v ti To the spatial eigenvector v (t+1)i The edge vector of .

4. The method for detecting camera occlusion in a box according to claim 1, characterized in that: According to a first focus area picture corresponding to the target focus area in the current monitoring image and a second focus area picture corresponding to the target focus area in the background image in the box, judging whether the target focus area is blocked by comparing the texture and brightness, including: According to the predetermined target focus area, a corresponding first focus area picture is intercepted from the current monitoring image, and a corresponding second focus area picture is intercepted from the background image in the box, wherein the background image in the box is previously acquired by the camera in the box when the box is empty; Dividing the target area of ​​interest into a plurality of sub-areas of the same size; For each of the plurality of sub-regions, according to the corresponding picture information in the first region of interest picture, a corresponding first directional gradient histogram eigenvalue and a first average grayscale value are calculated, and according to the corresponding picture information in the second region of interest picture, a corresponding second directional gradient histogram eigenvalue and a second average grayscale value are calculated; For each of the sub-regions, if it is determined that the similarity between the corresponding first directional gradient histogram feature value and the corresponding second directional gradient histogram feature value is less than the second preset threshold, it is determined that the corresponding first average grayscale value is less than the corresponding second average grayscale value, and it is determined that the corresponding first average grayscale value is less than the third preset threshold, it is determined that the corresponding sub-region is currently in an obscured state; Obtaining by counting a first ratio of the number of sub-regions currently in an obstructed state to the total number of sub-regions of the plurality of sub-regions; Determining whether the first ratio is greater than a fourth preset threshold; If so, it is determined that the target focus area is blocked.

5. A camera occlusion detection device in a box, characterized in that: It includes a feature vector extraction unit, a feature map construction unit, an occlusion behavior recognition unit, a first judgment unit, a second judgment unit and an event occurrence determination unit which are sequentially connected in communication; The feature vector extraction unit is used to extract spatial feature vectors of multiple human joints at the time of each frame image acquisition from the continuous multiple frames of the in-box monitoring images acquired by the in-box camera, wherein the in-box camera is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body; The feature map construction unit is used to construct a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected; The occlusion behavior recognition unit is used to input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and has been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior; The first judgment unit is used to judge whether the first confidence prediction value is greater than a first preset threshold; The second judgment unit is used to, when determining that the first confidence prediction value is greater than the first preset threshold, use the last frame image in the continuous multiple frames of in-box monitoring images as the current monitoring image, and judge whether the target focus area is blocked based on the first focus area picture in the current monitoring image and corresponding to the target focus area and the second focus area picture in the in-box background image and corresponding to the target focus area, by comparing the texture and brightness, wherein the in-box background image is pre-acquired by the in-box camera when the box is empty; The event occurrence determination unit is used to determine that a malicious camera blocking event occurs when it is determined that the target focus area is blocked.

6. An elevator operation control method, characterized in that: include: After the elevator car door is opened, a monitoring image inside the elevator collected in real time by a camera inside the elevator is obtained, wherein the camera inside the elevator is installed at the top of the elevator car and faces the elevator car door, and the camera field of view is fixed to cover the ground area inside the elevator car; Input the surveillance image in the elevator as an image sample to be tested into a trained target contraband recognition model based on a target detection algorithm to obtain a first recognition result reflecting whether there is a target contraband; If the first recognition result indicates that there is a target contraband, determining a contraband focus area according to the location of the target contraband in the elevator monitoring image, and controlling the elevator car to switch from a first state for allowing the elevator to operate normally to a second state for maintaining the elevator car door open; After switching to the second state, the in-elevator surveillance image captured by the in-elevator camera in real time continues to be acquired, and each time a new frame of in-elevator surveillance image is acquired, the in-elevator camera is used as the in-box camera, the contraband focus area is used as the target focus area, the in-elevator surveillance image captured by the in-elevator camera when the elevator car is empty is used as the in-box background image, and the continuous multiple frames of in-elevator surveillance images newly captured by the in-elevator camera are used as the continuous multiple frames of in-box surveillance images, and the in-box camera occlusion detection method as described in any one of claims 1 to 4 is started to be executed to obtain a determination result reflecting whether a malicious camera occlusion event has occurred; If the determination result indicates that a malicious camera blocking event has occurred, controlling the elevator car to switch from the second state to a third state for continuing to maintain the elevator car door open; After switching to the third state, the elevator surveillance image collected by the elevator camera in real time continues to be acquired, and each time a new elevator surveillance image is acquired, the new elevator surveillance image is input into the target contraband recognition model as another image sample to be tested, to obtain a second recognition result reflecting whether the target contraband exists, and based on the third focus area picture corresponding to the contraband focus area in the new elevator surveillance image and the fourth focus area picture corresponding to the contraband focus area in the box background image, through comparison results in texture and brightness, it is determined whether the contraband focus area is not blocked; If the second recognition result indicates that there is no target contraband and it is determined that the contraband focus area is not blocked, the elevator car is controlled to switch back to the first state.

7. The elevator operation control method according to claim 6, characterized in that: According to the third focus area picture in the monitoring image in the new elevator and corresponding to the contraband focus area and the fourth focus area picture in the background image in the box and corresponding to the contraband focus area, judging whether the contraband focus area is not blocked by comparing the texture and brightness, including: According to the contraband focus area, a corresponding third focus area picture is intercepted from the new elevator monitoring image, and a corresponding fourth focus area picture is intercepted from the box background image; Dividing the contraband area of ​​concern into a plurality of sub-blocks of the same size; For each of the plurality of sub-blocks, according to the corresponding picture information in the third region of interest picture, a corresponding third directional gradient histogram eigenvalue and a third average grayscale value are calculated, and according to the corresponding picture information in the fourth region of interest picture, a corresponding fourth directional gradient histogram eigenvalue, a fourth average grayscale value and a grayscale standard deviation are calculated; For each of the sub-blocks, if it is determined that the similarity between the corresponding third-direction gradient histogram feature value and the corresponding fourth-direction gradient histogram feature value is greater than a fifth preset threshold and it is determined that the absolute difference between the corresponding first average grayscale value and the corresponding second average grayscale value is less than the corresponding grayscale standard deviation, it is determined that the corresponding sub-block is currently in an unobstructed state; Obtaining by counting a second ratio of the number of sub-blocks currently in an unblocked state to the total number of sub-blocks of the plurality of sub-blocks; Determining whether the second ratio is greater than a sixth preset threshold; If so, it is determined that the contraband area of ​​interest is not blocked.

8. An elevator operation control device, characterized in that: It includes a monitoring image acquisition module, a contraband identification module, a focus area determination module, an elevator car control module, a malicious occlusion detection module and a judgment module, wherein the malicious occlusion detection module includes a feature vector extraction unit, a feature map construction unit, an occlusion behavior identification unit, a first judgment unit, a second judgment unit and an event occurrence determination unit which are sequentially connected in communication; The monitoring image acquisition module is used to acquire the monitoring image in the elevator collected by the elevator camera in real time after the elevator car door is opened, wherein the elevator camera is installed at the top of the elevator car and faces the elevator car door, and the camera field of view is fixed to cover the ground area in the elevator car; The contraband identification module is communicatively connected to the monitoring image acquisition module, and is used to input the monitoring image in the elevator as an image sample to be tested into a target contraband identification model that has been trained based on a target detection algorithm, and obtain a first identification result that reflects whether there is a target contraband; The focus area determination module is communicatively connected to the contraband identification module, and is used to determine the contraband focus area according to the location of the target contraband in the surveillance image in the ladder when the first identification result indicates the presence of the target contraband; The elevator car control module is communicatively connected to the contraband identification module, and is used to control the elevator car to switch from a first state for allowing the elevator to operate normally to a second state for maintaining the elevator car door open when the first identification result indicates the presence of target contraband; The malicious occlusion detection module is respectively communicatively connected to the focus area determination module, the elevator car control module and the monitoring image acquisition module, and is used to, after switching to the second state, continue to acquire the elevator monitoring image acquired in real time by the elevator camera, and each time a new elevator monitoring image is acquired, use the elevator camera as the box camera, use the prohibited items focus area as the target focus area, use the elevator monitoring image acquired by the elevator camera when the elevator car is empty as the box background image, and use the latest continuous multiple frames of elevator monitoring images acquired by the elevator camera as the continuous multiple frames of box monitoring images, start the feature vector extraction unit, the feature map construction unit, the occlusion behavior recognition unit, the first judgment unit, the second judgment unit and the event occurrence determination unit, and obtain a determination result reflecting whether a malicious camera occlusion event has occurred; The feature vector extraction unit is used to extract spatial feature vectors of multiple human joints at the time of each frame image acquisition from the continuous multiple frames of the in-box monitoring images acquired by the in-box camera, wherein the in-box camera is installed on the top of the box and the lens field of view is fixed to cover the ground area in the box, and the multiple human joints belong to the same human body and can represent the camera blocking behavior of the human body; The feature map construction unit is used to construct a skeleton sequence spatiotemporal feature map for identifying camera occlusion behavior according to the spatial feature vectors of the multiple human joints when each frame of image is collected; The occlusion behavior recognition unit is used to input the skeleton sequence spatiotemporal feature graph as a data sample to be tested into a camera occlusion behavior recognition model based on a neural network and has been trained, and output a corresponding first confidence prediction value that identifies the camera occlusion behavior; The first judgment unit is used to judge whether the first confidence prediction value is greater than a first preset threshold; The second judgment unit is used to, when determining that the first confidence prediction value is greater than the first preset threshold, use the last frame image in the continuous multiple frames of in-box monitoring images as the current monitoring image, and judge whether the target focus area is blocked based on the first focus area picture in the current monitoring image and corresponding to the target focus area and the second focus area picture in the in-box background image and corresponding to the target focus area, by comparing the texture and brightness, wherein the in-box background image is pre-acquired by the in-box camera when the box is empty; The event occurrence determination unit is used to determine that a malicious camera blocking event occurs when it is determined that the target focus area is blocked; The elevator car control module is further configured to control the elevator car to switch from the second state to a third state for continuing to maintain the elevator car door open when the determination result indicates that a malicious camera blocking event has occurred; The contraband identification module is further used to, after switching to the third state, continue to obtain the elevator surveillance image collected by the elevator camera in real time, and each time a new elevator surveillance image is obtained, input the new elevator surveillance image as another image sample to be tested into the target contraband identification model to obtain a second identification result reflecting whether the target contraband exists; The judgment module is respectively connected to the focus area determination module, the elevator car control module and the monitoring image acquisition module, and is used to continue to acquire the elevator monitoring image collected by the elevator camera in real time after switching to the third state, and each time a new elevator monitoring image is acquired, it is judged whether the contraband focus area is not blocked according to the third focus area picture corresponding to the contraband focus area in the new elevator monitoring image and the fourth focus area picture corresponding to the contraband focus area in the box background image through comparison results in texture and brightness; The elevator car control module is further configured to control the elevator car to switch back to the first state when the second recognition result indicates that there is no target prohibited item and it is determined that the prohibited item focus area is not blocked.

9. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive information, and the processor is used to read the computer program to execute the in-box camera occlusion detection method as described in any one of claims 1 to 4 or the elevator operation control method as described in any one of claims 6 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions. When the instructions are executed on the computer, the in-box camera occlusion detection method as described in any one of claims 1 to 4 or the elevator operation control method as described in any one of claims 6 to 7 is executed.