A method, device and electronic device for detecting occlusion of a surveillance camera in an elevator

Abstract

The present invention relates to an occlusion detection method, device, and electronic device for a surveillance camera in an elevator. An occlusion detection method for a surveillance camera in an elevator, comprising: S1, generating a template depth map through a picture collected by a camera; S2, performing self-occlusion detection based on the maximum connected domain difference comparison between the template depth map and the self-occlusion depth map; the self-occlusion depth map is generated by a picture collected by the camera; S3, performing occlusion detection based on the maximum connected domain difference comparison between the template depth map and the occluded depth map and the centroid coordinate difference comparison; the occluded depth map and the centroid coordinates are generated by a picture collected by the camera; the self-occlusion detection and the occlusion detection are run independently. It is implemented based on monocular depth estimation, and has low requirements on the picture acquisition and processing frequency (only needs to be greater than 1fps).

Description

Technical Field

[0001] The present invention relates to a camera occlusion detection method, and in particular to an occlusion detection method, device and electronic equipment for a surveillance camera in an elevator. Background Art

[0002] After strengthening the supervision and management of electric bicycles illegally entering indoor spaces, many software and hardware solutions based on traditional image processing technology or neural network deep learning technology have emerged, which can directly detect the presence of electric bicycles when the elevator door is opened to control the elevator. This type of solution has achieved good results in the past one or two years, and has indeed reduced the phenomenon of electric bicycles illegally entering indoor spaces from the source. However, after the technical principles of this solution were widely recognized, through exploration and trial, the endless obstruction of surveillance cameras has greatly reduced the original monitoring effect. In addition to electric bicycles riding elevators, there are many uncivilized and dangerous behaviors that cannot be effectively supervised.

[0003] To address this type of occlusion problem, some elevator control service providers have improved the algorithm that only detects the presence of electric bicycles, and developed an algorithm for identifying occlusion behavior using continuous frame images; some surveillance camera manufacturers have launched solutions based on ToF sensors, which directly use the depth information provided by additional sensors to determine occlusion behavior.

[0004] Under ideal conditions, both of the above solutions can produce certain positive effects on the detection of occlusion, but there are still deficiencies in actual production use. The former relies on high-frame-rate image data and algorithm processor performance due to the real-time requirements of elevator control alarms, and has high real-time requirements for image capture and processing services. If any link reaches a bottleneck, it will affect the detection effect; the latter requires the installation of sensors in advance during the production process, which, in addition to increasing costs, cannot provide help for monitoring platforms using old equipment. Summary of the invention

[0005] The present invention proposes a method, device, and electronic device for detecting occlusion of a surveillance camera in an elevator. The method does not require the installation of additional sensors or the replacement of hardware equipment, and has low requirements for the frequency of image acquisition and processing (greater than 1 fps) for detecting the occlusion of elevator surveillance cameras. The present application is based on monocular depth estimation. Depth estimation is a technology that restores the distance from a camera to the camera pixel by pixel from a 2D image taken by a camera. The 2D image composed of the obtained distance information is called a depth map. The method using only a single camera is called monocular depth estimation, and the obtained depth map is the relative depth at an infinite point in the relative image. The depth map obtained by performing depth estimation on the image returned by the capture service after the elevator door is opened is analyzed to determine whether there is an occlusion phenomenon and thus issue an elevator control alarm.

[0006] A method for detecting occlusion of a surveillance camera in an elevator, comprising:

[0007] S1, generate a template depth map through the picture collected by the camera;

[0008] S2, performing self-occlusion detection based on the maximum connected domain difference comparison between the template depth map and the self-occlusion depth map; the self-occlusion depth map is generated by a picture collected by a camera;

[0009] S3, performing occlusion detection based on the maximum connected domain difference comparison between the template depth map and the occluded depth map and the centroid coordinate difference comparison of the maximum connected domain; the occluded depth map and the centroid coordinates are generated from the picture captured by the camera;

[0010] The self-occlusion detection and the occlusion detection are run independently.

[0011] Furthermore, a template depth map is generated through the image collected by the camera, including:

[0012] S1.1 When the elevator confirms through the sensor that the elevator has kept the car door closed and stationary for a predetermined time, a camera is used to collect a picture of the interior of the car;

[0013] S1.2 Using the image in the car to detect people and objects through a people and objects detection model;

[0014] If the detection result is that there is no person and no object, the image is further sent to the car depth estimation model to generate a depth map, and the depth map is saved as a template depth map;

[0015] If the detection result is that there is a person or an object, the detection is terminated and the process returns to step S1.1 to continue collecting a picture of the interior of the car through the camera.

[0016] Furthermore, the specific steps of the self-occlusion detection are as follows:

[0017] S2.1 confirms through sensors whether the elevator is currently in a closed and stationary state;

[0018] S2.2 If yes, collect a picture of the car; if no, end the test and wait for the next hour to execute step S2.1;

[0019] S2.3 The image collected in step S2.2 is used for person and object detection through a person and object detection model;

[0020] S2.4 If the detection result is that there is no person and no object, the image is sent to the car depth estimation model to generate a self-occlusion depth map; if the detection result is that there is a person or an object, the detection is terminated;

[0021] S2.5 extract the maximum connected domain of the relative depth value in the first numerical range in the self-occlusion depth map, denoted as A1; extract the maximum connected domain of the relative depth value in the first numerical range in the template depth map, denoted as A2; extract the maximum connected domain of the relative depth value in the second numerical range in the self-occlusion depth map, denoted as A3; extract the maximum connected domain of the relative depth value in the second numerical range in the template depth map, denoted as A4;

[0022] S2.6 compares the areas of the two groups of maximum connected domains A1 and A2, A3 and A4 respectively; if the area differences of the two groups of maximum connected domains A1 and A2, A3 and A4 are less than 5%, the template depth map is not updated; if the area differences of the two groups of maximum connected domains A1 and A2, A3 and A4 are greater than the first difference value and less than the second difference value, the self-occlusion depth map is used as the template depth map; if the area difference of at least one group of the two groups of maximum connected domains A1 and A2 or A3 and A4 is greater than the second difference value, it is determined that self-occlusion occurs. The process of generating the template depth map can adapt to the environmental changes of the elevator car, thereby increasing the accuracy of self-occlusion detection and occlusion detection.

[0023] Furthermore, the specific steps of occlusion detection are as follows:

[0024] S3.1 obtains through the sensor that when the elevator car door is in the open state, the camera collects pictures with a collection frequency greater than 1 fps and stores them in the picture queue. The collection stops after the elevator door is closed;

[0025] S3.2 performs person and object detection on the images in the image queue in turn through the person and object detection model;

[0026] S3.3 If the detection result is that there is no person and no object, the detection is terminated; if the confidence corresponding to the dangerous person or dangerous object category in the output result of the person and object detection model is greater than the confidence value, the elevator control alarm is directly issued; otherwise, the collected image is sent to the car depth estimation model to generate an occluded depth map; the process of generating a template depth map can adapt to the environmental changes of the elevator car, thereby increasing the accuracy of self-occlusion detection and occlusion detection.

[0027] S3.4 extract the maximum connected domain with a relative depth value in the third numerical range in the occluded depth map, denoted as B1, and its centroid coordinates are denoted as C1; extract the maximum connected domain with a relative depth value in the third numerical range in the template depth map, denoted as B2, and its centroid coordinates are denoted as C2; extract the maximum connected domain with a relative depth value in the fourth numerical range in the occluded depth map, denoted as B3, and its centroid coordinates are denoted as C3; extract the maximum connected domain with a relative depth value in the fourth numerical range in the template depth map, denoted as B4, and its centroid coordinates are denoted as C4;

[0028] S3.5 compares the areas of the two groups of maximum connected domains B1 and B2 or B3 and B4 respectively, and compares the numerical coordinates of the two groups of centroids C1 and C2 or C3 and C4; if the area difference of at least one group of the two groups of maximum connected domains B1 and B2 or B3 and B4 is greater than the third difference value and the corresponding centroid coordinate numerical difference exceeds the fourth difference value of the resolution of the occluded depth map, it is determined that occlusion occurs; otherwise, this detection is terminated.

[0029] Furthermore, the establishment of the car depth estimation model includes:

[0030] By collecting depth map samples of various scenes of passengers and objects in the elevator;

[0031] Use deep learning convolutional neural network to train samples;

[0032] If the training accuracy meets the requirements, the output is the car depth estimation model;

[0033] If the accuracy does not meet the requirements, the model structure or parameters are optimized and retrained. If it still cannot meet the accuracy requirements, the sample size is increased for training until the accuracy meets the requirements.

[0034] Furthermore, the establishment of the person-object detection model includes:

[0035] Collect samples of people, no people, objects, no objects, dangerous people, and dangerous objects in the elevator;

[0036] Use deep learning convolutional neural network to train samples;

[0037] If the training accuracy meets the requirements, the output is a person and object detection model;

[0038] If the accuracy does not meet the requirements, the model structure or parameters are optimized and retrained. If it still cannot meet the accuracy requirements, the sample size is increased for training until the accuracy meets the requirements.

[0039] A device for detecting an obstruction of a surveillance camera in an elevator, comprising:

[0040] An acquisition module is used to acquire pictures inside the elevator car;

[0041] A calculation module, used to generate a template depth map, a self-occlusion depth map and an occluded depth map according to the picture in the elevator car;

[0042] The judgment module is used to calculate the occlusion situation according to the template depth map, the self-occlusion depth map and the occluded depth map.

[0043] An electronic device, comprising:

[0044] one or more processors;

[0045] A memory for storing one or more programs;

[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the above methods.

[0047] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the steps of any of the above methods.

[0048] The beneficial effects of this application are:

[0049] (1) No need to install additional sensors or replace hardware equipment, it is achieved based on monocular depth estimation;

[0050] (2) This application generates a template depth map, performs maximum connected domain difference comparison and centroid coordinate difference comparison based on the template depth map to perform self-occlusion detection and occlusion detection, with high accuracy and fast judgment speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a method for detecting occlusion of a surveillance camera in an elevator according to the present invention;

[0052] Figure 2 It is a car depth estimation model training process of an occlusion detection method of a monitoring camera in an elevator of the present invention;

[0053] Figure 3 This is a training process of a person and object detection model of an occlusion detection method of a surveillance camera in an elevator according to the present invention;

[0054] Figure 4 It is a module schematic diagram of a device for detecting an obstruction of a surveillance camera in an elevator according to the present invention;

[0055] Figure 5 It is a picture inside the elevator collected by a method for detecting an obstruction of a monitoring camera in an elevator according to the present invention;

[0056] Figure 6 It is a template depth map of an occlusion detection method of a surveillance camera in an elevator according to the present invention;

[0057] Figure 7 It is a picture inside the elevator collected by a method for detecting an obstruction of a monitoring camera in an elevator according to the present invention;

[0058] Figure 8 The invention discloses an occluded depth map collected by an occlusion detection method for a surveillance camera in an elevator. DETAILED DESCRIPTION

[0059] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0060] This application proposes a method for detecting the occlusion of elevator surveillance cameras that does not require the installation of additional sensors or replacement of hardware equipment and has low requirements on the frequency of image acquisition and processing (greater than 1 fps).

[0061] A method for detecting occlusion of a surveillance camera in an elevator, comprising:

[0062] S1, generate a template depth map through the picture collected by the camera;

[0063] S2, performing self-occlusion detection based on the maximum connected domain difference comparison between the template depth map and the self-occlusion depth map; the self-occlusion depth map is generated by a picture collected by a camera;

[0064] S3, performing occlusion detection based on the maximum connected domain difference comparison and centroid coordinate difference comparison between the template depth map and the occluded depth map; the occluded depth map and centroid coordinates are generated from the picture captured by the camera;

[0065] Self-occlusion detection and occlusion detection run independently.

[0066] Generate a template depth map from the image captured by the camera, including:

[0067] S1.1 When the sensor confirms that the elevator has kept the car door closed and stationary for a predetermined time, a camera is used to capture a picture of the interior of the car; the predetermined time is generally 5 minutes.

[0068] S1.2 Use the person and object detection model to detect people and objects in the car image;

[0069] If the detection result is that there is no person and no object, the image is further sent to the car depth estimation model to generate a depth map, and the depth map is saved as a template depth map;

[0070] If the detection result is that there is a person or an object, the detection is terminated and the process returns to step S1.1 to continue collecting a picture of the interior of the car through the camera.

[0071] The specific steps of self-occlusion detection are as follows:

[0072] S2.1 confirms through sensors whether the elevator is currently in a closed and stationary state;

[0073] S2.2 If yes, collect a picture of the car; if no, end the test and wait for the next hour to execute step S2.1;

[0074] S2.3 The image collected in step S2.2 is used for person and object detection through a person and object detection model;

[0075] S2.4 If the detection result is that there is no person and no object, the image is sent to the car depth estimation model to generate a self-occlusion depth map; if the detection result is that there is a person or an object, the detection is terminated;

[0076] S2.5 extract the maximum connected domain with relative depth values ​​in the first numerical range from the occlusion depth map, denoted as A1; extract the maximum connected domain with relative depth values ​​in the first numerical range from the template depth map, denoted as A2; extract the maximum connected domain with relative depth values ​​in the second numerical range from the occlusion depth map, denoted as A3; extract the maximum connected domain with relative depth values ​​in the second numerical range from the template depth map, denoted as A4;

[0077] The first value range is 0-0.2, and the second value range is 0.8-1.0.

[0078] S2.6 compares the areas of the two groups of maximum connected domains A1 and A2, A3 and A4 respectively; if the area differences of the two groups of maximum connected domains A1 and A2, A3 and A4 are less than 5%, the template depth map is not updated; if the area differences of the two groups of maximum connected domains A1 and A2, A3 and A4 are greater than 5% and less than 10%, the self-occlusion depth map is used as the template depth map; if the area difference of at least one of the two groups of maximum connected domains A1 and A2 or A3 and A4 is greater than 10%, it is determined that self-occlusion occurs. Generally, the first difference value is 5% and the second difference value is 10%, which can also be adjusted as needed.

[0079] The specific steps of occlusion detection are as follows:

[0080] S3.1 obtains information through the sensor when the elevator car door is in the open state, collects pictures through the camera, sets the collection frequency to be greater than 1 fps, and stores them in the picture queue, and stops collecting after the elevator door is closed; generally, a collection frequency greater than 1fps can meet subsequent judgment requirements, and the requirements for picture collection and processing frequency are not high (greater than 1 fps). In some embodiments, it can also be adjusted as needed.

[0081] S3.2 performs person and object detection on the images in the image queue in turn through the person and object detection model;

[0082] S3.3 If the detection result is that there is no person and no object, the detection is terminated; if the confidence level corresponding to the dangerous person or dangerous object category in the output result of the person and object detection model is greater than 0.5, the elevator control alarm is directly issued; otherwise, the collected image is sent to the car depth estimation model to generate an occluded depth map; generally, the confidence value is 0.5, which can also be adjusted as needed.

[0083] S3.4 extract the maximum connected domain with a relative depth value in the third numerical range in the occluded depth map, denoted as B1, and its centroid coordinates are denoted as C1; extract the maximum connected domain with a relative depth value in the third numerical range in the template depth map, denoted as B2, and its centroid coordinates are denoted as C2; extract the maximum connected domain with a relative depth value in the fourth numerical range in the occluded depth map, denoted as B3, and its centroid coordinates are denoted as C3; extract the maximum connected domain with a relative depth value in the fourth numerical range in the template depth map, denoted as B4, and its centroid coordinates are denoted as C4;

[0084] The third numerical range is 0-0.2, and the fourth numerical range is 0.2-0.4.

[0085] S3.5 compares the areas of the two groups of maximum connected domains B1 and B2 or B3 and B4, and compares the coordinate values ​​of the two groups of centroids C1 and C2 or C3 and C4; if the area difference of at least one of the two groups of maximum connected domains B1 and B2 or B3 and B4 is greater than 10% and the corresponding centroid coordinate value difference exceeds 25% of the resolution of the occluded depth map, it is determined that occlusion occurs; otherwise, the detection is terminated. Generally, the third difference value is 10% and the fourth difference value is 25%, which can also be adjusted as needed.

[0086] like Figure 1 ,The establishment of the car depth estimation model includes:

[0087] By collecting depth map samples of various scenes of passengers and objects in the elevator;

[0088] Use deep learning convolutional neural network to train samples;

[0089] If the training accuracy meets the requirements, the output is the car depth estimation model;

[0090] If the accuracy does not meet the requirements, the model structure or parameters are optimized and retrained. If it still cannot meet the accuracy requirements, the sample size is increased for training until the accuracy meets the requirements.

[0091] Specifically, the car depth estimation model refers to an algorithm model that can output the relative depth of the car interior scene by collecting depth map samples of various scenes of people and objects in the elevator, and then modifying the structure and retraining based on the public neural network model. The end condition of the model retraining process is that the training accuracy meets the actual use needs. If it does not meet the requirements, retrain after optimizing the structure and parameters. If it still cannot meet the accuracy requirements, increase the sample size for training, and repeat the cycle until the end condition is met. Figure 3 ,The specific process of establishing the car depth estimation model is as follows:

[0092] For an input image of size 388×388, a circle of pixels that are mirror-symmetric to the original edge pixels is added to the edge to fix the input image size to 572×572 pixels. Each convolution layer performs a 3×3 convolution operation with a stride of 1, the pooling layer performs a 2×2 maximum pooling operation with a stride of 2, and the upsampling layer performs a 2×2 deconvolution operation with a stride of 2. The activation function is ReLU, and each convolution layer is followed by a Batch Normalization operation.

[0093] The convolutional layer transforms the feature data of dimension 572×572×3 into 570×570×64.

[0094] The convolution layer transforms the feature data of dimension 570×570×64 into 568×568×64.

[0095] The feature data with dimension 568×568×64 is converted into feature data with dimension 282×282×128 through pooling layer and convolution layer;

[0096] The convolution layer converts the feature data of dimension 282×282×128 into 280×280×128.

[0097] The feature data with dimension 280×280×128 is converted into feature data with dimension 138×138×256 through pooling layer and convolution layer;

[0098] The convolution layer converts the feature data of dimension 138×138×256 into 136×136×256.

[0099] The feature data with dimension 136×136×256 is converted into feature data with dimension 66×66×512 through pooling layer and convolution layer;

[0100] The convolution layer converts the feature data with dimension 66×66×512 into 64×64×512.

[0101] The feature data with dimension 64×64×512 is converted into feature data with dimension 30×30×1024 through pooling layer and convolution layer;

[0102] The convolutional layer converts the feature data of dimension 30×30×1024 into 28×28×1024.

[0103] The feature data with dimension 28×28×1024 is converted into feature data with dimension 54×54×512 through upsampling layer and convolution layer;

[0104] The convolution layer converts the feature data of dimension 54×54×512 into 52×52×512.

[0105] The feature data with dimension 52×52×512 is converted into feature data with dimension 102×102×256 through upsampling layer and convolution layer;

[0106] The convolutional layer converts the feature data of dimension 102×102×256 into feature data of dimension 100×100×256;

[0107] The feature data with a dimension of 100×100×256 is converted into feature data with a dimension of 198×198×128 through upsampling layers and convolution layers;

[0108] The convolution layer converts the feature data with dimension 198×198×128 into 196×196×128.

[0109] The feature data with dimension 196×196×128 is converted into feature data with dimension 390×390×64 through upsampling layer and convolution layer;

[0110] The convolution layer converts the feature data of dimension 390×390×64 into 388×388×64.

[0111] The 388×388×64 dimension is converted into feature data of 388×388×1 through a 1×1 convolution layer. After normalization based on the maximum value in the data, a depth map with relative depth values ​​between 0 and 1 is finally obtained.

[0112] like Figure 2 ,The establishment of the person and object detection model includes:

[0113] Collect samples of people, no people, objects, no objects, dangerous people, and dangerous objects in the elevator;

[0114] Use deep learning convolutional neural network to train samples;

[0115] If the training accuracy meets the requirements, the output is a person and object detection model;

[0116] If the accuracy does not meet the requirements, the model structure or parameters are optimized and retrained. If it still cannot meet the accuracy requirements, the sample size is increased for training until the accuracy meets the requirements.

[0117] Specifically, the person and item detection model refers to an algorithm model that can detect the presence and category of people and items by collecting samples of people, people, items, and no items in the elevator, and modifying the structure and retraining based on the public neural network model. The end condition of the model retraining process is that the training accuracy meets the actual use needs. If it does not meet the requirements, retrain after optimizing the structure and parameters. If it still cannot meet the accuracy requirements, increase the sample size for training, and repeat the cycle until the end condition is met. The input image is finally output by the person and item detection model as an image divided into 7×7 grids, and each grid generates two sets of prediction boxes. The specific establishment process of the person and item detection model is as follows:

[0118] The input image is fixed to a pixel matrix of 448×448×3. The operation step size of the convolution layer defaults to 1. The pooling layer performs a maximum pooling operation of 2×2 with a step size of 2. The activation function is Leaky ReLU. Each convolution layer is followed by a BatchNormalization operation.

[0119] The feature data of dimension 448×448×3 is transformed into feature data of dimension 112×112×192 through a 7×7×64 convolutional layer and a pooling layer with a step size of 2;

[0120] The feature data with dimension 112×112×192 is converted into feature data with dimension 56×56×256 through 3×3×192 convolutional layers and pooling layers;

[0121] The feature data of dimension 56×56×256 is converted into feature data of dimension 28×28×512 through convolutional layers and pooling layers of 1×1×128, 3×3×256, 1×1×256, and 3×3×512;

[0122] The feature data of dimension 28×28×512 is converted into feature data of dimension 14×14×1024 through four convolutional layers of 1×1×256, 3×3×512, 1×1×512, and 3×3×1024, and pooling layers;

[0123] The dimension 14×14×1024 is converted into feature data of dimension 7×7×1024 through two convolutional layers of 1×1×512 followed by 3×3×1024, 3×3×1024, and 3×3×1024 with a stride of 2;

[0124] The dimension 7×7×1024 is converted into feature data of 1×1×4096 through the fully connected layer;

[0125] The fully connected layer transforms the feature data of dimension 1×1×4096 into dimension 7×7×30, of which 7×7×20 is used as the probability of the possible categories of people and objects in the 20 preset elevators, and the remaining 7×7×10 is the coordinates, size and target probability of the two sets of prediction boxes;

[0126] If the probability of the existence of any prediction box is greater than 0.5, the coordinates, size, and the maximum value of the probability of the 20 preset categories of people and objects that may appear in the elevator are used as the final output of the model. Otherwise, it is judged that there are no people or objects in the input image.

[0127] The categories of people and items that may appear in the 20 elevators are: cleaning staff, couriers, take-out delivery staff, elevator maintenance staff, other personnel, electric vehicles, bicycles, baby carriages, scooters, batteries, gas tanks, handbags, suitcases, large items, backpacks, express boxes, knives, sticks, umbrellas, and other items.

[0128] Correspondingly, such as Figure 4 The present application also provides a device for detecting an obstruction of a surveillance camera in an elevator, comprising:

[0129] An acquisition module is used to acquire pictures inside the elevator car;

[0130] A calculation module, used to generate a template depth map, a self-occlusion depth map and an occluded depth map according to the image in the elevator car;

[0131] The judgment module is used to calculate the occlusion situation according to the template depth map, the self-occlusion depth map and the occluded depth map.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0134] Accordingly, the present application also provides an electronic device, including:

[0135] one or more processors;

[0136] A memory for storing one or more programs;

[0137] When one or more programs are executed by one or more processors, the one or more processors implement any of the above methods.

[0138] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the steps of any of the above methods are implemented.

[0139] Figure 5 are pictures collected by this embodiment, and the template depth maps output after being processed by the method of this embodiment are respectively as follows Figure 6 As shown; Figure 7 The pictures collected by this embodiment, and the occluded depth maps output after being processed by the method of this embodiment are respectively as follows: Figure 8 In some cases, Figure 5 It can also be a picture collected under self-occlusion. Figure 6 Output the result for it.

[0140] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for detecting occlusion of a surveillance camera in an elevator, characterized in that: include: S1, generate a template depth map through the picture collected by the camera; S2, performing self-occlusion detection based on the maximum connected domain difference comparison between the template depth map and the self-occlusion depth map; the self-occlusion depth map is generated by a picture collected by a camera; S3, performing occlusion detection according to the maximum connected domain difference comparison between the template depth map and the occluded depth map and the centroid coordinate difference comparison of the maximum connected domain; The occluded depth map and centroid coordinates are generated from a picture captured by a camera; Generate a template depth map from the image captured by the camera, including: S1.1 When the elevator confirms through the sensor that the elevator has kept the car door closed and stationary for a predetermined time, a camera is used to collect a picture of the interior of the car; S1.2 Using the image in the car to detect people and objects through a people and objects detection model; If the detection result is that there is no person and no object, the image is further sent to the car depth estimation model to generate a depth map, and the depth map is saved as a template depth map; If the detection result is that there is a person or an object, then the detection is terminated and the process returns to step S1.1 of collecting a picture of the interior of the car through the camera; The specific steps of the self-occlusion detection are as follows: S2.1 confirms through sensors whether the elevator is currently in a closed and stationary state; S2.2 If yes, collect a picture of the car; if no, end the test and wait for the next hour to execute step S2.1; S2.3 The image collected in step S2.2 is used for person and object detection through a person and object detection model; S2.4 If the detection result is that there is no person and no object, the image is sent to the car depth estimation model to generate a self-occlusion depth map; if the detection result is that there is a person or an object, the detection is terminated; S2.5 extract the maximum connected domain of the relative depth value in the first numerical range in the self-occlusion depth map, denoted as A1; extract the maximum connected domain of the relative depth value in the first numerical range in the template depth map, denoted as A2; extract the maximum connected domain of the relative depth value in the second numerical range in the self-occlusion depth map, denoted as A3; extract the maximum connected domain of the relative depth value in the second numerical range in the template depth map, denoted as A4; S2.6 compares the areas of the two groups of maximum connected domains A1 and A2, and A3 and A4 respectively; if the area differences of the two groups of maximum connected domains A1 and A2, and A3 and A4 are all less than the first difference value, the template depth map is not updated; if the area differences of the two groups of maximum connected domains A1 and A2, and A3 and A4 are all greater than the first difference value and less than the second difference value, the self-occlusion depth map is used as the template depth map; if the area difference of at least one group of the two groups of maximum connected domains A1 and A2 or A3 and A4 is greater than the second difference value, it is determined that self-occlusion occurs; The specific steps of occlusion detection are as follows: S3.1 obtains information through the sensor when the elevator car door is open, collects pictures through the camera, sets the collection frequency, and stores them in the picture queue. The collection stops after the elevator door is closed; S3.2 performs person and object detection on the images in the image queue in turn through the person and object detection model; S3.3 If the detection result is that there is no person and no object, then the detection is terminated; if the confidence level corresponding to the dangerous person or dangerous object category in the output result of the person and object detection model is greater than the confidence value, then the elevator control alarm is directly issued; otherwise, the collected image is sent to the car depth estimation model to generate an occluded depth map; S3.4 extract the maximum connected domain with a relative depth value in the third numerical range in the occluded depth map, denoted as B1, and its centroid coordinates are denoted as C1; extract the maximum connected domain with a relative depth value in the third numerical range in the template depth map, denoted as B2, and its centroid coordinates are denoted as C2; extract the maximum connected domain with a relative depth value in the fourth numerical range in the occluded depth map, denoted as B3, and its centroid coordinates are denoted as C3; extract the maximum connected domain with a relative depth value in the fourth numerical range in the template depth map, denoted as B4, and its centroid coordinates are denoted as C4; S3.5 compares the areas of the two groups of maximum connected domains B1 and B2 or B3 and B4 respectively, and compares the numerical coordinates of the two groups of centroids C1 and C2 or C3 and C4; if the area difference of at least one group of the two groups of maximum connected domains B1 and B2 or B3 and B4 is greater than the third difference value and the corresponding centroid coordinate numerical difference exceeds the fourth difference value of the resolution of the occluded depth map, it is determined that occlusion occurs; otherwise, this detection is terminated.

2. The method for detecting occlusion of a surveillance camera in an elevator according to claim 1, characterized in that: The establishment of the car depth estimation model includes: Collect depth map samples for various scenes of passengers and objects in the elevator; Use deep learning convolutional neural network to train samples; If the training accuracy meets the requirements, the output is the car depth estimation model; If the accuracy does not meet the requirements, the model structure or parameters are optimized and retrained. If it still cannot meet the accuracy requirements, the sample size is increased for training until the accuracy meets the requirements.

3. The method for detecting occlusion of a surveillance camera in an elevator according to claim 1, characterized in that: The establishment of the person-object detection model includes: Collect samples of people, no people, objects, no objects, dangerous people, and dangerous objects in the elevator; Use deep learning convolutional neural network to train samples; If the training accuracy meets the requirements, the output is a person and object detection model; If the accuracy does not meet the requirements, the model structure or parameters are optimized and retrained. If it still cannot meet the accuracy requirements, the sample size is increased for training until the accuracy meets the requirements.

4. A device for implementing the occlusion detection method for a surveillance camera in an elevator according to claim 1, characterized in that: include: An acquisition module is used to acquire pictures inside the elevator car; A calculation module, used to generate a template depth map, a self-occlusion depth map and an occluded depth map according to the picture in the elevator car; The judgment module is used to calculate the occlusion situation according to the template depth map, the self-occlusion depth map and the occluded depth map.

5. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images