An elevator left-over detection method and system
By collecting images before and after the elevator is opened, using the Yolov8 and DINOv3 models to extract feature images, and calculating cosine similarity to generate binarized images, the problem of high false recognition rate and poor timeliness in elevator debris detection is solved, and efficient and accurate debris recognition is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHONGSHAN SIDA TECH CO LTD
- Filing Date
- 2026-01-14
- Publication Date
- 2026-06-05
AI Technical Summary
Existing methods for detecting items left in elevators are susceptible to environmental factors, have a high false recognition rate, and are not timely enough to effectively identify items left in elevator cars.
Images are collected before and after the elevator door opens. Humans are detected using the Yolov8 algorithm, and a rear image is collected when no one is present. Feature images are extracted using the DINOv3 model, and cosine similarity is calculated to generate a binarized image to determine whether any objects remain.
It significantly improves the accuracy of elevator debris detection, effectively eliminates the influence of environmental factors, and achieves rapid and accurate debris identification.
Smart Images
Figure CN122157140A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual recognition technology, and in particular to a method and system for detecting objects left behind in elevators. Background Technology
[0002] During elevator operation, it is common for items to be left behind in the elevator car. For example, during renovation and moving, packaging bags, liner sheets, construction waste, etc. may be left in the car, affecting the riding experience of subsequent passengers. There may also be cases where passengers drop personal items such as mobile phones and wallets without realizing it. If such items enter the elevator door gaps, they may damage the elevator and cause safety hazards.
[0003] Traditionally, removing items left in elevators relies on routine inspections by security or cleaning staff, or on backend monitoring to detect and notify relevant personnel, resulting in significant timeliness issues. To address this, existing solutions use image recognition for item detection; however, these methods are susceptible to environmental factors such as light and shadow from door opening and closing, and water droplets from mopping, leading to a high false recognition rate. Summary of the Invention
[0004] The first aspect of this embodiment discloses a method for detecting items left in an elevator, specifically including: Acquire front and rear images; Extract a front feature map from the front image and extract a back feature map from the back image; Calculate the cosine similarity between the front feature map and the back feature map; A binarized image is generated based on the cosine similarity; The presence of any remaining objects is determined based on the binarized image.
[0005] As an optional implementation, the acquisition of the front and rear images includes: Based on the opening action of the elevator door, a data acquisition command is triggered, and the preceding image is captured based on the surveillance video. Based on the closing action of the elevator door, a detection command is triggered, and the Yolov8 algorithm is used to continuously detect human bodies in the current monitoring image. When it is determined that there is no human body in the current monitoring image, a data acquisition command is triggered to capture the rear image based on the monitoring image.
[0006] As an optional implementation, the method further includes: Acquire clean images of the target elevator car; Interference materials were set for the target elevator car, and interference images were acquired; Leave behind materials and collect leftover images in the target elevator car; Based on the clean image, the interference image, and the residual image of the target elevator car, training cases corresponding to the target elevator car are established.
[0007] As an optional implementation, the extraction of a front feature map from the front image and a back feature map from the back image includes: The DINOv3 model is used to perform feature extraction training on the training cases of the target elevator to obtain the feature extraction model; The aforementioned feature extraction model is used to extract a front feature map from the front image and a back feature map from the back image.
[0008] As an optional implementation, generating a binarized image based on the cosine similarity includes: If the cosine similarity of any feature point in the preceding feature map corresponding to the following feature map is less than the difference threshold, then the feature point is recorded as a value of 1; otherwise, the feature point is recorded as a value of 0.
[0009] As an optional implementation, determining whether there are any remaining objects based on the binarized image includes: The difference ratio between the number of feature points with a value of 1 in the binarized image and the total number of feature points is statistically analyzed. If the difference ratio is higher than the residue threshold, then it is determined that there is residue. Conversely, if the elevator car is clean, it is determined that there are no items left behind.
[0010] As an optional implementation, before calculating the cosine similarity between the preceding feature map and the following feature map, the method further includes: Normalization is performed on each corresponding feature point in the pre-feature map and the post-feature map to obtain the pre-normalized feature map and the post-normalized feature map.
[0011] As an optional implementation, the method further includes: For each feature point corresponding to the pre-normalized feature map and the post-normalized feature map, calculate the cosine similarity to obtain the pre-similarity feature map and the post-similarity feature map. Using the pre-normalized feature map, the post-normalized feature map, the pre-similarity feature map, and the post-similarity feature map as input, a legacy assignment adapter is trained based on a multilayer perceptron.
[0012] As an optional implementation, the output of the legacy allocation adapter includes legacy increase, legacy decrease, and no change.
[0013] The second aspect of this embodiment discloses an elevator debris detection system, specifically including: The acquisition unit is used to acquire front and rear images; A feature extraction unit is used to extract a front feature map from the front image and a rear feature map from the rear image. A similarity calculation unit is used to calculate the cosine similarity between the front feature map and the back feature map; A binarization unit is used to generate a binarized image based on the cosine similarity; The judgment unit is used to determine whether there are any remaining objects based on the binarized image.
[0014] Compared with the prior art, this embodiment has the following beneficial effects: By collecting images before and after riding the elevator and extracting features, the similarity between the two images is used to determine whether there are any objects left behind, rather than identifying objects based on a single image. This effectively eliminates the influence of environmental factors and significantly improves the accuracy of recognition. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in this embodiment, the accompanying drawings used in the embodiment will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a schematic diagram of the workflow of an elevator debris detection method disclosed in this embodiment; Figure 2 This is a schematic diagram of the system structure of an elevator debris detection system disclosed in this embodiment. Detailed Implementation
[0017] The technical solutions in this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Example 1 Please see Figure 1 This embodiment discloses a method for detecting items left behind in elevators, including: 101. Acquire front and rear images.
[0019] In this embodiment, unlike existing image recognition schemes that only identify and interpret surveillance images at the current moment, this embodiment avoids the low accuracy of image recognition by collecting images before and after taking the elevator.
[0020] As an optional implementation method, the acquisition command is triggered based on the opening action of the elevator door, and the previous image is captured based on the monitoring video. Based on the closing action of the elevator door, a detection command is triggered, and the Yolov8 algorithm is used to continuously detect human bodies in the current monitoring image. When it is detected that there is no human body in the current monitoring image, a data acquisition command is triggered to capture a rear image based on the monitoring image.
[0021] Specifically, when the elevator receives an external command to open the elevator door while it is in standby mode, it indicates that there is a passenger about to board the elevator. At this time, the passenger has not yet entered the elevator car, and a front view image of the unoccupied person is captured.
[0022] Furthermore, when monitoring reveals that there is no one in the elevator car, it indicates that all passengers have left the elevator, and at this time, a rear-view image of the empty car is collected.
[0023] It is evident that, excluding human interference, good comparison and recognition can be achieved based on images of passengers before and after entering and exiting the elevator.
[0024] Here, assuming there are water stains on the elevator car caused by cleaning work, if the existing image recognition scheme is used to identify and interpret the current monitoring image of the idle elevator, the water stains may be identified as dark objects or reflective objects, leading to a misjudgment that there are leftovers in the elevator.
[0025] By collecting images before and after riding the elevator, water stains on the elevator car itself will be used as an initial condition for subsequent comparison, and will not be identified as leftovers on their own.
[0026] 102. Extract the front feature map from the front image and the back feature map from the back image.
[0027] In this embodiment, the acquired original image contains a lot of irrelevant information, so features are extracted from it first to reduce the computational burden of the subsequent identification and comparison process.
[0028] As an optional implementation method, clean images are acquired for the target elevator car; Interference materials were set for the target elevator car, and interference images were acquired; Leave behind materials and collect leftover images in the target elevator car; Based on the clean image, interfering image, and residual image of the target elevator car, training cases corresponding to the target elevator car are established.
[0029] Here, different training cases are set for different target elevators to reduce the misidentification rate of different elevators under different environmental factors.
[0030] For example, multiple elevators may exist in the same elevator lobby. The same external light source will produce different lighting effects inside elevators facing different directions. If the same training cases are used, the recognition accuracy will be difficult to improve substantially. Therefore, we collect training cases independently for each elevator and provide corresponding input / output interfaces to achieve targeted recognition and interpretation.
[0031] As an optional implementation, extracting a front feature map from the front image and extracting a back feature map from the back image includes: The DINOv3 model is used to perform feature extraction training on the training cases of the target elevator to obtain the feature extraction model; A feature extraction model is used to extract the front feature map from the front image and the back feature map from the back image.
[0032] Therefore, by employing a feature extraction model, high-level image features can be extracted from the original image.
[0033] 103. Calculate the cosine similarity between the front feature map and the back feature map.
[0034] In this embodiment, similarity calculation is performed on the feature points corresponding to the two.
[0035] Suppose a passenger leaves their phone in the elevator car. The feature points in the area where the phone is located will show significant differences in similarity between the preceding and following feature maps. Furthermore, the relevant feature points exhibit a clear continuity, meaning that feature points with significant differences in similarity constitute a continuous region.
[0036] 104. Generate binarized images based on cosine similarity.
[0037] In this embodiment, binarized images can further reduce the difficulty of recognition and interpretation, and improve the interpretation speed.
[0038] As an optional implementation, if the cosine similarity of any feature point in the preceding feature map corresponding to the following feature map is less than the difference threshold, then the feature point is recorded as a value of 1; otherwise, the feature point is recorded as a value of 0.
[0039] Here, regions with continuous features on the binarized image, denoted by a value of 1, can be used to characterize the addition of debris in the region after the passenger leaves the elevator.
[0040] 105. Determine the presence of remnants based on binarized images.
[0041] In this embodiment, the object is identified by using a binarized image, which can easily identify the object without needing to determine details such as what kind of object it is. The identification accuracy is high and the identification speed is significantly improved.
[0042] As an optional implementation, determining the presence of remnants based on a binarized image includes: The difference between the number of feature points with a value of 1 and the total number of feature points in a binary image is statistically analyzed. If the difference ratio is higher than the residue threshold, then it is determined that there is residue. Conversely, if the elevator car is clean, it is determined that there are no items left behind.
[0043] Here, by setting a difference ratio, a threshold limit can be set for the volume or surface area of the object to be determined.
[0044] For example, when there is construction or renovation work in a building, it is necessary to strictly identify whether there is construction waste left in the elevator car and whether the elevator car is cleaned in a timely manner after the removal is completed. In this case, the difference ratio can be lowered to enable the detection of small bricks, gravel, etc.
[0045] In daily operation and management, the difference ratio can be appropriately increased to conform to the specifications of objects such as mobile phones and wallets, so as to avoid frequent notifications for inspection and cleaning due to the detection of small items such as paper scraps.
[0046] As an optional implementation, before calculating the cosine similarity between the pre-feature map and the post-feature map, normalization processing is performed on each corresponding feature point in the pre-feature map and the post-feature map to obtain the pre-normalized feature map and the post-normalized feature map.
[0047] As an optional implementation, cosine similarity is calculated for each feature point corresponding to the pre-normalized feature map and the post-normalized feature map to obtain the pre-similarity feature map and the post-similarity feature map. Using the preceding normalized feature map, the following normalized feature map, the preceding similarity feature map, and the following similarity feature map as input, a legacy assignment adapter is trained based on a multilayer perceptron.
[0048] The output of the legacy allocation adapter includes legacy increase, legacy decrease, and no change.
[0049] Here, the normalized feature map and similarity feature map can be input for training to achieve higher-dimensional recognition and interpretation output. This not only allows us to know whether there are any leftovers, but also whether the number of leftovers has decreased or remained unchanged. In addition to timely notification of inspection and cleaning, this further enables functions such as cleaning effect evaluation.
[0050] Compared with the prior art, this embodiment has the following beneficial effects: By collecting images before and after riding the elevator and extracting features, the similarity between the two images is used to determine whether there are any objects left behind, rather than identifying objects based on a single image. This effectively eliminates the influence of environmental factors and significantly improves the accuracy of recognition.
[0051] Example 2 Please see Figure 2 This embodiment discloses an elevator debris detection system, comprising: The acquisition unit is used to acquire front and rear images; The feature extraction unit is used to extract the front feature map from the front image and the back feature map from the back image. The similarity calculation unit is used to calculate the cosine similarity between the front feature map and the back feature map; Binarization units are used to generate binary images based on cosine similarity; The judgment unit is used to determine whether there are any remaining objects based on the binarized image.
[0052] In this embodiment, since the images before and after riding the elevator are analyzed and interpreted, there are no specific requirements for the images and image acquisition equipment. The acquisition steps can be completed based on the monitoring equipment already in the elevator, which can be widely applied to various types of elevators, reducing the difficulty of upgrading and the cost of renovation.
Claims
1. A method for detecting items left behind in an elevator, characterized in that, include: Acquire front and rear images; Extract a front feature map from the front image and extract a back feature map from the back image; Calculate the cosine similarity between the front feature map and the back feature map; A binarized image is generated based on the cosine similarity; The presence of any remaining objects is determined based on the binarized image.
2. The method for detecting items left in an elevator according to claim 1, characterized in that, The acquisition of the front and back images includes: Based on the opening action of the elevator door, a data acquisition command is triggered, and the preceding image is captured based on the surveillance video. Based on the closing action of the elevator door, a detection command is triggered, and the Yolov8 algorithm is used to continuously detect human bodies in the current monitoring image. When it is determined that there is no human body in the current monitoring image, a data acquisition command is triggered to capture the rear image based on the monitoring image.
3. The method for detecting items left in an elevator according to claim 1, characterized in that, The method further includes: Acquire clean images of the target elevator car; Interference materials were set for the target elevator car, and interference images were acquired; Leave behind materials and collect leftover images in the target elevator car; Based on the clean image, the interference image, and the residual image of the target elevator car, training cases corresponding to the target elevator car are established.
4. The method for detecting items left in an elevator according to claim 3, characterized in that, The step of extracting a front feature map from the front image and extracting a back feature map from the back image includes: The DINOv3 model is used to perform feature extraction training on the training cases of the target elevator to obtain the feature extraction model; The aforementioned feature extraction model is used to extract a front feature map from the front image and a back feature map from the back image.
5. The method for detecting items left in an elevator according to claim 1, characterized in that, The generation of a binarized image based on the cosine similarity includes: If the cosine similarity of any feature point in the preceding feature map corresponding to the following feature map is less than the difference threshold, then the feature point is recorded as a value of 1; otherwise, the feature point is recorded as a value of 0.
6. The method for detecting items left in an elevator according to claim 1, characterized in that, The step of determining whether there are any remaining objects based on the binarized image includes: The difference ratio between the number of feature points with a value of 1 in the binarized image and the total number of feature points is statistically analyzed. If the difference ratio is higher than the residue threshold, then it is determined that there is residue. Conversely, if the elevator car is clean, it is determined that there are no items left behind.
7. The method for detecting items left in an elevator according to claim 5, characterized in that, Before calculating the cosine similarity between the preceding feature map and the following feature map, the method further includes: Normalization is performed on each corresponding feature point in the pre-feature map and the post-feature map to obtain the pre-normalized feature map and the post-normalized feature map.
8. The method for detecting items left in an elevator according to claim 7, characterized in that, The method further includes: For each feature point corresponding to the pre-normalized feature map and the post-normalized feature map, calculate the cosine similarity to obtain the pre-similarity feature map and the post-similarity feature map. Using the pre-normalized feature map, the post-normalized feature map, the pre-similarity feature map, and the post-similarity feature map as input, a legacy assignment adapter is trained based on a multilayer perceptron.
9. The method for detecting items left in an elevator according to claim 8, characterized in that, The method includes: The output of the legacy allocation adapter includes legacy increases, legacy decreases, and no change.
10. An elevator debris detection system, characterized in that, include: The acquisition unit is used to acquire front and rear images; A feature extraction unit is used to extract a front feature map from the front image and a rear feature map from the rear image. A similarity calculation unit is used to calculate the cosine similarity between the front feature map and the back feature map; A binarization unit is used to generate a binarized image based on the cosine similarity; The judgment unit is used to determine whether there are any remaining objects based on the binarized image.