Elevator door opening and closing state detection method, device, equipment and medium

Abstract

The invention provides an elevator door opening and closing state detection method and device, equipment and a medium. The method comprises the steps that a real-time monitoring image of a to-be-detected elevator door is obtained; a to-be-detected area image in the real-time monitoring image is determined, the to-be-detected area image is preprocessed, the to-be-detected area image comprises image data of two markers, and the two markers are symmetrically installed at the position, close to the door crack, of the to-be-detected elevator door; performing template matching on the preprocessed to-be-detected area image by adopting a pyramid image template area rapid search method, a nearest neighbor scale search method and a three-step search method to obtain a marker recognition result; feature point extraction is carried out on the markers based on the marker identification result, and feature point L2 norms of the two markers are calculated; and on the basis of the feature point L2 norm, the door opening and closing state detection result of the to-be-detected elevator door is determined. Therefore, the accuracy and the real-time performance of elevator door opening and closing state detection can be effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of elevator safety, and in particular to a method, device, equipment and medium for detecting the state of an elevator door opening and closing. Background Art

[0002] Elevators are an indispensable means of vertical transportation in modern urban buildings. The normal opening and closing of elevator doors is crucial to the safety of passengers. Door failure may cause dangerous situations such as pinching people or failure to close or open, so a reliable detection method or system is needed to ensure the normal operation of the door.

[0003] In the related art, when performing elevator door opening and closing status detection, the accuracy and real-time performance of the detection results cannot be guaranteed. Summary of the invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the purpose of the present disclosure is to propose a method, device, computer equipment and storage medium for detecting the state of an elevator door switch, thereby effectively improving the accuracy and real-time performance of the detection of the state of an elevator door switch.

[0006] To achieve the above-mentioned purpose, a method for detecting an elevator door opening and closing state is provided in a first aspect of the present disclosure, comprising:

[0007] Obtain real-time monitoring images of the elevator door to be inspected;

[0008] Determine an image of the area to be detected in the real-time monitoring image, and pre-process the image of the area to be detected, wherein the image of the area to be detected includes image data of two markers, and the two markers are symmetrically installed near the door gap of the elevator door to be detected;

[0009] The pyramid image template area fast search, nearest neighbor scale search and three-step search method are used to perform template matching on the preprocessed image of the area to be detected to obtain the marker recognition result;

[0010] Extracting feature points of the markers based on the marker recognition results, and calculating the L2 norms of the feature points of the two markers;

[0011] Based on the L2 norm of the feature point, a detection result of the door opening and closing state of the elevator door to be detected is determined.

[0012] To achieve the above-mentioned purpose, the elevator door opening and closing state detection device proposed in the second aspect of the present disclosure includes:

[0013] An acquisition module is used to acquire a real-time monitoring image of the elevator door to be detected;

[0014] A first determination module is used to determine an image of a region to be detected in the real-time monitoring image and pre-process the image of the region to be detected, wherein the image of the region to be detected includes image data of two markers, and the two markers are symmetrically installed near a door gap of the elevator door to be detected;

[0015] The template matching module is used to perform template matching on the pre-processed image of the area to be detected by using the pyramid image template area fast search, the nearest neighbor scale search and the three-step search method to obtain the marker recognition result;

[0016] A feature extraction module, used to extract feature points of the markers based on the marker recognition results, and calculate the L2 norm of the feature points of the two markers;

[0017] The second determination module is used to determine the door opening and closing state detection result of the elevator door to be detected based on the L2 norm of the feature point.

[0018] The computer device proposed in the third aspect embodiment of the present disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the elevator door opening and closing state detection method proposed in the first aspect embodiment of the present disclosure is implemented.

[0019] The fourth aspect embodiment of the present disclosure proposes a non-temporary computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the elevator door opening and closing state detection method proposed in the first aspect embodiment of the present disclosure is implemented.

[0020] The fifth aspect of the present disclosure provides a computer program product. When the instructions in the computer program product are executed by a processor, the elevator door opening and closing state detection method provided in the first aspect of the present disclosure is executed.

[0021] The elevator door opening and closing state detection method, device, computer equipment and storage medium provided by the present disclosure obtain the real-time monitoring image of the elevator door to be detected; determine the image of the area to be detected in the real-time monitoring image, and pre-process the image of the area to be detected, wherein the image of the area to be detected contains image data of two markers, and the two markers are symmetrically installed near the door gap of the elevator door to be detected; use the pyramid image template area fast search, nearest neighbor scale search and three-step search method to perform template matching on the pre-processed image of the area to be detected to obtain the marker recognition result; extract the feature points of the marker based on the marker recognition result, and calculate the L2 norm of the feature points of the two markers; determine the door opening and closing state detection result of the elevator door to be detected based on the L2 norm of the feature points. In this way, the accuracy and real-time performance of the elevator door opening and closing state detection can be effectively improved.

[0022] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 It is a flow chart of an elevator door opening and closing state detection method proposed in an embodiment of the present disclosure;

[0025] Figure 2 It is a flow chart of the detection method for opening and closing elevator doors and sensing abnormal states proposed in the present disclosure;

[0026] Figure 3 is a schematic diagram of a feature extraction process proposed in the present disclosure;

[0027] Figure 4 is a schematic diagram of a typical installation position of a marker according to the present disclosure;

[0028] Figure 5 It is a schematic diagram of the structure of the detection device for opening and closing elevator doors and sensing abnormal states proposed in the present disclosure;

[0029] Figure 6 It is a structural schematic diagram of an elevator door opening and closing state detection device proposed in an embodiment of the present disclosure;

[0030] Figure 7 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure, and are not to be construed as limitations of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications, and equivalents that fall within the spirit and connotation of the appended claims.

[0032] Figure 1 It is a flow chart of an elevator door opening and closing state detection method proposed in an embodiment of the present disclosure.

[0033] Among them, it should be noted that the executor of the elevator door opening and closing state detection method of this embodiment is the elevator door opening and closing state detection device, which can be implemented by software and / or hardware. The device can be configured in a computer device, and the computer device can include but is not limited to a terminal, a server, etc. For example, the terminal can be a mobile phone, a handheld computer, etc.

[0034] like Figure 1 As shown, the elevator door opening and closing state detection method includes:

[0035] S101: Acquire a real-time monitoring image of the elevator door to be inspected.

[0036] The elevator door to be detected may refer to the elevator door to be detected for the door opening and closing state in the embodiment of the present disclosure. The model of the elevator door to be detected is not limited in the embodiment of the present disclosure.

[0037] The real-time monitoring image refers to an image obtained in real time that contains information related to the elevator door to be inspected.

[0038] That is to say, in the embodiment of the present disclosure, a real-time monitoring image of the elevator door to be detected can be obtained as data input for subsequent elevator door opening and closing status detection.

[0039] S102: Determine an image of a region to be detected in a real-time monitoring image, and pre-process the image of the region to be detected, wherein the image of the region to be detected includes image data of two markers, and the two markers are symmetrically installed near a door gap of an elevator door to be detected.

[0040] The image of the area to be detected may refer to the area used for detecting the door opening and closing status in the embodiment of the present disclosure.

[0041] It is understandable that the real-time monitoring image may contain image information of a large area, and in the embodiment of the present disclosure, the door opening and closing state detection is performed with the marker as the analysis object, so only the area containing the image information related to the marker needs to be analyzed. In the embodiment of the present disclosure, when determining the image of the area to be detected in the real-time monitoring image, the workload of the subsequent image analysis process can be effectively reduced, thereby effectively improving the processing efficiency.

[0042] The marker may refer to an object that is pre-installed symmetrically near the door gap of the elevator door to be detected and is used to detect the opening and closing status of the elevator door. For example, it may be a sticker, colored tape, etc., and there is no limitation on this.

[0043] Optionally, in some embodiments, the image of the area to be detected is preprocessed, including at least one of the following: filtering the image of the area to be detected; performing brightness correction on the image of the area to be detected; performing contrast enhancement on the image of the area to be detected; performing affine transformation on the image of the area to be detected. In this way, the indication effect of the image of the area to be detected obtained after preprocessing on the marker image information can be effectively improved, thereby improving the accuracy of the elevator door opening and closing state detection.

[0044] S103: using pyramid image template area fast search, nearest neighbor scale search and three-step search method to perform template matching on the pre-processed image of the area to be detected to obtain a marker recognition result.

[0045] The marker recognition result can be used to indicate the marker in the image of the area to be detected.

[0046] For example, in the embodiment of the present disclosure, when the pyramid image template area fast search, the nearest neighbor scale search and the three-step search method are used to perform template matching on the preprocessed image of the area to be detected to obtain the marker recognition result, it can be based on the following steps:

[0047] 1. Quick search of pyramid image template area

[0048] 1.1 Building an image pyramid

[0049] Generate a pyramid from the image to be detected and the template image. This involves gradually reducing the original image to multiple resolution levels to form a pyramid structure.

[0050] Typically, the bottom layer of the pyramid is the original image, and the resolution decreases gradually as the layers go up (or down).

[0051] 1.2 Rough Matching

[0052] A rough match is performed in the low-resolution layer (smaller image size) of the pyramid. The goal of this step is to roughly locate the template position in the area to be detected and reduce the computational effort of subsequent high-resolution matching.

[0053] Use a suitable matching algorithm (such as normalized cross-correlation, square difference, etc.) to calculate the similarity between the template and the image of the area to be detected on the low-resolution image.

[0054] 2. Nearest Neighbor Scale Search

[0055] 2.1 Determine the initial match

[0056] Based on the rough matching results of the low-resolution layer of the pyramid, the possible matching area is determined. This preliminary matching result will provide an approximate range or location.

[0057] 2.2 Scale Search

[0058] In the adjacent scales of the pyramid (i.e. between different resolution layers), a refined matching search is performed. This step is to find the most suitable template matching position at different scales.

[0059] By matching images at different resolutions, the closest scale is found to ensure the best match is found in various image scaling situations.

[0060] 3. Three-step search method

[0061] 3.1 High-resolution fine matching

[0062] Using the initial position of the high-resolution layer of the pyramid as the starting point of the search, a three-step search method is performed on the high-resolution image of the area to be detected.

[0063] 3.2 Execute the three-step search method

[0064] Step 1: Match within a larger search range. This step can help you quickly find a rough matching position.

[0065] Step 2: Match within a smaller range around the matching area found in the first step to narrow the search scope and refine the results.

[0066] Step 3: Perform fine matching in a smaller range around the precise position found in step 2 to find the best template position.

[0067] S104: Extract feature points of the markers based on the marker recognition results, and calculate the L2 norms of the feature points of the two markers.

[0068] Among them, the L2 norm of the feature points of the two markers can be used to indicate the straight-line distance between the two feature points.

[0069] It is understandable that the distance between the two markers is different when the elevator door is in different open and closed states. Therefore, the embodiments of the present disclosure can provide a reliable analysis basis for determining the detection result of the open and closed door state of the elevator door to be detected.

[0070] S105: Based on the L2 norm of the feature point, determine the door opening and closing state detection result of the elevator door to be detected.

[0071] Among them, the door opening and closing state detection result can be used to indicate the door opening state, door closing state and door opening degree of the elevator door to be detected, and there is no restriction on this.

[0072] In the embodiment of the present disclosure, a relationship table may be pre-configured, and the door opening and closing state detection result corresponding to the feature point L2 norm may be determined based on the relationship table.

[0073] In this embodiment, by acquiring a real-time monitoring image of the elevator door to be detected; determining the image of the area to be detected in the real-time monitoring image, and preprocessing the image of the area to be detected, wherein the image of the area to be detected contains image data of two markers, and the two markers are symmetrically installed near the door gap of the elevator door to be detected; using the pyramid image template area fast search, the nearest neighbor scale search and the three-step search method to perform template matching on the preprocessed image of the area to be detected to obtain the marker recognition result; based on the marker recognition result, feature points of the marker are extracted, and the L2 norm of the feature points of the two markers is calculated; based on the L2 norm of the feature points, the door opening and closing state detection result of the elevator door to be detected is determined. In this way, the accuracy and real-time performance of the elevator door opening and closing state detection can be effectively improved.

[0074] Optionally, in some embodiments, when determining the door opening and closing state detection result of the elevator door to be detected based on the feature point L2 norm, it can be: determining the first detection result based on the feature point L2 norm; performing histogram statistical analysis on the pixels of the preprocessed image of the area to be detected to obtain pixel histogram statistical features, and obtaining the second detection result based on the pixel histogram statistical features; when the first detection result is consistent with the second detection result, the first detection result is used as the door opening and closing state detection result. Therefore, a secondary check of the first detection result can be implemented based on the histogram statistical analysis to ensure the accuracy of the obtained door opening and closing state detection result.

[0075] The first detection result and the second detection result refer to the door open / close state detection results obtained based on the feature point L2 norm and the pixel histogram statistical features, respectively.

[0076] Optionally, in some embodiments, when extracting feature points of the marker based on the marker recognition result, the maximum stable extreme value region may be used to extract feature points of the marker, thereby effectively reducing the computational cost of feature point extraction.

[0077] Optionally, in some embodiments, when the marker recognition result indicates an unrecognized marker, fill-in light processing can be performed and a new marker recognition result can be obtained; if the new marker recognition result indicates that the marker is recognized, a light abnormality alarm is issued; if the new marker recognition result indicates that the marker is not recognized, an occlusion alarm is issued. Thus, by performing fill-in light processing to obtain a new marker recognition result, rapid detection of light abnormalities and occlusion abnormalities can be achieved.

[0078] Optionally, in some embodiments, when the door opening and closing state detection result indicates that the elevator door to be detected is in the closed state, the actual position of the marker in the image of the area to be detected can be determined; the comparison result between the actual position and the preset reference position can be determined; based on the comparison result, the deflection angle of the camera can be determined, wherein the camera is used to obtain real-time monitoring images; and a lens deflection alarm can be issued when the deflection angle is greater than a first preset threshold. In this way, rapid detection of lens deflection can be achieved based on the comparison result between the actual position and the preset reference position.

[0079] The first preset threshold may refer to a threshold value configured for the deflection angle of the camera.

[0080] Optionally, in some embodiments, the clarity information and contrast information of the marker image in the preprocessed image of the area to be detected can be determined; based on the clarity information and contrast information, the lens contamination index can be calculated and determined; when the lens contamination index is greater than a second preset threshold, a lens contamination alarm can be issued. Thus, a lens contamination alarm can be issued when the lens contamination level is high, so that corresponding processing measures can be taken in time to maintain the availability of the lens.

[0081] Among them, the lens contamination index can be used to describe the degree of contamination of the lens. The larger the value of the lens contamination index is, the higher the degree of contamination is.

[0082] The second preset threshold refers to a threshold value pre-configured for the lens contamination index.

[0083] Based on the above embodiments, Figure 2 As shown, Figure 2 It is a flow chart of the detection method for elevator door opening and closing and abnormal state perception proposed in the present disclosure.

[0084] Among them, the elevator door opening and closing and abnormal state perception detection algorithm adopts the form of pattern recognition combined with active markers. The entire detection process includes:

[0085] First, the real-time color monitoring image of the elevator car is obtained through the surveillance camera and used as the input of the elevator door opening and closing detection algorithm. The entire elevator door opening and closing detection method includes the following modules:

[0086] S1, initialization module. (1) The algorithm model is created and internal parameters are loaded; (2) The ROI region (i.e., the image of the area to be detected) is automatically selected. The range of the area where the elevator door is located is determined so that the subsequent processing steps can focus on this area, reduce the interference of irrelevant information, and improve the processing speed. The ROI region can be automatically selected based on the pattern recognition results.

[0087] S2, feature extraction module. Figure 3 As shown, Figure 3It is a schematic diagram of the feature extraction process proposed in the present disclosure, which mainly includes the following steps:

[0088] S21, image preprocessing. (1) Image filtering. The filtering operation can realize the smoothing of the image, eliminate the random noise in the image, and make the image clearer and easier to observe; (2) Adaptive brightness correction. Adaptive brightness correction takes into account the local features and content in the image, and adjusts the brightness in each area separately, so as to retain more details and avoid overexposure or underexposure; (3) Contrast enhancement. Combine image information of multiple scales to balance local and global contrast enhancement, so as to avoid noise amplification and loss of details; (4) Affine transformation and image correction. Because the camera is installed in the left or right corner of the top of the car, the monitoring screen is tilted from top to bottom. In the image preprocessing disclosed in the present invention, an affine transformation is performed on the above-mentioned ROI area, so that the markers in the ROI area are in a front view state. The effect is tested on the verification set. The accuracy of template matching is improved by about 0.8%, and the matching accuracy of feature points is improved by about 3.5%.

[0089] S22, the marker has the characteristics of clear image, distinct outline, and reflective effect. The markers need to be used in pairs and installed on the elevator door. The installation must comply with the following key points: (1) Symmetrical installation: The two markers should be close to the elevator door gap and installed symmetrically; (2) Field of view and accessibility: The marker should be installed within the camera's field of view and the position should be convenient for the camera to capture a clear image; (3) Avoid obstruction: The marker should not be easily obstructed by passengers or objects. In addition, the installation of the marker does not have any rigid requirements for the lighting conditions in the elevator. Thanks to the internal algorithm that can automatically detect changes in light and the infrared fill light integrated on the camera mainboard, the fill light can be automatically turned on or off according to the light intensity in the elevator, so that the camera can clearly capture the marker under different lighting conditions, realizing door opening and closing detection under any lighting conditions around the clock. Combining the above conditions, if Figure 4 As shown, Figure 4 Schematic diagram of typical installation positions of markers proposed in the present disclosure.

[0090] S23, the pattern recognition adopts an improved template matching method. In order to better identify the marker, it is necessary to collect a diverse marker sample set for template learning. The marker sample set includes marker images at different angles, different heights, different background interferences, and different lighting conditions.

[0091] S24, the template matching is an improved fast template matching algorithm. The traditional template matching method has a huge amount of calculation and low efficiency when performing a full search on a large-size image. The present disclosure uses a pyramid image template area fast search, and by constructing an image pyramid, the horizontal large-scale search in the traditional template matching is converted into a local search in the vertical multi-scale space. When searching, it is not a layer-by-layer search on the Gaussian pyramid, but a nearest neighbor scale search method is used, that is, on the image layer of the best size that completed the template matching last time, for the image layer of this specific scale, the method of searching for the most similar feature points in the scale image adjacent to this layer is used. This method significantly reduces the search space and improves the search efficiency. When calculating the similarity of feature points, the corresponding similarity measurement criteria such as the normalized cross-correlation algorithm (NCC) are used to calculate the similarity between the template and the target area. In addition, NCC can better adapt to changes in illumination.

[0092] S25, the local area required for the template matching is the ROI area, and the ROI area can be automatically positioned according to the template matching result. First, during the power-on initialization process, when the elevator door is completely closed and fully opened, a global template matching will be performed to track the moving range of the marker. Considering that the marker is fixed on the elevator door and only moves to both sides with the opening and closing of the elevator door, the moving range is slightly expanded as the final ROI area. Subsequent template matching is only performed in the ROI area, which greatly reduces the amount of calculation and improves the processing speed.

[0093] S26, within the ROI area, the three-step search method (TSS) is used to complete motion estimation based on image block matching. Compared with other block matching algorithms, TSS is simple and efficient, gradually narrowing the search range and ultimately determining the best matching position, thereby realizing block matching in two-dimensional motion estimation.

[0094] S27 uses the maximum stable extreme region (MSER) ​​to extract features from the matched image blocks. MSER is superior to algorithms such as FAST, SURF, and SIFT in feature point detection speed, and is suitable for application scenarios with high stability and affine invariance such as in elevator cars.

[0095] S28, based on the feature points extracted from the markers on both sides of the door gap in the image, the L2 norm is calculated, and appropriate discrimination rules are set to determine the door opening and closing state of the elevator and give the degree of the elevator door opening.

[0096] S29. In addition, for the door opening and closing detection, considering that the light intensity and background of the elevator door accessories will change when the door is opened and closed, the algorithm also adds histogram statistical characteristics to analyze the degree of change of the image in the ROI area to assist in judging the door opening and closing results. Generally, elevators are mostly used indoors, and there is a constant lighting source in the car. In this case, the histogram statistical analysis of the image pixels in the ROI area can also obtain good calculation results with a very small amount of calculation, but it is not applicable to situations such as changes in outdoor lighting, flickering of light sources in the car, and power failure of light sources. However, the above-mentioned pattern recognition method can be used normally. The device disclosed in the present invention has an infrared fill light. Combined with the reflective characteristics of the marker, it can normally match the marker under any light to complete pattern recognition. In summary, the pattern recognition algorithm and the histogram statistical analysis method complement each other to achieve elevator door opening and closing detection under all-weather and various lighting conditions.

[0097] S3, elevator abnormality perception function recognition module. It has multiple abnormality perception functions, as follows:

[0098] S31, door opening and closing detection and door opening degree detection. As described above, pattern recognition and pixel histogram statistical features in the ROI area are used to realize real-time elevator door opening and closing detection and provide the elevator door opening degree.

[0099] S32, door opening and closing timeout detection. According to the above door opening and closing detection results, the number of door opening and closing within a certain period of time is counted, and a suitable alarm mechanism is set to implement door opening and closing timeout detection.

[0100] S33, light anomaly detection. Under normal use, if the marker is not matched in the ROI area, it may be caused by obstruction or insufficient lighting in the elevator or sudden light failure. At this time, the infrared fill light should be turned on. Because the marker has reflective properties, even if the external light is insufficient, the marker is still clearly visible in the video image. If the marker is matched in the ROI area after turning on the infrared fill light, it means that there is a light anomaly in the elevator car.

[0101] S34, occlusion detection. If no marker is matched in the ROI area under normal use and when the infrared fill light is turned on, it means that occlusion has occurred. There are two types of occlusion: one is that the camera lens is occluded, and the other is that the marker is occluded or falls off. At this time, the histogram statistical characteristics of the image pixels in the above ROI area are used for auxiliary judgment. If the video image clarity is good and the door opening and closing can be detected normally, it is considered that the marker is occluded or falls off. If the video clarity is poor and the door opening and closing cannot be detected normally, it is considered that the camera lens is occluded.

[0102] S35, door line detection. Use edge detection algorithms, such as the Canny operator, to identify the outline of the elevator door. Use straight line detection algorithms, such as the Hough transform, to identify the edge lines of the elevator door. Analyze the position changes of the door line feature points to assist in determining whether the elevator door is open or closed. On the other hand, it can also provide an approximate detection range for subsequent electric vehicle entry detection or human detection. In view of the small space inside the elevator, the elevator door line is easily blocked by passengers inside. Considering the large differences in color and texture between human skin, outer clothing, etc. and the elevator door, the present disclosure further adopts chromaticity detection and texture detection technology, which can further assist in determining whether the door line is blocked.

[0103] S36, lens deflection detection. When the elevator door is closed, the algorithm calculates the deflection angle of the camera by comparing the current position of the marker with the preset ideal center position. This calculation is based on affine transformation or homography matrix to ensure sub-pixel accuracy. If the position change is found to be beyond the normal range, it is judged as lens deflection. If lens deflection is detected, parameters can be provided for subsequent automatic adjustment of the camera angle according to the degree of deflection. When automatic correction cannot be performed or the deflection exceeds the preset threshold, an alarm is issued through the alarm module to remind relevant users to check.

[0104] S37, lens contamination detection. The position and shape of the marker in the ROI area can be obtained through the above-mentioned pattern recognition. Based on the analysis of the clarity and contrast of the marker image, the algorithm evaluates the degree of lens contamination. This step is completed by calculating the high-frequency components and texture, edge and other features of the image. It should be noted that lens contamination does not need to be detected frequently. It can be detected at a fixed time period, and the detection indicators can be added to a queue. By analyzing the curve changes of the detection indicators and setting reasonable thresholds, once the pollution is detected to exceed the preset threshold, the algorithm will trigger a lens contamination prompt to remind personnel to come and wipe it to ensure the maintenance of the best image quality.

[0105] S4, alarm output module: Determine whether the recognition result triggers an alarm mechanism.

[0106] S5, if the alarm mechanism is triggered, the abnormal alarm state of the elevator is established, and an alarm signal (sound, light and electricity) is issued to remind the personnel who check the alarm state.

[0107] S6, if the alarm mechanism is not triggered, the detection of the color video image to be recognized in the next week is performed.

[0108] In combination with the above embodiments, the present disclosure proposes a detection device for elevator door opening and closing and abnormal state perception, such as Figure 5 As shown, Figure 5 The present invention is a schematic diagram of a detection device for detecting an elevator door opening and closing and abnormal state perception, which includes:

[0109] The high-definition color camera 10 is used to collect visible light color video images.

[0110] The infrared fill light 20 can be automatically turned on to provide fill light when abnormal light is detected.

[0111] The speaker 30 notifies relevant personnel through voice playback when an abnormal state is detected.

[0112] Microphone 40, when there is an abnormality or emergency situation in the elevator, voice intercom with the remote monitoring platform can be achieved through the microphone.

[0113] The main control board 50 is electrically connected to the above four components. The main control board 50 includes a memory 501, a processor 502, a transceiver 504, a power module 505 and a peripheral interface 506. The computer program is stored in the memory 501. When the computer program is executed by the processor 502, the above elevator door opening and closing real-time detection method is implemented. The main control board 50 can be a Rockchip RV1106 core board, which contains a single-core ARM Cortex-A7 core and a built-in 0.5T computing power NPU.

[0114] In the present disclosure, the processor 502 is connected to the memory 501 and is connected via a bus 503. Optionally, the main control board 50 may further include a transceiver 504 and a backup power supply module 505. It should be noted that in actual applications, the transceiver 504 is not limited to one, and the structure of the main control board 50 does not constitute a limitation of the present disclosure. The peripheral connector 506 is connected to various data tail lines to achieve data interaction with the outside or the platform end.

[0115] The processor 502 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure in this disclosure. The processor 502 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0116] The bus 503 may include a path to transmit information between the above components. The bus 503 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 503 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0117] The present disclosure also proposes a real-time detection system for elevator door opening and closing and abnormality perception, which uses advanced image acquisition and processing technology to automatically monitor the opening and closing status of elevator doors. The system captures the image of the elevator door in real time through a high-definition camera, and after preprocessing and feature extraction, uses a professional door line detection algorithm to analyze the opening and closing status of the elevator door. The intelligent analysis unit further optimizes the accuracy of state recognition through pattern recognition. The system adopts real-time performance optimization technology to ensure low latency and high throughput data processing. The anomaly detection and alarm unit can issue an alarm in time when an abnormal state is detected, reminding maintenance personnel to check. The user interface provides intuitive operation and monitoring functions, and supports historical data query and report generation. In addition, the system integrates data management and communication technologies to achieve efficient collaboration and remote monitoring capabilities between modules. Overall, the system provides an automated and intelligent solution for the safety management of elevators, effectively improving the safety and reliability of elevator operation. The system includes:

[0118] 1. Image acquisition unit, with a high-definition camera integrated on the main board, installed at the corner inside the elevator or on the top of the car to capture the image of the elevator door in real time;

[0119] 2. The video processing unit performs operations such as denoising, enhancement, and binarization on the image to improve the accuracy of subsequent processing, and recognizes features such as the edge and contour of the elevator door to provide a basis for status judgment;

[0120] 3. Intelligent analysis unit, including learning and identifying the movement pattern of elevator doors based on markers, distinguishing normal door opening and closing from abnormal behavior, and establishing door line detection algorithms such as Hough transform based on the above image processing technology to assist in detecting the opening and closing status of elevator doors;

[0121] 4. Anomaly detection and alarm unit, which sets the threshold of state change and triggers the alarm when the threshold is exceeded. When an abnormal state is detected, relevant personnel are notified through sound and light signals or the network;

[0122] 5. System integration and communication unit, which realizes data exchange and communication between system modules through middleware, and supports wired and wireless network connections to realize remote monitoring and data transmission;

[0123] In addition, the system also includes data management and user interface, which establishes a database system to store historical data of elevator door status for later audit and analysis. The user interface provides real-time video display, status monitoring and alarm information, and supports remote access and control.

[0124] In combination with the above embodiments, the key points and points to be protected in this disclosure mainly include the following aspects:

[0125] Key Points:

[0126] 1. Image preprocessing: Use technologies such as image filtering, adaptive brightness correction, contrast enhancement, and automatic ROI area positioning to improve image quality, reduce irrelevant information, and focus on the elevator door area.

[0127] 2. Markers and pattern recognition: By installing a pair of reflective markers on the elevator door and combining them with an improved template matching method, accurate identification of the elevator door’s open and closed status can be achieved.

[0128] 3. Fast template matching algorithm: It adopts pyramid image template area fast search and nearest neighbor scale search, combined with NCC similarity measurement, to improve the efficiency and illumination adaptability of template matching.

[0129] 4.ROI area positioning and optimization: Automatically locate the ROI area based on template matching results to reduce calculation amount and improve processing speed.

[0130] 5. Feature extraction and motion estimation: Use TSS (three-step search method) for image block matching, MSER algorithm to extract feature points, and L2 norm calculation to determine the door open and close state.

[0131] 6. Histogram statistical analysis: Assists in judging the results of door opening and closing, and adapts to the detection needs under different lighting conditions.

[0132] 7. Rich abnormality perception functions: including light anomaly, occlusion, lens deflection and pollution detection to ensure the stability and accuracy of the system.

[0133] 8. Alarm output and system integration: Abnormal detection triggers the alarm mechanism, and system integration enables data exchange and remote monitoring capabilities.

[0134] Innovations of this publication:

[0135] 1. Improved template matching algorithm: fast search of pyramid image template area and nearest neighbor scale search improve matching speed and accuracy.

[0136] 2. Installation and use of markers: The specific installation location and use conditions of the markers ensure that the elevator door status can be detected normally under various lighting conditions.

[0137] 3. Automatic positioning and optimization of ROI area: Automatically locate the ROI area according to the moving range of the marker to improve processing efficiency.

[0138] 4. Feature extraction and state determination method: Use the MSER algorithm to extract feature points, and combine the L2 norm to calculate the switch state and opening degree of the elevator door.

[0139] 5. Anomaly detection algorithm: Detection methods for light anomalies, occlusion, lens deflection and contamination to ensure the robustness and stability of the system.

[0140] 6. Provides a complete set of detection methods, devices and systems for elevator door opening and closing and abnormal state perception: including high-definition cameras, infrared fill lights, main control boards, transceivers and other hardware components, as well as data management and communication technologies, to achieve efficient collaboration between modules and remote monitoring capabilities.

[0141] Through these key points and points to be protected, the present disclosure aims to provide an efficient, accurate and adaptable elevator door opening and closing and abnormal state perception detection method, system and device to improve the safety and reliability of elevator operation.

[0142] The present disclosure can at least achieve the following technical effects:

[0143] 1. High-precision recognition. By using the improved template matching method and the fast template matching algorithm, the present invention can accurately identify the marker and maintain a high recognition rate even under different angles, heights, background interference and lighting conditions.

[0144] 2. Good real-time performance. The disclosed method can monitor the status of elevator doors in real time. Through optimized algorithms and calculation processes, the detection efficiency and real-time performance are ensured. The automatic positioning and local search strategy of the ROI area further reduce the amount of calculation and improve the processing speed. The pyramid image template area fast search technology is used to significantly reduce the search space and improve the search efficiency.

[0145] 3. Strong adaptability and robustness. The present invention takes into account a variety of practical application scenarios, such as light changes, occlusion, lens deflection and pollution, and enhances the adaptability and robustness of the system by combining pattern recognition and histogram statistical analysis. In addition, the use of infrared fill light ensures normal detection even in low light conditions.

[0146] 4. Complete abnormality detection and alarm function. The present invention can not only detect the door opening and closing status of the elevator, but also sense various abnormal conditions, such as abnormal light, occlusion, lens deflection and pollution, and send out alarm signals in time, thus improving the safety of the elevator. The design of the alarm output module enables the system to respond quickly when an abnormality is detected, and remind relevant personnel to handle it.

[0147] 5. Easy to install and maintain. This elevator abnormality sensing camera contains intelligent algorithms, and the system supports remote upgrades. The camera is installed in the inner corner of the right or left wall of the elevator; the camera image needs to cover the entire elevator door and cover as much of the car floor as possible. The marker should be installed symmetrically close to the elevator door gap, and the height should be higher than the height of ordinary passengers to reduce the probability of the marker being blocked.

[0148] Figure 6 It is a structural schematic diagram of an elevator door opening and closing state detection device proposed in an embodiment of the present disclosure.

[0149] like Figure 6 As shown, the elevator door opening and closing state detection device 60 includes:

[0150] An acquisition module 601 is used to acquire a real-time monitoring image of the elevator door to be detected;

[0151] The first determination module 602 is used to determine the image of the area to be detected in the real-time monitoring image and pre-process the image of the area to be detected, wherein the image of the area to be detected includes image data of two markers, and the two markers are symmetrically installed near the door gap of the elevator door to be detected;

[0152] The template matching module 603 is used to perform template matching on the pre-processed image of the area to be detected by using a pyramid image template area fast search, a nearest neighbor scale search and a three-step search method to obtain a marker recognition result;

[0153] A feature extraction module 604 is used to extract feature points of the markers based on the marker recognition results, and calculate the L2 norm of the feature points of the two markers;

[0154] The second determination module 605 is used to determine the door opening and closing state detection result of the elevator door to be detected based on the L2 norm of the feature point.

[0155] It should be noted that the aforementioned explanation of the elevator door opening and closing state detection method is also applicable to the elevator door opening and closing state detection device of this embodiment, and will not be repeated here.

[0156] In this embodiment, by acquiring a real-time monitoring image of the elevator door to be detected; determining the image of the area to be detected in the real-time monitoring image, and preprocessing the image of the area to be detected, wherein the image of the area to be detected contains image data of two markers, and the two markers are symmetrically installed near the door gap of the elevator door to be detected; using the pyramid image template area fast search, the nearest neighbor scale search and the three-step search method to perform template matching on the preprocessed image of the area to be detected to obtain the marker recognition result; based on the marker recognition result, feature points of the marker are extracted, and the L2 norm of the feature points of the two markers is calculated; based on the L2 norm of the feature points, the door opening and closing state detection result of the elevator door to be detected is determined. In this way, the accuracy and real-time performance of the elevator door opening and closing state detection can be effectively improved.

[0157] Figure 7 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Figure 7 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0158] like Figure 7 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0159] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0160] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0161] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 Not shown, often called a "hard drive").

[0162] although Figure 7 Not shown, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (Compact Disc Read Only Memory; hereinafter referred to as: CD-ROM), a digital versatile disc read only memory (Digital Video Disc Read Only Memory; hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0163] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0164] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a human body to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0165] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the elevator door opening and closing state detection method mentioned in the above embodiment.

[0166] In order to implement the above embodiments, the present disclosure also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the elevator door opening and closing state detection method proposed in the above embodiments of the present disclosure is implemented.

[0167] In order to implement the above embodiments, the present disclosure further proposes a computer program product. When the instruction processor in the computer program product is executed, the elevator door opening and closing state detection method proposed in the above embodiments of the present disclosure is executed.

[0168] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0169] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0170] It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0171] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0172] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0173] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0174] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0175] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0176] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0177] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A method for detecting the state of an elevator door opening and closing, characterized in that: include: Obtain real-time monitoring images of the elevator door to be inspected; Determine an image of the area to be detected in the real-time monitoring image, and pre-process the image of the area to be detected, wherein the image of the area to be detected includes image data of two markers, and the two markers are symmetrically installed near the door gap of the elevator door to be detected; The pyramid image template area fast search, nearest neighbor scale search and three-step search method are used to perform template matching on the preprocessed image of the area to be detected to obtain the marker recognition result; Extracting feature points of the markers based on the marker recognition results, and calculating the L2 norms of the feature points of the two markers; Based on the L2 norm of the feature point, a detection result of the door opening and closing state of the elevator door to be detected is determined.

2. The method according to claim 1, characterized in that The preprocessing of the image of the area to be detected includes at least one of the following: Performing filtering processing on the image of the area to be detected; Performing brightness correction processing on the image of the area to be detected; Performing contrast enhancement processing on the image of the area to be detected; Perform affine transformation processing on the image of the area to be detected.

3. The method according to claim 1, characterized in that The step of determining the door opening and closing state detection result of the elevator door to be detected based on the feature point L2 norm includes: Determine a first detection result based on the L2 norm of the feature point; Performing histogram statistical analysis on the pixels of the preprocessed image of the area to be detected to obtain pixel histogram statistical features, and obtaining a second detection result based on the pixel histogram statistical features; When the first detection result is consistent with the second detection result, the first detection result is used as the door opening and closing state detection result.

4. The method according to claim 1, characterized in that The extracting feature points of the marker based on the marker recognition result includes: The maximum stable extreme value region is used to extract feature points of the marker.

5. The method according to claim 1, characterized in that The method further comprises: When the marker recognition result indicates that the marker is not recognized, fill light processing is performed, and a new marker recognition result is obtained; If the new marker recognition result indicates that the marker is recognized, an abnormal light alarm is issued; If the new marker recognition result indicates that the marker is not recognized, an occlusion alarm is issued.

6. The method according to claim 1, characterized in that The method further comprises: When the door opening / closing state detection result indicates that the elevator door to be detected is in a door closing state, determining the actual position of the marker in the image of the area to be detected; Determine a comparison result between the actual position and a preset reference position; Based on the comparison result, determining a deflection angle of a camera, wherein the camera is used to obtain the real-time monitoring image; When the deflection angle is greater than a first preset threshold, a lens deflection alarm is issued.

7. The method according to claim 1, characterized in that The method further comprises: Determining the clarity information and contrast information of the marker image in the preprocessed image of the area to be detected; Calculate and determine a lens contamination index according to the clarity information and the contrast information; A lens contamination alarm is issued when the lens contamination index is greater than a second preset threshold.

8. An elevator door opening and closing state detection device, characterized in that: include: An acquisition module is used to acquire a real-time monitoring image of the elevator door to be detected; A first determination module is used to determine an image of a region to be detected in the real-time monitoring image and pre-process the image of the region to be detected, wherein the image of the region to be detected includes image data of two markers, and the two markers are symmetrically installed near a door gap of the elevator door to be detected; The template matching module is used to perform template matching on the pre-processed image of the area to be detected by using the pyramid image template area fast search, the nearest neighbor scale search and the three-step search method to obtain the marker recognition result; A feature extraction module, used for extracting feature points of the markers based on the marker recognition result, and calculating the L2 norm of the feature points of the two markers; The second determination module is used to determine the door opening and closing state detection result of the elevator door to be detected based on the L2 norm of the feature point.

9. A computer device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images