A contactless human-machine interaction system and method for an elevator

Abstract

This application relates to the technical field of elevators, and particularly to a contactless human-machine interaction system and method for elevators, which includes a floor display terminal, a portable terminal, an image collector, an edge processing module, and an elevator controller. The system generates a marked image by the portable terminal photographing the floor display terminal, and after being collected by the image collector, the edge processing module analyzes the current floor and target floor information to control the operation of the elevator, without external communication throughout the process. This application adopts technologies such as color marking, perspective correction, and conflict arbitration to ensure an efficient and accurate interaction experience, and improves the system stability and security through visual feedback and floor instruction sharing logic. This solution avoids the dependence on the external environment of traditional non-contact methods and realizes a hygienic, stable, and efficient elevator interaction.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular, to a contactless human-machine interaction system and method for elevators. Background Art

[0002] In modern buildings and public places, elevators, as important vertical transportation tools, the convenience, hygiene, and reliability of their human-machine interaction methods directly affect the user experience and public health. The traditional elevator human-machine interaction method usually relies on passengers to send ride requests by pressing the call buttons at the elevator entrances on each floor, and then select the destination floor through the internal floor selection panel after entering the elevator. However, this contact-based interaction method has obvious deficiencies: on the one hand, passengers need to directly touch the elevator buttons, which is easy to cause cross-transmission of germs, especially in the context of frequent public health incidents, this problem is particularly prominent; on the other hand, when there are many passengers in the elevator, passengers far from the floor selection panel may be unable to conveniently select the destination floor because it is difficult to reach the panel, reducing the interaction efficiency and user experience.

[0003] To solve the above problems, a series of contactless elevator human-machine interaction solutions have emerged in the market in recent years. For example, the invention patent with the application number 202020755266.8 discloses a contactless human-machine interaction system for elevators based on voice recognition and face recognition, and realizes data transmission between the automatic control device, the voice recognition device, and the face interaction device through a Bluetooth module. However, this solution has limitations in practical applications: when there are many passengers in the elevator, the system needs to process the interaction requirements of multiple users simultaneously, and it is easy to cause recognition failures due to environmental noise and user occlusion, thus affecting the reliability and stability of the interaction. In addition, this solution has high requirements for hardware devices, increasing the complexity and cost of the system.

[0004] Another typical contactless solution is a call system based on two-dimensional codes disclosed in the invention patent with the application number 201710816120.2, which realizes human-machine interaction through two-dimensional codes combined with a mobile phone APP. However, this solution also has significant defects: firstly, passengers need to download and register a dedicated APP in advance, which is cumbersome and unfriendly for unfamiliar visitors; secondly, this solution highly depends on the communication network environment, and in scenarios with weak signals such as elevator shafts, the system may fail to function due to inability to access the server, restricting its scope of application.

[0005] In summary, although the existing contactless human-machine interaction technology for elevators has solved the hygiene and convenience problems of traditional contact-based interaction to a certain extent, there are still many deficiencies, including low interaction reliability, strong dependence on hardware and communication environments, and poor user experience. Therefore, developing an elevator human-machine interaction system that is contactless, does not rely on a communication network, can adapt to multi-user scenarios, and has high reliability has become a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The present application provides an elevator contactless human-machine interaction system and a human-machine interaction method. Through the system and method provided by the present application, efficient, hygienic, and direct-contact-free interaction between passengers and the elevator can be achieved.

[0007] In a first aspect, the present application provides an elevator contactless human-machine interaction system, which includes a floor display terminal, a portable terminal, an image collector, an edge processing module, and an elevator controller. Among them: The floor display terminal is fixedly installed on the elevator hall wall, and is used to display all reachable floor numbers, the current floor number area marked with a first color, and the dynamically updated elevator car position; The portable terminal has a camera and a touch screen, and is used to take a first image generated by the floor display terminal and display the first image on the touch screen. The user generates a second image by clicking on the target floor number area for second color marking; The image collector is arranged on the side of the floor display terminal and is used to collect the second image displayed on the screen of the portable terminal; The edge processing module is electrically connected to the image collector and is built with a color segmentation unit and an OCR recognition unit, and is used to extract the first color area based on a preset HSV threshold to identify the current floor, the second color area to identify the target floor, and perform local digital recognition on the segmented area; The elevator controller receives the current floor and target floor data output by the edge processing module and controls the operation of the elevator, without the need for data communication between the portable terminal and the elevator control system throughout the process. In this way, through the local calculation of the edge processing module, the dependence on the external communication environment in the traditional non-contact solution is avoided, and the stability and response speed of the system are improved.

[0008] Furthermore, the floor display terminal in the present application is designed with reachable floor numbers arranged in a 3×4 matrix. A red border is set around the current floor number, and the elevator car position is displayed through a dynamic arrow icon. The arrow color is bound to the running direction: blue when going up and yellow when going down. This visual design is not only intuitive and clear but also can effectively distinguish different information levels and improve the user experience.

[0009] In addition, the image collector in this application uses a wide-angle camera, and the coverage range includes the area 1.5 meters in front of the floor display terminal. The edge processing module performs perspective correction and screen reflection suppression on the collected second image. Perspective correction processes the second image through the following formula to eliminate the distortion caused by the inclined shooting of the portable terminal screen:

[0010] ;

[0011] where x and y are the coordinates of the original image, and x' and y' are the coordinates after correction. 、 are the principal point coordinates, is the focal length, and k is the distortion coefficient.

[0012] For screen reflection suppression, the polarization filtering algorithm is used to extract the effective annotation area to ensure the accurate extraction of annotation information.

[0013] The second color annotation of the portable terminal needs to meet specific conditions. Its hue value H = 120° ± 10°, saturation S > 80%, transparency = 0.6 - 0.8; the area of the annotation area accounts for 50% - 150% of the target floor digital display area. If it is detected that the same digital area is clicked multiple times, the transparency of the annotation color increases according to the formula: increases, where n is the number of clicks and .

[0014] This mechanism not only enhances the visualization effect of the annotation but also avoids the possibility of misoperation.

[0015] The edge processing module also has a conflict arbitration unit. When multiple second images are collected at the same time, the instruction with the center point of the annotation area closest to the geometric center of the floor number is preferentially selected; if there are overlapping annotations, the one with a higher saturation of the annotation color is used as the valid instruction. Conflict arbitration calculates the priority through the following formula: ;

[0016] where, is the instruction priority score; is the Euclidean distance between the center point of the annotation area and the geometric center of the floor number; the saturation of the annotation color; is the weight coefficient.

[0017] This algorithm ensures the fairness and accuracy of interaction in a multi-user scenario.

[0018] In a second aspect, the present application provides a human-computer interaction method based on the above system, including an elevator calling stage and a floor selection stage. In the elevator calling stage, the passenger uses a portable terminal to capture an image of the floor display terminal, generating a first image with the current floor marked. After being captured by the image collector, the edge processing module analyzes the current floor and schedules the car to arrive. In the floor selection stage, the passenger clicks on the area corresponding to the target floor number on the touch screen, generating a second image with a second color-marked area. The edge processing module analyzes the target floor and executes the transportation. All operations only change the image displayed locally on the portable terminal and do not send any electrical signals externally, thus ensuring data security and privacy.

[0019] Further, the edge processing module analyzing the target floor and executing the transportation includes: performing a region-based HSV color space conversion on the second image, extracting the current floor number that meets the first color threshold and the target floor number that meets the second color threshold, and comparing the numbers extracted by OCR with the reachable floor database of the floor display terminal to determine the current floor and the target floor. In addition, the system is also equipped with an operation feedback mechanism. When the target floor is successfully analyzed, the corresponding floor number on the floor display terminal is controlled to flash three times; if the analysis fails, a red breathing light effect is displayed on the edge LED light strip of the floor display terminal. All feedback information is transmitted only through the visual channel without the participation of the portable terminal.

[0020] Finally, the present application sets up a floor instruction sharing logic. When the target floor is successfully analyzed, the floor display terminal lights up the target floor number to remind the passenger that this floor has been selected; this avoids passengers with the same target floor from performing repeated floor selection operations, improving the interaction efficiency while also avoiding the data processing burden caused by repeated floor selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 : Schematic diagram of the structure of the elevator contactless human-computer interaction system.

[0022] Figure 2 : Schematic diagram of the working principle of the edge processing module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] The present invention provides a detailed implementation of an elevator contactless human-computer interaction system, in combination with the attached Figure 1 to the attached Figure 2The system composition, operating principle, and operation process are described in detail. In this embodiment, the system includes a floor display terminal, a portable terminal, an image collector, an edge processing module, and an elevator controller. The specific function implementation of each component and their interaction process will be described one by one below.

[0025] The floor display terminal is fixedly installed on the wall of the elevator hall and is used to provide intuitive floor information and car status to passengers. As Figure 1 shown, this terminal adopts a 3×4 matrix layout design, displays all reachable floor numbers, and marks the current floor number with a red border. The color parameters of the red border are H = 0° ± 5°, S > 90%, V > 90%, ensuring clear visibility under different lighting conditions. In addition, a dynamic arrow icon is used to indicate the car position and running direction, and the arrow color is bound to the running direction: blue when going up and yellow when going down. This visual design differentiates information levels through color, enhancing the user experience. For example, in an office building scenario, when the elevator is on the 1st floor and about to go up, the floor display terminal will display the 1st floor number with a red border, and at the same time, the dynamic arrow icon will be displayed in blue and point to the area above the 1st floor.

[0026] The portable terminal is the core tool for users to interact with the system. It has functions such as a camera and a touch screen, such as smartphones, tablets, and laptops. First, the user uses the camera of the portable terminal to take a picture of the floor display terminal to generate a first image. The first image is then displayed on the touch screen of the portable terminal, and the user generates a second image of the second color annotation area by clicking on the target floor number area. The hue value of the annotation area needs to satisfy H = 120° ± 10°, saturation S > 80%, transparency = 0.6 - 0.8, and the area of the annotation area accounts for 50% - 150% of the target floor number display area. If the user clicks on the same number area multiple times, the transparency of the annotation color increases according to the formula where n is the number of clicks and .

[0027] This mechanism not only enhances the visualization effect of the annotation but also effectively avoids the possibility of misoperation. For example, in actual applications, when the user stands about 1 meter in front of the elevator hall, uses the portable terminal to take a picture of the floor display terminal, and clicks on the target floor to generate a green annotation area, the generated second image contains the annotation information of the current floor and the target floor.

[0028] The image collector is set on the side of the floor display terminal, and the coverage range includes an area 1.5 meters in front of the floor display terminal, and is used to collect the second image displayed on the screen of the portable terminal. To eliminate the distortion caused by the inclined shooting of the portable terminal screen, the edge processing module performs perspective correction on the collected second image. The perspective correction formula is as follows ;

[0029] Where x and y are the coordinates of the original image, and x' and y' are the coordinates after correction. , are the coordinates of the principal point, is the focal length, and k is the distortion coefficient.

[0030] For example, if the image collector is a binocular camera for obtaining the spatial coordinates of the portable terminal, the coordinates of the principal point , the focal length f = 800 pixels, and the distortion coefficient k = 0.12. If the original coordinates are (400, 300), the corrected coordinates are (399.25, 299.44), indicating that the original coordinates (400, 300) are located above and to the right of the principal point (320, 240), and the corrected coordinates (399.25, 299.44) are slightly contracted towards the principal point, meeting the expected effect of distortion correction.

[0031] In addition, screen reflection suppression extracts the effective annotation area through a polarization filtering algorithm to ensure the accurate extraction of annotation information. For example, in a shopping mall scenario, when the user stands in a strong light environment and uses the portable terminal to generate a second image, the image collector can accurately capture the annotation information and transmit the data to the edge processing module.

[0032] The edge processing module is built-in with a color segmentation unit and an OCR recognition unit, which are used to extract the first color area based on a preset HSV threshold to identify the current floor, the second color area to identify the target floor, and perform local digital recognition on the segmented area. Specifically, the parsing process includes performing a sub-region HSV color space conversion on the second image, extracting the current floor number that meets the first color threshold, and the target floor number that meets the second color threshold. Subsequently, the numbers extracted through OCR recognition are compared with the reachable floor database of the floor display terminal to determine the current floor and the target floor. For example, in a hospital scenario, when the edge processing module receives the second image transmitted by the image collector, it first extracts the annotation information of the current floor "3" and the target floor "8" through HSV color space conversion, then confirms the floor number through OCR recognition and compares it with the database, and finally outputs the data that the current floor is the 3rd floor and the target floor is the 8th floor.

[0033] The conflict arbitration unit is an important part of the edge processing module, which is used to solve the instruction conflict problem in a multi-user scenario. When multiple second images are collected at the same time, the instruction with the center point of the annotation area closest to the geometric center of the floor number is preferentially selected; if there is overlapping annotation, the one with higher color saturation is taken as the valid instruction. The conflict arbitration calculates the priority through the following formula: ;

[0034] Where is the instruction priority score; is the Euclidean distance between the center point of the marked area and the geometric center of the floor number; The saturation of the marked color (range 0 - 100%); is the weight coefficient (which needs to be adjusted according to the actual scenario, for example .

[0035] For example, during the peak hours in an office building, if the mark of user A is closer to the center but has a lower saturation; while the mark of user B is slightly farther away but has a higher saturation, the system will determine the priority by comprehensively considering the scores of both. Both users generate the second image using a portable terminal at the same time. The edge processing module calculates the priority through a conflict arbitration formula and selects the instruction with the marked area center point closer to the geometric center of the floor number and a higher color saturation as the valid instruction, thereby ensuring the fairness and accuracy of the interaction in a multi-user scenario.

[0036] The elevator controller receives the current floor and target floor data output by the edge processing module and controls the operation of the elevator. There is no need for data communication between the portable terminal and the elevator control system throughout the process, thus ensuring the security and privacy of the data. For example, in a residential community scenario, when the edge processing module resolves that the current floor is the 1st floor and the target floor is the 10th floor, the elevator controller schedules the car to descend from the current position to the 1st floor to pick up passengers, and then ascends to the 10th floor to complete the transportation task after the passengers enter the car.

[0037] This system also includes an operation feedback mechanism. When the target floor is successfully resolved, it controls the corresponding floor number on the floor display terminal to flash 3 times; if the resolution fails, a red breathing light effect is displayed on the LED strip at the edge of the floor display terminal. All feedback information is transmitted only through the visual channel without the participation of the portable terminal. For example, in a hotel scenario, when the user generates the second image, if the edge processing module successfully resolves that the target floor is the 7th floor, the number 7 on the floor display terminal will flash 3 times to prompt the user that the operation is successful; if the resolution fails, the LED strip at the edge of the floor display terminal will display a red breathing light effect to prompt the user to operate again.

[0038] The system also includes instruction sharing logic. After successfully parsing the target floor, the display terminal lights up the target floor number to remind passengers that this floor has been selected, preventing passengers with the same target floor from repeating the floor selection operation, which improves the interaction efficiency and also avoids the data processing burden caused by repeated floor selection. For example, during the peak hours in an office building, there are many passengers waiting for the elevator in the elevator hall. The target floors of passenger A and passenger B are both the 10th floor. When passenger A places the second image in front of the image collector and the edge processing module successfully parses the target floor as the 10th floor, the display terminal lights up the number of the 10th floor, indicating that the 10th floor has been selected. After observing that the 10th floor has been selected, passenger B does not need to perform the elevator call and floor selection operations, which not only avoids the reduction of interaction efficiency caused by repeated operations but also avoids the data processing burden brought by redundant data generated by repeated floor selection.

[0039] In summary, through the collaborative work of the floor display terminal, portable terminal, image collector, edge processing module, and elevator controller, the present invention realizes an efficient, hygienic, and non-direct-contact interaction between passengers and the elevator. The system exhibits excellent stability and response speed in actual application scenarios. At the same time, by means of local calculation, it avoids dependence on the external communication environment, significantly improving the user experience and security.

[0040] The present invention also provides a human-computer interaction method based on the above elevator non-contact human-computer interaction system, including an elevator call stage and a floor selection stage. In the elevator call stage, the passenger uses the portable terminal to capture the floor display terminal to generate a first image with the current floor marked. After being captured by the image collector, the edge processing module parses the current floor and schedules the car to arrive. In the floor selection stage, the passenger clicks on the area corresponding to the target floor number on the touch screen to generate a second image with a second color marked area. The edge processing module parses the target floor and executes the transportation. All operations only change the images displayed locally on the portable terminal and do not send any electrical signals to the outside, thus ensuring the security and privacy of the data.

[0041] Further, in the present application, after being captured by the image collector, the edge processing module parses the current floor and schedules the car to arrive, including: performing a regional HSV color space conversion on the first image, extracting the current floor number that meets the first color threshold, and extracting the current floor number through OCR recognition to determine the current floor.

[0042] Further, in the present application, the edge processing module parses the target floor and executes the transportation, including: performing a regional HSV color space conversion on the second image, extracting the current floor number that meets the first color threshold and the target floor number that meets the second color threshold, and comparing the numbers extracted through OCR recognition with the reachable floor database of the floor display terminal to determine the current floor and the target floor.

[0043] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only the specific embodiments. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the relevant art can well understand and utilize the present invention.

Claims

1. A contactless human-machine interaction system for an elevator, characterized in that, The system includes a floor display terminal, a portable terminal, an image collector, an edge processing module, and an elevator controller, where: The floor display terminal is fixedly installed on the wall of the elevator hall, and is used to display all reachable floor numbers, the current floor number area marked with a first color, and the dynamically updated elevator car position; The portable terminal has a camera and a touch screen. The camera is used to capture a first image of the floor display terminal and display the first image on the touch screen. The user generates a second image by clicking on the target floor number area for a second color marking; The second color annotation of the portable terminal shall meet the conditions: hue value H = 120° ± 10°, saturation S > 80%, transparency ; the area of the annotation region accounts for 50% - 150% of the digital display region of the target floor number; if it is detected that the same digital region is clicked multiple times, the transparency of the annotation color increases according to the formula where n is the number of clicks and ; The image collector is arranged on the side of the floor display terminal and is used to collect the second image displayed on the screen of the portable terminal; The edge processing module is electrically connected to the image collector and is built-in with a color segmentation unit and an OCR recognition unit, and is used to extract the first color area based on a preset HSV threshold to identify the current floor, the second color area to identify the target floor, and perform local digital recognition on the segmented area; The edge processing module is provided with a conflict arbitration unit. When multiple second images are collected at the same time, the instruction with the center point of the marked area closest to the geometric center of the floor number is preferentially selected; if there is an overlapping mark, the one with a higher color saturation of the mark is used as the valid instruction; the conflict arbitration calculates the priority through the following formula: ； Among them, is the instruction priority score; is the Euclidean distance between the center point of the marked area and the geometric center of the floor number; is the saturation of the marked color; is the weight coefficient; The elevator controller receives the current floor and target floor data output by the edge processing module and controls the operation of the elevator.

2. The contactless human-machine interaction system for an elevator according to claim 1, wherein The floor display terminal is designed with reachable floor numbers arranged in a 3×4 matrix. A red border is set around the current floor number, and the elevator car position is displayed through a dynamic arrow icon. The arrow color is bound to the running direction: blue when going up and yellow when going down.

3. The contactless human-machine interaction system for an elevator according to claim 1, wherein The image collector uses a wide-angle camera, and the coverage range includes an area 1.5 meters in front of the floor display terminal. The edge processing module performs perspective correction and screen reflection suppression on the collected second image.

4. The contactless human-machine interaction system for an elevator according to claim 3, wherein, The perspective correction eliminates the distortion caused by the tilted shooting of the portable terminal screen through the following formula: ； Where x and y are the original image coordinates, and x' and y' are the corrected coordinates. is the principal point coordinate, is the focal length, and k is the distortion coefficient.

5. The contactless human-machine interaction system for an elevator according to claim 1, wherein The system also includes an operation feedback mechanism. When the target floor is successfully parsed, the corresponding floor number of the floor display terminal is controlled to flash 3 times; if the parsing fails, a red breathing light effect is displayed on the LED light strip at the edge of the floor display terminal.

6. The contactless human-machine interaction system for elevator according to claim 5, wherein, The system sets a floor instruction sharing logic. When the target floor is successfully parsed, the floor display terminal lights up the target floor number to remind the passengers that this floor has been selected.

7. A human-computer interaction method for an elevator non-contact human-computer interaction system according to any one of claims 1 to 6, characterized in that, It includes the elevator call stage and the floor selection stage: In the elevator call stage, the passenger uses the portable terminal to capture the floor display terminal to generate a first image with the current floor marked. After the image collector captures it, the edge processing module parses out the current floor and schedules the car to arrive; In the floor selection stage, the passenger clicks on the corresponding area of the target floor number on the touch screen to generate a second image with a second color marked area, and the edge processing module parses the target floor and performs the transportation.

8. The human-computer interaction method according to claim 7, wherein The edge processing module parses the target floor and performs the transportation including: Perform sub-region HSV color space conversion on the second image, extract the current floor number that meets the first color threshold and the target floor number that meets the second color threshold, and compare the numbers extracted by OCR with the reachable floor database of the floor display terminal to determine the current floor and the target floor.

Images