Elevator non-contact man-machine interaction system and man-machine interaction method

Abstract

The invention relates to the technical field of elevators, in particular to an elevator non-contact man-machine interaction system and man-machine interaction method.The elevator non-contact man-machine interaction system comprises a floor display terminal, a portable terminal, an image collector, an edge processing module and an elevator controller. According to the system, the portable terminal shoots the floor display terminal to generate annotation images, the image collector collects the annotation images, the edge processing module analyzes information of the current floor and the target floor, elevator operation is controlled, and external communication is not needed in the whole process. According to the method, technologies such as color marking, perspective correction and conflict arbitration are adopted, efficient and accurate interaction experience is ensured, and the stability and safety of the system are improved through visual feedback and floor instruction sharing logic. According to the scheme, dependence of a traditional non-contact mode on the external environment is avoided, and sanitary, stable and efficient elevator interaction is achieved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a contactless human-computer interaction system and a human-computer interaction method for an elevator. Background Art

[0002] In modern buildings and public places, elevators are important vertical transportation tools. The convenience, hygiene and reliability of their human-computer interaction methods directly affect user experience and public health. Traditional elevator human-computer interaction methods usually rely on passengers to issue a ride request through the elevator call button at the elevator entrance, and select the target floor through the internal floor selection panel after entering the elevator. However, this contact interaction method has obvious shortcomings: on the one hand, passengers need to directly touch the elevator buttons, which can easily lead to cross-transmission of germs, especially in the context of frequent public health events. This problem is particularly prominent; on the other hand, when there are many elevator passengers, passengers far away from the floor selection panel may not be able to conveniently select the target floor because it is difficult to touch the panel, which reduces the interaction efficiency and user experience.

[0003] In order to solve the above problems, a series of contactless elevator human-computer interaction solutions have appeared on the market in recent years. For example, the invention patent with application number 202020755266.8 discloses an elevator contactless human-computer interaction system based on voice recognition and face recognition, which realizes data transmission between the automatic control device, the voice recognition device and the face interaction device through a Bluetooth module. However, this solution has limitations in practical applications: when there are many passengers in the elevator, the system needs to handle the interaction needs of multiple users at the same time, which is prone to recognition failure due to environmental noise and user occlusion, thereby affecting the reliability and stability of the interaction. In addition, the solution has high requirements for hardware equipment, which increases the complexity and cost of the system.

[0004] Another typical contactless solution is a QR code-based elevator call system disclosed in the invention patent with application number 201710816120.2, which realizes human-computer interaction through QR code combined with mobile phone APP. However, this solution also has significant defects: first, passengers need to download and register a dedicated APP in advance, which is cumbersome and unfriendly for unfamiliar visitors; second, the solution is highly dependent on the communication network environment, and in scenes with weak signals such as elevators, the system may fail to function due to inability to access the server, limiting its scope of application.

[0005] In summary, although the existing elevator contactless human-computer interaction technology has solved the hygiene and convenience problems of traditional contact interaction to a certain extent, it still has many shortcomings, including low interaction reliability, strong dependence on hardware and communication environment, and poor user experience. Therefore, developing an elevator human-computer interaction system that does not require contact, does not rely on communication networks, can adapt to multi-user scenarios, and has high reliability has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The present application provides a contactless human-computer interaction system and a human-computer interaction method for an elevator. Through the system and method provided by the present application, efficient, hygienic and direct contact-free interaction between passengers and elevators can be achieved.

[0007] In the first aspect, the present application provides an elevator contactless human-computer interaction system, which includes a floor display terminal, a portable terminal, an image collector, an edge processing module and an elevator controller, wherein: the floor display terminal is fixedly installed on the wall of the elevator hall, and is used to display all accessible floor numbers, the current floor number area marked with a first color, and the dynamically updated elevator car position; the portable terminal has a camera and a touch screen, which are used to shoot the floor display terminal to generate a first image, and display the first image on the touch screen, and the user generates a second image by clicking on the target floor number area for a second color mark; the image collector is arranged on the side of the floor display terminal, and is used to collect the second image displayed on the portable terminal screen; the edge processing module is electrically connected to the image collector, and has a built-in color segmentation unit and an OCR recognition unit, which are used to extract the first color area to identify the current floor and the second color area to identify the target floor based on a preset HSV threshold, and perform localized digital recognition on the segmented area; the elevator controller receives the current floor and target floor data output by the edge processing module, controls the elevator operation, and does not require data communication between the portable terminal and the elevator control system throughout the process. In this way, through the localized calculation of the edge processing module, the dependence on the external communication environment in traditional contactless solutions is avoided, and the stability and response speed of the system are improved.

[0008] Furthermore, the floor display terminal in this application is designed as a 3×4 matrix of accessible floor numbers, with a red border around the current floor number, and the elevator car position is displayed through a dynamic arrow icon, the arrow color is bound to the running direction: the arrow is blue when going up and yellow when going down. This visual design is not only intuitive and clear, but also can effectively distinguish different information levels and improve user experience.

[0009] In addition, the image collector in this application uses a wide-angle camera, and the coverage area includes the 1.5-meter area in front of the floor display terminal. The edge processing module performs perspective correction and screen reflection suppression on the collected second image. The perspective correction processes the second image through the following formula to eliminate the distortion caused by the tilted shooting of the portable terminal screen: ;

[0010] Among them, x and y are the original image coordinates, and x' and y' are the corrected coordinates. , are the principal point coordinates, is the focal length, and k is the distortion coefficient.

[0011] Screen reflection suppression uses a polarization filtering algorithm to extract the effective marking area, ensuring accurate extraction of marking information.

[0012] The second color labeling of the portable terminal must meet specific conditions: its hue value H=120°±10°, saturation S>80%, transparency =0.6-0.8; the area of ​​the marked area occupies 50%-150% of the digital display area of ​​the target floor. If the same digital area is detected to be clicked multiple times, the transparency of the marked color increases according to the formula: Incrementally, where n is the number of clicks and .

[0013] This mechanism not only enhances the visualization effect of annotation, but also avoids the possibility of misoperation.

[0014] The edge processing module is also equipped with a conflict arbitration unit. When multiple second images are collected at the same time, the instruction whose center point of the marked area is closest to the digital geometric center of the floor is given priority; if there are overlapping annotations, the one with higher color saturation is the valid instruction. The conflict arbitration calculates the priority through the following formula: ;

[0015] in, Score the instruction priority; It is the Euclidean distance between the center point of the marked area and the digital geometric center of the floor; Mark the saturation of the color; is the weight coefficient.

[0016] The algorithm ensures the fairness and accuracy of interactions in multi-user scenarios.

[0017] On the second aspect, the present application provides a human-computer interaction method based on the above system, including an elevator call stage and a floor selection stage. In the elevator call stage, the passenger uses a portable terminal to shoot the floor display terminal to generate a first image containing the current floor mark. After the image collector captures it, the edge processing module parses the current floor and dispatches the elevator to arrive; in the floor selection stage, the passenger clicks on the area corresponding to the target floor number on the touch screen to generate a second image of a second color marked area. The edge processing module parses the target floor and executes the transportation. All operations only change the image displayed locally on the portable terminal and do not send any electrical signals to the outside, thereby ensuring the security and privacy of the data.

[0018] Furthermore, the edge processing module analyzes the target floor and performs transportation, including: performing regional HSV color space conversion on the second image, extracting the current floor number that meets the first color threshold, and the target floor number that meets the second color threshold, and comparing the extracted number with the accessible floor database of the floor display terminal through OCR recognition to determine the current floor and the target floor. In addition, the system is also equipped with an operation feedback mechanism. When the target floor is successfully analyzed, the corresponding floor number of the floor display terminal is controlled to flash 3 times; if the analysis fails, the LED light strip at the edge of the floor display terminal displays a red breathing light effect. All feedback information is transmitted only through the visual channel, without the participation of the portable terminal.

[0019] Finally, this application sets up floor instruction sharing logic. When the target floor is successfully parsed, the display terminal lights up the target floor number to remind passengers that the floor has been selected; it avoids passengers with the same target floor from repeatedly selecting the floor, thereby improving the interaction efficiency and avoiding the data processing burden caused by repeated floor selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 : Schematic diagram of the structure of the elevator contactless human-computer interaction system.

[0021] Figure 2 : Schematic diagram of the working principle of the edge processing module. DETAILED DESCRIPTION

[0022] The technical scheme of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The present invention provides a specific implementation of a contactless human-machine interaction system for an elevator. Figure 1 To Attachment Figure 2The system composition, operating principle and operation process are described in detail. In this embodiment, the system includes a floor display terminal, a portable terminal, an image collector, an edge processing module and an elevator controller. The specific function implementation of each component and its interaction process will be described one by one below.

[0024] The floor display terminal is fixedly installed on the wall of the elevator hall to provide passengers with intuitive floor information and car status. Figure 1 As shown in the figure, the terminal adopts a 3×4 matrix layout design, displays all accessible floor numbers, and marks the current floor number with a red border. The color parameters of the red border are H=0°±5°, S>90%, and V>90%, ensuring that it can be clearly seen under different lighting conditions. In addition, the dynamic arrow icon is used to indicate the position and direction of the car, and the color of the arrow is bound to the direction of travel: the arrow is blue when going up and yellow when going down. This visual design distinguishes information levels by color and improves the user experience. For example, in an office building scene, when the elevator is on the 1st floor and is ready to go up, the floor display terminal will display the 1st floor number with a red border, and the dynamic arrow icon will be displayed in blue and point to the area above the 1st floor.

[0025] The portable terminal is the core tool for users to interact with the system. It has camera and touch screen functions, such as smart phones, tablets and laptops. The user first uses the camera of the portable terminal to shoot the floor display terminal to generate the first image. The first image is then displayed on the touch screen of the portable terminal. The user clicks the target floor number area to generate the second image of the second color marked area. The hue value of the marked area must meet H=120°±10°, saturation S>80%, and transparency =0.6-0.8, and the marked area occupies 50%-150% of the target floor digital display area. If the user clicks on the same digital area multiple times, the transparency of the marked color is calculated according to the formula Incrementally, where n is the number of clicks and .

[0026] This mechanism not only enhances the visualization effect of annotation, but also effectively avoids the possibility of misoperation. For example, in actual application, the user stands about 1 meter in front of the elevator hall, uses a portable terminal to shoot the floor display terminal, and clicks on the target floor to generate a green annotation area. The generated second image contains the annotation information of the current floor and the target floor.

[0027] The image collector is set on the side of the floor display terminal, covering an area of ​​1.5 meters in front of the floor display terminal, and is used to collect the second image displayed on the portable terminal screen. In order to eliminate the distortion caused by the tilted shooting of the portable terminal screen, the edge processing module performs perspective correction on the collected second image. The perspective correction formula is as follows ;

[0028] Among them, x and y are the original image coordinates, and x' and y' are the corrected coordinates. , are the principal point coordinates, is the focal length, and k is the distortion coefficient.

[0029] For example, the image collector is a binocular camera, which obtains the spatial coordinates of the portable terminal and the coordinates of the principal point. , focal length f=800 pixels, distortion coefficient k=0.12, if the original coordinates are (400,300), then the corrected coordinates are (399.25,299.44), indicating that the original coordinates (400,300) are located to the upper right of the principal point (320,240), and the corrected coordinates (399.25,299.44) are slightly shrunk toward the principal point, which is in line with the expected effect of distortion correction.

[0030] In addition, screen reflection suppression extracts effective annotation areas through a polarization filtering algorithm to ensure accurate extraction of annotation information. For example, in a shopping mall scenario, when a user stands in a strong light environment and uses a portable terminal to generate a second image, the image collector can accurately capture the annotation information and transmit the data to the edge processing module.

[0031] The edge processing module has a built-in color segmentation unit and an OCR recognition unit, which are used to extract the first color area to identify the current floor and the second color area to identify the target floor based on the preset HSV threshold, and perform localized digital recognition on the segmented area. Specifically, the parsing process includes performing regional HSV color space conversion on the second image, extracting the current floor number that meets the first color threshold, and the target floor number that meets the second color threshold. Subsequently, the numbers extracted by OCR recognition are compared with the reachable floor database of the floor display terminal to determine the current floor and the target floor. For example, in a hospital scenario, when the edge processing module receives the second image transmitted by the image collector, it first extracts the annotation information of the current floor "3" and the target floor "8" through HSV color space conversion, and then confirms the floor number through OCR recognition and compares it with the database, and finally outputs data that the current floor is the 3rd floor and the target floor is the 8th floor.

[0032] The conflict arbitration unit is an important part of the edge processing module and is used to resolve command conflicts in multi-user scenarios. When multiple second images are acquired at the same time, the command whose center point of the annotation area is closest to the digital geometric center of the floor is given priority; if there are overlapping annotations, the one with higher annotation color saturation is the valid command. The conflict arbitration calculates the priority by the following formula: ;

[0033] in, Score the instruction priority; It is the Euclidean distance between the center point of the marked area and the digital geometric center of the floor; The saturation of the marked color (range 0-100%); is the weight coefficient (needs to be adjusted according to the actual scenario, for example .

[0034] For example, during peak hours in an office building, if user A's annotation is closer to the center but has lower saturation, and user B's annotation is slightly farther away but has higher saturation, the system will combine the scores of the two to determine the priority. When two users use a portable terminal to generate a second image at the same time, the edge processing module calculates the priority through a conflict arbitration formula and selects the instruction whose center point of the annotation area is closer to the digital geometric center of the floor and has higher color saturation as the valid instruction, thereby ensuring the fairness and accuracy of interaction in multi-user scenarios.

[0035] The elevator controller receives the current floor and target floor data output by the edge processing module and controls the operation of the elevator. During the entire process, no data communication is required between the portable terminal and the elevator control system, thus ensuring the security and privacy of the data. For example, in a residential community scenario, when the edge processing module parses that the current floor is the 1st floor and the target floor is the 10th floor, the elevator controller dispatches the car from the current position to the 1st floor to pick up passengers, and after the passengers enter the car, it goes up to the 10th floor to complete the transportation task.

[0036] The system also includes an operation feedback mechanism. When the target floor is successfully parsed, the corresponding floor number on the floor display terminal is controlled to flash three times; if the parsing fails, the LED light strip at the edge of the floor display terminal will display a red breathing light effect. All feedback information is transmitted only through the visual channel without the participation of the portable terminal. For example, in a hotel scenario, when the user generates the second image, the edge processing module successfully parses the target floor as the 7th floor, and the 7th floor number on the floor display terminal will flash three times to prompt the user that the operation is successful; if the parsing fails, the LED light strip at the edge of the floor display terminal will display a red breathing light effect to prompt the user to re-operate.

[0037] The system also includes instruction sharing logic. When the target floor is successfully parsed, the display terminal lights up the target floor number to remind passengers that the floor has been selected. This prevents passengers with the same target floor from repeatedly selecting floors, improving interaction efficiency while also avoiding the data processing burden caused by repeated floor selection. For example, during peak hours in a certain office building, there are many passengers waiting for the elevator in the elevator hall. Passenger A and Passenger B both have the 10th floor as their target floor. When passenger A places the second image in front of the image collector and the edge processing module successfully parses the target floor as the 10th floor, the display terminal lights up the target floor number of the 10th floor, indicating that the 10th floor has been selected. After passenger B observes that the 10th floor has been selected, there is no need to call the elevator or select a floor, avoiding the interaction efficiency being reduced due to repeated operations, and also avoiding the data processing burden caused by redundant data generated by repeated floor selection.

[0038] In summary, the present invention realizes efficient, sanitary and direct contact-free interaction between passengers and elevators through the collaborative work of floor display terminals, portable terminals, image collectors, edge processing modules and elevator controllers. The system shows excellent stability and response speed in actual application scenarios, and avoids dependence on external communication environments through localized computing, significantly improving user experience and security.

[0039] The present invention also provides a human-computer interaction method based on the above-mentioned elevator contactless human-computer interaction system, including an elevator call stage and a floor selection stage. In the elevator call stage, the passenger uses a portable terminal to shoot the floor display terminal to generate a first image containing the current floor mark. After the image collector captures it, the edge processing module parses the current floor and dispatches the car to arrive; in the floor selection stage, the passenger clicks on the area corresponding to the target floor number on the touch screen to generate a second image of a second color marked area. The edge processing module parses the target floor and performs transportation. All operations only change the image displayed locally on the portable terminal, and do not send any electrical signals to the outside, thereby ensuring the security and privacy of the data.

[0040] Furthermore, in the present application, after the image collector captures the image, the edge processing module parses the current floor and schedules the arrival of the elevator, including: performing regional HSV color space conversion on the first image, extracting the current floor number that meets the first color threshold, and extracting the current floor number through OCR recognition to determine the current floor.

[0041] Furthermore, in the present application, the edge processing module parses the target floor and executes the transportation, including: performing regional HSV color space conversion on the second image, extracting the current floor number that meets the first color threshold, and the target floor number that meets the second color threshold, and comparing the extracted number with the reachable floor database of the floor display terminal through OCR recognition to determine the current floor and the target floor.

[0042] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.

Claims

1. A contactless human-computer interaction system for an elevator, characterized in that: The system includes a floor display terminal, a portable terminal, an image collector, an edge processing module and an elevator controller, wherein: The floor display terminal is fixedly installed on the wall of the elevator hall, and is used to display all accessible floor numbers, the current floor number area marked with a first color, and the dynamically updated elevator car position; The portable terminal has a camera and a touch screen, the camera is used to shoot the floor display terminal to generate a first image, and the first image is displayed on the touch screen, and the user clicks on the target floor number area to mark the second color to generate a second image; The image collector is arranged on the side of the floor display terminal and is used to collect the second image displayed on the screen of the portable terminal; The edge processing module is electrically connected to the image collector, and has a built-in color segmentation unit and an OCR recognition unit, which are used to extract the first color area to identify the current floor and the second color area to identify the target floor based on a preset HSV threshold, and perform localized digital recognition on the segmented area; The elevator controller receives the current floor and target floor data output by the edge processing module and controls the operation of the elevator.

2. The elevator contactless human-computer interaction system according to claim 1, characterized in that: The floor display terminal is designed as a 3×4 matrix arrangement of accessible floor numbers. A red border is set around the current floor number. The elevator car position is displayed through a dynamic arrow icon, and the arrow color is bound to the running direction: the arrow is blue when going up and yellow when going down.

3. The elevator contactless human-computer interaction system according to claim 1, characterized in that: The image collector uses a wide-angle camera, and the coverage area includes an area of ​​1.5 meters in front of the floor display terminal. The edge processing module performs perspective correction and screen reflection suppression on the collected second image.

4. The elevator contactless human-computer interaction system according to claim 3, characterized in that: The perspective correction eliminates the distortion caused by tilted shooting of the portable terminal screen through the following formula: ； Among them, x and y are the original image coordinates, and x' and y' are the corrected coordinates. , are the principal point coordinates, is the focal length, and k is the distortion coefficient.

5. The elevator contactless human-computer interaction system according to claim 1, characterized in that: The second color labeling of the portable terminal must meet the following conditions: hue value H=120°±10°, saturation S>80%, transparency =0.6-0.8; the area of ​​the marked area occupies 50%-150% of the digital display area of ​​the target floor; if it is detected that the same digital area is clicked multiple times, the transparency of the marked color is calculated according to the formula Incrementally, where n is the number of clicks and .

6. The elevator contactless human-computer interaction system according to claim 1, characterized in that: The edge processing module is provided with a conflict arbitration unit. When multiple second images are collected at the same time, the instruction whose center point of the marked area is closest to the digital geometric center of the floor is preferentially selected; if there are overlapping marks, the one with higher color saturation is the valid instruction; the conflict arbitration calculates the priority by the following formula: ； in, Score the instruction priority; It is the Euclidean distance between the center point of the marked area and the digital geometric center of the floor; Mark the saturation of the color; is the weight coefficient.

7. The elevator contactless human-computer interaction system according to claim 1, characterized in that: The system also includes an operation feedback mechanism. When the target floor is successfully parsed, the corresponding floor number of the floor display terminal is controlled to flash three times; if the parsing fails, a red breathing light effect is displayed on the LED light strip at the edge of the floor display terminal.

8. The elevator contactless human-machine interaction system according to claim 7, characterized in that: The system is provided with floor instruction sharing logic. When the target floor is parsed successfully, the display terminal lights up the target floor number to remind the passenger that the floor has been selected.

9. A human-machine interaction method based on the elevator contactless human-machine interaction system according to any one of claims 1 to 8, characterized in that: It includes the elevator calling stage and floor selection stage: During the elevator call phase, the passenger uses a portable terminal to capture the first image of the floor display terminal with the current floor label. After the image collector captures it, the edge processing module analyzes the current floor and dispatches the elevator to arrive. During the floor selection stage, the passenger clicks on the area corresponding to the target floor number on the touch screen to generate a second image of the area marked with a second color. The edge processing module analyzes the target floor and executes the transportation.

10. The human-computer interaction method according to claim 9, characterized in that: The edge processing module analyzes the target floor and executes the transportation, including: Perform regional HSV color space conversion on the second image, extract the current floor number that meets the first color threshold and the target floor number that meets the second color threshold, and compare the extracted numbers with the reachable floor database of the floor display terminal through OCR recognition to determine the current floor and the target floor.

Images