Multi-input call panel for elevator system

Abstract

A multi-input call panel for controlling operation of an elevator system is disclosed. The multi-input call panel includes a touchable interface associated with a plurality of touchable inputs arranged at different locations on the multi-input call panel, a non-touch interface including a processor configured to receive readings of a sensor that detects motion in proximity to the touchable interface, and the non-touch interface executes a probabilistic classifier trained to output a corresponding probability that a received reading is of an intent to touch one or more of the plurality of touchable inputs, and a controller configured to control operation of the elevator system in accordance with a control command associated with a touchable input of the plurality of touchable inputs when the touchable input is touched on the touchable interface, when the classifier outputs a probability that the intent to touch the touchable input is above a threshold, or both.

Description

Technical Field

[0001] This disclosure generally relates to vertical transport technology, and more specifically to a multi-input call panel for controlling the operation of an elevator system. Background Technology

[0002] Control panels are used to operate various types of equipment, such as elevators, factory automation machines, and information kiosks. Control panels can include physical buttons arranged on the panel itself or virtual buttons displayed on a touchscreen. For hygiene and disease control reasons, non-touch operation of such button panels may be desirable. For this purpose, multiple sensors such as thermal sensors (e.g., infrared (IR) sensors), motion sensors, and light sensors can be used to operate elevators non-touch. The sensors can detect non-touch input from users operating the elevator. However, implementing non-touch control for elevators may require a complete replacement of existing button-based control panels, which can be expensive and inefficient. In some cases, control panels can be customized through application programming. However, control panel customization can be time-consuming and labor-intensive. For example, technicians or technical experts may be needed to implement the application programming for customization, which can also be expensive for rapid deployment.

[0003] Therefore, there is a need for a technical solution to control the operation of elevator systems or other equipment in an efficient and feasible manner. Summary of the Invention

[0004] The purpose of this disclosure is to provide a contactless interface for retrofitting existing contact-based control panels, such as button panels in elevator systems. For this purpose, the contactless interface (hereinafter interchangeably referred to as a non-touch interface) can use any sensor to detect non-contact or non-touch input from a user. The sensor can detect non-touch input when user input (such as a user's finger) crosses a horizontal plane in space in front of the button panel and is approximately parallel to the button panel at a specified distance. After detection, the sensor begins recording corresponding readings. While recording readings, a correspondence is established between the sensor readings and one or more buttons on the button panel intended to be pressed. This correspondence can be established based on a minimal set of demonstrations performed during the installation of the contactless interface. This set of demonstrations may include data points manually entered by the installer of the contactless interface. For example, this set of demonstrations may include instructions for the regular operation of the button panel. Regular operation may correspond to pressing each button at the appropriate time, while detecting these button presses and recording them in a database. After the correspondence has been established, it can be stored in a computing device for periodic use in a non-touch operation mode.

[0005] During touchless operation mode, the computing device continuously monitors sensor readings and calculates the probability that the user intends to press each button on the button panel. When one of the probabilities exceeds a threshold, it indicates that the user has registered a button press, without the user physically touching the button.

[0006] In some cases, a user's intention to press one or more buttons on a button panel may be ambiguous. For example, a user's finger may be between two buttons. Therefore, some implementations aim to identify the user's intention to press one or more buttons. To this end, evidence of the user's intention can be collected and accumulated in a computing device based on sensor readings. Evidence of intention can be accumulated until the probability of the intention exceeds a threshold, at which point the button press is recorded.

[0007] Therefore, some embodiments aim to provide a multi-input call panel for controlling the operation of an elevator system. In various embodiments, the multi-input call panel is configured to receive inputs, such as call commands from two types of input interfaces. These two types of input interfaces include touch-sensitive interfaces (i.e., button panels) and non-touch-sensitive interfaces (i.e., contactless interfaces). Touch-sensitive interfaces are associated with multiple touch-sensitive inputs (e.g., buttons) arranged at different locations on the touch-sensitive interface. Each of the multiple touch-sensitive inputs corresponds to a predetermined destination, such as a floor in a building. For example, a button marked "5" corresponds to the fifth floor of a building. Touch-sensitive inputs trigger commands to control the elevator's movement to the destination floor when touched or pressed by the elevator user and / or operator. Some non-limiting embodiments of touch-sensitive interfaces include button panels, where each button acts as a touch-sensitive input responsive to touch inputs (such as those pressed by an operator's finger). Some other embodiments of touch-sensitive interfaces may include keyboard-based control panels, keypad-based control panels, etc. Non-touch-sensitive interfaces may include touch-sensitive screens, where different portions of the screen may correspond to different destination floors. Furthermore, the touchless interface can be operatively connected to sensors that detect the space near the multi-input call panel. The touchless interface can be configured to translate sensor readings into commands for controlling elevator operation.

[0008] Some implementations are based on the understanding that such multi-input call panels provide synergy when using one and / or a combination of touchable and non-touchable input interfaces. Furthermore, multi-input call panels can enable the retrofitting of existing button panels used for operating elevators. In addition, the synergy provides for the joint use of touchable and non-touchable interfaces. This joint use allows for the configuration, training, and utilization of the non-touchable interface to use the guidance provided by the touchable interface. For example, sensor readings near the call panel can be interpreted relative to the location of various touchable inputs such as buttons. In this way, the user's intention to press a button can be translated into a control command associated with the corresponding button before the user touches it. For this purpose, the use of touchable and non-touchable interfaces is synchronized, and the non-touchable interface becomes intuitive for elevator users. In this way, the multi-input call panel can be operated in a touchable manner that the user is accustomed to, or in a non-touch manner when the user desires it. For example, during a pandemic, users may prefer to operate the elevator using a non-touchable interface for hygiene and safety reasons.

[0009] Some implementations are based on the further understanding that, in order to achieve synergy in the operation of a multi-input call panel, the functionality of the non-touch interface can be trained in a specific way that mimics actual touch on a touchable interface. To this end, another objective of some implementations is to provide a trained probability classifier that maps sensor readings to the user's intention to touch a specific touchable input.

[0010] In some cases, touch inputs may be densely arranged on a multi-input call panel. For example, buttons on a touch interface may be closely spaced from each other. This dense arrangement of touch inputs may affect a user's ability to select multiple paths or gestures for pressing a particular touch input on the multi-input call panel. Therefore, some implementations are based on the understanding that the operator's actual intention to press a particular button is unclear before the operator actually touches it during classifier training.

[0011] In some implementations, training can be performed in response to touches on buttons on a touchable interface. For example, sensor readings at the moment of and / or prior to a touch can be associated with the touched button when a touch is detected. In this way, when different buttons are touched, sensor readings can be labeled with the identifiers of different buttons using ground reality information used during classifier training. Classifier training can allow sensor readings to be explicitly labeled with the intent indicated by the actual press. Classifier training can also allow association with buttons, not only the location of the reading and the number of readings near the multi-input call panel. In this way, accidental sensor readings can be prevented. For example, when the user's shoulder is within the sensor's field of view, the classifier detects the user's intent before the user physically touches a button on the multi-input call panel.

[0012] In some implementations, a probability classifier can be trained to detect a user's intention to touch a touchable input, i.e., a button when the probability of such a touch is above a threshold. In some exemplary implementations, noise from sensor readings can be taken into account when training the probability classifier. Combined, the probability classifier and the threshold for detecting intent can be trained end-to-end to achieve a balance between declaring intent too quickly or too late.

[0013] Some implementations are based on the understanding that different control panels may vary in type, structure, and installation. For this purpose, a probabilistic classifier can be trained in the field. For example, when installing a multi-input call panel to control an elevator system, a probabilistic classifier can be trained. In field training, the probabilistic classifier is trained in response to touchable inputs. Such field training can be performed by the installer during the installation and / or maintenance of the multi-input call panel without the need for additional measurements or tools.

[0014] Therefore, in some implementations, the multi-input call panel can be configured to have two operating modes. These two operating modes can include a training mode and a control mode. During the training mode, touch inputs on the buttons of the touchable interface, as well as sensor readings prior to the touch input, are collected. A probability classifier is trained based on the collected touch inputs and sensor readings. In various implementations, such touch inputs do not invoke operations that alter the elevator system. During the control mode, the touch inputs and outputs of the probability classifier are used to control the operation of the elevator system. This training provides the flexibility to modify non-touch interfaces using different types of touchable interfaces.

[0015] Alternatively, some implementations are based on the understanding that a probability classifier can be pre-trained for a specific type of touchable interface and during a calibration training mode specific to the installation. During the training mode of a multi-input call panel, the installer can touch the same button to create a transformation function. The transformation function transforms the readings received when the multi-input call panel is installed into the corresponding readings used during training. During control mode, the sensor readings are transformed by the transformation function before being submitted to the probability classifier.

[0016] In different implementations, the probability classifier can be trained in different ways. For example, in one implementation, the sensor can be arranged to sense a plane parallel to the multi-input call panel at a fixed distance (e.g., 20 mm). The sensor readings can then record the location of the user's input, such as the position of the user's fingertip on the plane. This location can correspond to x, y coordinates in the plane. These x, y coordinates can be fed as input to the probability classifier. The x, y coordinates can correspond to the category label of the button the user ultimately presses during training. This training eliminates the need for a technically skilled installer.

[0017] Alternatively, the probability classifier can transform readings collected at the time of (or before) a touch into the user's intention to touch the button. In this way, the x and y coordinates in a direction perpendicular to the multi-input call panel, as well as different types of readings including time-series readings leading to the touch, are considered. In this way, the probability classifier becomes robust to different paths of different fingers from different users touching different buttons.

[0018] In some implementations, sensor readings can be represented in a coordinate system that includes x and y spatial coordinates. In some implementations, sensor readings can correspond to the zigzag path taken by a user's fingertip pressing a button on a button panel. This zigzag path can correspond to a trajectory that can also be represented in a coordinate system. In the readings, the point corresponding to the user's input can be the closest to the plane of the button panel. The point of the user's fingertip closest to the plane can correspond to the smallest z-coordinate (z = 0 is the plane where the button is touched).

[0019] For example, if point p = (x, y, z) are the spatial coordinates of point p, then the corresponding x, y, and z coordinates can be fed as input to a probability classifier. The probability classifier can generate a clear probability distribution for points where the user's intent is clear and unambiguous. However, the probability classifier may also generate a more ambiguous distribution, which can occur when the intent to press a button is ambiguous. For example, ambiguity may arise when a user approaches a multi-input call panel from approximately the same position for multiple buttons before accurately locating the intended button. To address this, the probability classifier quantifies the ambiguity and only registers a button press when it is determined that the user intended to press the corresponding button. However, the probability classifier may require a large amount of training data to return the probability distribution. In some cases, the probability classifier may be sensitive to data such as the user's height. For example, the trajectory of different users' fingertips may depend heavily on their height. Shorter users may begin moving with a lower trajectory, and taller users may begin moving with a higher trajectory. For this reason, some implementations use multiple parallel planes to perform the probability classifier.

[0020] Alternatively, some implementations may use relationships between different readings at different planes to extract sensor readings for training and control of a multi-input call panel. For example, the sequence of XY positions (or the centroid of the finger) of a finger in the Z plane (in the case of multiple planes of the sensor) or at times T1, T2, T3…Tn can be detected and extrapolated to calculate a predicted “touch hit point” at the Z=0 plane of the button. This predicted touch hit point (PTIP) can be used to train a probability classifier, detect user button requests, or both.

[0021] In some other implementations, points on the trajectory corresponding to the user's touch input can be fed as input to the probability classifier. For this purpose, the x and y coordinates of the trajectory can be replaced with the x and y coordinates of the intended touch on the button, while retaining the actual z-values. This allows the probability classifier to distinguish between imprecise guesses for large z-values ​​and precise guesses for small z-coordinate values. Therefore, the probability classifier can return a clear probability distribution for some values ​​of z and a fuzzy probability distribution for larger values ​​of z.

[0022] Alternatively or concurrently, some implementations disclose adaptive correction of the position coordinates corresponding to a touch input intended to press a button on a touchable interface.

[0023] Therefore, one embodiment discloses a multi-input call panel for controlling the operation of an elevator system. The multi-input call panel includes a touchable interface associated with a plurality of touchable inputs arranged at different locations on the multi-input call panel. The multi-input call panel also includes a non-touchable interface, which includes a processor operatively connected to receive readings from sensors arranged to sense movement approaching the touchable interface. The non-touchable interface is configured to execute a probability classifier in response to receiving the readings, the probability classifier being trained to output a probability corresponding to the received reading and an intention to touch one or more of the plurality of touchable inputs. The multi-input call panel further includes a controller configured to control the operation of the elevator system according to a control command associated with one of the plurality of touchable inputs when the probability classifier outputs a probability of an intention to touch the touchable input when the touchable input is touched on the touchable interface, or both.

[0024] Another embodiment discloses a method for controlling the operation of an elevator system using a multi-input call panel. The method includes receiving readings from sensors of the non-touch interface of the multi-input call panel, the sensors being arranged to sense movement near a touchable interface of the multi-input call panel. The method includes, in response to receiving the readings, executing a probability classifier trained to output a probability corresponding to the received readings and an intention to touch one or more of a plurality of touchable inputs arranged at different locations on the touchable interface of the multi-input call panel. The method further includes controlling the operation of the elevator system according to a control command associated with one of the touchable inputs when the probability classifier outputs a probability of intention to touch the touchable input higher than a threshold, or both, when the touchable input is touched on the touchable interface.

[0025] Other features and advantages will become more apparent when described in detail below in conjunction with the accompanying drawings.

[0026] The present disclosure is further described in the following detailed description with reference to the accompanying drawings, which illustrate non-limiting embodiments of exemplary implementations, wherein like reference numerals denote similar parts in several views of the drawings. The drawings are not necessarily drawn to scale, but generally focus on illustrating the principles of the embodiments currently disclosed. Attached Figure Description

[0027] [ Figure 1 ]

[0028] Figure 1 An environmental representation for controlling the operation of an elevator system according to some embodiments of the present disclosure is shown.

[0029] [ Figure 2A ]

[0030] Figure 2A A block diagram of a system for controlling the operation of an elevator system using a multi-input call panel, according to an exemplary embodiment of the present disclosure, is shown.

[0031] [ Figure 2B ]

[0032] Figure 2B A schematic diagram of a switcher for a multi-input call panel according to an exemplary embodiment of the present disclosure is shown.

[0033] [ Figure 3 ]

[0034] Figure 3 A flowchart is shown illustrating a process that corresponds to a training mode of a multi-input call panel according to an exemplary embodiment of the present disclosure.

[0035] [ Figure 4 ]

[0036] Figure 4 A flowchart is shown illustrating a process that corresponds to the control mode of a multi-input call panel according to an exemplary embodiment of the present disclosure.

[0037] [ Figure 5A ]

[0038] Figure 5A A training scenario depicting a multi-input call panel according to an exemplary embodiment of the present disclosure is shown.

[0039] [ Figure 5B ]

[0040] Figure 5B A training scenario depicting a multi-input call panel according to another exemplary embodiment of this disclosure is shown.

[0041] [ Figure 6 ]

[0042] Figure 6 A tabular representation of the coordinate system corresponding to the sensor and the coordinate system of the touchable interface of the multi-input call panel according to an exemplary embodiment of the present disclosure is shown.

[0043] [ Figure 7 ]

[0044] Figure 7A tabular representation of a mapping of non-touch inputs on a touchable interface of a multi-input call panel, intended to be touched, is shown according to an exemplary embodiment of the present disclosure.

[0045] [ Figure 8 ]

[0046] Figure 8 A flowchart illustrating a method for controlling the operation of an elevator system using a multi-input call panel according to an exemplary embodiment of the present disclosure is shown.

[0047] [ Figure 9 ]

[0048] Figure 9 A block diagram of a device for controlling the operation of an elevator system according to an exemplary embodiment of the present disclosure is shown.

[0049] [ Figure 10 ]

[0050] Figure 10 A scenario is illustrated where a device using a multi-input call panel is used to control the operation of an elevator system according to an exemplary embodiment of this disclosure.

[0051] [ Figure 11 ]

[0052] Figure 11 This illustrates a scenario where the operation of a conveyor system is controlled by means of a device according to another exemplary embodiment of the present disclosure. Detailed Implementation

[0053] While the accompanying drawings illustrate the currently disclosed embodiments, other embodiments are contemplated as noted in the discussion. This disclosure presents illustrative embodiments by way of representation, not limitation. Those skilled in the art can devise many other variations and embodiments falling within the scope and spirit of the principles of the currently disclosed embodiments.

[0054] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without these specific details. In other instances, apparatus and methods are shown in block diagram form only to avoid obscuring this disclosure.

[0055] As used in this specification and claims, the terms “for example,” “like,” and “such as,” as well as the verbs “comprising,” “having,” and other verb forms thereof, when used in conjunction with a list of one or more components or other items, are each interpreted as open-ended, meaning that the list is not considered to exclude other additional components or items. The term “based on” means at least partially based on. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. Any headings used in this description are for convenience only and have no legal or limiting effect.

[0056] Figure 1 An environment representation 100 for controlling the operation of an elevator system 102 according to some embodiments of the present disclosure is shown. The environment representation 100 includes a user 106 who is at a service floor to access the elevator system 102 (hereinafter interchangeably referred to as elevator 102) and move to another service floor of the building. Figure 1 (Not shown). Elevator 102 can be operated using a contact-based panel 104A implemented inside elevator 102. For example, the contact-based input panel 104A may include buttons indicating the corresponding floor of elevator 102 and other operation buttons for elevator 102, such as an open button for opening the doors of elevator 102, a close button for closing the doors of elevator 102, an emergency call button, a lobby button, etc. The contact-based input panel 104A may also include a display screen to display outputs indicating the service floor of elevator 102. For example, when user 106 presses the corresponding button indicating the first floor, the specific service floor, such as the first floor, may be displayed as "1" on the display screen. When user 106 can press the corresponding operation button (such as the lobby button or emergency call button on the contact-based input panel 104A), the display screen also displays the output indicating the operation. A similar contact-based panel 104B may be installed outside elevator 102 to receive input from user 106 to operate elevator 102, such as... Figure 1 As shown in the image.

[0057] In some cases, elevator 102 can be operated via contactless input. For this purpose, elevator 102 can be equipped with a multi-input call panel that includes both contact-based and contactless functions for operating elevator 102. For example, contactless functions can be implemented via the multi-input call panel onto an existing contact-based input panel 104A.

[0058] This implementation of contactless functionality in the multi-input call panel prevents replacement of the contact-based input panel 104A. (Reference) Figure 2A This multi-input call panel is described in further detail.

[0059] Figure 2AA block diagram of a system 200 for controlling the operation of an elevator system 102 according to an exemplary embodiment of the present disclosure is shown. System 200 includes a multi-input call panel 202, which includes a touchable interface 204, a non-touchable interface 206, and a controller 208. The touchable interface 204 is associated with a plurality of touchable inputs arranged at different locations on the touchable interface 204. The touchable interface 204 corresponds to a contact-based input panel 104A having touchable inputs (such as buttons, keypads, etc.). The non-touchable interface 206 includes a processor 210 operatively connected to a sensor 212; and a memory 214 storing a probability classifier 216. The controller 208 is configured to control the operation of the elevator system 102 according to control commands associated with the touchable inputs among the plurality of touchable inputs.

[0060] In some exemplary embodiments, the touch interface 204 may include a plurality of mechanical switches, electrically controlled relays, or switching transistors that can be wired in parallel with each mechanical switch. Furthermore, the state of each mechanical switch (i.e., open or closed state) can be detected and recorded in a database by means of suitable electronic circuitry added to the terminals of the mechanical switch, or input via an auxiliary path, such as a selector switch, scroll wheel, keypad, or a debugging or programming interface running on an external device (such as a computer, laptop computer, etc.). When the touch interface 204 is implemented using touchscreen software, the software can record the coordinates of the corresponding point touched by the user 106 on the screen and can simulate touch input on the screen. For this purpose, the touch interface 204 has the aforementioned wired arrangement, and the sensor 212 can be mounted in a suitable location close to the touch interface 204. For example, the sensor 212 can be attached to the same wall on which the touch interface 204 is mounted.

[0061] In some implementations, sensor 212 is arranged to sense movement in the vicinity of touchable interface 204. In one exemplary implementation, sensor 212 may detect the location of an input, such as a fingertip or gesture of user 106 in front of touchable interface 204. Sensor 212 may include: a thermal sensor (e.g., an infrared (IR) sensor); a motion sensor; a red, green, and blue depth (RGBD) sensor camera; a light detection and ranging (LIDAR) sensor, etc. Sensor 212 may output a depth field of the visual scene in front of touchable interface 204. The depth field may be represented in a reference coordinate system attached to sensor 212. In some cases, sensor 212 may obtain depth information using triangulation techniques. Using triangulation techniques, multiple sensors of sensor 212 may be combined to obtain depth information as user 106's fingertip moves in front of sensor 212. For example, a LIDAR sensor may be paired with other sensors, such as an RGBD camera. The LIDAR may emit a laser beam that sweeps across the space in front of sensor 212, and the RGBD camera may detect and capture fingertips approaching sensor 212. When a fingertip falls onto the laser beam, its position or depth information can be recorded. This position or depth information of the user's fingertip can be used to detect movement near the touchable interface 204.

[0062] In some implementations, processor 210 is configured to receive readings from sensor 212. Processor 210 is further configured to execute probabilistic classifier 216 in response to receiving the readings. Some non-limiting embodiments of probabilistic classifier 216 may include Naive Bayes classifiers, k-nearest neighbor classifiers, Gaussian mixture model classifiers, support vector machine classifiers, Parzen kernel density estimation-based classifiers, and various types of neural network classifiers.

[0063] In some implementations, the probability classifier 216 can be trained based on a training program that can be stored in memory 214. In some exemplary implementations, the processor 210 can be configured to record physical button presses on the touch interface 204 based on readings from sensor 212 and to register button presses on behalf of user 106.

[0064] In some implementations, controller 208 is configured to control the operation of elevator system 102. Control is executed based on control commands associated with one of a plurality of touchable inputs. When touchable interface 204 receives user input (i.e., touchable input), probability classifier 216 outputs the probability, or both, that the intention to touch the touchable input is higher than a threshold. The threshold can act as a buffer between the readings of sensor 212 and the touchable input of touchable interface 204, because the smaller the threshold, the greater the distance between the sensor reading and the touchable input when probability classifier 216 detects an intention.

[0065] In some cases, the touch interface 204 may have different types and / or structures. For example, the arrangement of control buttons, floor buttons, displays, etc., may differ for different types of touch interfaces. In such cases, the installation of the multi-input call panel 202 may vary due to the difference in the type of touch interface. To address this, a probability classifier 216 can be trained on-site during the installation of the multi-input call panel 202. During on-site training, the probability classifier 216 can be trained in response to touches on the buttons of the touch interface 204. This on-site training of the probability classifier 216 can prevent additional steps, such as measurements and / or additional resources (such as instruments for measurement) associated with non-touch inputs to the non-touch interface 206. In this way, the overall installation and / or maintenance process of the multi-input call panel 202 can be improved in a cost-effective and feasible manner. Alternatively or additionally, the installation can be performed by the installer during the installation and / or maintenance of the multi-input call panel 202.

[0066] Therefore, the multi-input call panel 202 can be configured to operate in two different modes, referring to... Figure 2B This will be described in further detail.

[0067] Figure 2B A schematic diagram 218 is shown of a switcher 220 for a multi-input call panel 202 according to an exemplary embodiment of the present disclosure. The switcher 220 is configured to change the operating mode of the multi-input call panel 202. In some embodiments, the multi-input call panel 202 includes a switcher, such as switcher 220. Switcher 220 is configured to change the operating mode of the multi-input call panel 202.

[0068] The operating modes include training mode 222 and control mode 224. In some exemplary embodiments, during training mode 222, a touch input dataset and a sensor dataset are collected from sensor 212. The touch input dataset corresponds to multiple touch inputs with respect to the touchable interface 204, and the sensor dataset corresponds to sensor readings preceding the multiple touch inputs. The readings in the touch input dataset and sensor dataset are timestamped, thereby establishing a temporal correspondence between the multiple touch inputs and the sensor readings immediately preceding the multiple touch inputs. This allows sequences of sensor inputs to be labeled with corresponding button numbers registered via touch input.

[0069] For example, information from the sensor dataset corresponding to touch input on the first button of the touchable interface 204 may include markers such as "1". This information from the sensor dataset is associated with a marker on the first button indicating the first floor of the building.

[0070] During control mode 224, multiple touch inputs and outputs of probability classifier 216 are used to control the operation of elevator system 102.

[0071] refer to Figure 3 The steps for training the probability classifier 216 in training mode 222 are further described.

[0072] Figure 3 A process 300 is shown that corresponds to the execution of training mode 222 of a multi-input call panel 202 according to an exemplary embodiment of the present disclosure. Process 300 begins at step 302. The steps of process 300 can be executed by the processor 210 of the non-touch interface 206 to train the probability classifier 216 in memory 214. In some cases, the probability classifier 216 may be in training mode 222 during the installation and / or maintenance of the multi-input call panel 202. In this case, the installer can perform on-site training of the probability classifier 216. In some other cases, the probability classifier 216 can be pre-trained offline. In this case, training of the probability classifier 216 can begin in response to receiving touch input on a button of the touchable interface 204.

[0073] In other cases, data collection for training the probability classifier 216 can be conducted during normal touch-based operation of the multi-input call panel 202, while this data collection is performed by a regular user (such as user 106). The collected data can be categorized into training and test datasets. The probability classifier 216 can be trained based on the training dataset. The test dataset can be used to test the ability of the probability classifier 216 to correctly predict a button touch before the button touch occurs. Once the accuracy of the predictions on the test dataset exceeds a threshold (e.g., 99.99%), the probability classifier 216 can declare itself ready for non-touch operation in control mode 224.

[0074] In both on-site and offline training scenarios, the installer can input a minimal demo set, which may include multiple touch inputs made in a conventional manner at appropriate times on each button of the touchable interface 204. For example, the installer may provide multiple touch inputs on the same button, such as touching the button multiple times in different ways. The touch inputs on each button can be detected by sensor 212 and recorded as readings.

[0075] At step 304, a touch input dataset from the touchable interface 204 and a sensor dataset from the sensor 212 are received. The touch input dataset may include touch inputs from one or more buttons on the touchable interface 204. The sensor dataset may include readings from the sensor 212 prior to the touch input. The collected touch input dataset and sensor dataset may be stored in memory 214.

[0076] In step 306, a probability classifier 216 is trained based on the touch input dataset and the sensor dataset. In step 308, process 300 ends.

[0077] The probability classifier 216 is trained to classify non-touch inputs (intended to press buttons on the non-touch interface 206) based on actual touch inputs from the touchable interface 204. The actual touch inputs from the touchable interface 204 guide the probability classifier 216, making the non-touch interface 206 intuitive for users (such as user 106 in elevator 102). This guidance of touch inputs from the touchable interface 204 eliminates the need for installation technicians, thus saving deployment time in a cost-effective and feasible manner. In this way, the multi-input call panel 202 provides combined use of the touchable interface 204 and the non-touch interface 206.

[0078] After training, a probability classifier 216 is deployed for routine operations of elevator 102, referencing... Figure 4 This will be described in further detail.

[0079] Figure 4 A flowchart illustrating a process 400 corresponding to a control mode 224 of a multi-input call panel 202 according to an exemplary embodiment of the present disclosure is shown. At step 402, process 400 begins. The steps of flowchart 400 are executed by the processor 210 of the non-touch interface 206.

[0080] At step 404, sensor 212 continuously monitors non-touch inputs near multi-input call panel 202.

[0081] In some exemplary embodiments, sensor 212 measures readings of coordinate points, such as spatial coordinates along the x, y, and z axes corresponding to non-touch input (e.g., user 106 approaching the fingertip of non-touch interface 206). Spatial coordinates can be derived from (x... j y j , z j ) represents, where j = 1, m.

[0082] At step 406, the readings of sensor 212, including spatial coordinates, are recorded in the coordinate system of the touchable interface 204. For example, the spatial coordinates of a gesture are in the coordinate system of a button panel. In this coordinate system, the plane z = 0 corresponds to the plane of the touchable interface 204.

[0083] At step 408, select the point with the minimum z-coordinate from the spatial coordinates (i.e., z... k =min j (z j ), k = argmin j (zj ),j = 1, m).

[0084] At step 410, the point with the minimum value z k is compared with a predetermined threshold (dz) (such as 10 mm). At step 412, the non-touch input is terminated. At step 414, if z k < dz, the x, y coordinates (x k , y k ) of the point are given as the input to the probability classifier 216.

[0085] At step 416, the probability classifier 216 determines the probability Pr(b i |x k , y k ), i = 1, n indicating the likelihood that the user 106 is targeting each of the possible n buttons of the touchable interface 204.

[0086] At step 418, the probability P from the probabilities is compared with a threshold, i.e., p i > dp, where p i = the Pr(b i |x i , y k , y k ), and dp is the threshold.

[0087] In some alternative embodiments, probabilities can be used to accumulate evidence for detecting intent over multiple instances. In one exemplary embodiment, evidence can be accumulated based on Bayes' rule.

[0088] According to Bayes' rule, the probability of an event (such as the result corresponding to pressing a button of the touchable interface 204) is based on prior knowledge of the conditions corresponding to the event. To this end, before any readings are measured by the sensor 212, it is assumed that Bayes' rule can be used to represent the chance of contact with each button of the touchable interface 204 by a prior probability p i (0).

[0089] In some cases, for all buttons of the touchable interface 204 (for each i = 1, n, p i [[ID=�9]](0) = 1 / n,), these probabilities can be uniform. For example, the frequency of pressing all buttons is uniform. In some other cases, the probabilities can be non-uniform. For example, some buttons can be pressed more frequently than others, and statistical information about their relative frequencies can be available. For example, the button corresponding to the lobby of a building may be pressed more frequently than any other button in the touchable interface 204.

[0090] Furthermore, using Bayesian rules, based on the nearest point in the spatial coordinates of the user's hand (i.e., [x]), k (t), y k The projection of (t)]) is used to update the posterior probability Pr[b i |x k (t), y k (t)]). When at time t, the nearest point is detected when the distance between the user's hand tip on the plane and the touchable interface 204 is within the threshold dz. Therefore, the posterior probability Pr[b i |x k (t), y k [t] is updated to:

[0091] Pr[b i |x k (t),y k [(t)]=Pr(b) i )Pr[x k (t),y k (t)|b i ] / Pr[x k (t),y k (t)]

[0092] Here, Pr(b) i ) = p i (0) is button b of the touchable interface 204 as the target. i The prior probability, and Pr[x] k (t), y k (t)|b i ] is when b i When the expected button is pressed, the sensor 212 should register the nearest point [x] k (t), y k The posterior probability of [(t)].

[0093] In Bayesian rules, the posterior probability Pr[x] k (t), y k [(t)] is a constant that can be estimated as a normalization factor. In the operation of probability Pr(b)... i )Pr[x k (t), y k (t)|b i After that, the total posterior probability is 1. The posterior probability may not depend on the category label of the target button. The first spatial coordinates [x] of the fingertip indicating an intention to press the button are... k (t), y k After [(t)], the posterior probability p can be calculated. i (t)=Pr[b i |x k(t), y k (t)]. Similarly, spatial coordinates corresponding to fingertips intending to press different buttons can be accumulated for evidence at time t+dt, where dt represents the time interval. The accumulation of evidence can be equal to the sampling rate of sensor 212. Therefore, the probability of intending to press different buttons based on the accumulated evidence can be expressed as:

[0094] Pr[b i |x k (t+dt),y k (t+dt),x k (t),y k [(t)]=Pr(b) i |x k (t),y k (t))Pr[x k (t+dt),y k (t+dt)|b i ] / Pr[x k (t+dt),y k (t+dt)]

[0095] Alternatively, based on the last sensing event registered at time t+dt from the moment his / her hand first approached the button panel [x] k (t+dt), y k All evidence collected within (t+dt) indicates the user's intent to press button b. i The probability can be expressed as

[0096] P i (t+dt)=Pr[b i |x k (t+dt),y k (t+dt),x k (t),y k (t)],…]

[0097] Using a simple recursive update rule for the above probabilities, starting from the prior probabilities, we can obtain the following:

[0098]

[0099] Where a' is the chosen normalization constant, such that

[0100] The accumulation of evidence can continue until the posterior probability exceeds the threshold (dp), or until the moment when sensor 212 no longer indicates that any part of the user's hand is near the touchable interface 204 (possibly because the user has given up pressing the button), in which case the posterior probability can be reset back to the prior probability to await future sensing events caused by the same or other users.

[0101] In other implementations, the probability of pressing the target button can be estimated using a generated probability model. The generated probability model can be learned from touchable inputs to the touchable interface 204 and readings from the sensor 212 prior to the touch input. In some other implementations, during training mode 222, the probability can be estimated based on Naive Bayes, Gaussian mixture models, deep generative models, etc.

[0102] Regardless of which method is used to interpret the output of the probability classifier 216, the multi-input call panel 202 operates in a continuous loop to monitor for the user's hand approaching a portion of the touchable interface 204, and registers a button press when it is sufficiently determined that this is the user's intention.

[0103] At step 420, if the probability is less than a threshold, process 400 terminates. At step 422, the intent of the touch input corresponding to the touchable input is detected.

[0104] At step 424, the control command associated with the touch input is executed. At step 426, process 400 ends.

[0105] Figure 5A A scenario 500A depicting the training of a multi-input call panel 202 according to an exemplary embodiment of the present disclosure is shown. In the illustrative example scenario 500A, sensor 212 may monitor non-touch input, such as a gesture 502A of a user (e.g., user 106). Gesture 502A may be intended to press a button indicating the sixth floor of a building. In some cases, gesture 502A may be intended to press an operation button indicating an emergency call button (not shown). Sensor 212 may detect non-touch input when gesture 502A crosses a plane (e.g., a plane 504A parallel to the multi-input call panel 202). Plane 504A may be fixed at a predetermined distance, such as 20 mm. In some exemplary embodiments, the button that user 106 intends to press may be highlighted before user 106 actually touches the button. Figure 5A As shown, buttons can be highlighted using colored light.

[0106] In some cases, different users issuing gesture 502B may also intend to press the same button. Since the user 106 corresponding to gesture 502A may be shorter than the user corresponding to gesture 502B, the heights of gestures 502A and 502B can be different, such as... Figure 5A As shown in the diagram. This height difference can affect the probability classifier 216 when generating the corresponding output of the intent to press the button. For this purpose, some implementations may use multiple planes (such as plane 504B parallel to each other) to perform the probability classifier 216, as... Figure 5A As shown in the image.

[0107] First, during installation, a correspondence is established between the coordinate system of sensor 212 and the coordinate system of touch interface 204 (e.g., Figure 6 (As shown in the diagram). When establishing the correspondence between coordinate systems, the origin of the coordinate system of sensor 212 is a point on the touchable interface 204. The coordinate system of sensor 212 can be projected onto the coordinate plane of sensor 212, where the z-axis of the coordinate system is perpendicular to the coordinate plane of sensor 212.

[0108] In some exemplary embodiments, a correspondence can be established based on a general calibration method. This general calibration method can calibrate the correspondence based on the type of sensor 212. Thus, the establishment of the correspondence is independent of the layout of the touchable interface 204. For this purpose, a set of flags 506 can be attached to the touchable interface 204 to define the coordinate plane corresponding to the coordinate system of the touchable interface 204. In some exemplary embodiments, the correspondence can be defined by a rigid body transformation that maps the coordinate system of the sensor 212 to the coordinate system of the touchable interface 204. Once the correspondence is established, all readings of the sensor 212 are mapped to the coordinate system of the touchable interface 204. (See reference...) Figure 6 The mapping of the coordinate systems of sensor 212 and touch interface 204 is further described.

[0109] Specifically, when gestures 502A and 502B cross corresponding planes 504A and 504B, sensor 212 begins recording readings including the position (i.e., the position of gestures 502A and 502B). The position can be represented in spatial coordinates (such as x, y coordinates). For this purpose, the spatial coordinates of the corresponding gestures 502A and 502B, as well as the corresponding label of the button that user 106 intends to press during training, are input into probability classifier 216.

[0110] In some exemplary embodiments, the relationship between different readings at different planes can be used to extract sensor readings for training and controlling multi-input call panels, which in Figure 5B The explanation was provided in the text.

[0111] Figure 5BA scenario 500B is illustrated, describing the training of a multi-input call panel 202 according to another exemplary embodiment of the present disclosure. In some exemplary embodiments, extrapolation curves of the positions of gestures 502A and 502B terminating at corresponding buttons on the touchable interface 204 can be used to train a probability classifier. For example, the positions of gestures 502A and 502B spanning each of planes 504A and 504B can be extrapolated to produce extrapolation curves, such as extrapolation curve 508A corresponding to gesture 502A and extrapolation curve 508B corresponding to gesture 502B, as shown. Figure 5B As shown in the image.

[0112] In an exemplary scenario, the position sequence (x, y coordinates) corresponding to gestures 502A and 502B in the Z-plane or different time series T1, T2, T3...Tn is detected and extrapolated to obtain extrapolated curves 508A and 508B. For example... Figure 5B As shown, the sequences of x and y coordinates of gestures 502A and 502B at corresponding times T1 and T2 can be extrapolated. In one exemplary embodiment, processor 210 can extrapolate the sequences of x and y coordinates of gestures 502A and 502B based on one or a combination of linear regression, Catmull-Rom splines, cubic Hermite splines, or other similar methods. In some implementations, extrapolation can use cubic splines, where the last few (e.g., four points) are sufficient to fit a cubic curve, and extrapolation is performed for z = 0. In some implementations, the extrapolation of the x and y coordinate sequences can be used to calculate a predicted “touch hit point” at the Z = 0 plane of the button on the touchable interface 204. The predicted touch hit point (PTIP) can be used for training mode 222 of the probabilistic classifier 216.

[0113] Furthermore, coordinates corresponding to the extrapolated curves 508A and 508B can be provided to train the probability classifier 216. This training based on the extrapolated curves 508A and 508B on different planes 504A and 504B of different time series enables the probability classifier 216 to become robust to different ways in which different users touch the buttons.

[0114] As mentioned above, during the installation of the multi-input call panel 202, a correspondence is established between the coordinate system of the sensor 212 and the coordinate system of the touch interface 204. Figure 6 The coordinate system of sensor 212 and the coordinate system of touch interface 204 are shown in the figure.

[0115] Figure 6A tabular representation 600 of the coordinate systems corresponding to the touchable interface 204 and non-touchable interface 206 of a multi-input call panel 202 according to an exemplary embodiment of this disclosure is shown. The tabular representation 600 includes a coordinate system 602 corresponding to readings from sensor 212 and a coordinate system 604 corresponding to the touchable interface 204. In some exemplary embodiments, the readings from sensor 212 may record the location of input by user 106, such as the location of a point where user 106 places a finger on a plane (e.g., plane 504A or plane 504B) in front of sensor 212. This location may be represented as x, y, z coordinates in coordinate system 604. Furthermore, each coordinate in coordinate system 602 can be obtained through a rigid body transformation. A rigid body transformation maps coordinate system 602 to coordinate system 604 and defines the correspondence between coordinate systems 602 and 604. This correspondence between coordinate systems 602 and 604 can be established based on a minimal demo set performed by the installer during the installation of the multi-input call panel 202 without any technical or programming skills.

[0116] Furthermore, during training mode 222, the coordinates of the sensing point in coordinate system 604 can be input to the probability classifier 216. In control mode 224, the probability classifier 216 can use coordinate systems 602 and 604 to map the user 106's intention to touch a button on the touchable interface 204 to the corresponding category label of the intended button, which is shown in Figure 7 middle.

[0117] Figure 7 A tabular representation 700 is shown illustrating a mapping of non-touch inputs intended to touch a button on a touchable interface 204, according to an exemplary embodiment of this disclosure. The tabular representation 700 includes columns 702 and 704. Column 702 corresponds to the x, y coordinates of the location of the user 106's intention to touch the button, such as gesture 502A or gesture 502B. Column 704 corresponds to the category label of the corresponding button that the user 106 ultimately presses. For example, when gesture 502A is at a position (x1, y1) in plane 504A (or plane 504B), that position is mapped to button (b1).

[0118] Alternatively, a nearest neighbor classifier can be applied to the position coordinates [x(t), y(t)] of column 702. The position coordinates can be compared with previously learned coordinates (such as those in coordinate system 602). The position coordinates can be corrected to [x...] based on this comparison. k (t), y k (t)]→b i Mapping. You can select the location with the closest distance (e.g., "Euclidean distance") [x k (t), y k (t)] corresponds to button bi The category label. The Euclidean distance and the previously learned mapping can be calculated as r. err =SQRT((x(t)–x) k (t)) 2 +(y(t)-y k (t)) 2 Therefore, when the error radius (r) of the best match err () greater than the maximum permissible error radius r max At that time, the position radius r of gesture 502A or gesture 502B can be forced by registering zero hits. max The maximum permissible error in the process.

[0119] In some cases, the buttons on the touch interface 204 can be densely arranged. Due to the dense arrangement of the buttons, the user's finger can be positioned between two buttons. In this case, a simulation of button pressing can be performed during training mode 222. The button presses during the simulation can be recorded and stored in memory 214. Furthermore, the stored information of the pressed buttons can be used to update the coordinates of the positions in column 702 by adding a correction increment [x'(t), y'(t)], thereby generating corrected position coordinates [x'(t)]. k (t),y' k (t)]. The correction increment [x'(t), y'(t)] is the error [(x(t) – x k (t)),(y(t)-y k The vector in the direction of (t))] and its length r update Approximately 0.1 mm. Then, [x' k (t),y' k (t)] Copy to [x k (t),y k [(t)] allows the use of new position coordinates.

[0120] This adaptive correction can be achieved by using the initially learned position coordinates [x] k (t),y k (t)] is stored again as [x kinitial (t),y kinitial (t)] and the corrected position coordinates [x' k (t),y' k Copy (t) to the new location coordinates [x k (t),y k (t)] is restricted beforehand. The corrected position coordinates [x'] are... k (t),y' k (t)] can be learned at the location coordinates [x kinitial (t),ykinitial The maximum correction radius r of (t)] maxcorr Within this context, and if this is not the case, copying operations are prohibited. For example, for a touchable interface 204 with buttons spaced 40 to 50 mm apart, an initial r... maxcorr The thickness ranges from 10mm to 25mm.

[0121] Adaptive correction can be applied separately to each learned position coordinate [xk(t), yk(t)]. In some other cases, sensor 212 may exhibit low-frequency changes over time, such as global drift. This adaptive correction can be used to avoid global drift in sensor 212. Adaptive correction can improve the performance of sensor 212.

[0122] Figure 8 A flowchart is shown of a method 800 for controlling the operation of an elevator (e.g., elevator 102) using a multi-input call panel 202 according to an exemplary embodiment of the present disclosure. Method 800 includes operations 802 to 806 performed by a controller 208 of the multi-input call panel 202.

[0123] At operation 802, readings from a sensor (e.g., sensor 212) are received via a non-touch interface (e.g., non-touch interface 206). Sensor 212 is arranged to sense movement of a touchable interface (e.g., touchable interface 204) approaching the multi-input call panel 202.

[0124] At operation 804, a probability classifier (e.g., probability classifier 216) is executed in response to a received reading. Probability classifier 216 is trained to output the probability that the received reading corresponds to the intent of one or more of a plurality of touchable inputs arranged at different locations on the touchable interface 204.

[0125] At operation 806, when a touchable input is received on the touchable interface 204, when the probability classifier 216 outputs an intention to touch the touchable input that is higher than a threshold, or both, the operation of elevator 102 is controlled according to a control command associated with one of the multiple touchable inputs.

[0126] Figure 9 A block diagram of a device 900 for controlling the operation of an elevator (e.g., elevator 102) according to an exemplary embodiment of the present disclosure is shown. Device 900 corresponds to... Figure 2A and Figure 2B System 200. Device 900 includes processor 902, memory 904, and sensor 910. Memory 904 may include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system.

[0127] Device 900 is configured to implement functions for operating both a touchable interface and a non-touchable interface of elevator 102. For this purpose, device 900 may include an input interface 920 corresponding to the touchable interface 204 and the non-touchable interface 206. In some embodiments, processor 902 is configured to receive readings from sensor 910. Sensor 910 corresponds to sensor 212. Sensor 910 is configured to sense movement near the touchable interface. In some embodiments, sensor 910 may include an IR sensor, a light sensor, etc. Additionally or alternatively, sensor 910 may include a camera, such as camera 924. Some embodiments of camera 924 may include an RGBD camera.

[0128] Processor 902 can be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Processor 902 is also configured to execute probability classifier 906 in memory 904 in response to a received reading. Probability classifier 906 corresponds to probability classifier 216. In some embodiments, memory 904 may be configured to store a training program for training probability classifier 906. In some embodiments, probability classifier 906 may be trained on-site by an installer. Probability classifier 906 is trained to output the probability of a received reading corresponding to the intent to touch a button on touchable interface 204. In some embodiments, probability classifier 906 may have two operating modes, such as training mode 222 and control mode 224.

[0129] In one implementation, a human-machine interface (HMI) 914 within device 900 connects device 900 to camera 924. Alternatively, a network interface controller (NIC) 918 may be adapted to connect device 900 to network 928 via bus 916. In one implementation, sensor readings 912 may be received via input interface 920 of device 900.

[0130] Alternatively, device 900 may include a display screen 926 configured to display a floor value indicating the destination floor selected by user 106. Display screen 926 may be connected to device 900 via output interface 922. Alternatively, output interface 922 may include an audio interface that outputs an audio signal corresponding to the selected destination floor displayed on display screen 926. Alternatively, output interface 922 may be configured to emit colored light indicating a highlight on a button that user 106 intends to press on touchable interface 204. The highlight may correspond to the colored light emitted on the corresponding button. In some exemplary embodiments, display screen 926 may be configured to display the direction of elevator service of elevator 102, indicate the opening and / or closing of elevator 102, etc.

[0131] Alternatively, device 900 may include storage device 908, configured to store records of: current readings of sensor 910; previous readings of sensor 910; multiple touch inputs from user 106 during training mode 222; and touch inputs received from different users during control mode 224. Alternatively, storage device 908 may be configured to store a coordinate system corresponding to sensor 910 and touchable interface 204. Storage device 908 may also be configured to store a mapping between the intent to press one or more touchable inputs (e.g., buttons) on touchable interface 204 and corresponding category labels of one or more buttons. Data stored in storage device 908 may be accessed via network 928 for further processing. For example, processor 902 may access storage device 908 via network 928.

[0132] Figure 10 This illustration depicts a scenario where the operation of an elevator 1000 is controlled by a device 900 according to an exemplary embodiment of this disclosure. For example... Figure 10 As shown, elevator 1000 is equipped with a multi-input call panel 1002 (e.g., multi-input call panel 202), such as Figure 10 As shown in the illustration. In the example scenario, user 1004 enters elevator 1000. User 1004 approaches multi-input call panel 1002 to press a button (such as button 5 on multi-input call panel 1002) to operate elevator 1000.

[0133] When user 1004 reaches out to press a button on multi-input call panel 1002, sensor 1006 (e.g., sensor 212) detects hand movement near multi-input call panel 1002. Before user 1004 actually touches the button, multi-input call panel 1002 displays the button that user 1004 intends to press. In some cases, the intended button may be highlighted by colored light to indicate the button that user 1004 intends to press.

[0134] In this way, user 1004 can operate elevator 1000 efficiently and practically via multi-input call panel 1002 without physical touch input. This implementation of multi-input call panel 1002 is not limited to controlling elevator 1000 designed to transport people between different floors of a building. In some embodiments, elevator systems are widely used for transporting people and / or goods.

[0135] In different implementations, different elevator systems can implement such a multi-call panel 1002 that supports both contact-based and contactless panel functionality. For example, a transport system that controls the transport of goods or loads via a conveyor belt can implement such a multi-call panel 1002 in a cost-effective and feasible manner. Furthermore, refer to... Figure 11 The implementation of the multi-call panel is further described.

[0136] Figure 11 According to another exemplary embodiment of this disclosure, a scenario 1100 is shown where the operation of a conveyor system 1102 is controlled by a device 900. For example... Figure 11 As shown, the conveyor system 1102 is equipped with a motor 1104 and a multi-input call panel 1106 (e.g., multi-input call panel 202), such as Figure 11 As shown in the diagram, a multi-input call panel is configured to control multiple operations of the conveyor system 1102 to transport goods (such as box 1108) to one or more destinations. For this purpose, the multi-input call panel 1106 is used to provide input. Therefore, the motor 1104 can operate and transport box 1108.

[0137] In an illustrative example scenario, if a user (not shown) approaches the multi-input call panel 1106 to press a button on the multi-input call panel 1106 to operate the conveyor system 1102, sensor 1106a (e.g., sensor 212) detects hand movement near the multi-input call panel 1106. Before the user actually touches the button, the multi-input call panel 1106 displays the button the user intends to press on display 1106b. In some cases, the intended button may be highlighted by colored light to indicate the button the user intends to press.

[0138] In this way, users can operate the transmitter system 1102 efficiently and practically via the multi-input call panel 1106 without physical touch input.

[0139] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. Various changes to the function and arrangement of the elements may be contemplated without departing from the spirit and scope of the subject matter disclosed as set forth in the appended claims.

[0140] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that embodiments can be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form so as not to obscure the embodiments with unnecessary details. In other cases, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. Furthermore, the same reference numerals and designations in the various figures indicate the same elements.

[0141] Furthermore, each implementation can be described as a process, depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. Although a flowchart can describe operations as a sequential process, many operations can be performed in parallel or concurrently. Moreover, the order of operations can be rearranged. A process may terminate upon completion of its operations, but may have additional steps not discussed or included in the diagram. Furthermore, not all operations in any specifically described process may occur in all implementations. A process can correspond to a method, function, procedure, subroutine, subroutine, etc. The termination of a function may correspond to the function returning to the calling function or the main function.

[0142] Furthermore, implementations of the disclosed subject matter can be carried out, at least partially, manually or automatically. Manual or automatic implementation can be performed or at least assisted by using machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented as software, firmware, middleware, or microcode, program code or code segments that perform the necessary tasks can be stored in a machine-readable medium. A processor can perform the necessary tasks.

[0143] The various methods or processes outlined herein can be encoded as software that can be executed on one or more processors employing any of a variety of operating systems or platforms. Furthermore, such software can be written using a variety of suitable programming languages ​​and / or programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine. Typically, the functionality of program modules can be combined or distributed as needed in various implementations.

[0144] The embodiments of this disclosure can be embodied as a method, and embodiments of the method have been provided. Actions performed as part of the method can be ordered in any suitable manner. Therefore, embodiments can be constructed that perform actions in a different order than those shown, even if some actions are shown sequentially in the illustrative embodiments; the embodiments may also include performing these actions concurrently. Furthermore, the use of sequential terms such as "first," "second," etc., to modify claim elements in the claims does not in itself imply any priority, precedence, or order of one claim element relative to another claim element, nor does it imply a chronological order of actions of a method, but is merely used as a marker to distinguish one claim element with a specific name from another element with the same name (but using ordinal numbers).

[0145] Although this disclosure has been described with reference to certain preferred embodiments, it should be understood that various other modifications and variations may be made within the spirit and scope of this disclosure. Therefore, the aspects of the appended claims cover all such changes and variations that fall within the true spirit and scope of this disclosure.

Claims

1. A multi-input call panel for controlling operation of an elevator system, the multi-input call panel comprising: a touchable interface associated with a plurality of touchable inputs arranged at different locations on the multi-input call panel; a non-touch interface including a processor operably connected to receive readings of a sensor arranged to sense motion proximate the touchable interface, and the processor configured to execute a probabilistic classifier responsive to receiving the readings, the probabilistic classifier trained to output a corresponding probability that the received readings are of an intent to touch one or more of the plurality of touchable inputs; and a controller configured to control operation of the elevator system in accordance with control commands associated with a touchable input of the plurality of touchable inputs when the touchable input is touched on the touchable interface, when the probabilistic classifier outputs a probability that the intent to touch the touchable input is above a threshold, or both.

2. The multi-input call panel of claim 1, further comprising: a switch configured to change an operating mode of the multi-input call panel, wherein the operating mode includes a training mode and a control mode, wherein during the training mode a plurality of touch inputs of the touchable inputs and readings of the sensor prior to the plurality of touch inputs are collected and used to train the probabilistic classifier, and wherein during the control mode the plurality of touch inputs and outputs of the probabilistic classifier are used to control operation of the elevator system.

3. The multiple-input call panel of claim 2, wherein, the processor coupled with a memory configured to store a pre-trained probabilistic classifier and training readings of the sensor used for the training, wherein during the training mode readings of the sensor are mapped to the training readings to generate a transformation function, and wherein during the control mode readings of the sensor are transformed by the transformation function prior to submission to the probabilistic classifier.

4. The multiple-input call panel of claim 2, wherein, the processor coupled with a memory configured to store a training program for training the probabilistic classifier, wherein during the training mode readings of the sensor that result in touching a respective touchable input are labeled with the respective touchable input, wherein the training program trains the probabilistic classifier upon receiving a plurality of pairs of readings and the respective touchable input.

5. The multiple-input call panel of claim 2, wherein, the sensor arranged to sense a plane parallel to the multi-input call panel and located at a fixed distance from the multi-input call panel, wherein readings of the sensor submitted to the probabilistic classifier during the training mode or the control mode identify a location of a user's finger tip across the plane.

6. The multiple-input call panel of claim 4, wherein, The sensors are arranged to sense a set of planes parallel to the multi-input call panel and located at different distances from the multi-input call panel, wherein the readings of the sensors submitted to the probabilistic classifier during the training mode or the control mode include a position of a user's finger tip across each plane of the set of planes.

7. The multiple-input call panel of claim 6, wherein, The position of the user's finger tip across each plane of the set of planes is extrapolated to generate an extrapolated curve ending in the respective touchable input, wherein during the training mode, the probabilistic classifier is trained using the extrapolated curve ending in the respective touchable input, and wherein during the control mode, the extrapolated curve is submitted to the probabilistic classifier to estimate a touch hit point.

8. The multiple-input call panel of claim 1, wherein, The sensors include one or more of thermal sensors, motion sensors, light detection and ranging (LIDAR) sensors, and cameras.

9. The multiple-input call panel of claim 1, wherein, The probabilistic classifier corresponds to a Naive Bayes classifier, a k-Nearest Neighbors (KNN) classifier, a Gaussian Mixture Model (GMM) classifier, a Support Vector Machine (SVM) classifier, and a Parzen kernel density estimation based classifier.

10. A method for controlling operation of an elevator system using a multi-input call panel, the method comprising: receiving, via a non-touch interface of the multi-input call panel, readings of sensors of the non-touch interface, the sensors arranged to sense motion in a vicinity of a touchable interface of the multi-input call panel; in response to receiving the readings, executing a probabilistic classifier trained to output corresponding probabilities of the received readings being indicative of touching one or more touchable inputs of a plurality of touchable inputs arranged at different locations on the touchable interface of the multi-input call panel; and controlling operation of the elevator system in accordance with a control command associated with a touchable input of the plurality of touchable inputs when the touchable input is touched on the touchable interface, when the probabilistic classifier outputs a probability of the intention to touch the touchable input being above a threshold, or both.

11. The method of claim 10, further comprising: changing, via a switch of the multi-input call panel, an operational mode of the multi-input call panel, wherein the operational mode includes a training mode and a control mode, wherein during the training mode, a plurality of touch inputs of the touchable inputs and readings of the sensors prior to the plurality of touch inputs are collected and used to train the probabilistic classifier, and wherein during the control mode, the plurality of touch inputs and outputs of the probabilistic classifier are used to control operation of the elevator system.

12. The method of claim 11, further comprising: storing a pre-trained probability classifier and training readings of a sensor used for training of the probability classifier in a memory of the non-touch interface, wherein during the training mode, readings of the sensor are mapped to the training readings to generate a transformation function, and wherein during the control mode, readings of the sensor are transformed by the transformation function prior to submission to the probability classifier.

13. The method of claim 11, further comprising: storing a training program for training the probability classifier, wherein during the training mode, readings of the sensors that result from touching respective touchable inputs are labeled with the respective touchable inputs, wherein the training program is to train the probability classifier upon receiving a plurality of pairs of readings and respective touchable inputs.

14. The method of claim 11, further comprising: arranging the sensors to sense a plane parallel to the multi-input call panel and located at a fixed distance from the multi-input call panel, wherein readings of the sensors submitted to the probability classifier during the training mode or the control mode identify a location of a user's finger tip across the plane.

15. The method of claim 14, further comprising: arranging the sensors to sense a set of planes parallel to the multi-input call panel and located at different distances from the multi-input call panel, wherein readings of the sensors submitted to the probability classifier during the training mode or the control mode include a location of a user's finger tip across each of the set of planes.

16. The method of claim 15, further comprising: extrapolating the location of a user's finger tip across each of the set of planes to generate an extrapolated curve ending in a respective touchable input, wherein during the training mode, the probability classifier is trained using the extrapolated curve ending in the respective touchable input, and wherein during the control mode, the extrapolated curve is submitted to the probability classifier to estimate a touch hit point.

17. An apparatus corresponding to a multi-input call panel for controlling operation of an elevator system, the apparatus comprising: a touchable interface associated with a plurality of touchable inputs arranged at different locations on the multi-input call panel; a non-touch interface including a processor operably connected to receive readings of a sensor arranged to sense motion proximate to the touchable interface, and the processor configured to execute a probability classifier in response to receiving the readings, the probability classifier trained to output a corresponding probability that the received readings are of an intent to touch one or more of the plurality of touchable inputs; and and a controller configured to control operation of the elevator system in accordance with control commands associated with touch inputs of the plurality of touch inputs when the touch inputs are touched on the touchable interface, when the probability classifier outputs a probability of an intent to touch a touch input is higher than a threshold, or both.

18. The apparatus of claim 17, further comprising: a switch configured to change an operating mode of the multiple-input call panel, wherein the operating mode comprises a training mode and a control mode, wherein during the training mode, a plurality of touch inputs of the touch inputs and readings of the sensor prior to the plurality of touch inputs are collected and used to train the probability classifier, and wherein during the control mode, the plurality of touch inputs and the output of the probability classifier are used to control operation of the elevator system.

19. The apparatus of claim 18, wherein, the processor is coupled with a memory configured to store a pre-trained probability classifier and training readings of the sensor used for the training, wherein during the training mode, readings of the sensor are mapped to the training readings by means of a transformation function, and wherein during the control mode, readings of the sensor are transformed by the transformation function before being submitted to the probability classifier.

20. The apparatus of claim 18, wherein, the processor is coupled with a memory configured to store a training program for training the probability classifier, wherein during the training mode, readings of the sensor that result from touching respective touch inputs are labeled with the respective touch inputs, wherein the training program, once receiving a plurality of pairs of readings and respective touch inputs, will train the probability classifier.

Images