Personnel mover system configured to provide distribution of solutions to technical operation issues using augmented reality and cumulative data

Abstract

A system configured for addressing technical anomalies in a personnel mover system, the system having: a sensor configured for generating sensor data indicative of operating parameters associated with the personnel mover system; and a system controller module configured to receive status data indicative of operating parameters obtained from the sensor data over the first communication channel, apply the status data as input to the generated AI model, and identify an abnormal condition in the personnel mover system, wherein the system controller module is configured to communicate instruction data to the mobile device over the second communication channel, wherein the instruction data includes one or more of the following: identifying where to locate the anomalies written instructions; or data for identifying abnormal conditions, including sound data; or image data; or one or more of the video data; and corrective measures for addressing abnormal conditions.

Description

Technical Field

[0001] The embodiments relate to a distributed personnel mobility system, and more specifically, to a distributed personnel mobility system configured to use augmented reality and accumulated data to provide solutions to technical operational problems. Background Technology

[0002] For technicians, handling (e.g., resolving) technical operational problems on-site can often be challenging. After attempting to locate the technical problem and apply corrective measures, technicians may need to repeatedly ride the elevator to confirm that the location and corrective measures were correctly identified and successfully implemented. This can cause delays and lead to assessment errors, resulting in additional downtime for the affected equipment. Summary of the Invention

[0003] A system configured to resolve technical anomalies in a personnel mobility device system, the system comprising: a sensor configured to generate sensor data indicating operating parameters associated with the personnel mobility device system; and a system controller module configured to receive state data indicating the operating parameters obtained from the sensor data via a first communication channel, apply the state data as input to generate an AI model, and identify anomalies in the personnel mobility device system, wherein the system controller module is configured to transmit instruction data to the mobility device via a second communication channel, wherein the instruction data includes one or more of: a written instruction to identify the location of the anomaly; or data used to identify the anomaly, including one or more of sound data; image data; or video data; and corrective measures for resolving the anomaly.

[0004] In addition to one or more aspects of the system, or as an alternative, the system controller module is configured to transmit alarm data to the mobile device when an abnormal condition is identified and the instruction data is sent separately.

[0005] In addition to one or more aspects of the system, or as an alternative, the guidance data further identifies the location of the anomaly.

[0006] In addition to one or more aspects of the system, or as an alternative, the personnel mover system includes a personnel mover, which is an escalator, elevator, or mobile platform, and the status data indicates the operating status of the personnel mover.

[0007] In addition to one or more aspects of the system, or as an alternative, the personnel mover is the elevator, the personnel mover system includes an elevator machine, and the status data indicates the operating status of the elevator machine.

[0008] In addition to one or more aspects of the system, or as an alternative, the system controller module is configured to switch the personnel mover system to maintenance mode and instruct the personnel mover to the location of the malfunction, thereby transporting a technician to the location of the malfunction to maintain the personnel mover system.

[0009] In addition to one or more aspects of the system, or as an alternative, the mobile device is a mobile phone or peripheral device configured to process multimodal data, including one or more of text, sound, images, and video in the instruction data, for transmitting the instruction data as augmented reality to a technician via the mobile device.

[0010] In addition to one or more aspects of the system, or as an alternative, when the personnel mobility system is in the maintenance mode, the controller module repeatedly receives and processes the status data, generates updates to the instruction data, and transmits the updated instruction data to the mobility device until the controller module determines that the status data indicates that the operating parameters of the personnel mobility system are within a predetermined normal operating range.

[0011] In addition to one or more aspects of the system, or as an alternative, the generated AI model is trained on one or more of the following data: debugging data, heartbeat data, code version update data, and maintenance data.

[0012] In addition to one or more aspects of the system, or as an alternative, the sensors include one or more of a speed sensor, a vibration sensor, a load sensor, a door operation sensor, and a health sensor, wherein the health sensor is configured to sense mechanical faults in the personnel mobility system; and the first communication channel is a wired or wireless channel, and the second communication channel is a wireless channel that is the same as or different from the first communication channel.

[0013] Disclosed is a method for resolving technical anomalies in a personnel mobility device system, comprising: a system controller module receiving, via a first communication channel, status data indicating operating parameters associated with the personnel mobility device system, the status data being obtained by a sensor configured to sense the operating parameters; the system controller module applying the status data as input to a generated AI model to identify abnormal conditions in the personnel mobility device system; and the system controller module transmitting instruction data to the mobile device via a second communication channel, the instruction data including one or more audio data, image data, or video data for identifying the abnormal condition, and corrective measures for resolving the abnormal condition.

[0014] In addition to one or more aspects of the method, or as an alternative, the method includes the system controller module transmitting alarm data to the mobile device upon recognizing the abnormal condition, wherein the transmission of the alarm data is separate from the transmission of the instruction data.

[0015] In addition to one or more aspects of the method, or as an alternative, the instruction data further identifies the location of the anomaly.

[0016] In addition to one or more aspects of the method, or as an alternative, the personnel mover system includes a personnel mover, which is an escalator, elevator, or mobile platform, and the status data indicates the operating status of the personnel mover.

[0017] In addition to one or more aspects of the method, or as an alternative, the personnel mover is an elevator, the personnel mover system includes an elevator machine, and the status data further indicates the operating status of the elevator machine.

[0018] In addition to one or more aspects of the method, or as an alternative, the method includes switching the personnel mover system to a maintenance mode by the system controller module and instructing the personnel mover to the location of the malfunction, thereby transporting a technician to the location of the malfunction to maintain the personnel mover system.

[0019] In addition to one or more aspects of the method, or as an alternative, the mobile device is a mobile phone or peripheral device configured to process multimodal data including one or more of the text, sound, image, and video in the instruction data, for transmitting the instruction data as augmented reality to a technician via the mobile device.

[0020] In addition to one or more aspects of the method, or as an alternative, the method includes, when the personnel mobility system is in the maintenance mode, the controller module repeatedly receiving and processing the status data, generating an update to the instruction data, and transmitting the updated instruction data to the mobility device, until the controller module determines that the status data indicates that the operating parameters of the personnel mobility system are within a predetermined normal operating range.

[0021] In addition to one or more aspects of the method, or as an alternative, the generative AI model is trained on one or more of the following data: debugging data, heartbeat data, code version update data, and maintenance data.

[0022] In addition to one or more aspects of the method, or as an alternative, the sensor includes one or more of a speed sensor, a vibration sensor, a load sensor, a door operation sensor, and a health sensor, wherein the health sensor is configured to sense mechanical faults in the personnel mobility system; and the first communication channel is a wired or wireless channel, and the second communication channel is a wireless channel that is the same as or different from the first communication channel. Attached Figure Description

[0023] This disclosure is illustrated by way of example and is not limited to the accompanying drawings, in which similar reference numerals indicate similar elements.

[0024] Figure 1 These are schematic diagrams of elevator systems that can be implemented using various embodiments of this disclosure; Figure 2 A distributed system according to an embodiment is shown, which is configured to use augmented reality and accumulated data to solve technical problems related to the operation of a personnel mobility system; Figure 3 This is a schematic diagram illustrating the process by which a distributed system performs actions to solve technical problems related to the operation of a people mobility system using augmented reality and accumulated data; and Figure 4 It is a flowchart of a method for solving technical problems related to the operation of a personnel mobility system using augmented reality and accumulated data. Detailed Implementation

[0025] Figure 1 This is a perspective view of an elevator system 101, which includes a passenger mover 102 (possibly an elevator car 103), a counterweight 105, a tension member 107, guide rails (or track system) 109, a machine (or machine system) 111, a position reference system 113, and an electronic elevator controller (controller) 115. The elevator car 103 and the counterweight 105 are connected to each other via the tension member 107. The tension member 107 may include or be configured as, for example, ropes, cables, and / or coated steel strips. The counterweight 105 is configured to balance the load of the elevator car 103 and to facilitate simultaneous and opposite movement of the elevator car 103 relative to the counterweight 105 within the elevator shaft (or shaft) 117 and along the guide rails 109.

[0026] Tension member 107 engages machine 111, which is part of the overhead structure of elevator system 101. Machine 111 is configured to control movement between elevator car 103 and counterweight 105. Position reference system 113 may be mounted on a fixed portion at the top of elevator shaft 117, such as on a support rail or guide rail, and may be configured to provide a position signal relating to the position of elevator car 103 within elevator shaft 117. In other embodiments, position reference system 113 may be directly mounted to a moving component of machine 111, or may be positioned in other locations and / or configurations as known in the art. As known in the art, position reference system 113 can be any device or mechanism for monitoring the position of elevator car and / or counterweight. For example, and not limitingly, as those skilled in the art will appreciate, position reference system 113 can be an encoder, sensor, or other system, and can include speed sensing, absolute position sensing, etc.

[0027] As shown, controller 115 may be located in controller room 121 of elevator shaft 117 and configured to control the operation of elevator system 101 and, in particular, the operation of elevator car 103. It should be understood that controller 115 does not need to be in controller room 121, but may be in the shaft or other location within the elevator system. For example, controller 115 may provide drive signals to machine 111 to control the acceleration, deceleration, leveling, stopping, etc., of elevator car 103. Controller 115 may also be configured to receive position signals from position reference system 113 or any other desired position reference device. As the elevator car 103 moves up or down along guide rail 109 within elevator shaft 117, it may stop at one or more landings 125 as controlled by controller 115. Although shown in controller room 121, those skilled in the art will appreciate that controller 115 may be located and / or configured in other locations or positions within elevator system 101. In one embodiment, the controller may be remotely located or in the cloud.

[0028] Machine 111 may include a motor or similar drive mechanism. According to embodiments of this disclosure, machine 111 is configured to include an electrically driven motor. The power supply for the motor can be any power source, including the power grid, which is supplied to the motor in conjunction with other components. Machine 111 may include a traction pulley that transmits force to the tension member 107 to move the elevator car 103 within the elevator shaft 117.

[0029] Although illustrated and described using a rope system including tension member 107, embodiments of this disclosure may also be employed in elevator systems employing other methods and mechanisms for moving the elevator car within an elevator shaft. For example, embodiments may be employed in cordless elevator systems using linear motors to move the elevator car. Embodiments may also be employed in cordless elevator systems using hydraulic lifts to move the elevator car. Furthermore, embodiments may be employed in cordless elevator systems using self-propelled elevator cars (e.g., elevator cars equipped with friction wheels, clamping wheels, or traction wheels). Figure 1 These are non-restrictive examples presented merely for illustrative and explanatory purposes.

[0030] In other embodiments, the system includes a transport system that moves passengers between floors and / or along a single floor. Such a transport system may include escalators, moving walkways, etc. Therefore, the embodiments described herein are not limited to those described herein. Figure 1 Elevator systems, such as the elevator system shown herein. In one example, the embodiments disclosed herein may be applicable transportation systems, such as elevator system 101 and transportation equipment of a transport system, such as the elevator car 103 of elevator system 101. In another example, the embodiments disclosed herein may be applicable conveying systems, such as escalator systems and / or moving walkways and transportation equipment of transport systems, such as moving stairs of escalator systems and / or moving walkways.

[0031] Turn Figure 2 The publicly disclosed information pertains to distributed (e.g., cloud) systems 200. Although in Figure 2 The diagram illustrates various modules for performing discrete functions, but it should be understood that two or more functions can be combined into a common module, or alternatively, the functions can be further divided into additional modules.

[0032] System 200 includes network 210, which may be a wide area network, such as the Internet. Personnel mover 102 is shown as elevator cars 103A-103C (typically 103), but as a non-limiting embodiment, personnel mover 102 may be an escalator, moving platform, or moving walkway, etc. That is, while this disclosure may refer to elevator car 103, this is a non-limiting example of a personnel mover system 102 to which this disclosure is applicable.

[0033] The elevator car 103 can be an IoT (Internet of Things) device, i.e., a device operatively coupled to the Internet via a first communication channel (e.g., a network) 175A. The first communication channel 175 can utilize a wired channel (e.g., Ethernet) or a wireless channel (e.g., a wide area network or a cellular network), which will be discussed in more detail below. Each of the elevator cars 103 may have device controllers 150A-150C (typically 150) and sensors 155A-155C (typically 155), configured to transmit sensor data 156A-156C (typically 156). Sensors 155 may include one or more of a speed sensor, vibration sensor, microphone, pressure sensor, load sensor, door operation sensor, and health sensor, for example, configured to sense mechanical faults in the elevator car 103.

[0034] Additional sensors (typically grouped with sensor 155) in sensors 155D-155F may be located in hoistway 117, distributed between the pit (bottom) 117A and the top 117B of hoistway 117, i.e., where machine 111 may be located. Sensor data 156D-156F (typically grouped with sensor data 156) transmitted by sensors 155 in hoistway 117, for example, together with the remainder of sensor data 156, may include video, audio, vibration, and other sensing information. This information may relate to operating parameters of the elevator car 103 traveling in or around sensor 155, as well as any other machinery associated with the operation of hoistway 117 and elevator car 103, such as machine 111.

[0035] System 200 may include a controller module (or service) 220, an IoT central module 230 or a similar platform, and an IoT application and data storage module 240 (for simplicity, an application module or IoT app module 240). The IoT central module 230 is a known IoT application platform-as-a-service (aPaaS) with a user-interactive dashboard that centralizes device data, allowing for data-driven workflows and the creation of custom apps. The IoT central module 230 can integrate various components such as sensors, processors, memory, and communication interfaces. These modules play a crucial role in enabling devices to interconnect and communicate within the Internet of Things (IoT) ecosystem. The IoT application module 240 is used for storage and other processes running in cloud services.

[0036] Telemetry messages (data) 250 may originate from elevator car 103 (e.g., as a non-limiting example, in raw format or as processed data), and are extracted, converted into a readable / storable format, and loaded onto a database for consumption and distribution by front-end applications. Controller module 220 may instruct IoT central module 230 to register elevator car 103 with IoT app module 240, enabling IoT app module 240 to receive telemetry data 250A-250C (typically 250) including sensor data 156 from elevator car 103, and to transmit code 260, such as an update, to elevator car 103. Elevator cars 103 may also interact with each other and the cloud in other ways, such as requesting updates, voice communication, etc.

[0037] There may be hundreds of thousands of elevator cars 103, each sending production (e.g., actual) telemetry data 250 to the IoT app module 240. Each message may relate to different aspects of the elevator car 103 throughout the day, such as the operating status of brakes, doors, etc.

[0038] The IoT app module 240 can periodically (e.g., daily) generate logs 245 indicating received telemetry data 250 and transmitted codes 260. A monitoring and capture metric module (for simplicity, a monitoring module) 280 can monitor the logs generated by the IoT app module 240. Logs 245 can be forwarded to a metric storage module 310, from which telemetry metric data 315 is derived. As a non-limiting embodiment, a query module 450 can generate a report 320 from the telemetry metric data 315, which can be viewed via an interactive performance dashboard 297 on, for example, a mobile phone 298A or another portable smart device 298, such as smart glasses 298B, and accessed via a web interface module 290. Reference to mobile phone 298A herein should be considered to include other smart devices 298. Using this configuration, errors or alarm conditions recorded in communications can be identified by a user 295 who is likely a technician. The user can contact the controller module 220 via an API module (or gateway) 300. Mobile phone 298A can communicate via a second communication channel 175B, which can be a cellular network or a wide area network. In one embodiment, the first and second channels 175A and 175B can utilize the same protocol, and in another embodiment, they can be common channels.

[0039] To understand, mobile phone 298A may have onboard sensors 301, including microphone 301A, video input 301B, and other sensors 301C such as motion sensors. Mobile device data 302 captured by mobile phone 298A can be transmitted along with telemetry data 250 via a second communication channel 175B for processing by system 200. Furthermore, technician 295 can input user input data 303 indicating conditions detected on or in the operating environment of elevator car 103, such as along hoistway 117. This information can be input into interactive performance dashboard 297 for use by system 200 with telemetry data 250.

[0040] More specifically, the diagram illustrates a user 295 interacting with the web interface module 290 to communicate with the controller module 220 via the API module 300, and viewing a performance dashboard 297, for example, on their mobile phone 298. The IoT central module 230 registers the elevator car 103 with the IoT app module 240. Registration establishes trust in device connectivity and allows messages to travel back and forth between the device and the cloud (i.e., device-to-cloud and cloud-to-device) based on predefined load scenarios. The monitoring module 280 can monitor the telemetry logs 245 generated by the IoT app module 240. The query module 450 generates telemetry measurement data 315 and a report 320 from the telemetry measurement data 315. The telemetry measurement data 315 can be stored on the measurement storage module 310 and visualized on the performance dashboard 297 via the web interface module 290 to identify errors logged at the IoT app module 240 over the past day (as an example).

[0041] According to an embodiment, a machine learning model (MLM) 325 (commonly referred to as a neural network model or generative AI model 324) may be located within the AI ​​module 327 or in one of other identified modules, reporting 320. When a report is requested, for example, seeking a solution to a technical operational problem related to the operation of elevator car 103, user 295 may engage MLM 325 using natural language. In response, MLM 325 may, for example, provide recommendations on remediating problems identified in telemetry log 245 based on accumulated data used to train MLM 325.

[0042] As can be understood, the MLM 325 can operate in learning mode (training mode), where it is trained on a dataset, such as a dataset obtained from sensor data 156 or other data identified below. In this mode, the MLM 325 learns patterns and relationships within the data to make accurate predictions or decisions. In this mode, the parameters of the MLM 325 are adjusted based on the input data and the desired output. In production mode (inference mode), once the MLM 325 has been trained and validated and deployed, it uses the learned parameters to make predictions on new, unseen data and provide real-time or batch predictions to end users or other systems.

[0043] Information sources for the learning mode of MLM 325 may include data 326, including device maintenance data 326A, such as data obtained from a mechanic contact app on their phone 298, which transmits relevant data to controller module 220. Maintenance data 326A can identify technical problems, successful and unsuccessful solutions to the technical problems. This data may include voice, image, and video data, such as multimodal data, which can be segmented using a segmentation model performed by an encoder and decoder, which is related to the progressive resolution of the identified technical problem. Data 326 may include code download (e.g., update) data 326B, such as data obtained from IoT app module 240. Data 326 may include heartbeat data 326C from telemetry data 250, which includes performance, alarm, and event data. Data 326 may also include initial commissioning data 326D, such as data obtained from IoT central module 230 when registering elevator car 103. As mentioned above, data 326 may also include mobile device data 302 and user input data 303.

[0044] Similarly, according to embodiments, AI module 327 or, for example, query module 450, may be equipped with a large language model (LLM) 455 as another generative AI model 324. Typical techniques can be used to train the LLM 455, such as collecting and processing datasets related to the operation and maintenance of elevator car 103, applying model architectures such as transformers capable of handling long-range correlations in text, applying hyperparameter tuning to batches of training data to adjust the size and configuration of the training data, applying optimization techniques to improve accuracy, and subsequently iteratively tuning the LLM 455. The LLM 455 can be trained to respond to technician 295, who, for example, submits reports 320 to controller module 220 regarding the current, historical, and predictable (e.g., statistical) future operational status of system 200, as well as queries to resolve proactive technical maintenance problems. That is, while the MLM 325 can be used to identify and resolve technical operational problems in system 200 and recommend solutions, the LLM 455 can be used to enable communication exchange with technician 295 using natural language.

[0045] To understand this, an MLM 325 can be trained to respond to natural language input without requiring a separate MLM 455, for example, using Natural Language Processing (NLP). NLP is a subfield of machine learning that focuses on the interaction between computers and human language.

[0046] As indicated, although in Figure 2 Various modules for performing discrete functions are shown, but it should be understood that two or more functions can be combined into a common module, or alternatively, functions can be further divided into additional modules. Therefore, references to controller module 220 herein may imply functions applicable to controller module 220 or other modules.

[0047] Turn Figure 3 According to an embodiment, the trained generative AI model 324 may be able to receive telemetry data 250, sensor data 156, mobile device data 302, and user data 303 (collectively referred to as status data 304) as input in production mode. Based on the status data 304, the generative AI model 324 can identify technical anomalies (e.g., technical malfunctions), such as those identified herein in the operation of the technical system on the elevator car 103 or within the hoistway 117. For example, via controller module 220, system 200 can control elevator system 101 to enter maintenance mode via communication through network connection 175A, for example, locking normal use by passengers, and transmitting alarm data 305 to mechanic 295 via a second communication channel 175B. Alarm data 205 can identify malfunctions.

[0048] The controller module 220 can control the elevator car 103, equipped with the mechanic 295 thereon, to travel to an abnormal location in the shaft 117. For example, the car 103 can travel to the pit 117A or top of the shaft 117 near the machine 111, or the mechanic 295 can resolve other abnormal locations, such as detecting an abnormality at a specific height in the shaft. Multimodal instruction data 306 can be transmitted to the mechanic 295's mobile device 298 (e.g., mobile phone 298A or smart glasses 298B) to resolve the problem, for example, with sound (e.g., voice), images, and video. The instruction data 306 can include multimodal data representing the abnormal condition, i.e., the appearance and operating status of the elevator system 101, including one or more of the car 103 or associated equipment, such as the machine 111, as it operates under abnormal conditions. The instruction data 306 can include steps for resolving the abnormal condition, as well as the appearance and operating status of the elevator system 101 when operating within a predetermined normal operating range. During the problem-solving period, controller module 220 can continue to receive telemetry data 250 from sensor 255, device data 302 from smart device 298, and user input data 303 from technician 295. This updated status data 304 can be fed as input to the generating AI model 324 to determine whether an anomaly is still detected, and whether it is the same or a different anomaly. This loop can continue until the generating AI model 324 determines that no anomaly has been detected, for example, that elevator system 101 is operating within acceptable operating limits.

[0049] In other words, the disclosed system 200 collects real-time feedback, such as status data, through sensors 155 to detect ride quality and other technical problems and anomalies, such as operations exceeding acceptable thresholds or ranges. System 200 presents insights related to the detection and resolution of technical problems and depicts the resolution on an augmented reality device 298, which can represent the real-time elevator system 101 and AI recommendations, such as showing components and faults. The mechanic 295 uses this data to address and resolve ride quality and other mechanical problems. System 200 helps the mechanic 295 perceive the elevator system 101 in real time and resolve ride quality and other technical problems more effectively. Based on the insights, system 200 instructs the elevator car 103 to take the mechanic 295 directly to the abnormal location and displays the problem to the mechanic 130 using augmented reality, providing steps to resolve the problem. This cycle can be repeated until updated status data 304 received by system 200 indicates that no further problems exist. It can be understood that system 200 provides an optimized method for resolving technical problems in elevator system 101. System 200 enables time and workload savings and provides technicians 295 with more accurate and reliable solutions.

[0050] Turn Figure 4 The flowchart illustrates a method for solving technical problems related to the operation of a personnel mobility system 200 (for simplicity, System 200) using augmented reality and accumulated data. Dashed boxes (if present) in the flowchart indicate further explanation of one or more preceding steps and are not intended to limit the scope of the embodiments.

[0051] As shown in block 510, the method includes receiving status data 304 via a first communication network 175A by the system controller module 220. Status data 304 indicates operating parameters associated with the system 200, which are obtained, for example, by a sensor 155 configured to sense operating parameters. In other words, status data 304 can indicate the operating status of the personnel mover 102.

[0052] As indicated, system 200 includes a personnel mover 102, which is an escalator, elevator 103, or moving platform. Also as indicated, personnel mover 102 is elevator 103, system 200 includes elevator machine 111, and status data 304 indicates the operating status of elevator machine 111. As indicated, sensor 155 includes one or more of a speed sensor, vibration sensor, load sensor, door operation sensor, and health sensor. The health sensor is configured to sense mechanical faults in system 200.

[0053] As shown in box 520, the method includes applying state data 304 as input to a generative AI model 324 by the system controller module 220. Using this input data, model 324 identifies abnormal conditions in system 200. As indicated, the generative AI model is trained on data 326, which includes one or more of debug data 326D, heartbeat data 326C, code version update data 326B, and maintenance data 326A.

[0054] As shown in box 530, the method includes transmitting instruction data 306 from system controller module 220 to mobile device 298 via a second communication channel (e.g., network) 175B. Instruction data 306 contains written instructions to identify the location of an anomaly. Additionally, or alternatively, instruction data 306 includes data for identifying the anomaly, including one or more of sound data, image data, or video data. Additionally, or alternatively, instruction data 306 includes corrective measures for resolving the anomaly. As also indicated, the first communication channel 175A is a wired or wireless channel, and the second communication channel 175B is the same as or different from the first communication channel 175A. As indicated, mobile device 298 is a mobile phone 298A or a peripheral device 298B configured to process multimodal data, including one or more of text, sound (e.g., voice), images, and video in instruction data 306. This enables instruction data 306 to be transmitted as augmented reality to technician 295 via mobile device 298.

[0055] As shown in box 540, the method includes transmitting alarm data 305 from system controller module 220 to mobile device 298 upon identification of an abnormal situation. The alarm data transmission may be separate from the transmission of command data 306. For example, the alarm message may be transmitted over the network with a higher priority than other messages. As indicated, command data 306 further identifies the location of the abnormal situation.

[0056] As shown in box 550, the method includes switching system 200 to maintenance mode by system controller module 220 and instructing personnel mover 102 to move to the location of the malfunction. This set of steps transports technician 295 to the location of the malfunction to maintain system 200.

[0057] As shown in box 560, the method includes the controller module 220 repeatedly receiving and processing status data 304, generating an update to instruction data 306, and transmitting the updated instruction data 306 to the mobile device 298. This cycle continues when the system 200 is in maintenance mode until the controller module 220 determines that the status data 304 indicates that the operating parameters of the system 200 are within a predetermined normal operating range.

[0058] Regarding the implementation of artificial intelligence (AI) identified explicitly or inherently herein, machine learning models (e.g., part of an AI system) may be utilized in embodiments. AI systems use digital computers or machines controlled by digital computers to simulate human intelligence in sensing the environment, such as using available sensors including speed, acceleration, vibration, sound, video, etc., and acquire knowledge and use that knowledge to obtain optimal results. AI infrastructure includes technologies such as sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technologies, operating / interaction systems, mechatronics, etc. Some implementations of AI according to embodiments utilize computer vision, speech processing, natural language processing, machine learning / deep learning, etc.

[0059] Some implementations of the AI ​​according to the embodiments utilize pre-trained (PT) machine translation models that employ a neural network-based sequence-to-sequence (sequence-to-sequence, or SS) framework. The SS framework is a framework that includes an encoder-decoder structure. The encoder-decoder structure transforms an input sequence into another sequence as output. In this framework, the encoder transforms the input sequence into a vector, and the decoder takes the vector and generates the output sequence in chronological order. The encoder and decoder can utilize the same type of neural network model, or they can utilize different types of neural network models. The neural network model can be a CNN (Convolutional Neural Network) model, an RNN (Redundant Neural Network) model, a Long Short-Term Memory (LSTM) model, a Delayed Network model, a Gated CNN model, etc.

[0060] Once trained, a machine learning model can analyze input data and predict and / or represent features included in the sensed data in one or more aspects. In the case of video, in a non-limiting example, the sensed data may include sequential images and / or encoded video data (e.g., using digital video file / stream formats and / or codecs, such as MP4, MOV, AVI, WEBM, AVCHD, OGG, etc., including combinations and / or multiple thereof). The prediction and / or representation of features may include segmenting the video data. In some instances, one or more trained machine learning models include or are associated with preprocessing or enhancement performed prior to segmenting the video data (e.g., intensity normalization, resizing, cropping, etc., including combinations and / or multiple thereof). The output of one or more trained machine learning models may include predictions of aspects of the video data, localization and / or position of aspects within the video data, and / or state of aspects. The localization may be a set of coordinates in an image / frame in the video data. In one or more examples, the trained machine learning model is trained to perform higher-level prediction and tracking.

[0061] Similar predictions about the device's operational state can be made by analyzing sensor data captured during device utilization and applying that data to a trained machine learning model. For example, using service learning techniques, a model can be trained on known inputs and outputs from legacy events to predict future outputs based on future inputs. The model can be evaluated, allowing variables to be weighted or reweighted to more accurately correlate inputs and outputs, and can be retrained as more inputs and outputs are collected. For instance, predictions about the state of multiple devices in an operationally integrated system can be obtained using a trained model. Data can be captured for one (or fewer) of the devices, including operational sounds, vibrations, etc., and the captured data can be used to run a trained model that is trained to identify the impact (constructive and destructive) of the devices on each other in their respective operational states, including when they are operating within and outside acceptable tolerances.

[0062] Regarding the telecommunications implementations explicitly or inherently identified herein, the wireless connections identified above may apply protocols including Local Area Network (LAN, or WLAN for wireless LAN) protocols and / or Private Area Network (PAN) protocols. LAN protocols include WiFi technology based on the Section 802.11 standard from the Institute of Electrical and Electronics Engineers (IEEE). PAN protocols include, for example, Bluetooth Low Energy (BTLE), a wireless technology standard designed and marketed by the Bluetooth Special Interest Group (SIG) for exchanging data over short distances using short-wavelength radio waves. PAN protocols also include Zigbee, a technology based on the Section 802.15.4 protocol from IEEE, representing a set of advanced communication protocols for creating personal area networks with small, low-power digital wireless devices for low-power, low-bandwidth requirements. Such protocols also include Z-Wave, a wireless communication protocol supported by the Z-Wave Alliance, which uses mesh networking and applies low-energy radio waves to communicate between devices such as electrical appliances, thereby allowing for wireless control of them.

[0063] Other applicable protocols include Low Power WAN (LPWAN), which is a wireless wide area network (WAN) designed to allow long-range communication at low bit rates, enabling terminal devices to operate on battery power for extended periods (years). Long Range WAN (LoRaWAN) is a type of LPWAN maintained by the LoRa Alliance and is a Media Access Control (MAC) layer protocol used to transmit management and application messages between network servers and application servers. Such wireless connectivity can also include Radio Frequency Identification (RFID) technology, used for communication with integrated chips (ICs) on, for example, RFID smart cards. Furthermore, Sub-1GHz RF devices operate in the ISM (Industrial, Scientific, and Medical) spectrum band below Sub-1GHz (typically in the 769–935 MHz, 315 MHz, and 468 MHz frequency ranges). This sub-1GHz spectrum band is particularly useful for RF IoT (Internet of Things) applications. Other LPWAN-IoT technologies include Narrowband IoT (NB-IoT) and Cat M1-IoT. Wireless communication used in the disclosed systems may include cellular technologies, such as 2G / 3G / 4G (etc.). The above is not intended to limit the scope of applicable wireless technologies.

[0064] The wired connections identified above can include connections (cables / interfaces) under RS-422 (Recommended Standard), also known as TIA / EIA-422, a technical standard supported by the Telecommunications Industry Association (TIA) and initiated by the Electronic Industries Alliance (EIA), which specifies the electrical characteristics of digital signaling circuits. Wired connections can also include connections (cables / interfaces) under the RS-232 standard for serial communication transmission of data. The RS-232 standard formally defines the signaling connection between a DTE (Data Terminal Equipment), such as a computer terminal, and a DCE (Data Circuit Termination Equipment or Data Communication Equipment), such as a modem. Wired connections can also include connections (cables / interfaces) under the Modbus serial communication protocol managed by the Modbus organization. Modbus is a server / client protocol designed for use by programmable logic controllers (PLCs) and is a commonly available means of connecting industrial electronic devices. Wireless connections can also include connectors (cables / interfaces) under the PROFibus (Process Fieldbus) standard managed by PROFIBUS & PROFINET International (PI). PROFibus is a standard for fieldbus communication in automation technology, publicly published as part of IEC (International Electrotechnical Commission) 61158. Wired communication can also be achieved via Controller Area Network (CAN) bus. CAN is a vehicle bus standard that allows microcontrollers and devices to communicate with each other in applications without a host computer. CAN is a message-based protocol published by the International Organization for Standardization (ISO). The above is not intended to limit the scope of applicable wired technologies.

[0065] As indicated, when data is transmitted over a network between terminal processors, the data may be transmitted in its raw form, or it may be processed, in whole or in part, at any of the terminal processors or intermediate processors (e.g., at a cloud service or other processor). The data may be parsed, partially or completely processed or compiled at any of the processors, and may then be concatenated together or maintained as separate information packets.

[0066] Regarding the computing technologies explicitly or inherently identified herein, each processor identified herein can be, but is not limited to, a single-processor or multi-processor system of any of a variety of possible architectures, including field-programmable gate arrays (FPGAs), central processing units (CPUs), application-specific integrated circuits (ASICs), digital signal processors (DSPs), or graphics processing units (GPUs) hardware arranged homogeneously or heterogeneously. The memory identified herein can be, but is not limited to, random access memory (RAM), read-only memory (ROM), or other electronic, optical, magnetic, or any other computer-readable medium. Embodiments can take the form of processes implemented by a processor and means for practicing those processes, such as a processor. Embodiments can also take the form of modules based on computer code, such as computer program code containing instructions (e.g., a computer program product) embodied in a tangible medium (e.g., a non-transitory computer-readable medium) (e.g., a floppy disk, CD-ROM, hard disk drive), as firmware on processor registers, or in any other non-transitory computer-readable medium, wherein when the computer program code is loaded into and executed by the computer, the computer becomes means for practicing the embodiments. Embodiments may also take the form of computer program code, for example, whether stored in a storage medium, loaded into a computer, and / or executed by a computer, or transmitted via a transmission medium (e.g., via wires or cables, via optical fibers, or via electromagnetic radiation), wherein when the computer program code is loaded into and executed by the computer, the computer becomes an apparatus for practicing exemplary embodiments. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits.

[0067] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The term “about” is intended to include the degree of error associated with a measurement of a particular quantity and / or with manufacturing tolerances of the equipment available at the time of filing. As used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context clearly indicates otherwise. It will also be understood that the terms “comprises and / or comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

Claims

1. A system configured to resolve technical anomalies in a personnel mobility system, the system comprising: Sensors configured to generate sensor data indicating operating parameters associated with the personnel mobility system; as well as The system controller module is configured to receive status data indicating the operating parameters obtained from the sensor data via a first communication channel, apply the status data as input to generate an AI model, and identify abnormal conditions in the personnel mobility device system. The system controller module is configured to transmit instruction data to the mobile device via a second communication channel, wherein the instruction data includes: One or more of the following: A written instruction to identify and locate the anomaly; or The data used to identify the abnormal situation includes one or more of the following: audio data; image data; or video data; and Corrective measures to address the aforementioned abnormal situation.

2. The system as claimed in claim 1, wherein, The system controller module is configured to transmit alarm data to the mobile device when an abnormal situation is identified and the instruction data is sent separately.

3. The system as described in claim 1, wherein, The guidance data further identifies the location of the anomaly.

4. The system as described in claim 3, wherein, The personnel mobility system includes a personnel mobility device, which is an escalator, elevator, or mobile platform, and the status data indicates the operating status of the personnel mobility device.

5. The system as described in claim 4, wherein, The personnel mover is the elevator, the personnel mover system includes the elevator machine, and the status data indicates the operating status of the elevator machine.

6. The system of claim 4, wherein, The system controller module is configured to switch the personnel mover system to maintenance mode and instruct the personnel mover to the location of the abnormal situation, thereby transporting technicians to the location of the abnormal situation to maintain the personnel mover system.

7. The system of claim 4, wherein, The mobile device is a mobile phone or peripheral device configured to process multimodal data, including one or more of text, sound, images, and video in the instruction data, for transmitting the instruction data as augmented reality to a technician via the mobile device.

8. The system of claim 6, wherein, When the personnel mobility system is in the maintenance mode, the controller module repeatedly receives and processes the status data, generates updates to the instruction data, and transmits the updated instruction data to the mobility device until the controller module determines that the status data indicates that the operating parameters of the personnel mobility system are within a predetermined normal operating range.

9. The system of claim 1, wherein: The generated AI model is trained on one or more of the following data: debugging data, heartbeat data, code version update data, and maintenance data.

10. The system of claim 1, wherein: The sensors include one or more of a speed sensor, a vibration sensor, a load sensor, a door operation sensor, and a health sensor, wherein the health sensor is configured to sense mechanical faults in the personnel mobility system; and The first communication channel is a wired or wireless channel, and the second communication channel is a wireless channel that is the same as or different from the first communication channel.

11. A method for resolving technical anomalies in a personnel mobility system, comprising: The system controller module receives status data indicating operating parameters associated with the personnel mobility device system via a first communication channel, the status data being obtained by sensors configured to sense the operating parameters; The system controller module takes the status data as input and applies it to generate an AI model to identify abnormal conditions in the personnel mobility device system; and The system controller module transmits instruction data to the mobile device via a second communication channel. The instruction data includes: One or more of the following: A written instruction to identify and locate the anomaly; or The data used to identify the abnormal situation includes one or more of the following: audio data; image data; or video data; and Corrective measures to address the aforementioned abnormal situation.

12. The method of claim 11, further comprising transmitting alarm data to the mobile device by the system controller module when the abnormal condition is identified, wherein the transmission of the alarm data is separate from the transmission of the instruction data.

13. The method of claim 11, wherein, The instruction data further identifies the location of the abnormal situation.

14. The method of claim 13, wherein, The personnel mobility system includes a personnel mobility device, which is an escalator, elevator, or mobile platform, and the status data indicates the operating status of the personnel mobility device.

15. The method of claim 14, wherein, The personnel mover is an elevator, the personnel mover system includes an elevator machine, and the status data further indicates the operating status of the elevator machine.

16. The method of claim 14, further comprising switching the personnel mobility system to a maintenance mode by the system controller module and instructing the personnel mobility device to move to the location of the abnormal situation, thereby transporting a technician to the location of the abnormal situation to maintain the personnel mobility device system.

17. The method of claim 11, wherein, The mobile device is a mobile phone or peripheral device configured to process multimodal data, including one or more of the text, sound, image, and video in the instruction data, for transmitting the instruction data as augmented reality to a technician via the mobile device.

18. The method of claim 16, further comprising, when the personnel mobility system is in the maintenance mode, having the controller module repeatedly receive and process the status data, generate an update to the instruction data, and transmit the updated instruction data to the mobility device, until the controller module determines that the status data indicates that the operating parameters of the personnel mobility system are within a predetermined normal operating range.

19. The method of claim 11, wherein, The generated AI model is trained on one or more of the following data: debugging data, heartbeat data, code version update data, and maintenance data.

20. The method of claim 11, wherein: The sensor includes one or more of a speed sensor, a vibration sensor, a load sensor, a door operation sensor, and a health sensor, wherein the health sensor is configured to sense mechanical failures in the personnel mobility system. as well as The first communication channel is a wired or wireless channel, and the second communication channel is a wireless channel that is the same as or different from the first communication channel.

Images