An elevator operation multi-modal data processing method and device for implementing AI timing-free processing

Abstract

The application discloses a kind of processing method and device of elevator operation multi-modal data of AI timing processing implementation, through maintenance personnel identity authentication, elevator identification code binding, operation area NFC trigger, monitoring component automatic switching, complete four-level space alignment, and based on NTP unified time service carries out time window check, from source intercept personnel inconsistency, elevator mismatch, area error, component mismatch and pre-recorded supplement record and other abnormal data;Then adopt the method of nearest neighbor timestamp matching combined with frame interpolation completion to execute timing alignment, generate alignment dataset;Alignment dataset is synchronously mapped to the same shared screen picture according to standardized fixed mapping rule;Same screen picture is used as the input of AI big model, so that AI big model is exempted from any timing processing.The present application completely preposes space-time credible verification and cross-modal alignment, greatly reduces the AI big model computing power cost and discriminant time delay, and is suitable for elevator operation batch intelligent supervision scene.

Description

Technical Field

[0001] This invention relates to the field of operational data processing, and specifically to a method and apparatus for processing multimodal elevator operational data that enables AI-based time-series-free processing. Background Technology

[0002] Elevator maintenance quality is directly related to public safety, and relevant laws and regulations require that the maintenance process be traceable and the actions verifiable. However, elevator maintenance data has typical heterogeneous and multimodal characteristics, including elevator identification text, maintenance work order text, IoT signals, and on-site operation videos, which are scattered across different systems. To achieve intelligent judgment of maintenance compliance, the problem of time-series synchronization and alignment of cross-modal data must first be solved.

[0003] Currently, existing technologies generally adopt the technical approach of "AI inference side completing temporal alignment and fusion". After data collection, only routine preprocessing (such as noise reduction and format normalization) is performed, and then the data is input into a large AI model, which completes temporal feature extraction, cross-modal alignment, feature fusion and violation detection.

[0004] The aforementioned existing technologies expose a long-standing, entrenched understanding in the field: the temporal synchronization and alignment of multimodal heterogeneous data must rely on AI models to complete this during the inference phase. Preprocessing can only achieve noise reduction and format standardization for single data points, failing to achieve accurate spatiotemporal alignment across modalities. This understanding forces the AI ​​to simultaneously undertake the dual tasks of temporal alignment and compliance judgment, resulting in high computational costs, significant judgment latency (the average end-to-end latency for a single elevator reaches hundreds of milliseconds to over one second, and increases linearly with the number of elevators), and unstable alignment accuracy, making it difficult to meet the stringent legal requirements for special equipment regulation. Therefore, a new solution is urgently needed that can fundamentally reconstruct the multimodal data processing workflow and free AI from the burden of temporal processing. Summary of the Invention

[0005] To address the problems existing in the prior art, the purpose of this invention is to provide a method and apparatus for processing multimodal data of elevator operations that enables AI to perform time-series processing-free processing, thereby reducing AI computing costs and discrimination latency.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for processing multimodal elevator operation data without time-series processing using AI includes the following steps: Step 1, Data Acquisition: Maintenance personnel log in to the work terminal to complete identity authentication and obtain maintenance personnel identity information as well as maintenance items and process text data corresponding to the task work order; The system triggers the scanning of elevator sensor identifiers to obtain elevator basic ID text data and initiates on-site video data collection for maintenance operations. Upon entering a specific work area, the work area identifier is obtained by triggering the sensor device identifier of that area. Based on the current maintenance project, the IoT monitoring terminal is automatically triggered to switch to the corresponding monitoring component and obtain the IoT monitoring signal data of that component; Step 2, Spatial Alignment: Perform the following checks sequentially. If any level fails to match, intercept and issue an alarm, terminating subsequent processing: Qualification Alignment: Verify whether the identity of the maintenance personnel matches the personnel specified in the task order; Elevator identification code alignment: Verify whether the elevator identification code carried in all data matches the target elevator; Work area alignment: Verify whether the work area triggered by NFC matches the work area required by the maintenance project; Monitoring component alignment: Verify whether the current IoT monitoring components match the components required to be monitored in the maintenance project; Step 3, Time Verification: Based on NTP unified time synchronization, verify whether the current job time is within the job time window specified in the task order. If not, intercept and issue an alarm. Step 4, Spatiotemporal Joint Filtering and Triage: Data that passes the verification in Steps 2 and 3 is determined to be compliant data and proceeds to subsequent processing; data that fails the verification is subject to abnormal triage, including real-time alarms and encrypted archiving. Step 5, Timing Alignment: Using a unified timestamp as the time anchor point and the video stream timeline as the main reference axis, the data streams with different acquisition frequencies are aligned to the same timeline by using nearest neighbor timestamp matching combined with frame interpolation to generate an aligned dataset. Step 6, Synchronous Output of Screen Mapping: The aligned dataset is synchronously mapped to the same shared screen according to the standardized fixed mapping rules that correspond one-to-one with the data modal, so that all data in the screen retains the unified elevator identification code and timestamp identifier, and maintains a strict spatiotemporal synchronization state; the shared screen image with completed screen mapping and its unified identifier are synchronously output as input data for the AI ​​large model.

[0007] In step 2, the work area includes the elevator machine room, car top, car, and pit. Area identification is achieved by deploying encrypted sensing devices in different areas. The monitoring component includes at least one of a brake, a gantry crane, and a frequency converter; In step 3, NTP unified time synchronization uses an Internet NTP server or an intranet NTP service.

[0008] In step 4, the abnormal traffic handling includes: pushing the data that fails the verification to the regulatory end for alarm in real time, and encrypting and archiving it for storage, without participating in subsequent alignment and screen mapping.

[0009] In step 5, the specific method of nearest neighbor timestamp matching is as follows: for each video frame, within a preset time window before and after it, search for the text data record and IoT monitoring signal data record with the closest timestamp and associate them; The specific method of frame interpolation completion is as follows: if no matching IoT monitoring signal data is found within the time window, the two most recent valid signal sampling points before and after the timestamp of the target video frame are taken, and the completion value of the video frame is calculated by linear interpolation according to the time ratio. Before being output to the same-screen mapping, the aligned dataset synchronously generates a hash certificate value and appends it to the end of the data packet.

[0010] In step 6, the standardized fixed layout of the shared screen includes at least: a main visual area for displaying video data, a device information area for displaying elevator basic ID text data, a work project area for displaying maintenance project and process text data, and an IoT data area for displaying IoT monitoring signal data.

[0011] When multiple elevators need to be monitored in batches, step 6 further includes: setting the shared screen image as a grid cell matrix, with each grid cell corresponding to one elevator, each elevator independently executing steps 1 to 5 to generate its own alignment dataset, and mapping each alignment dataset to the corresponding grid cell, with all grid cells on the screen maintaining spatiotemporal synchronization through a globally unified timestamp.

[0012] A multimodal data processing device for elevator operations that enables AI-based time-series processing, comprising: The data acquisition module is used for maintenance personnel to log in to the work terminal to complete identity authentication and obtain maintenance personnel identity information and maintenance project and process text data corresponding to the task work order; to trigger the acquisition of elevator basic ID text data and start the collection of on-site video data of maintenance operation by scanning the elevator sensor device identifier; to trigger the acquisition of work area identifier by the sensor device identifier when entering a specific work area; and to automatically trigger the IoT monitoring terminal to switch to the corresponding monitoring component to obtain the IoT monitoring signal data of that component according to the current maintenance project. The spatial alignment module, connected to the data acquisition module, is used to sequentially perform entity qualification alignment verification, elevator identification code alignment verification, work area alignment verification, and monitoring component alignment verification. If any level does not match, it will be blocked and an alarm will be triggered. The time verification module, connected to the spatial alignment module, is used to verify whether the current job time is within the job time window specified in the task order based on NTP unified time synchronization. If it is not, it will be intercepted and an alarm will be triggered. The spatiotemporal filtering and diversion module, connected to the time verification module, is used to determine the data that passes spatial alignment and time verification as compliant data and proceed to subsequent processing, while the data that fails verification is diverted and handled as abnormal data. The timing alignment module, connected to the spatiotemporal filtering and splitting module, is used to align data streams with different acquisition frequencies to the same time axis using a unified timestamp as the time anchor point and the video stream time axis as the main reference axis. It employs a method of nearest neighbor timestamp matching combined with frame interpolation to generate an aligned dataset. The screen mapping module, connected to the time alignment module, is used to synchronously map the aligned dataset to the same shared screen according to a standardized fixed mapping rule that corresponds one-to-one with the data modality, so that all data in the screen retains a unified elevator identification code and timestamp identifier, and maintains a strict spatiotemporal synchronization state. The output module, connected to the screen mirroring module, is used to synchronously output the shared screen image after screen mirroring and its unified identifier, as input data for the AI ​​large model.

[0013] In the spatial alignment module, the work area alignment unit identifies the machine room, car top, car, and pit areas based on the encrypted sensing device identifier; the monitoring component alignment unit is used to verify whether the current IoT monitoring component matches the component required to be monitored by the maintenance project.

[0014] The spatiotemporal filtering and diversion module is also used to push data that fails verification to the monitoring terminal for alarm in real time, and to encrypt and archive the data for storage. The device also includes a hash evidence module, which is used to synchronously generate hash evidence values ​​and append them to the end of the data packet before the aligned dataset is output to the same-screen mapping module.

[0015] For scenarios involving the batch monitoring of multiple elevators, the shared screen mapping module is also used to preset the shared screen image as a grid cell matrix, with each grid cell corresponding to one elevator, and to map the aligned dataset of each elevator to the corresponding grid cell.

[0016] By adopting the above scheme, this invention first uses four-level spatial alignment (subject qualification, elevator identification code, work area, and monitoring components) and NTP time verification to accurately identify "the person who should perform maintenance, the elevator that should be maintained, the area that must be operated, and the parts that must be operated" and the compliant time window from the data source, effectively intercepting abnormal data such as personnel mismatch, elevator mismatch, area error, component mismatch, and pre-recorded / supplemented data. Then, through nearest neighbor timestamp matching and frame interpolation completion, it synchronizes multimodal data of different frequencies to the same time axis, with an alignment error of ≤1 frame. Finally, through screen mapping, the spatiotemporally synchronized data is rendered to the shared screen, allowing the AI ​​large model to directly read the data for compliance judgment without any time-series processing. This invention completely front-ends spatiotemporal reliable verification and cross-modal alignment, significantly reducing the computing power cost and judgment latency of the AI ​​large model, and meeting the legal requirements for batch intelligent supervision of elevator operations.

[0017] Furthermore, for multi-elevator batch monitoring scenarios, the data of all elevators is integrated into the same grid matrix screen. AI can capture the entire screen at once and complete the compliance judgment of all elevators in parallel. The processing latency does not increase linearly with the number of elevators, which completely solves the computing power bottleneck of existing technologies in city-level large-scale monitoring. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the device principle of the present invention. Detailed Implementation

[0019] This invention discloses a method for processing multimodal elevator operation data without time-series processing in AI. In the data acquisition stage, it acquires maintenance personnel identities, elevator basic IDs, work area identifiers, and IoT monitoring component signals through multi-level acquisition. Subsequently, it performs four-level spatial alignment: subject qualification alignment, elevator identification code alignment, work area alignment, and monitoring component alignment, along with time verification using NTP unified time synchronization. Spatiotemporal joint filtering ensures the reliability of the data source. Then, using the video stream timeline as the main reference axis, it performs temporal alignment using nearest neighbor timestamp matching combined with frame interpolation to generate an aligned dataset with an alignment accuracy ≤ 1 frame. The aligned dataset is rendered onto the same shared screen according to standardized fixed mapping rules, ensuring strict spatiotemporal synchronization of all data within the screen. This screen directly serves as input to the AI ​​large-scale model, allowing the AI ​​large-scale model to read the synchronized data for compliance judgment without performing any further temporal feature extraction or cross-modal alignment calculations. The invention will be further described in detail below with specific implementation details.

[0020] Step 1: Data Acquisition.

[0021] (1) Maintenance personnel log in to the work terminal to complete identity authentication and obtain maintenance personnel identity information and maintenance project and process text data corresponding to the task work order.

[0022] (2) By scanning the elevator sensor device identifier (e.g., QR code or NFC), the elevator basic ID text data is obtained, and the on-site video data collection for maintenance operations is initiated.

[0023] (3) When entering a specific work area (such as the machine room, car top, car, pit), the work area identifier is obtained by triggering the encrypted sensing device identifier (such as NFC tag) deployed in that area.

[0024] (4) Based on the current maintenance project, automatically trigger the IoT monitoring terminal to switch to the corresponding monitoring component (such as brake, door machine, frequency converter, etc.) and obtain the IoT monitoring signal data of the component.

[0025] The above four types of data (elevator basic ID text data, maintenance project and process text data, IoT monitoring signal data, and maintenance operation site video data) each carry an elevator identification code (or can be indirectly obtained through binding relationship) and a collection timestamp during collection, and are aggregated to the processing end in the form of independent data streams.

[0026] Step 2, spatial alignment.

[0027] The following four levels of checks are performed sequentially. If any level fails to match, the process is blocked and an alarm is triggered, terminating subsequent processing: (1) Qualification Alignment: Verify whether the identity of the maintenance personnel matches the personnel specified in the task order. If they do not match, it is determined as "qualification mismatch", and the system is immediately blocked and an alarm is pushed.

[0028] (2) Elevator identification code alignment: Verify whether the elevator identification code carried in all data packets is consistent with the target elevator identification code. If they do not match, the data packet is temporarily stored in the queue to be reviewed, does not participate in subsequent processing, and triggers an alarm.

[0029] (3) Work Area Alignment: Verify whether the work area triggered by NFC matches the work area required by the maintenance project. For example, if the maintenance project is "brake inspection", the work area is required to be the machine room; if it is "car lighting inspection", the area is required to be the car. If they do not match, it is determined to be "non-compliant area data", and the data is intercepted and an alarm is triggered.

[0030] (4) Monitoring component alignment: Verify whether the current IoT monitoring components match the components required to be monitored in the maintenance project. For example, the brake detection project should match the brake circuit sensor, and the gantry crane commissioning project should match the gantry crane inverter signal. If they do not match, it is determined as "component mismatch", and the system is blocked and an alarm is triggered.

[0031] Once spatial alignment is complete, all data to be processed is locked to the same operator, elevator, work area, and monitoring component, laying a solid spatial foundation for subsequent processing.

[0032] Step 3: Time verification.

[0033] Based on NTP unified time synchronization (which can use an internet NTP server or an intranet NTP service), obtain the current accurate timestamp. Verify whether the current job time is within the job time window specified in the task order (e.g., between the job start and end times specified in the task order). If the current time is not within the window (e.g., pre-recording or supplementary recording), it is determined to be abnormal data, and interception is performed with a real-time alarm pushed; if it is within the window, proceed to the next step.

[0034] Step 4: Spatiotemporal joint filtering and diversion.

[0035] Data that passes the verification in steps 2 and 3 will be deemed compliant and proceed to the next processing stage. Data that fails verification will be subject to anomaly diversion: alarm information will be pushed to the regulatory terminal in real time, and the abnormal data will be encrypted, archived, and stored for subsequent auditing, and will not be included in subsequent alignment and screen mapping.

[0036] Step 5: Timing alignment.

[0037] Time alignment is performed on compliant data. First, single-modal preprocessing is carried out: format verification, spatial noise reduction, and keyframe extraction are performed on video data; Kalman filtering and baseline calibration are performed on IoT monitoring signal data; and format standardization and duplicate data removal are performed on text data.

[0038] The purpose of timing alignment is to unify data streams with different acquisition frequencies onto the same time axis, ensuring that each set of data frames is strictly synchronized in the time dimension. The timing alignment method of this invention employs "nearest neighbor timestamp matching + frame interpolation completion," specifically as follows: Time anchor setting: A globally unified timestamp is used as the unique time anchor. Timestamp verification is performed on all data packets, and linear interpolation calibration is applied to timestamps that are obviously abnormal.

[0039] Reference axis selection: The time axis of the video stream is used as the primary reference axis. Since the video frame rate is higher than the sensor sampling frequency, using video frames as the reference ensures the highest temporal resolution, keeping the alignment error ≤ 1 frame.

[0040] Nearest neighbor matching: For each video frame, within a preset time window (e.g., ±50ms) before and after it, search for the entry with the closest timestamp in both text data records and IoT monitoring signal data records. If a matching record is found, the text or signal data is directly associated with that video frame.

[0041] Frame interpolation completion: If no matching IoT monitoring signal data is found within the time window (e.g., due to a large sensor sampling interval resulting in no sampling points within the window), linear interpolation is used for completion. Specifically: Take the two most recent valid signal sampling points before and after the timestamp of the target video frame. Using the value of the previous sampling point as a reference, add the difference between the values ​​of the two sampling points multiplied by (the difference between the timestamp of the target video frame and the timestamp of the previous sampling point), divided by the difference between the timestamps of the next sampling point and the previous sampling point. The result is the completed value for that video frame moment. Missing text data is less common; if missing and unable to be interpolated, it is marked as empty.

[0042] Alignment accuracy: After the above matching and completion, ensure that each set of data frames (video frame + corresponding text field + corresponding sensor value) has the exact same timestamp (or the same interpolated time base). The alignment error does not exceed the time interval between adjacent video frames (≤1 frame, approximately 33ms). After the temporal alignment is completed, generate an aligned dataset that is uniquely bound to the target elevator and is spatiotemporally synchronized with all data.

[0043] In addition, to ensure the integrity of the evidence chain for regulatory enforcement, a hash evidence value (such as SHA-256) can be generated synchronously before the aligned dataset is output to the same screen mapping, and appended to the end of the data packet for subsequent anti-tampering verification.

[0044] Step 6: Simultaneous output via screen mirroring.

[0045] The aligned datasets are synchronously mapped onto the same shared screen according to predetermined mapping rules. These mapping rules are standardized, fixed layouts that correspond one-to-one with the data modalities, and include at least the following areas: the main visual area displays video data, typically occupying about 70% of the central screen area; the equipment information area displays elevator basic ID text data, typically located in the upper left corner; the work project area displays maintenance project and process text data, typically located in the vertical area on the right side of the screen; and the IoT data area displays IoT monitoring signal data (real-time values ​​and trend waveforms), typically located in the horizontal area at the bottom of the screen. All data mapped to the screen retains the original elevator identification code and timestamp. Because the input source data has already undergone spatial and temporal alignment, the data in all areas of the screen share the same elevator entity and the same time base, maintaining strict spatiotemporal synchronization. The mapping rules are pre-stored in a configuration file for subsequent AI large-scale model reading to locate the corresponding data area. The mapping rules remain consistent across different display terminals (local screen of the work terminal, large screen of the monitoring center, web terminal, mobile terminal) to ensure the universality of AI reading parameters.

[0046] When multiple elevators (e.g., N elevators, N≥2) need to be monitored in batches, the shared screen uses a pre-defined grid of cells (e.g., a 10×10 grid matrix), with each cell corresponding to a unique elevator. Each elevator independently executes steps 1 to 5 to generate its own aligned dataset; the aligned dataset is then mapped to the corresponding grid cell. All grid cells on the screen maintain spatiotemporal synchronization through a globally unified timestamp, allowing direct comparison of data between any two grid cells in the time dimension.

[0047] The shared screen image, along with its unified identifier, is synchronously output after screen mirroring and used as input data for the AI ​​model. Since the input data has undergone four levels of spatial alignment (entity qualification, elevator identification code, work area, and monitoring components) and time verification, it ensures a high degree of matching between the operators, elevators, work areas, monitoring components, and maintenance projects from the source, effectively blocking invalid data from non-compliant personnel, unauthorized elevators, incorrect areas, and incorrect components. Simultaneously, time-series alignment has achieved precise synchronization of cross-modal data, eliminating the need for the AI ​​model to perform any time-series feature extraction or cross-modal alignment calculations. It can directly read the synchronized screen image for compliance determination, significantly reducing computational overhead and judgment latency. Actual testing shows that the end-to-end processing latency for a single elevator, from data acquisition to outputting the screen image, is consistently within 200ms. For multi-elevator batch monitoring scenarios, the AI ​​model can capture the entire screen image at once and complete the compliance determination for all elevators in parallel, with processing latency not increasing linearly with the number of elevators.

[0048] like Figure 2 As shown, corresponding to the above method, the present invention also discloses a multimodal data processing device for elevator operations that enables AI-based time-series processing, comprising the following modules: The data acquisition module is used by maintenance personnel to log in to the work terminal to complete identity authentication and obtain maintenance personnel identity information and maintenance project and process text data corresponding to the task work order; to trigger the acquisition of elevator basic ID text data and start the collection of on-site video data of maintenance operation by scanning the elevator sensor device identifier; to trigger the acquisition of work area identifier by the sensor device identifier when entering a specific work area; and to automatically trigger the IoT monitoring terminal to switch to the corresponding monitoring component to obtain the IoT monitoring signal data of that component according to the current maintenance project.

[0049] The spatial alignment module, connected to the data acquisition module, is used to sequentially perform entity qualification alignment verification, elevator identification code alignment verification, work area alignment verification, and monitoring component alignment verification. If any level does not match, it will be blocked and an alarm will be triggered. Among them, the work area alignment verification is based on the encrypted sensing device identification to identify the machine room, car top, car, and pit areas. The monitoring component alignment verification is used to verify whether the current IoT monitoring component matches the component required to be monitored by the maintenance project.

[0050] The time verification module, connected to the spatial alignment module, is used to verify whether the current job time is within the job time window specified in the task order based on NTP unified time synchronization. If not, it will be intercepted and an alarm will be issued.

[0051] The spatiotemporal filtering and diversion module, connected to the time verification module, is used to determine the data that passes spatial alignment and time verification as compliant data and proceed to subsequent processing, and to handle the abnormal diversion of data that fails verification, including real-time push to the regulatory end for alarms and encrypted archiving and storage.

[0052] The timing alignment module, connected to the spatiotemporal filtering and splitting module, is used to align data streams with different acquisition frequencies to the same time axis using a unified timestamp as the time anchor point and the video stream time axis as the main reference axis. It employs a method of nearest neighbor timestamp matching combined with frame interpolation to generate an aligned dataset with an alignment error of ≤1 frame.

[0053] The screen mapping module, connected to the time-series alignment module, is used to synchronously map the aligned dataset to the same shared screen according to a standardized fixed mapping rule that corresponds one-to-one with the data modality. This ensures that all data within the screen retains a unified elevator identification code and timestamp, maintaining strict spatiotemporal synchronization. For scenarios involving batch monitoring of multiple elevators, the screen mapping module is also used to preset the shared screen as a grid cell matrix, with each grid cell corresponding to one elevator, and to map the aligned dataset of each elevator to the corresponding grid cell.

[0054] The output module, connected to the screen mirroring module, is used to synchronously output the shared screen image after screen mirroring and its unified identifier, as input data for the AI ​​large model.

[0055] Optionally, the device further includes a hash evidence module for synchronously generating hash evidence values ​​and appending them to the end of data packets before the aligned dataset is output to the same-screen mapping module.

[0056] In summary, this invention first uses four-level spatial alignment (subject qualification, elevator identification code, work area, and monitoring components) and NTP time verification to accurately identify "the person who should perform maintenance, the elevator that should be maintained, the area that must be operated, and the components that must be operated" and the compliant time window from the data source, effectively intercepting abnormal data such as personnel mismatch, elevator mismatch, area error, component mismatch, and pre-recorded / supplemented data. Then, through nearest neighbor timestamp matching and frame interpolation for temporal alignment, it synchronizes multimodal data of different frequencies to the same time axis, with an alignment error ≤1 frame. Finally, through screen mapping, the spatiotemporally synchronized data is rendered to a shared screen, allowing the AI ​​large model to directly read the data for compliance judgment without any temporal processing. This invention completely front-ends spatiotemporal reliable verification and cross-modal alignment, significantly reducing the computing power cost and judgment latency of the AI ​​large model, and meeting the legal requirements for batch intelligent supervision of elevator operations.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing multimodal elevator operation data without time-series processing using AI, characterized in that, Includes the following steps: Step 1, Data Acquisition: Maintenance personnel log in to the work terminal to complete identity authentication and obtain maintenance personnel identity information as well as maintenance items and process text data corresponding to the task work order; The system triggers the scanning of elevator sensor identifiers to obtain elevator basic ID text data and initiates on-site video data collection for maintenance operations. Upon entering a specific work area, the work area identifier is obtained by triggering the sensor device identifier of that area. Based on the current maintenance project, the IoT monitoring terminal is automatically triggered to switch to the corresponding monitoring component and obtain the IoT monitoring signal data of that component; Step 2, Spatial Alignment: Perform the following checks sequentially. If any level fails to match, intercept and issue an alarm, terminating subsequent processing: Qualification Alignment: Verify whether the identity of the maintenance personnel matches the personnel specified in the task order; Elevator identification code alignment: Verify whether the elevator identification code carried in all data matches the target elevator; Work area alignment: Verify whether the work area triggered by NFC matches the work area required by the maintenance project; Monitoring component alignment: Verify whether the current IoT monitoring components match the components required to be monitored in the maintenance project; Step 3, Time Verification: Based on NTP unified time synchronization, verify whether the current job time is within the job time window specified in the task order. If not, intercept and issue an alarm. Step 4, Spatiotemporal Joint Filtering and Triage: Data that passes the verification in Steps 2 and 3 is determined to be compliant data and proceeds to subsequent processing; Data that fails verification is handled through anomaly routing, including real-time alerts and encrypted archiving; Step 5, Timing Alignment: Using a unified timestamp as the time anchor point and the video stream timeline as the main reference axis, the data streams with different acquisition frequencies are aligned to the same timeline by using nearest neighbor timestamp matching combined with frame interpolation to generate an aligned dataset. The specific method for nearest neighbor timestamp matching is as follows: For each video frame, within a preset time window before and after it, search for and associate the text data record and IoT monitoring signal data record with the closest timestamps. The specific method of frame interpolation completion is as follows: if no matching IoT monitoring signal data is found within the time window, the two most recent valid signal sampling points before and after the timestamp of the target video frame are taken, and the completion value of the target video frame is calculated by linear interpolation according to the time ratio. Before the aligned dataset is output to the same-screen mapping, a hash certificate value is synchronously generated and appended to the end of the data packet; Before time alignment, single-modal preprocessing is performed: format verification, spatial noise reduction, and key frame extraction are performed on video data; Kalman filtering and baseline calibration are performed on IoT monitoring signal data; and format standardization and duplicate data removal are performed on text data. Step 6, Synchronous Output of Screen Mapping: The aligned dataset is synchronously mapped to the same shared screen according to the standardized fixed mapping rules that correspond one-to-one with the data modal, so that all data in the screen retains the unified elevator identification code and timestamp identifier, and maintains a strict spatiotemporal synchronization state; the shared screen image with completed screen mapping and its unified identifier are synchronously output as input data for the AI ​​large model. When multiple elevators need to be monitored in batches, step 6 further includes: setting the shared screen image as a grid cell matrix, with each grid cell corresponding to one elevator, each elevator independently executing steps 1 to 5 to generate its own alignment dataset, and mapping each alignment dataset to the corresponding grid cell, with all grid cells on the screen maintaining spatiotemporal synchronization through a globally unified timestamp.

2. The method for processing multimodal elevator operation data without time-series processing according to claim 1, characterized in that, In step 2, the work area includes the elevator machine room, car top, car, and pit. Area identification is achieved by deploying encrypted sensing devices in different areas. The monitoring component includes at least one of a brake, a gantry crane, and a frequency converter; In step 3, NTP unified time synchronization uses an Internet NTP server or an intranet NTP service.

3. The method for processing multimodal elevator operation data without time-series processing according to claim 1, characterized in that, In step 4, the abnormal traffic handling includes: pushing the data that fails the verification to the regulatory end for alarm in real time, and encrypting and archiving it for storage, without participating in subsequent alignment and screen mapping.

4. The method for processing multimodal elevator operation data without time-series processing according to claim 1, characterized in that, In step 6, the standardized fixed layout of the shared screen includes at least: a main visual area for displaying video data, a device information area for displaying elevator basic ID text data, a work project area for displaying maintenance project and process text data, and an IoT data area for displaying IoT monitoring signal data.

5. A multimodal data processing device for elevator operations that enables AI-based time-series processing, characterized in that, include: The data acquisition module is used for maintenance personnel to log in to the work terminal to complete identity authentication and obtain maintenance personnel identity information and maintenance project and process text data corresponding to the task work order; to trigger the acquisition of elevator basic ID text data and start the collection of on-site video data of maintenance operation by scanning the elevator sensor device identifier; to trigger the acquisition of work area identifier by the sensor device identifier when entering a specific work area; and to automatically trigger the IoT monitoring terminal to switch to the corresponding monitoring component to obtain the IoT monitoring signal data of that component according to the current maintenance project. The spatial alignment module, connected to the data acquisition module, is used to sequentially perform entity qualification alignment verification, elevator identification code alignment verification, work area alignment verification, and monitoring component alignment verification. If any level does not match, it will be blocked and an alarm will be triggered. The time verification module, connected to the spatial alignment module, is used to verify whether the current job time is within the job time window specified in the task order based on NTP unified time synchronization. If it is not, it will be intercepted and an alarm will be triggered. The spatiotemporal filtering and diversion module, connected to the time verification module, is used to determine the data that passes spatial alignment and time verification as compliant data and proceed to subsequent processing, while the data that fails verification is diverted and handled as abnormal data. The timing alignment module, connected to the spatiotemporal filtering and splitting module, is used to align data streams with different acquisition frequencies to the same time axis using a unified timestamp as the time anchor point and the video stream time axis as the main reference axis. It employs a method of nearest neighbor timestamp matching combined with frame interpolation to generate an aligned dataset. The specific method for nearest neighbor timestamp matching is as follows: For each video frame, within a preset time window before and after it, search for and associate the text data record and IoT monitoring signal data record with the closest timestamps. The specific method of frame interpolation completion is as follows: if no matching IoT monitoring signal data is found within the time window, the two most recent valid signal sampling points before and after the timestamp of the target video frame are taken, and the completion value of the target video frame is calculated by linear interpolation according to the time ratio. Before time alignment, single-modal preprocessing is performed: format verification, spatial noise reduction, and key frame extraction are performed on video data; Kalman filtering and baseline calibration are performed on IoT monitoring signal data; and format standardization and duplicate data removal are performed on text data. The hash evidence module is used to synchronously generate hash evidence values ​​and append them to the end of the data packet before the aligned dataset is output to the same screen mapping module; The screen mapping module, connected to the time alignment module, is used to synchronously map the aligned dataset to the same shared screen according to a standardized fixed mapping rule that corresponds one-to-one with the data modality, so that all data in the screen retains a unified elevator identification code and timestamp identifier, and maintains a strict spatiotemporal synchronization state. For scenarios involving the batch monitoring of multiple elevators, the shared screen mapping module is also used to preset the shared screen image as a grid cell matrix, with each grid cell corresponding to one elevator, and to map the aligned dataset of each elevator to the corresponding grid cell. All grid cells on the screen maintain spatiotemporal synchronization through a globally unified timestamp. The output module, connected to the screen mirroring module, is used to synchronously output the shared screen image after screen mirroring and its unified identifier, as input data for the AI ​​large model.

6. The elevator operation multimodal data processing device for realizing AI-based time-series processing according to claim 5, characterized in that, In the spatial alignment module, the work area alignment unit identifies the machine room, car top, car, and pit areas based on the encrypted sensing device identifier; the monitoring component alignment unit is used to verify whether the current IoT monitoring component matches the component required to be monitored by the maintenance project.

7. The multimodal data processing device for elevator operation that achieves AI-based time-series-free processing according to claim 5, characterized in that, The spatiotemporal filtering and diversion module is also used to push data that fails verification to the regulatory end for alarm in real time, and to encrypt and archive the data for storage.

Images