Elevator inspection process supervision system based on multi-source information fusion

Abstract

The invention belongs to the technical field of elevator supervision and inspection and artificial intelligence crossing, and provides an elevator inspection process supervision system. The system is composed of a local monitoring terminal, a data and algorithm supporting platform and a supervision and inspection interaction system. By fusing the face information of the inspector, the nameplate information of the inspected elevator, the Beidou positioning information of the local terminal equipment and the field monitoring video information, the system realizes the intelligent judgment of the execution compliance of the inspection task, effectively ensures the correct execution of the inspection task, and avoids the illegal behaviors such as missing inspection, counterfeiting and the like. And more convenience is provided for the backtracking inspection process through information archiving and series connection of multi-source information, and the efficiency is improved. According to the system, the intelligent and automatic level of elevator supervision and inspection is remarkably improved, the inspection efficiency is greatly improved by means of collaborative application of multi-source data fusion and an intelligent algorithm, the risks of behaviors such as inspection counterfeiting and inspection result tampering are reduced, meanwhile, the error rate and cost caused by manual intervention are reduced, and the system is suitable for popularization and application. The method is especially suitable for supervision and inspection requirements of large-scale elevator clusters.

Description

Technical Field

[0001] This invention addresses the intersection of elevator inspection and artificial intelligence, specifically a software and hardware system for monitoring elevator inspection compliance based on multi-source information fusion. Specifically, to address the requirements of elevator inspection tasks—supervising whether inspection activities are being carried out in accordance with specifications—this system integrates multi-source and multi-modal information, including Beidou positioning information, video streams, images, and computer time watermarks, to analyze and monitor the geographic location, identity, time, and data consistency and compliance of elevator inspection tasks. Background Art

[0002] Elevators, as a frequently used means of public transportation, have a direct bearing on the safety of people's lives and property. my country has established an elevator safety management system covering factory delivery, installation, regular inspections, and maintenance. Inspection, as a key component, should provide a safety net. However, in practice, due to insufficient oversight and outdated technology, the inspection process still presents risks such as opaque processes, inadequate execution, and non-compliant records. This has made the credibility of the inspection itself a new source of potential safety risks for elevators.

[0003] "Elevator supervision inspection" refers to the supervision of elevator inspection agencies during installation, commissioning, and regular inspections by government special equipment supervisory agencies, in accordance with relevant regulations on elevator supervision inspection. This is done to standardize elevator inspections and ensure safe operation. Specifically, at the elevator inspection site, supervisors must strictly follow standards and specifications to oversee the inspections and ensure that the elevators meet performance standards after standardized inspections.

[0004] The elevator inspection process monitoring system mentioned in this patent is a means of "supervising" the inspection process. It provides a remote, unmanned method for real-time monitoring, compliance verification, and full-process traceability of elevator inspection and maintenance operations. Unlike equipment status monitoring, it focuses not on the elevator equipment itself but rather on whether the inspection activities are being carried out in accordance with specifications.

[0005] At present, research in this field is still in its infancy, and existing achievements are mainly concentrated in the fields of elevator operation status monitoring and fault prediction. For example, sensor-based elevator operation data collection, health assessment and anomaly identification methods (such as Patent No.: CN202010403406 A video-assisted system for elevator inspection). There is a relative lack of systems and solutions for supervising the inspection process itself.

[0006] In most cities, supervision still relies primarily on manual inspections and paper records, resulting in heavy workloads, low supervisory efficiency, and a lack of process traceability. For example, one pilot area previously employed traditional record-keeping methods, with inspectors using paper documents and dashcam videos. This resulted in fragmented data and an average accident tracing time of 4.5 hours.

[0007] The few existing supervisory inspection systems focus on remote video display, digital traceability, and task scheduling of the elevator inspection process, focusing on workflow management and digital record keeping (e.g., Patent No. CN202323283732, an elevator inspection system based on AR technology). However, they lack the ability to supervise the standardization of inspection work. Some studies have introduced image recognition to assist in inspection process analysis (e.g., Patent No. CN202410720791, an artificial intelligence inspection and testing method for elevators). However, these studies focus solely on AI-assisted automated determination of key inspection items, without forming a system-level inspection supervision plan or considering intelligent supervision of the standardization and authenticity of inspection behavior.

[0008] In general, as the core link to ensure the safe operation of elevators, there are many problems with the current technical means and implementation models of elevator inspection supervision: 1) Supervision behavior is heavily dependent on manual labor, and the vast majority of government special equipment supervision agencies still use manual on-site inspections and paper records as the main means. Supervisors need to visit the inspection site in person to verify the standardization and authenticity of operations, which limits the scope of supervision and makes it difficult to cover large-scale elevator groups; 2) There is a lack of supervision on the standardization of inspection behavior, and there is a lack of verification of key nodes in the inspection process (such as personnel identity, equipment positioning, and operation time). It is impossible to verify whether the inspection actually occurred in the designated elevator and at the designated time, and whether the inspector is the designated person, which may lead to violations such as fraud and tampering. 3) There is a lack of archiving of inspection tasks. After the inspection task is completed, there is no systematic archiving and storage of the inspection process data, making it difficult to trace back the inspection process later.

[0009] Therefore, there is an urgent need to build a digital elevator inspection process compliance supervision system to achieve remote, unmanned supervision of the standardization and authenticity of the entire elevator inspection process. The present invention is a set of elevator inspection process supervision systems designed based on Beidou satellite positioning, 5G digital networks, and machine vision algorithms. It integrates Beidou positioning information, facial video streams, elevator nameplate images, computer time watermarks, and other multi-source information to monitor the geographic location, identity, time, and data consistency and compliance of elevator inspection tasks, establish a traceable electronic archive system from task release, inspection execution to result archiving, and form a closed-loop supervision system to prevent cheating and tampering; utilize 5G networks to transmit multi-terminal high-definition video streams to build a "multi-terminal full-process remote monitoring" supervision paradigm; and use web technology to support synchronous remote supervision of multiple computer rooms in large urban areas. Summary of the Invention

[0010] The present invention constructs a compliance supervision system for elevator inspection processes based on multi-source information fusion. The system integrates multimodal technology to achieve the following core functions: 1) Comprehensively judge the compliance of inspection processes and data based on multi-source information such as Beidou positioning information, image and video data, and computer time watermarks; 2) Utilize 5G wireless video communication technology to build a new supervision model of "multi-terminal full-process remote monitoring"; 3) Rely on cloud computing and Web software technology to support synchronous remote supervision of multiple computer rooms in large urban areas and multi-task process data archiving.

[0011] The elevator inspection process supervision system based on multi-source information fusion provided by the present invention is composed of a local monitoring terminal, a data and algorithm support platform, and a supervision and inspection interactive system.

[0012] The local monitoring terminal is responsible for real-time monitoring of the elevator inspection site using both global and first-person perspectives. The terminal equipment includes a 5G surveillance camera, several 5G safety helmets, and a handheld 5G recorder. The 5G surveillance camera collects global video streams of the elevator inspection site, while the 5G safety helmets worn by inspectors collect localized video streams. Both the 5G surveillance camera and the safety helmets are equipped with Beidou and GPS dual-mode positioning modules to capture the equipment's latitude and longitude location information. The handheld 5G recorder allows inspectors to capture videos or photos of key inspection items. The local monitoring terminal collects real-time on-site video, photos, and location information, and uploads it to the data and algorithm support platform via the 5G wireless communication network.

[0013] The data and algorithm support platform consists of four supporting modules: a data storage module, a video processing module, a Beidou positioning module, and a compliance verification module. The data storage module is responsible for receiving terminal device information data and inspection images for the current inspection task. The video processing module mainly uses the built video server to realize video reception, storage, and streaming. That is, it receives real-time video stream data from the terminal and pushes it to the upper-level supervisory inspection interactive system for screen display. At the same time, it records the video stream data in segments at regular intervals and stores it together with the inspection project video data in an object storage system indexed by the task. The Beidou positioning module is responsible for receiving Beidou positioning information transmitted by the local monitoring terminal and converting the data into a unified coordinate system to obtain the precise location coordinates of the equipment. The compliance verification module uses the data from the above three modules to perform machine vision-based facial recognition and elevator code recognition. In combination with computer time watermark data, it comprehensively determines the current inspection location, identity, time, and consistency and compliance of the inspection data to ensure inspection specifications and data authenticity.

[0014] The supervision and inspection interactive system consists of information management, task management, video center and data dashboard. Information management is responsible for managing information related to elevator supervision and inspection, such as information on the inspected elevator, inspection agency information, inspection personnel information and system user information. Task management is responsible for the creation, release, dynamic scheduling and status update of supervision and inspection tasks, and combines the compliance verification results to ensure the standardization of inspections. The video center displays multi-channel on-site video footage of all ongoing inspection tasks in real time (including the perspectives of control balls, safety helmets and handheld recorders), and archives historical inspection video data by task, supporting one-click backtracking inspection. The data dashboard summarizes and displays the list of all ongoing tasks and core status information (abnormal alarms, supervision statistics, etc.) in real time, and integrates the geographic information system to dynamically present the spatial distribution and location details of inspection tasks, providing supervisors with global command and decision-making support. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 : A schematic structural diagram of the system of the present invention;

[0016] Figure 2 : Flowchart of compliance determination of the system of the present invention;

[0017] Figure 3 : Geographic location consistency verification flow chart;

[0018] Figure 4 : Flowchart of identity verification of inspectors and inspected elevators;

[0019] Figure 5 : Flowchart for authenticity and standardization verification; DETAILED DESCRIPTION

[0020] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0021] The system of the present invention consists of three parts: supervision and inspection interactive system 3, data and algorithm support platform 2, local monitoring terminal 1, such as Figure 1 shown.

[0022] Local monitoring terminal 1 is responsible for collecting real-time video of the inspection site, Beidou positioning data for the inspection task, and capturing key images during the inspection process. After preliminary processing of the collected video and image data, the terminal transmits it to data and algorithm support platform 2, laying the data foundation for subsequent verification of the standardization of the inspection process and data authenticity. Data and algorithm support platform 2 is responsible for classifying and storing the multi-source and multi-modal data of terminal 1, and invoking the compliance determination algorithm to implement compliance verification. Its data and determination results are made available to the supervisory inspection interactive system 3. Supervisory inspection interactive system 3 calls the data and compliance verification results stored in platform 2 to establish a traceable electronic archive from task issuance, inspection execution, to result archiving, realizing remote visual supervision of inspection tasks and intelligent and accurate judgment of compliance.

[0023] From an interactive perspective, the local monitoring terminal 1 uploads real-time video, recorded video, image data, and Beidou positioning data to the data and algorithm support platform 2. After receiving the data, the platform performs preprocessing (including adding time watermarks) based on different task IDs and groups and classifies the data into corresponding modules: image data and its time watermark are stored in the data storage module 21, video data is stored in the video processing module 22, and Beidou positioning data and its processed results are stored in the Beidou positioning module 23. The compliance verification module 24 calls upon the elevator nameplate image data (including time watermarks) in the data storage module 21, the facial video data (including time watermarks) in the video processing module 22, the inspection item video file (including time watermarks), and the positioning data in the Beidou positioning module 23 to verify the geographic location, identity, time, and data consistency and compliance according to the compliance determination algorithm process. In the supervisory inspection interactive system 3, the video center 33 accesses the real-time video stream and historical video files from the video processing module 22 to visualize the inspection process and retrieve historical files. Task management 32 implements human-computer interaction for creating and publishing inspection tasks, dynamically scheduling, and tracking status. It also accesses the results of the compliance verification module 24 to ensure inspection compliance. The data dashboard 31 and information management 34 display data and statistically analyze inspection-related information based on the data and algorithm support platform 2.

[0024] Figure 2 This is a flow chart of the compliance determination algorithm of the system of the present invention. The system includes the following key processes:

[0025] S1. Geographic location consistency verification

[0026] A1. Inspection tasks are usually issued by supervisors. The specific process requires supervisors to select corresponding information from the list of inspected elevators (including the elevator's location coordinates), the list of inspection agencies, and the list of inspectors provided by the Information Management 31 module. After confirming the accuracy of the information at these key nodes one by one, they can finally create and issue the inspection task.

[0027] A2. After receiving the task, the inspector arrives at the task site and first turns on the device in the local monitoring terminal 1. When the device is turned on, the system will automatically execute the Beidou positioning verification process (such as Figure 3 As the control ball has high positioning accuracy and rarely moves after deployment, its position is used as the actual coordinate reference for the entire inspection mission and for Beidou positioning verification.

[0028] A3.Beidou positioning verification first requires coordinate conversion because different terminal devices use different coordinate systems (which need to be unified to the GCJ-02 coordinate system). For coordinate data derived from the WGS84 coordinate system, its nonlinear offset must be calculated in advance. The calculation formula for the latitude offset is:

[0029]

[0030] The longitude offset calculation formula is:

[0031]

[0032] Among them, Δx and Δy are the offsets between the coordinate longitude and latitude and the reference point (105°E, 35°N). After calculating the longitude and latitude offsets, it is necessary to perform ellipsoidal projection correction to convert the above offsets into angles. The conversion calculation formula is:

[0033]

[0034] Where a is the semi-major axis of the Earth's ellipsoid, e is the Earth's eccentricity, and other variables (θ, k, r) are converted using the following formula:

[0035]

[0036] Finally, add the original coordinates and the offset to get the converted coordinates:

[0037]

[0038] A4. After completing the coordinate conversion, the system will bind the task to the device: In the published task list, the task's specified coordinates (equivalent to the coordinates of the elevator under inspection in step A1) are fuzzy matched with the Beidou positioning coordinates of the enabled terminal device. If the offset between the coordinates of a task and the reference point coordinates returned by a control ball (obtained in step A4) is less than a preset threshold c1, the task is bound to the control ball, and the system prompts that the task's geographical location is consistent:

[0039] d((lat task ,lon task ),(lat device ,londevice ))<c1

[0040]

[0041] Among them, R is the radius of the earth (taken as 6371000 meters), lat r is the radian value corresponding to the latitude, Δlon r = is the arc value corresponding to the longitude difference. If the task fails to match any control spheres, it indicates that there are no pending tasks at the inspector's current location. The system prompts that the inspection task's geographic location is inconsistent and requires the inspector to go to the correct location and restart the Beidou positioning verification process. The 5G helmet device is bound to this task in the same way.

[0042] A5. After the task and device are bound, the system will implement electronic fence monitoring: the electronic fence will be set according to the task coordinate offset threshold set by the supervisor:

[0043]

[0044] Where lat is latitude, lon is longitude, and c is the threshold set by the supervisor. The Beidou positioning information returned by the control ball is continuously monitored until the mission is completed. If the returned coordinates exceed the geo-fence range, the system will record the mission location anomaly and trigger an alarm.

[0045] S2. Identity verification of inspectors and inspected elevators

[0046] Figure 4 Flowchart for identity verification of inspectors and inspected elevators.

[0047] B1. After the task's geographic location consistency verification is passed, the system will verify that the identities of the inspector and the elevator being inspected are consistent with the published task. The inspector uses a handheld 5G recorder to capture a video of their own face and a photo of the elevator nameplate, and uploads it through the Supervisory Inspection Interactive System 3.

[0048] B2. First, perform data validity verification: After the system receives the video and image data from B1, it extracts the data's time watermark (the extraction method varies depending on the device) and verifies the data's validity using the following formula to eliminate the risk of video and image failure or tampering.

[0049] T data ≠T exist &T end >T data >T start

[0050] Where T data is the data upload time, T exist is the time when the item was uploaded, T startis the task start time, T end The end time of the task.

[0051] B3. After verifying the data's validity, the system uses facial recognition algorithms (RetinaFace + ArcFace) to verify the identity of the inspector and character recognition algorithms (DBnet + CRNN) to parse the elevator nameplate information and verify the identity of the inspected elevator. The obtained personnel identity information is then compared with the inspected elevator number and the task release information.

[0052] B4. The system implements triple verification of the inspector and the inspected elevator's identity through steps B2 and B3, strictly ensuring that the current inspector's identity and the inspected elevator's number meet the task's specified objectives, thereby effectively preventing others from taking over and repeatedly inspecting the same elevator.

[0053] S3. Verification of authenticity and compliance

[0054] Figure 5 Verify the flow chart to test the authenticity and standardization of the project.

[0055] After C1.S1 and S2 are verified, the inspection task officially begins. The system automatically activates real-time video streaming on the terminal device. Supervisors can remotely monitor the inspection site in real time through the interactive system's video center 33. If any irregularities are found in the inspection process, supervisors can conduct remote video calls through the inspectors' 5G safety helmets to provide guidance on the on-site inspection.

[0056] C2. When inspecting each inspection item, the inspector needs to use a handheld 5G recorder to record the implementation process and the movement of the elevator equipment (such as the traction wheel, steel rope, buffer, etc.) into a video, and upload the video file through the interactive system 3.

[0057] C3. The system verifies the validity of the inspection data uploaded in C2 using the same method as in B2. This step ensures that each inspection item is truly and in real time, preventing violations such as duplicate uploads of inspection data from other elevators.

[0058] C4. Establish a time evidence chain for inspection items: The system accurately records and stores the specific time and data files of each inspection item, establishing an evidence chain indexed by the timeline of occurrence, effectively preventing missed inspections, fraud, and other behaviors, and also providing a reliable time basis for process backtracking.

[0059] C5. When all inspection tasks are completed, the system will automatically start the task information archiving process, organize and save various types of data generated during the entire inspection process according to the task ID, establish a timeline of key nodes of the inspection task, and provide a complete basis for one-click backtracking.

[0060] The above-mentioned compliance determination process significantly improves the accuracy of identity verification, process anti-counterfeiting capabilities and judgment efficiency. Its core lies in the construction of a strict data authenticity assurance mechanism: from the strict matching of personnel, organization, and elevator information in the task creation stage, to the on-site Beidou positioning, face recognition and elevator nameplate information triple verification, and then to the time watermark anti-counterfeiting embedded in all image data, layer by layer to ensure the authenticity and reliability of the data. At the same time, relying on the remote supervision capability achieved by real-time video push throughout the process, and the complete traceable evidence chain formed by automatically integrating all images, project status and process data after the task is completed, this process not only greatly improves the inspection efficiency, but also fully meets the statutory compliance requirements for special equipment inspection. Furthermore, in the retrospective link, by connecting the global and local data at the time of the inspection behavior (such as project playback and corresponding time period monitoring video), the system provides supervisors with a comprehensive perspective, significantly improving their efficiency in judging the compliance of inspection behaviors.

Claims

1. An elevator inspection process compliance supervision system based on multi-source information fusion, characterized by: (1) Integrate Beidou positioning information, facial video stream, elevator nameplate image, inspection video data, computer time watermark and other multi-source and multi-modal information to analyze and supervise the geographic location, identity, time and data consistency and compliance of elevator inspection tasks; (2) Utilize 5G networks to transmit multi-terminal high-definition video streams to realize the "multi-terminal full-process remote real-time monitoring" supervision paradigm; (3) Utilize web technology to support synchronous remote supervision of multiple computer rooms in large urban areas, establish a traceable electronic archive system from task release, inspection execution to result archiving, and form a closed-loop supervision to prevent cheating and tampering.

2. In the compliance determination process described in claim 1, the task geographic location consistency verification is characterized by: (1) obtaining the real coordinate reference of the inspection task through the Beidou positioning verification process; (2) realizing the pairing of the current task and the enabled device through task and device binding; (3) realizing real-time verification of the geographic location of the inspection process through electronic fence monitoring.

3. In the compliance determination process described in claim 1, the identity verification of the inspector and the inspected elevator is characterized by: (1) data validity verification, using a computer time watermark to verify the validity of the identity verification data; (2) implementing the identity verification of the inspector through a face recognition algorithm (RetinaFace+ArcFace); (3) implementing the identity verification of the inspected elevator by parsing the elevator nameplate information using a character recognition algorithm (DBnet+CRNN).

4. In the compliance determination process described in claim 1, the verification of the authenticity and standardization of the inspection items is characterized by: (1) remote real-time monitoring of the inspection site through multi-terminal video streams to promptly discover irregular behaviors and provide remote guidance and collaboration; (2) data validity verification of the uploaded inspection item data to ensure that each inspection item is indeed occurring in real time; (3) establishing a time evidence chain for the inspection items to prevent violations such as missed inspections and falsification.

Images