A method and system for video labeling, positioning and segment association of potential safety hazards along a railway

Abstract

The application discloses a railway safety hidden danger video marking, positioning and segment association method and system, and relates to the technical field of video monitoring data processing. The method comprises the following steps: establishing a video recording session and generating a session number; in response to a hidden danger type trigger operation during recording, calculating a relative millisecond time and generating a marking object, determining the start and end milliseconds of a segment; after stopping recording, batch binding the marking objects with empty video paths in the same session and updating them to a replayable state; drawing marking points on the track trajectory according to the line position ratio and supporting segment playback. The application realizes on-site synchronous hidden danger classification and time positioning, associates multiple hidden danger events through a session number, supports quick search based on spatial positions, and solves the technical problem of data loss caused by page mis-cutting in the prior art.

Description

Technical Field

[0001] This invention relates to the field of railway inspection informatization and video surveillance data processing technology, and in particular to a method and system for annotating, locating and associating video clips of safety hazards along railway lines. Background Technology

[0002] With the rapid increase in the operational mileage of high-speed railways in my country, the investigation and management of safety hazards in the external environment along railway lines has become a crucial aspect of ensuring railway transportation safety. Traditional railway safety hazard inspections primarily rely on manual patrols and post-incident registration: inspectors walk along the line carrying paper logs, recording hazards through written descriptions, hand-drawn diagrams, or individual photographs, and then entering the information into the system upon returning to the office. This model has several drawbacks: First, the time lag between the occurrence and recording of a hazard can easily lead to discrepancies in the description of the hazard's location, type, and on-site condition. Second, the separation of video capture and hazard registration means inspectors must review the entire video afterward to pinpoint the hazard, often taking more than 3 minutes to verify a single hazard. Third, a single video file may contain multiple hazard events, and the traditional method lacks a mechanism for structurally associating multiple hazards with the same video file, resulting in low video resource utilization and difficulty in retrieval. Fourth, in environments with weak or no network connectivity, data collected on-site cannot be uploaded in real time, easily leading to data loss or omissions.

[0003] To address the aforementioned issues, existing technologies have proposed various solutions for video surveillance and hazard management along railway lines. Dong Xuejuan (Railway Computer Applications, 2022, Vol. 31, No. 4, pp. 26-35, DOI: 10.3969 / j.issn.1005-8451.2022.04.05) proposed a high-speed railway infrastructure video big data application system. This system provides a structured analysis platform for surveillance videos, achieving structured organization and storage management through video data annotation, with intelligent video analysis for infrastructure anomaly alarms and closed-loop management as its core functions. However, Dong's solution focuses on the structured analysis of videos in the back-end monitoring center, neglecting a real-time acquisition mechanism for synchronously classifying hazards and associating video segments via mobile terminals at the inspection site, and also lacking session-level association management for multiple hazard events within the same video.

[0004] The railway protection system released by the China Security and Protection Industry Association on September 30, 2024, integrates high-point video surveillance, IoT AIoT monitoring, AI recognition, route map matching, and mobile APP response functions, enabling precise location and rapid response to incidents along railway lines. The system utilizes AR real-scene, edge computing, and AI recognition technologies to accurately perceive and locate the status of all elements, including people, vehicles, roads, animals, and the environment, and precisely matches the incident location information with the railway line map through route map matching. However, the system's route map matching function is primarily used to determine the geographical location of the incident and cannot perform rapid spatial location-based searches.

[0005] A company's "Railway Visualized Flood and Disaster Prevention Solution," released on September 27, 2025, employs augmented reality tagging technology, supporting hierarchical classification, search, and location of tags. However, the tagging in this solution involves overlaying AR tags onto pre-recorded video footage, which falls under the category of post-processing tagging.

[0006] In the field of general video processing, EU patent EP3306495B1 discloses a method for associating recorded video with highlight moments and event tags to facilitate playback services, but it does not address session number management, batch binding, and state machine mechanisms specific to railway scenarios. The OBS-TimeMarker open-source project provides the ability to mark content at the current time point when recording video in OBS Studio, but only for single time points. Hikvision's "Tag Playback" function allows recording information at a specific time point during playback, but this is a tag addition during the playback phase. Chinese invention patent CN112419226A discloses a railway external environment hazard monitoring system and method, including a video acquisition module, a real-scene image hazard labeling module, and a satellite image hazard labeling module, but its hazard labeling relies on post-event analysis of real-scene images and satellite images.

[0007] In summary, the existing technology has at least the following technical defects: 1. The lack of technical means to simultaneously classify hazards and locate the time of occurrence of hazards at the collection site leads to low efficiency in manually reviewing the entire video afterward; 2. The lack of a mechanism to associate multiple potential incidents occurring consecutively in the same recording session with the same video file leads to difficulties in video segment retrieval and low review efficiency; 3. There is a lack of algorithms to map hazard markings to spatial locations along the line, making it impossible to quickly retrieve hazards by location. 4. The lack of protection mechanisms against abnormal situations such as accidental page switching, camera malfunction, or network interruption during recording makes it easy to lose labeled data.

[0008] Therefore, there is an urgent need for a complete technical solution that can simultaneously classify, locate, spatially map, and associate segments of potential hazards during the collection of video footage from railway patrols. Summary of the Invention

[0009] To address the aforementioned deficiencies in existing technologies, this invention provides a method and system for video annotation, location, and segment association of safety hazards along railway lines. The aim is to simultaneously classify hazards and pinpoint their occurrence time at the on-site video collection stage during railway line inspections, avoiding the need for manual review of entire video segments and manual transcription, thereby significantly improving inspection efficiency and reducing omission rates. Furthermore, this invention addresses how to associate and manage multiple consecutive hazard events within the same recording session with the same video file while maintaining session consistency, enabling the same video file to be associated with multiple hazard events, each with independent timeline location, thus improving the structuring of video resources and retrieval and verification efficiency. It also addresses how to map hazard annotations to spatial locations along the railway line, enabling rapid retrieval and visualization based on spatial location, facilitating hazard distribution analysis by management personnel according to line sections. Finally, it addresses how to prevent data loss due to accidental page switching, camera malfunctions, or network interruptions during recording, ensuring continuous operation even in weak network environments and data synchronization after network recovery.

[0010] To achieve the above technical objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for video annotation, location, and segment association of safety hazards along railway lines, comprising the following steps: S1. Establish a video recording session: In response to the recording trigger operation, establish a video recording session, generate a session number to uniquely identify the video recording session, and record the recording start time; When a trigger operation for a preset hazard type is received while the system is not in recording mode, the trigger operation for that preset hazard type directly triggers the initiation of short video capture and establishes the video recording session. Step S1 generates and maintains session numbers through the session management module, ensuring that multiple labeled objects generated in the same video recording session carry the same session number, thereby guaranteeing accurate identification of all labeled objects belonging to the same session during subsequent batch binding. The session number is generated in the format of session-millisecond timestamp-random number, where the timestamp is taken as millisecond-level Date.now(), and the random number is taken as a local random sequence converted into a short string. This dual mechanism of timestamp and random number reduces the probability of number collisions. The label is generated in the format of marker-millisecond timestamp-random number. If a label with the same name is detected before writing, it is regenerated to ensure global uniqueness.

[0011] S2. Generate annotation objects During recording, in response to a trigger operation for a preset hazard type, the system obtains the current time, calculates the relative millisecond time between the current time and the recording start time, and generates a label object corresponding to the triggered hazard type. The label object includes at least a label identifier, hazard type, relative millisecond time, recorder, route start and end points, session number, video path field, segment start and end milliseconds, status field, and route position ratio. The formula for calculating the relative millisecond time is: elapsedMs = now - recordingStartAt, Where now is the millisecond-level timestamp of the moment the trigger operation occurred, and recordingStartAt is the recording start time recorded in step S1. The relative millisecond time provides a millisecond-level reference for the alignment of the annotation with the video timeline, enabling the positioning of subsequent video segments to have accurate time coordinates.

[0012] The start and end milliseconds of the video segment corresponding to the labeled object are determined based on the relative millisecond time: For labeled objects manually triggered during recording, the segment start millisecond is max(0, elapsedMs - 5000), and the segment end millisecond is elapsedMs + 5000, covering 5 seconds before and after the hazard occurs; for the first labeled object automatically created by triggering short video capture based on a preset hazard type, the segment start millisecond is 0, and the segment end millisecond is elapsedMs + 5000. The recording duration timer refreshes the interface display every 500ms, but the segment start and end values ​​are calculated using millisecond-level timestamps to ensure that the interface refresh frequency does not affect the labeling accuracy. Multiple labeled objects generated in the same video recording session carry the same session number, realizing a unified association between multiple hazard events and sessions.

[0013] S3, Batch Binding Video Paths After stopping recording and obtaining the video path of the video file, the annotation list is traversed, and annotation objects with session numbers equal to the session number of the current video recording session and empty video path fields are selected. The video path is written to each of the selected annotation objects in batches, and their status fields are updated to playable status. Then, the annotations are persisted.

[0014] The core of the batch binding logic is as follows: after recording stops, only annotations with sessionId equal to the current session and videoPath empty are selected as unassociated annotations. These records are written to the same videoPath and set to ready. Historical annotations with existing videoPaths are not overwritten, and annotations from other sessions are not bound, thus ensuring batch association of multiple potential issues within the same video session while avoiding cross-session misbinding and historical data overwriting. Persistent storage persists the annotation object array to the local key lcoas:video-markers, retaining a maximum of the most recent 30 records, and synchronizing them to the backend database with the same field structure. When the page is re-entered, the local annotation object array is read, and the latest record is selected as the default detail. If the local data format is abnormal, the local storage key is cleared to prevent erroneous records from affecting subsequent collection.

[0015] S4. Positioning and Segment Playback Marking points are drawn on the route trajectory according to the route position ratio of each marked object; in response to the selection operation of the marking point, the video path and segment start and end milliseconds of the corresponding marked object are read, the corresponding video segment in the video file is located, and playback is performed. The route position ratio is determined by an interval ratio mapping algorithm: obtaining the interval start mileage startMileage, interval end mileage endMileage, and marking mileage markerMileage, the calculation formula for the route position ratio is: positionPercent = clamp((markerMileage - startMileage) / (endMileage- startMileage) × 100, 0, 100), The `clamp` function restricts the calculation result to the range [0, 100]. Kilometer markers represented in the form of "Kx+y" are first converted to metric mileage in meters (x × 1000 + y meters) before being included in the calculation. When no reliable mileage is available, a temporary display ratio is generated between 8% and 92% according to the generation order of the marked objects to ensure that the interface can still express the marking order. The calculation is recalculated after the accurate mileage is obtained. When responding to playback operations, the video control is opened by selecting the currently selected marked object or the marked object with a non-empty field in the most recent video path. Playback begins from the start millisecond of the corresponding segment and automatically pauses when the playback progress reaches the end millisecond of the segment, thus only reviewing segments related to potential hazards.

[0016] In addition, this method also includes an interface locking step: When recording starts, the recording component sends a busy-change=true message to the parent page. If the current tab is the video recording tab and the busy status is true, the parent page will refuse to switch tabs and prompt "Please end the current video recording first". When recording ends, fails, or the recording status is reset, send `busy-change=false`. This interface lock only restricts tab switching and does not prevent the continued triggering of preset hazard types to add labeled objects during the recording process, thus ensuring the continuity of hazard recording while preventing accidental operation.

[0017] Secondly, this invention provides a system for labeling, locating, and associating video clips of safety hazards along railway lines, comprising the following modules: The video capture module is used to respond to recording trigger operations, establish a video recording session, record the recording start time, and call the terminal camera to generate a temporary video file; The hazard type triggering module is used to provide a triggering entry for a preset hazard type and output the triggered hazard type. When not in recording mode, the triggering operation of the preset hazard type directly triggers the video acquisition module to start short video acquisition. The annotation event generation module is used to respond to the triggering of potential hazard types during the recording process, obtain the current time and calculate the relative millisecond time with the recording start time, generate annotation objects, and determine the start and end milliseconds of the segment based on the relative millisecond time; The session management module is used to generate and maintain session numbers, so that multiple labeled objects in the same session carry the same session number. The positioning and route mapping module is used to obtain positioning information and map the mileage corresponding to the marked object to the route location ratio according to the interval ratio; The annotation state machine module is used to manage the transitions of the annotation object's state field between the acquisition state, the pending file state, and the replayable state. The storage module is used to filter out annotation objects whose session number is equal to the current session and whose video path field is empty after stopping recording and obtaining the video path, write them in batches to the video path and update them to a playable state, and persist the annotation objects to storage. The retrieval and playback module is used to draw annotation points on the line trajectory according to the line position ratio of each annotation object, and locate and play back the corresponding video segment according to the video path and the start and end milliseconds of the selected annotation object. The interface locking module is used to lock tab switching during recording or when processing is stopped.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. In this invention, a dual-mode mechanism is triggered by a hazard button, allowing inspection personnel to complete the full registration of hazard type, occurrence time, route location, and recorder with a single click. When recording is in progress, annotations are created instantly; when not recording, the hazard button triggers short video capture and automatically creates the first annotation. This mechanism transforms traditional continuous recording and subsequent manual registration into a synchronous recording and annotation mode. Combined with relative millisecond time calculation and segment boundary algorithms, the average time for a complete record of a single hazard is significantly reduced, substantially decreasing the workload of reviewing the entire video and manual transcription. Using `recordingStartAt` as a baseline, a millisecond-level alignment relationship between the annotation and the video timeline is established using `elapsedMs = now - recordingStartAt`, ensuring that each hazard event has a time coordinate accurate to the millisecond level, eliminating the need for manual retrieval afterward.

[0019] 2. In this invention, a mechanism linking session IDs with relative millisecond timestamps allows multiple potential incidents to be associated with the same video while maintaining session consistency. The session management module generates a sessionId and ensures that multiple annotations within the same recording segment use the same session ID. The batch binding logic, after recording stops, only selects annotations with a sessionId equal to the current session and an empty videoPath and writes them to the video path in batches. Existing historical annotations on the videoPath are not overwritten, and annotations from other sessions are not included in the binding. This mechanism enables the structured association of multiple potential incident segments within the same video file. Each segment has independent clipStartMs and clipEndMs, allowing direct location of the corresponding segment during retrieval and playback, rather than the entire video, greatly improving retrieval, review, and transfer efficiency.

[0020] 3. This invention projects hazard markers onto the track trajectory using an interval proportional mapping algorithm. The algorithm `positionPercent = clamp((markerMileage - startMileage) / (endMileage - startMileage) × 100, 0, 100)` converts the railway-specific kilometer markers (Kx+y) into metric mileage, calculates the relative positional proportion of the hazard within the track interval, and plots the marker point on the track trajectory. This facilitates location-based hazard management. When no reliable mileage is available, the system uses the current marker number to generate a temporary display ratio between 8% and 92%, ensuring the interface still reflects the marker order. The calculation is repeated once the accurate mileage is obtained, balancing reliability and user experience.

[0021] 4. The annotation state machine and acquisition window timer of this invention ensure that potential incidents are fully preserved even when annotations return before the video file. After manual annotation is generated, it enters the collecting state, and a 5-second acquisition window timer is started simultaneously. If the videoPath is not obtained by the timer expires, the state changes to waiting_file, ensuring that potential incidents are preserved even when annotations return before the video file. If the camera fails or the user cancels recording, annotations that have been generated but have no video are kept in waiting_file for subsequent re-upload or manual processing, solving the time difference problem between the asynchronous return of the video file and the instantaneous generation of annotations, ensuring zero loss of potential incidents.

[0022] 5. The recording status locking mechanism of this invention reduces the risk of data loss due to accidental page switching. At the start of recording, the component sends `busy-change=true` to the parent page. If the current tab is for video recording and `recordBusy` is true, the parent page refuses to switch tabs and prompts "Please end the current video recording first." This locking only restricts tab switching and does not prevent the hazard button from continuing to add annotations. This mechanism prevents inspection personnel from accidentally touching tabs and causing recording interruptions while ensuring the continuity of hazard recording.

[0023] 6. The local persistence mechanism of this invention ensures continuous operation even in weak network environments. Each time a new annotation is added, a state transition occurs, or batch binding is performed, the storage module persists the annotation array to the local key lcoas:video-markers, retaining a maximum of the most recent 30 records. Subsequent records can be synchronized to the backend database using the same fields. Local persistence ensures that all new annotations added within 30 minutes of network outage are retained, and the system successfully synchronizes to the backend after network recovery, guaranteeing continuous operation capabilities even in weak network environments. Attached Figure Description

[0024] Figure 1 A schematic diagram of the framework of a video annotation, location and segment association system for safety hazards along railway lines provided by the present invention; Figure 2 A flowchart illustrating a method for video annotation, location, and segment association of safety hazards along railway lines provided by the present invention; Figure 3 This invention provides a schematic diagram of the mapping between line location and potential hazard marking. Figure 4 This is a schematic diagram of the annotation record retrieval and video playback process provided by the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0028] Example 1 Please refer to the appendix. Figure 1 This embodiment proposes a system for video annotation, location, and segment association of safety hazards along railway lines. The system is deployed on handheld terminals used by inspection personnel, employing a front-end / back-end separation architecture. The front-end is a mobile web application based on Vue.js, while the back-end provides line mileage data and data synchronization services. The system includes a video acquisition module, a hazard type triggering module, an annotation event generation module, a session management module, a location and line mapping module, an annotation state machine module, a storage module, a retrieval and playback module, and an interface locking module.

[0029] Please refer to the attached document. Figure 2 A method for video annotation, location, and segment association of safety hazards along railway lines, including the following steps: Step 1: Initialization After inspection personnel enter the video recording page, the system reads the operation summary information [including the current train number, recorder's name, route start and end points, and current kilometer markers], loads the marker list from the local storage key lcoas:video-markers, and initiates location polling. The positioning and route mapping module outputs positioning results including latitude, longitude, altitude, accuracy, speed, and address. If positioning fails, the locationMode is marked as "Positioning Failed," but manual annotation is not blocked, ensuring that potential hazards can still be recorded normally in areas with weak GPS signals, such as tunnels and under viaducts. If the local data format is abnormal, for example, JSON parsing fails or the array structure is corrupted, the storage module clears the key to avoid erroneous records affecting subsequent data collection. When the page is re-entered, the latest marker is selected as the default detail, allowing inspection personnel to quickly view the previous record.

[0030] Step 2: Start Recording and Establish a Session When a user clicks the "Start Recording" button, or directly clicks a hazard type button while not recording (e.g., when "Illegal Construction" is detected), the system enters a dual-mode triggering mechanism. The session management module generates a session ID (sessionId) in the format "session-" + timestamp + "-" + random, where timestamp is a millisecond-level Date.now() and random is a 6-character short string generated by Math.random().toString(36).substring(2, 8). The video acquisition module records `recordingStartAt = Date.now()`, sets the recording status to "recording", and starts a 500ms cycle timer to refresh the recording duration. The camera prioritizes the rear camera of the terminal, with a maximum recording duration of 10 seconds (short video mode) or 60 seconds (long video mode), pre-selected by the inspection personnel based on the scenario. The annotation event generation module generates annotation IDs in the format "marker-" + timestamp + "-" + random. If an ID with the same name already exists in the annotation array before writing, it is regenerated to ensure uniqueness when clicked consecutively on the same terminal. When recording begins, the interface locking module sends a `postMessage({type: 'busy-change', busy: true})` to the parent page. The parent page listens for this message, and if the current tab is the video recording tab and `busy` is true, it intercepts the tab switching event and prompts the parent page with the message "Please end the current video recording first." This locking only restricts tab switching and does not prevent the "potentially problematic" button from adding annotations. When recording ends, fails, or is reset, `busy-change=false` is sent to unlock the page.

[0031] Step 3: Generate annotation objects When a user clicks a button for a specific hazard type during recording, for example, when clicking "Hazardous Materials," the event generation module reads `now = Date.now()` and calculates the relative millisecond time `elapsedMs = now - recordingStartAt`. This `elapsedMs` is the millisecond-level offset of the current hazard event relative to the start of the video, providing a precise baseline for subsequent segment extraction. A marker object is constructed with the following data structure: `marker = {id: String, type: String, time: String, timestamp: Number, trainNo: String, recorder: String, locationMode: String, locationName: String, location: Object, km: String, speed: Number, from: String, to: String, sessionId: String, clipStartMs:Number, clipEndMs: Number, videoPath: String, status: String, autoCapture:Boolean, positionPercent: Number}`.

[0032] For manual annotation, when a user clicks the "Hazard" button during recording, the start and end milliseconds of the clip are set as follows: clipStartMs = max(0, elapsedMs - 5000), clipEndMs = elapsedMs + 5000. This setting covers 5 seconds before and after the hazard occurs, ensuring the video clip includes the complete hazard occurrence process and its surrounding context. For short videos automatically triggered by the "Hazard" button, for example, if the user clicks the "Hazard" button directly without clicking "Start Recording," the system first starts short video capture. The first annotation is set as follows: clipStartMs = 0, clipEndMs = elapsedMs + 5000, where elapsedMs is the relative millisecond time from the start of recording to the button trigger time. Since short video capture is triggered by the button, elapsedMs is usually small, so the clip covers from the beginning of the video to 5 seconds after the hazard occurs. The recording duration timer refreshes the interface display every 500ms, but the clip start and end values ​​are calculated according to the above millisecond timestamps to ensure that the interface refresh frequency does not affect the annotation accuracy. Multiple tags in the same recorded session share the same sessionId, ensuring that they can be identified as members of the same session when binding in batches later.

[0033] Step 4: Confirmation of data acquisition window and state machine transition After manual annotation is generated, the annotation state machine module sets its status to "collecting" and simultaneously starts a 5-second acquisition window timer. When this timer expires, if the annotation has not yet obtained a videoPath (i.e., the camera has not yet returned the video file path), the status changes from "collecting" to "waiting_file," ensuring that potential events are retained even if the annotation returns before the video file. This mechanism solves the time difference problem between the asynchronous return of the video file by the mobile camera and the immediate generation of the annotation. Once the camera returns a valid videoPath or batch binding is complete, the status changes to "ready." If the camera fails or the user cancels recording, the recording status resets to "preview," and annotations that have been generated but have no video are kept in "waiting_file" for later re-uploading or manual processing. The recording control status includes "preview" (preview to be started), "recording" (recording in progress), and "stopping" (processing stopped), corresponding to three interface states.

[0034] Step 5: Stop recording and batch bind. After the user clicks "Stop Recording," the camera returns a `tempFilePath`, which is then uploaded by the backend to obtain the final `videoPath`. The storage module performs batch binding: it retrieves the current `sessionId` and `videoPath`, iterates through the local marker array `markers`, and selects only records where `item.sessionId === currentSessionId` and `item.videoPath ===` as unassociated markers. These records are written to the same `videoPath`, and the `status` is updated from `collecting` or `waiting_file` to `ready`. Subsequently, local persistence is performed to `lcoas:video-markers`. Historical markers in existing `videoPath`s are not overwritten, and markers from other sessions are not included in the binding, thus ensuring batch association of multiple potential issues within the same video session. If no file has been returned when recording is stopped, these records are added to `waiting_file` and can be re-uploaded or re-bound later.

[0035] Step 6: Line Mapping, Retrieval and Replay Referring to Appendices 3 and 4, the positioning and route mapping module executes the interval proportional mapping algorithm. If the backend or positioning service provides the interval start mileage (startMileage), end mileage (endMileage), and marker mileage (markerMileage), then: positionPercent = clamp((markerMileage - startMileage) / (endMileage - startMileage) × 100, 0, 100), where the clamp function is defined as: clamp(value, min, max) = min(max(value, min), max). The kilometer marker Kx+y is first converted to metric mileage in meters (x × 1000 + y meters) before being used in the calculation. For example, kilometer marker K12+345 is converted to 12 × 1000 + 345 = 12345 meters. If the starting point of the interval is K10+000 (10000 meters) and the ending point is K20+000 (20000 meters), then the position ratio of this marker is: (12345 - 10000) / (20000 - 10000) × 100 = 23.45%. When there is no reliable mileage, the system uses the current marker number index to generate a temporary display ratio between 8% and 92%: tempPercent = 8 + (index % 10) × (84 / 10), ensuring that the interface can still express the marker order. The ratio will be recalculated after the accurate mileage is obtained. The retrieval and playback module displays markers on the route track by positionPercent. Markers that are not ready (status: collecting or waiting_file) are displayed as pending. Clicking on a marker displays the hazard type, location method, time, location, and video clip start-end information. When playback is initiated, the system selects the current marker or the most recent marker with videoPath to open the video control, starting playback from clipStartMs and automatically pausing when the playback progress reaches clipEndMs, thus reviewing only the segments related to the hazard. Markers in the waiting_file status are given the message "Video pending re-upload, please try again later or contact the administrator."

[0036] Step 7: Data Persistence and Recovery Each time a new annotation is added, a status transition occurs, or a batch binding is performed, the storage module persists the annotation array to the local key `lcoas:video-markers`, retaining a maximum of the most recent 30 records. A queue-based management system is used; when more than 30 records are added, the oldest record is removed. Subsequent records can be synchronized to the backend database using the same fields. Local persistence ensures continuous operation even in weak network environments. After the network recovers, the system checks the online status and batch synchronizes annotations in the local `waiting_file` and `ready` states to the backend, achieving eventual data consistency.

[0037] Simulation Examples To verify the technical effectiveness of this invention, the following simulation experimental environment was constructed: A railway section from K10+000 to K20+000 was selected [startMileage = 10000 meters, endMileage = 20000 meters]. Three inspection personnel were simulated patrolling the line, and a total of 120 potential hazard events were injected. Among these, 30 scenarios were randomly generated [where 2 to 4 potential hazards occurred consecutively within the same recording session]. During the experiment, abnormal situations such as weak network conditions [network restored after a 30-minute outage], accidental page switching during recording (20 times), and delayed return of video files from the camera [delay of 3-8 seconds, involving 12 annotations] were simulated. The traditional method of continuous recording of the entire section followed by manual post-recording was used as the control group, and the method of this invention was used as the experimental group. The statistical results are as follows.

[0038] Efficiency comparison experiment Table 1 shows the efficiency comparison results between the control group and the present invention based on a sample of 120 potential incidents: Table 1 As shown in Table 1, this invention reduces the average time for complete recording of a single hazard from 95 seconds to 8 seconds, a reduction of 91.6%; the average time for locating the target hazard video segment from 210 seconds to 12 seconds, a reduction of 94.3%; the hazard event omission rate from 7.5% to 0.8%; and the average time for reviewing a single hazard from 180 seconds to 15 seconds. These efficiency improvements are mainly due to the following technical mechanisms: relative millisecond time calculation and segment boundary algorithms automatically align hazards with the video timeline, eliminating the need for manual retrieval afterward; session numbering and batch binding mechanisms automatically and structurally associate multiple hazards in the same video, eliminating the need for manual matching; and line location ratio mapping visualizes hazards by spatial location, supporting rapid location-based positioning.

[0039] Accuracy and reliability experiments Regarding accuracy and reliability, the experimental results are shown in Table 2: Table 2 The annotation time accuracy error did not exceed ±50 ms, verifying the effectiveness of the technique of calculating the start and end mills of segments using millisecond-level timestamps and decoupling it from the 500ms refresh rate of the interface. Session association accuracy and batch binding success rate both reached 100%, verifying the correctness of the session number uniqueness rule and batch binding logic. The annotation retention rate was 100% in weak network environments, verifying the effectiveness of the local persistence mechanism. The number of data interruptions caused by accidental page switching was 0, verifying the reliability of the interface locking mechanism. The retention rate of potential problems during video delay return was 100%, verifying the annotation state machine and acquisition window timer's ability to handle asynchronous exceptions.

[0040] Fragment offset parameter optimization experiment For the preset offset of milliseconds at the start and end of the segment, three sets of parameters were set for 3 seconds, 5 seconds, and 8 seconds before and after. The reviewers judged whether the truncated segment completely covered the process of the potential problem, and the review coverage rate and average redundancy time were statistically analyzed. Table 3 As shown in Table 3, the offset of 5 seconds before and after achieves a better balance between the review coverage rate of 98.6% and the redundancy duration of 4.1 seconds. Further increasing the offset has limited effect on improving the coverage rate, only 0.4%, but significantly increases the redundancy by 2.7 seconds. Therefore, in this embodiment, 5000 milliseconds before and after is preferred as the preset offset.

[0041] Simulation results show that the present invention simultaneously completes hazard classification and time location at the data collection site. The time for recording a single hazard is reduced from 95 seconds to 8 seconds, the time for video segment retrieval and location is reduced from 210 seconds to 12 seconds, and the hazard omission rate is reduced from 7.5% to 0.8%. The session association accuracy, batch binding success rate, and weak network retention rate all reach 100%, and accidental page switching during recording does not cause data interruption. The above data corroborates the expected technical effects of the present invention, indicating that the technical solution of the present invention can effectively solve the technical problems of low efficiency of manual review after the event, difficulty in retrieval and verification, and easy data loss in video inspections along railway lines.

[0042] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0043] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0045] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0047] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0048] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for video annotation, location, and segment association of safety hazards along railway lines, characterized in that, Includes the following steps: S1. Establish a video recording session In response to the recording trigger operation, establish a session, generate a session number, and record the recording start time; When not in recording mode, if a preset hazard type trigger operation is received, the operation will directly trigger the start of short video capture and establish a session. S2. Generate annotation objects During recording, the system responds to the operation of generating annotation objects, obtains the current time, calculates the relative millisecond time with the recording start time, and generates annotation objects. The annotation objects include at least the annotation identifier, hazard type, relative millisecond time, recorder, line start and end point, session number, video path field, segment start and end milliseconds, status field, and line position ratio. Multiple labeled objects within the same session carry the same session number; S3, Batch Binding Video Paths After stopping recording and obtaining the video path, filter the labeled objects whose session number is equal to the current session and whose video path field is empty, write the video path in batches and update the status field to playable status, and then persist the storage. S4. Positioning and Segment Playback Mark points on the route trajectory according to the proportion of the route location; In response to the selection operation, read the video path and the start and end milliseconds of the segment, locate the corresponding video segment, and play it back.

2. The method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, The session number generated in step S1 is in the format of: session-millisecond timestamp-random number; The format for generating the marker is: marker-millisecond timestamp-random number. If a marker with the same name already exists before writing, it will be regenerated.

3. The method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, The method for determining the start and end milliseconds of the segment in step S2 includes: For manually generated annotation objects, the starting milliseconds of a fragment are the relative millisecond time minus the preset front offset, and the ending milliseconds of a fragment are the relative millisecond time plus the preset back offset. For the first labeled object automatically created by short video collection triggered by a preset hazard type, the starting millisecond of the segment is 0, and the ending millisecond of the segment is the relative millisecond time plus the preset offset. Both the preset forward offset and the preset backward offset are 5000 milliseconds, and the start and end milliseconds of the segment are calculated using millisecond-level timestamps, which are independent of the frequency at which the interface refreshes the recording duration display at a 500ms cycle.

4. The method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, The values ​​of the status field include acquisition status, pending file status, and replayable status; After the labeled object is generated, it enters the acquisition state and starts the acquisition window timer of the preset duration; If the video path field is still empty when the acquisition window timer expires, the status field will change from acquisition status to pending file status. After writing a valid video path, the status field is changed to a playable state; If camera capture fails or recording is canceled, the generated but not written annotation objects remain in a pending file state.

5. The method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, The line location ratio is determined using an interval ratio mapping algorithm: The formula for obtaining the starting mileage (startMileage), ending mileage (endMileage), and marker mileage of the interval is as follows: positionPercent=clamp((markerMileage-startMileage) / (endMileage-startMileage)×100, 0, 100); Among them, the kilometer markers in the form of Kx+y are first converted into metric mileage of x×1000+y meters before participating in the calculation; when there is no reliable mileage, a temporary display ratio is generated between 8% and 92% according to the generation order, and the calculation is recalculated after the accurate mileage is obtained.

6. The method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, It also includes a screen locking step: When recording begins, the recording component sends a busy status message to the parent page. If the parent page is currently on the video recording tab and is in a busy state, it will refuse the tab switching operation and output a prompt message. When recording ends, fails, or is reset, a message to clear the busy state is sent; the interface lock only restricts tab switching and does not prevent the continued triggering of preset hidden danger types to add labeled objects.

7. The method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, The persistent storage mentioned in step S3 includes the following steps: The array of labeled objects is persisted to the local storage key, retaining a maximum of the most recent 30 records, and synchronized to the backend database with the same field structure; When the page is re-entered, the local annotation object array is read and the latest one is selected as the default detail; if the local data format is abnormal, the local storage key is cleared.

8. A method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, In step S4, when responding to the playback operation, select the currently selected annotation object or the annotation object whose most recent video path field is not empty to open the video control, start playing from the beginning milliseconds of the clip, and automatically pause when the playback progress reaches the end milliseconds of the clip; For labeled objects that are in the pending file state, their labeled points are displayed in the ready style, and a message indicating that they need to be uploaded is output.

9. A method for video annotation, location, and segment association of safety hazards along railway lines according to claim 1, characterized in that, The triggering operation for the preset potential hazard type is a dual-mode triggering mechanism: When the system is in recording mode, the trigger operation of the preset hidden danger type immediately creates a labeled object; When the system is not in recording mode, the trigger operation of the preset hidden danger type directly triggers the start of short video collection and automatically creates the first labeled object. The starting millisecond of the segment of the first labeled object is 0, and the ending millisecond of the segment is the relative millisecond time plus the preset offset.

10. A system for video annotation, location, and segment association of safety hazards along railway lines, used to implement the method for video annotation, location, and segment association of safety hazards along railway lines as described in any one of claims 1-9, characterized in that, Includes the following modules: The video capture module is used to respond to recording trigger operations, establish a video recording session, record the recording start time, and call the terminal camera to generate a temporary video file; The hazard type triggering module is used to provide triggering entry points for preset hazard types and output the triggered hazard types; When not in recording mode, the trigger operation of the preset hidden danger type directly triggers the video acquisition module to start short video acquisition; The annotation event generation module is used to respond to the triggering of potential hazard types during the recording process, obtain the current time and calculate the relative millisecond time with the recording start time, generate annotation objects, and determine the start and end milliseconds of the segment based on the relative millisecond time; The session management module is used to generate and maintain session numbers, so that multiple labeled objects in the same session carry the same session number. The positioning and route mapping module is used to obtain positioning information and map the mileage corresponding to the marked object to the route location ratio according to the interval ratio; The annotation state machine module is used to manage the transitions of the annotation object's state field between the acquisition state, the pending file state, and the replayable state. The storage module is used to filter out annotation objects whose session number is equal to the current session and whose video path field is empty after stopping recording and obtaining the video path, write them in batches to the video path and update them to a playable state, and persist the annotation objects to storage. The retrieval and playback module is used to draw annotation points on the line trajectory according to the line position ratio of each annotation object, and locate and play back the corresponding video segment according to the video path and the start and end milliseconds of the selected annotation object. The interface locking module is used to lock tab switching during recording or when processing is stopped.

Images