Escalator safety early warning method based on Deepseek

Abstract

An escalator safety early warning method based on Deepseek belongs to the field of escalator safety early warning, and comprises the following steps: step 1, collecting real-time video data of an escalator operation area; step 2, inputting the video frames into a lightweight computer vision detection model, identifying standing, stooping, squatting and falling behaviors of the passengers, and generating structured information containing behavior types and confidence coefficients; 3, carrying out preliminary risk assessment, and triggering a voice alarm for a high-confidence dangerous behavior; 4, constructing an intelligent decision-making module based on a DeepSeek large language model, converting the structured information into a structured prompt text through a standard prompt word template, and calling DeepSeek through an API to generate a decision-making result containing risk levels, emergency measures and safety suggestions; and 5, triggering an alarm according to a decision result to realize real-time early warning and intervention. The safety monitoring precision and the response efficiency in the escalator operation process are improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring and escalator safety early warning technology, and in particular, a method for early warning of escalator safety. Background Technology

[0002] With the widespread use of escalators in public transportation and commercial facilities such as subway stations, shopping malls, and airports, safety accidents caused by passengers' improper behaviors, such as falls, bending over to pick up items, going against the flow of traffic, and leaning on handrails, occur frequently. Traditional escalator safety management mainly relies on manual video inspections or simple threshold alarm devices, such as infrared detection and emergency stop buttons. However, these methods have the following shortcomings: First, manual monitoring is lagging and subjective, making it difficult to detect dangerous behaviors in a timely manner; second, traditional sensor alarm methods rely on fixed rules and cannot identify complex or potential abnormal behaviors, such as passengers' imbalance or tendency to fall in crowded situations; third, existing intelligent video analysis systems mostly remain at the target detection level, lacking a deep understanding of behavioral semantics and risk levels, leading to frequent false alarms and missed alarms.

[0003] In recent years, the development of computer vision technology has provided new solutions for intelligent safety monitoring of escalators. Deep learning-based detection models (such as the YOLO series) can achieve real-time object detection, but their output is limited to object location and category, and they cannot perform semantic judgment and strategy generation based on the safety risks of passenger behavior. Furthermore, these models typically rely on a large number of labeled samples and lack high-level decision-making capabilities, making it difficult to dynamically adjust response methods according to different scenarios.

[0004] With the rise of Large Language Model (LLM) technology, it has demonstrated powerful generalization capabilities in natural language understanding, logical reasoning, and multimodal information fusion. DeepSeek models, in particular, possess multi-layered semantic reasoning and contextual understanding capabilities, enabling efficient mapping between structured data and natural language. However, there is currently no mature solution for deeply integrating LLM with computer vision detection results for escalator safety warnings.

[0005] Therefore, there is an urgent need for an escalator safety early warning method that can combine the real-time detection capabilities of computer vision with the semantic reasoning capabilities of large language models, so as to realize an intelligent closed loop from abnormal behavior recognition to safety strategy generation and improve the safety and response efficiency of escalator operation in public places. Summary of the Invention

[0006] To address the shortcomings of existing escalator safety monitoring systems, such as poor real-time performance, low intelligence, and inability to perform semantic-level safety analysis, this invention proposes an escalator safety early warning method based on the fusion of a large language model (DeepSeek) and computer vision detection. This method combines the object detection results of a deep learning visual recognition model with the semantic reasoning capabilities of a large language model, achieving an intelligent closed-loop early warning process from passenger behavior recognition to safety policy generation, thereby improving the accuracy and response efficiency of safety monitoring during escalator operation.

[0007] The technical solution adopted by this invention to solve its technical problem is: A Deepseek-based escalator safety early warning method includes the following steps: Step 1: Collect real-time video data of the escalator operating area; The second step involves inputting the video frames collected in the first step into a lightweight computer vision detection model to identify passenger behavior, extract behavioral features including standing, bending over, squatting, and falling, and generate structured detection information containing behavior type and confidence level. The third step is to conduct a preliminary risk assessment based on the structured detection information obtained in the second step, set the risk level according to the confidence level, and set falling, squatting, and bending over as dangerous behaviors, triggering an immediate voice warning for dangerous behaviors; The fourth step is to build an intelligent decision-making module based on the DeepSeek large language model. The structured detection information output by the visual detection module in the second step is converted into structured prompt text using the constructed standardized prompt template. Using the preset prompt word template, the DeepSeek large oracle model is called through the API to generate structured output containing risk level, emergency measures and safety suggestions. Fifth, based on the decision results output by the DeepSeek module, trigger voice broadcasts, display prompts, or light alarms to achieve real-time early warning and intervention for abnormal behavior.

[0008] Furthermore, the method also includes the following steps: Step 6: Use timestamps to ensure that warning content is regenerated for each detection, process all behavior types, ensure that warning content is updated in real time, provide backup preset content when API calls fail, and ensure system reliability.

[0009] Furthermore, in the second step, a lightweight computer vision detection model is used to analyze the input video frames in real time to identify the types of passenger behavior on the escalator. The types of behavior include standing, bending over, squatting, and falling. The detection module outputs structured data, which includes behavior type, location coordinates, confidence level, and timestamp information.

[0010] Furthermore, in the third step, the structured detection information obtained in the second step is subjected to a preliminary risk assessment. Detection results with a confidence level > 0.9 are set as extremely high risk, detection results with a confidence level > 0.7 are set as high risk, detection results with a confidence level > 0.5 are set as medium risk, and detection results with a confidence level > 0.5 are set as low risk. Instant voice warnings are triggered for dangerous behaviors.

[0011] The process of the fourth step is as follows: Step (4.1) converts the structured detection information output by the visual detection module in the second step into structured prompt text using the constructed standardized prompt template. The constructed standardized prompt template is as follows: Passenger behavior was detected at the escalator location, with a confidence level of 2f. Step (4.2) Design of preset prompt word template. The preset prompt word template includes four aspects: system role definition, task clarification, output structure specification, and language style constraints. The system role definition sets Deepseek as a "professional escalator safety monitoring system AI assistant". The task clarification requires Deepseek to provide "accurate and serious safety analysis and warnings". The output structure specification clarifies the three parts of the output content: risk analysis, immediate measures, and safety recommendations. The language style constraints require Deepseek to use "professional and concise language". Step (4.3) uses the API to call the DeepSeek oracle model to generate a structured output containing risk level, emergency measures, and safety recommendations. The API calls the DeepSeek oracle model to utilize preset prompt word templates and employ streaming response processing to achieve real-time updates of the safety warning content, generating a three-part structured safety warning content containing risk analysis, emergency measures, and safety recommendations. The risk analysis includes a detailed analysis of the potential dangers and consequences of this behavior; the emergency measures include a list of specific actions that staff should take immediately; and the safety recommendations include suggestions for long-term prevention of such behavior.

[0012] In the fifth step, based on the decision results output by the DeepSeek module, voice broadcasts, display prompts, or light alarms are triggered to achieve real-time early warning and intervention for abnormal behaviors. The voice broadcasts generate targeted voice warnings based on the behavior type, including: "Serious warning! Passenger has fallen. Please stop the escalator immediately and provide assistance" for falling behavior; "Attention! A passenger is squatting on the escalator. Please remind them to stand up immediately" for squatting behavior; "Attention! A passenger is bending over on the escalator. Please remind them to maintain a normal standing posture" for bending over behavior; and "Attention! Standing behavior has been detected. Please pay attention to the passenger's condition." for standing behavior. The display prompts include highlighting the detected dangerous behavior on the detection interface and triggering a visual alarm effect for high-confidence dangerous behaviors, which is a red background flashing effect on the detection interface. The text warnings are AI-generated safety suggestions displayed in a dedicated area on the right side of the detection interface. The safety suggestions include key information such as time, location, behavior type, and confidence level, and are displayed in segments as risk analysis, immediate measures, and safety suggestions.

[0013] In the sixth step, the system records timestamp information in each detection cycle to ensure that new warning content is regenerated with each detection, processing and updating all behavior types, including normal behavior. When API calls to the DeepSeek model fail or communication anomalies occur, the system automatically switches to a local backup strategy, outputting preset security warning content to ensure the warning system continues to operate even in network anomalies, thereby improving system stability and reliability.

[0014] The main benefits of this invention are: improving the accuracy and response efficiency of safety monitoring during escalator operation. Attached Figure Description

[0015] Figure 1 This is a flowchart of the overall process for the escalator safety early warning method based on Deepseek. Detailed Implementation

[0016] The present invention will now be further described with reference to the accompanying drawings.

[0017] Reference Figure 1 A safety early warning method for escalators based on Deepseek includes the following steps: Step 1: Collect real-time video data of the escalator operating area; In this embodiment, monitoring cameras installed above or on both sides of the escalator collect passenger video data in real time in the escalator operating area. The collected video frames are input to the subsequent detection module in time sequence form for passenger behavior recognition and anomaly detection. The second step involves inputting the video frames collected in the first step into a lightweight computer vision detection model to identify passenger behavior, extract behavioral features including standing, bending over, squatting, and falling, and generate structured detection information containing behavior type and confidence level. In this embodiment, a lightweight computer vision detection model is used to analyze the input video frames in real time to identify the types of passenger behavior on the escalator. These behavior types include standing, bending over, squatting, and falling. The detection module outputs structured data, which includes information such as behavior type, location coordinates, confidence level, and timestamp. The third step is to conduct a preliminary risk assessment based on the structured detection information obtained in the second step, set the risk level according to the confidence level, and set falling, squatting, and bending over as dangerous behaviors, triggering an immediate voice warning for dangerous behaviors (such as falling, squatting, and bending over). In this embodiment, the structured detection information obtained in the second step is used for preliminary risk assessment. Detection results with a confidence level > 0.9 are set as extremely high risk, detection results with a confidence level > 0.7 are set as high risk, detection results with a confidence level > 0.5 are set as medium risk, and detection results with a confidence level > 0.5 are set as low risk. Instant voice warnings are triggered for dangerous behaviors (such as falling, squatting, bending over). The fourth step is to build an intelligent decision-making module based on the DeepSeek large language model. The structured detection information output by the visual detection module in the second step is converted into structured prompt text using the constructed standardized prompt template. Using the preset prompt word template, the DeepSeek large oracle model is called through the API to generate structured output containing risk level, emergency measures and safety suggestions. The fourth step of this embodiment is as follows: Step (4.1) converts the structured detection information output by the visual detection module in the second step into structured prompt text using the constructed standardized prompt template. The constructed standardized prompt template is as follows: Passenger behavior was detected at the escalator location, with a confidence level of 2f. Step (4.2) Design of preset prompt word template. The preset prompt word template includes four aspects: system role definition, task clarification, output structure specification, and language style constraints. The system role definition sets Deepseek as a "professional escalator safety monitoring system AI assistant". The task clarification requires Deepseek to provide "accurate and serious safety analysis and warnings". The output structure specification clarifies the three parts of the output content: risk analysis, immediate measures, and safety recommendations. The language style constraints require Deepseek to use "professional and concise language". Step (4.3) uses the API to call the DeepSeek oracle model to generate a structured output containing risk level, emergency measures, and safety recommendations. The API call to the DeepSeek oracle model utilizes preset prompt word templates and employs streaming response processing to achieve real-time updates of the safety warning content, generating a structured safety warning content consisting of three parts: risk analysis, emergency measures, and safety recommendations. The risk analysis includes a detailed analysis of the potential dangers and consequences of this behavior; the emergency measures list specific actions that staff should take immediately; and the safety recommendations provide suggestions for long-term prevention of such behavior. Fifth step: Based on the decision results output by the DeepSeek module, trigger voice broadcast, display prompts or light alarms to achieve real-time early warning and intervention for abnormal behavior; In the fifth step of this embodiment, based on the decision results output by the DeepSeek module, voice broadcasts, display prompts, or light alarms are triggered to achieve real-time early warning and intervention for abnormal behaviors. The voice broadcasts generate targeted voice warnings based on the behavior type, including issuing "Serious warning! Passenger fall detected. Please stop the escalator immediately and provide assistance" for falling behavior; "Attention! A passenger is squatting on the escalator. Please remind them to stand up immediately" for squatting behavior; "Attention! A passenger is bending over on the escalator. Please remind them to maintain a normal standing posture" for bending over behavior; and "Attention! Standing behavior detected. Please pay attention to the passenger's condition." for standing behavior. The display prompts include highlighting the detected dangerous behavior on the detection interface and triggering a visual alarm effect for high-confidence dangerous behaviors, which is a red background flashing effect on the detection interface. The text warnings are AI-generated safety suggestions displayed in a dedicated area on the right side of the detection interface. The safety suggestions include key information such as time, location, behavior type, and confidence level, and are displayed in segments as risk analysis, immediate measures, and safety suggestions.

[0018] Step 6: Use timestamps to ensure that warning content is regenerated for each detection, process all behavior types (including normal behavior), ensure that warning content is updated in real time, provide backup preset content when API call fails, and ensure system reliability; In the sixth step of this embodiment, the system records timestamp information in each detection loop to ensure that new warning content is regenerated for each detection, processing and updating all behavior types, including normal behavior. When the API call to the DeepSeek model fails or communication is abnormal, the system automatically switches to a local backup strategy and outputs preset security prompts, ensuring that the warning system can continue to operate even in network anomalies, thereby improving the system's stability and reliability.

[0019] The implementation process of this example is as follows: Step 1: Video Acquisition and Input Processing. Monitoring cameras installed above or on either side of the escalator collect real-time video data of passengers in the escalator's operating area. The acquired video frames are input as a time sequence to the subsequent detection module for passenger behavior recognition and anomaly detection.

[0020] Step 2: Passenger behavior detection and recognition. Video frames are input into a lightweight computer vision detection model (such as an improved YOLOv8n structure). The model identifies and classifies passenger behaviors, outputting behavior categories such as standing, bending over, squatting, and falling, and generating structured detection results that include behavior type, location region, and confidence level.

[0021] Step 3: Preliminary risk assessment and immediate warning. The system classifies risks based on the detection confidence level. For high-risk and extremely high-risk dangerous behaviors, the system immediately triggers a voice broadcast, such as "Serious warning! Passenger fall detected. Please stop the elevator immediately and provide assistance."

[0022] Step 4: DeepSeek Semantic Decision Making and Prompt Text Generation. The structured detection information output by the detection module is converted into standardized structured natural language prompt text. This prompt text calls the DeepSeek large language model through the API interface. The model performs semantic understanding and logical reasoning to generate a structured output result containing three parts: "risk analysis, immediate measures, and safety recommendations".

[0023] Step 5: Multimodal safety early warning output. The system parses the DeepSeek output results and triggers different early warning methods according to the risk level. The early warning content is displayed synchronously on the right side of the detection interface, showing three parts of structured information: "risk analysis, immediate measures, and safety recommendations" to facilitate quick response by on-duty personnel.

[0024] Step Six: The system automatically generates a timestamp in each detection cycle to ensure that the warning content is updated in real time. If the API call to the DeepSeek model fails or the network is interrupted, the system automatically activates the local preset strategy and outputs the voice prompt "Please pay attention to safety and maintain order" to ensure the continuous operation of the system under abnormal conditions.

[0025] Step 7: Verify the running effect. Through extensive testing and result analysis, this method can accurately and robustly detect various passenger behaviors in escalator environments. Whether it's routine standing behavior or abnormal behaviors that could lead to danger, such as falls, bending over, or squatting, the system can capture and classify these behaviors in real time with high precision using a lightweight computer vision detection model, combined with intelligent analysis and early warning using the DeepSeek model. For example, upon detecting a "fall" behavior, the system immediately triggers a safety alarm and automatically generates a detailed report including potential hazard analysis and emergency response suggestions (such as immediately stopping the escalator and assessing the injured person's condition). In complex scenarios, the system can even simultaneously identify and handle concurrent behaviors like "bending over" and "falling," demonstrating its effectiveness in handling multiple overlapping risks. The test results fully validate the system's superior detection performance, rapid response capabilities, and highly intelligent decision support level in real-world scenarios, providing reliable technical assurance for the safe operation and maintenance of escalators.

[0026] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A safety early warning method for escalators based on Deepseek, characterized in that, The method includes the following steps: Step 1: Collect real-time video data of the escalator operating area; The second step involves inputting the video frames collected in the first step into a lightweight computer vision detection model to identify passenger behavior, extract behavioral features including standing, bending over, squatting, and falling, and generate structured detection information containing behavior type and confidence level. The third step is to conduct a preliminary risk assessment based on the structured detection information obtained in the second step, set the risk level according to the confidence level, and set falling, squatting, and bending over as dangerous behaviors, triggering an immediate voice warning for dangerous behaviors; The fourth step is to build an intelligent decision-making module based on the DeepSeek large language model. The structured detection information output by the visual detection module in the second step is converted into structured prompt text using the constructed standardized prompt template. Using the preset prompt word template, the DeepSeek large oracle model is called through the API to generate structured output containing risk level, emergency measures and safety suggestions. Fifth, based on the decision results output by the DeepSeek module, trigger voice broadcasts, display prompts, or light alarms to achieve real-time early warning and intervention for abnormal behavior.

2. The escalator safety early warning method based on Deepseek as described in claim 1, characterized in that, The method further includes the following steps: Step 6: Use timestamps to ensure that warning content is regenerated for each detection, process all behavior types, ensure that warning content is updated in real time, provide backup preset content when API calls fail, and ensure system reliability.

3. The escalator safety early warning method based on Deepseek as described in claim 1 or 2, characterized in that, In the second step, a lightweight computer vision detection model is used to analyze the input video frames in real time to identify the passenger's behavior categories on the escalator. The behavior categories include standing, bending over, squatting, and falling. The detection module outputs structured data, which includes behavior type, location coordinates, confidence level, and timestamp information.

4. The escalator safety early warning method based on Deepseek as described in claim 1 or 2, characterized in that, In the third step, the structured detection information obtained in the second step is subjected to a preliminary risk assessment. Detection results with a confidence level > 0.9 are set as extremely high risk, detection results with a confidence level > 0.7 are set as high risk, detection results with a confidence level > 0.5 are set as medium risk, and detection results with a confidence level > 0.5 are set as low risk. Instant voice warnings are triggered for dangerous behaviors.

5. The escalator safety early warning method based on Deepseek as described in claim 1 or 2, characterized in that, The process of the fourth step is as follows: Step (4.1) converts the structured detection information output by the visual detection module in the second step into structured prompt text using the constructed standardized prompt template. The constructed standardized prompt template is as follows: Passenger behavior was detected at the escalator location, with a confidence level of 2f. Step (4.2) Design of preset prompt word template. The preset prompt word template includes four aspects: system role definition, task clarification, output structure specification, and language style constraints. The system role definition sets Deepseek as a "professional escalator safety monitoring system AI assistant". The task clarification requires Deepseek to provide "accurate and serious safety analysis and warnings". The output structure specification clarifies the three parts of the output content: risk analysis, immediate measures, and safety recommendations. The language style constraints require Deepseek to use "professional and concise language". Step (4.3) calls the DeepSeek oracle model via API to generate a structured output containing risk level, emergency measures and security recommendations. The DeepSeek oracle model is called via API to utilize preset prompt word templates and adopt streaming response processing to realize real-time updates of security warning content and generate a three-part structured security warning content containing risk level, emergency measures and security recommendations.

6. The escalator safety early warning method based on Deepseek as described in claim 1 or 2, characterized in that, In the fifth step, based on the decision results output by the DeepSeek module, voice broadcasts, display prompts, or light alarms are triggered to achieve real-time early warning and intervention for abnormal behaviors. The voice broadcasts generate targeted voice warnings based on the behavior type. The display prompts include highlighting the detected dangerous behaviors on the detection interface and triggering visual alarm effects for high-confidence dangerous behaviors. The visual alarm effects involve flashing a red background on the detection interface. The text warnings are AI-generated safety suggestions displayed in a dedicated area on the right side of the detection interface. The safety suggestions include key information such as time, location, behavior type, and confidence level, and are presented in segments, including risk analysis, immediate measures, and safety recommendations.

7. The escalator safety early warning method based on Deepseek as described in claim 2, characterized in that, In the sixth step, the system records timestamp information in each detection cycle to ensure that new warning content is regenerated for each detection, and processes and updates all behavior types, including normal behavior. When the API call to the DeepSeek model fails or communication is abnormal, the system automatically switches to the local backup strategy and outputs preset security prompts to ensure that the early warning system can continue to operate even in the event of network anomalies.

Images