Dangerous state monitoring method and system and storage medium

Abstract

The invention provides a dangerous state monitoring method and system and a storage medium, and the method comprises the steps: obtaining passenger image data and cabin environment data in real time, and carrying out the real-time monitoring of the passenger image data based on an action recognition model, so as to obtain a passenger action state; and carrying out dangerous state evaluation according to the passenger action state and the cabin environment data, and carrying out corresponding alarm feedback operation according to an evaluation result. According to the method, the technical problems that in the prior art, behavior states of the passengers in the back row of the automobile are insufficient in monitoring, and potential dangerous behaviors cannot be detected and early warned in real time are effectively solved, the dangerous behaviors of the passengers in the back row of the automobile are monitored in real time, the safety of the passengers is ensured, and the use experience of the automobile is improved. Through different alarm feedback operations, the alarm feedback operations are more flexible, interference to passengers is reduced, and the environmental comfort of the vehicle cabin is improved under the condition that the safety of the passengers is guaranteed.

Description

Technical Field

[0001] This application belongs to the technical field of automotive safety, and particularly relates to a method, system, and storage medium for monitoring dangerous states. Background Art

[0002] With the rapid development of the automotive industry, the comfort and intelligence level of vehicles have been significantly improved. However, passenger safety remains an important area of social concern. In particular, improper behaviors of children or pets in the vehicle, such as sticking body parts out of the window, may lead to serious safety hazards. Currently, most of the safety measures on the market focus on the monitoring of drivers and front seats. For example, a driver fatigue monitoring system can judge whether a driver is fatigued by detecting the eye closing time, head posture, etc. of the driver; the safety monitoring system for the co-driver seat can detect whether a passenger has fastened the seat belt or whether there are other dangerous actions. However, the protection of these systems for rear passengers is significantly insufficient, and there are monitoring blind spots, making rear passengers lack sufficient safety protection during driving. Most of the existing technologies for monitoring rear passengers are limited to simple passenger locking or in-vehicle personnel quantity monitoring, and cannot detect and warn of potential dangerous behaviors in real time. Especially in the case of multiple passengers, the current systems are difficult to cover all rear seat positions. This monitoring defect may lead to the failure to intervene in dangerous behaviors in time, thus increasing the risks for passengers, especially children and pets. Summary of the Invention

[0003] To solve the above technical problems, this application proposes a method, system, and storage medium for monitoring dangerous states, aiming to solve the technical problems in the prior art that the monitoring of the behavior states of rear passengers in a vehicle is insufficient and potential dangerous states cannot be monitored and warned in real time.

[0004] Specifically, this application proposes a method for monitoring dangerous states, including:

[0005] Obtaining passenger image data and cabin environment data in real time.

[0006] Based on an action recognition model, performing real-time monitoring on the passenger image data to obtain the passenger action state.

[0007] Performing a dangerous state assessment based on the passenger action state and the cabin environment data, and taking corresponding alarm feedback operations according to the assessment result.

[0008] In the above technical solution, the action recognition model is used to monitor the passenger image data in real time, realizing the real-time monitoring and high-precision recognition of the behavior states of rear passengers. It can accurately capture dangerous behaviors such as passengers' hands or heads protruding out of the window, improving the accuracy of identifying passengers' dangerous behaviors and effectively ensuring the safety of passengers. Through the dangerous state assessment, the normal behaviors and dangerous behaviors of passengers are effectively distinguished, reducing unnecessary misjudgments and improving the user experience. By taking corresponding alarm feedback operations based on the evaluation results, the alarm feedback is made more flexible, reducing interference to passengers while ensuring passenger safety and improving the comfort of the cabin environment.

[0009] As an implementation manner, after the passenger image data and the cabin environment data are obtained in real time, it further includes:

[0010] Preprocessing the passenger image data and the cabin environment data.

[0011] Based on the preprocessed passenger image data, perform time synchronization and spatial calibration operations on the cabin environment data to obtain the final cabin environment data.

[0012] By preprocessing the passenger image data, the noise in the passenger image data can be removed, improving the image quality and thus the accuracy of subsequent processing and analysis. By performing time synchronization operations on the cabin environment data, the consistency of the data in time is ensured, thereby improving the ability to identify and analyze the subsequent behavior states of passengers. Through the spatial calibration operation, the image data and the cabin environment data are aligned in space, making the monitoring of passengers' behavior states more accurate.

[0013] Furthermore, the pre-trained action recognition model includes:

[0014] Construct an action recognition model based on a deep learning algorithm, obtain historical passenger image data, and train the action recognition model through the historical passenger image data; perform optimization processing on the trained action recognition model based on a preset optimization algorithm to obtain the final action recognition model.

[0015] By training the action recognition model through the historical passenger image data, the action recognition model can learn to identify different passenger action state features from the image data. The action recognition model constructed based on the deep learning algorithm makes the recognition of passenger action states more accurate, improving the accuracy and reliability of passenger action state monitoring. Through the preset optimization algorithm, the generalization ability of the action recognition model is improved, optimizing the complexity and running speed of the action recognition model. The robustness and efficiency of the action recognition model are improved.

[0016] Further, the risk assessment based on the passenger motion state and the cabin environment data includes:

[0017] Fusing the passenger motion state and the final cabin environment data to obtain fused data.

[0018] Performing a risk assessment on the fused data according to a preset rule base and a preset risk assessment algorithm to obtain an assessment result.

[0019] By fusing the passenger motion state and the final cabin environment data, it is possible to comprehensively identify the changes in the passenger's behavior and the cabin environment. By combining the preset rule base with the preset risk assessment algorithm, the assessment of the passenger's dangerous behavior state is made more accurate, enabling the precise identification of the passenger's dangerous behavior actions in various different situations.

[0020] Further, the assessment result includes at least a dangerous state and a safe state; the dangerous state includes at least a first-level dangerous state and a second-level dangerous state.

[0021] Performing a risk assessment based on the fused data. If it is detected that the passenger has not left the seat and no body part has protruded out of the window, it is determined that the passenger motion state is a safe state.

[0022] If it is detected that the passenger has left the seat and / or a body part has protruded out of the window but not exceeded the preset range, it is determined that the passenger motion state is a first-level dangerous state.

[0023] If it is detected that the passenger has left the seat, a body part has protruded out of the window and exceeded the preset range, it is determined that the passenger motion state is a second-level dangerous state.

[0024] By dividing the dangerous state into a first-level dangerous state and a second-level dangerous state, it is possible to conduct a refined assessment of the passenger motion state, enabling timely differentiation and handling in different degrees of dangerous situations, thereby improving the overall response efficiency. By monitoring whether the passenger has left the seat and whether a body part has protruded out of the window, it is determined whether the passenger is in a dangerous state, avoiding false alarms and missed alarms of dangerous behaviors and ensuring the accuracy of monitoring the passenger motion state.

[0025] Further, the corresponding alarm feedback operation based on the assessment result includes:

[0026] When the passenger motion state is a first-level dangerous state, a first alarm feedback operation is taken, and the first alarm feedback operation includes at least an audible and visual alarm operation.

[0027] When the passenger action state is in the secondary danger state, a second alarm feedback operation is taken, and the second alarm feedback operation at least includes an audible and visual alarm operation, a seat vibration alarm operation, and an alarm display operation.

[0028] By taking different alarm feedback operations according to different passenger action states, when in the primary danger state, the passenger is reminded to pay attention to safety by means of an audible and visual alarm. This kind of alarm feedback operation is not too intense, avoiding causing too much interference to other passengers. When in the secondary danger state, the reminder effect is enhanced through the seat vibration alarm and the alarm display operation, which can effectively remind the passenger.

[0029] Further, after taking the corresponding alarm feedback operation according to the evaluation result, it also includes:

[0030] Based on the evaluation result and the alarm feedback operation, record the passenger action state and the alarm feedback information, pack and compress the passenger action state and the alarm feedback information to obtain a data packet.

[0031] Encrypt and store the data packet according to the data encryption algorithm.

[0032] Among them, the passenger action state at least includes the state time and the state type; the alarm feedback information at least includes the taken alarm operation and the response time.

[0033] Encrypting and storing the data packet through the data encryption algorithm ensures the confidentiality and integrity of the data, effectively protecting the privacy and security of users. By packing and compressing the passenger action state machine alarm feedback information, the occupation of storage space can be significantly reduced, improving the efficiency of data storage. The stored passenger action state and alarm feedback information can be used to train and optimize the action recognition model to improve the accuracy of recognizing the passenger action state.

[0034] Further, after taking the corresponding alarm feedback operation according to the evaluation result, it also includes:

[0035] Calibrate and calibrate each sensor device according to the cabin environment data after time synchronization and space calibration. By calibrating and calibrating each sensor device, the accuracy and reliability of collecting passenger image data and cabin environment data are improved.

[0036] Based on the same inventive concept, the present application also proposes a system for a dangerous state monitoring method, and the system includes:

[0037] A data acquisition module, configured to acquire passenger image data and cabin environment data in real time.

[0038] An action recognition module, configured to perform real-time monitoring on the passenger image data based on an action recognition model to obtain the passenger action status.

[0039] A risk assessment module, configured to perform a dangerous state assessment according to the passenger action status and the cabin environment data.

[0040] And an alarm module, configured to perform corresponding alarm feedback operations according to the evaluation result.

[0041] Based on the same inventive concept, the present application also provides a computer-readable storage medium storing computer-executable instructions that can be read and executed by a domain controller to perform the dangerous state monitoring method.

[0042] Compared with the prior art, the present application has at least the following beneficial effects:

[0043] The dangerous state monitoring method provided by the present application solves the technical problems in the prior art that the monitoring of the action status of rear passengers in a vehicle is insufficient and the potential dangerous state cannot be monitored and warned in real time. By performing real-time monitoring on the passenger image data through an action recognition model, the real-time monitoring and high-precision recognition of the behavior status of rear passengers are realized, the dangerous behavior of the passenger's body part extending out of the window is accurately captured, the accuracy of identifying the dangerous behavior of the passenger is improved, and the safety of the passenger is effectively guaranteed. Through the dangerous state assessment, the normal behavior and dangerous behavior of the passenger are effectively distinguished, unnecessary misjudgments are reduced, and the user experience is improved. By performing corresponding alarm feedback operations according to the evaluation result, the alarm feedback is made more flexible, the interference to the passenger can be reduced on the premise of ensuring the safety of the passenger, and the comfort of the cabin environment is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of the dangerous state monitoring method shown in an embodiment of the present application.

[0045] Figure 2 is a schematic diagram of the dangerous state monitoring system shown in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present application belong to the scope of protection of the present application.

[0047] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] Embodiment 1:

[0049] Please refer to Figure 1 , the dangerous state monitoring method mainly includes steps S1 to S3.

[0050] Among them, step S1 includes: obtaining passenger image data and cabin environment data in real time. Among them, the passenger image data and cabin environment data can be mainly collected by a multi-sensor device. The multi-sensor device mainly can include an infrared sensor, an ultrasonic sensor, and a camera. The infrared sensor is used to detect the temperature change in the cabin in real time for identifying the passenger position and action state. The infrared sensor can be mainly arranged around the seats and on the top of the cabin, and the acquisition range covers each vehicle seat. The ultrasonic sensor is used to measure the distance change between the passenger and the seat and the window, and can be mainly installed on the top and sides of the cabin. The camera is used to collect the image data of the passenger in real time. The camera can be a combination of a high-resolution infrared camera and a visible light camera installed on both sides of the cabin. Those skilled in the art can select other sensor devices and adjust the installation positions of the sensor devices according to the actual situation, and are not limited thereto. The data collected by the multi-sensor device can be transmitted through a low-latency bus. For example, the data can be transmitted through an I2C bus (Inter-Integrated Circuit, communication between integrated circuits) and a CAN bus (Controller Area Network, controller area network).

[0051] Step S2 includes: Based on the action recognition model, the passenger image data is monitored in real time to obtain the passenger action state. The action recognition model can mainly be a deep learning model, such as a neural network model, a YOLO model (You Only Look Once), and an SSD model (Single Shot MultiBox Detector). The action recognition model can be deployed to an in-vehicle edge computing device to monitor the passenger image data in real time. For example, the action recognition model can be deployed to an (NVIDIA Jetson Embedded Computing Platform) or a Google Edge TPU (Google Edge Tensor Processing Unit).

[0052] Step S3 includes: Conduct a dangerous state assessment based on the passenger action state and the cabin environment data, and take corresponding alarm feedback operations according to the assessment results. Among them, the assessment results can include the danger level and the required response time. The alarm feedback operations can be to issue a sound alarm through the in-vehicle buzzer, flash an alarm through the LED indicator, and give an alarm through the seat vibration. The intensity and method of the alarm can be adjusted according to the driver or passenger to adapt to different user preferences and special needs. For example, the passenger can choose to only use seat vibration and turn off the sound and light alarms.

[0053] In the specific implementation process, infrared sensors, ultrasonic sensors, and cameras can be installed on the top and both sides of the cabin to collect passenger image data and cabin environment data in real time. The passenger image data is monitored in real time through an action recognition model constructed in advance using a neural network model to obtain the passenger action state. A dangerous state assessment is conducted based on the passenger action state and the cabin environment data to obtain the danger level of the passenger action state, and sound and light alarm operations or seat vibration alarms are taken based on the danger level of the passenger action state.

[0054] In some embodiments, after the passenger image data and the cabin environment data are obtained in real time, it further includes:

[0055] Preprocess the passenger image data and the cabin environment data.

[0056] Based on the preprocessed passenger image data, perform time synchronization and spatial calibration operations on the cabin environment data to obtain the final cabin environment data.

[0057] The preprocessing may at least include grayscale processing, denoising processing, and contrast enhancement processing through filtering techniques to improve the quality of image data. The filtering technique may be a Kalman filtering technique. The cabin environment data can be time-synchronized based on the timestamps in the preprocessed passenger image data, and the cabin environment data can be spatially calibrated based on the spatial positions of the preprocessed passenger image data.

[0058] Optionally, the pre-trained action recognition model includes:

[0059] Construct an action recognition model based on a deep learning algorithm.

[0060] Obtain historical passenger image data to train the action recognition model with the historical passenger image data.

[0061] Optimize the trained action recognition model based on a preset optimization algorithm to obtain the final action recognition model.

[0062] Among them, the deep learning algorithm may be a convolutional neural network, a recurrent neural network, etc. By annotating the historical passenger image data, the action recognition model is trained with the annotated dataset. The specific annotation content may include information such as passenger action types and time. When training the action recognition model in actual situations, to improve the generalization ability of the model, it is necessary to ensure the diversity of historical passenger image data. The action recognition model is trained with the image data of passengers in various different situations. For example, the action recognition model is trained after annotating the image data of passengers under different lighting conditions, different perspectives, different ages, genders, and heights.

[0063] The preset optimization algorithm may include data augmentation techniques, pruning processing, and quantization techniques. Through the data augmentation techniques, the generalization ability of the action recognition model can be effectively improved. By pruning the action recognition model, the complexity of the model is reduced. By optimizing the action recognition model through quantization techniques, the running efficiency of the action recognition model is improved.

[0064] Optionally, the risk status assessment according to the passenger action status and cabin environment data includes:

[0065] Fuse the passenger action status with the final cabin environment data to obtain fused data.

[0066] Perform a risk assessment on the fused data according to a preset rule base and a preset risk assessment algorithm to obtain an assessment result.

[0067] The passenger action status and the final cabin environment data can be integrated through the ROS (Robot Operating System) platform, and the passenger action status and the final cabin environment data can be converted into a unified format. The passenger action status and the final cabin environment data can be fused through a Kalman filter. The preset risk assessment algorithm can be a time series analysis algorithm based on XGBoost (eXtreme Gradient Boosting) or LSTM (Long Short-Term Memory).

[0068] Optionally, the evaluation result includes at least a dangerous state and a safe state; the dangerous state includes at least a first-level dangerous state and a second-level dangerous state.

[0069] Based on the fused data, a risk assessment is performed. If it is detected that the passenger has not left the seat and no body part extends out of the window, it is determined that the passenger action status is a safe state.

[0070] If it is detected that the passenger leaves the seat and / or a body part extends out of the window but does not exceed the preset range, it is determined that the passenger action status is a first-level dangerous state.

[0071] If it is detected that the passenger leaves the seat, a body part extends out of the window and exceeds the preset range, it is determined that the passenger action status is a second-level dangerous state.

[0072] When it is determined that the passenger action status is a safe state, it indicates that the current passenger's actions are normal and no potential dangerous behaviors have occurred. When the passenger leaves the seat or a body part extends out of the window, it indicates that there are potential dangers for the current passenger. The preset range can be 15 cm or 10 cm. For example, when a child sitting on the rear seat stands up from the seat or extends his head or hand out of the window by 5 cm, it is determined that the action status of the current child is a first-level dangerous state. Those skilled in the art can adjust the preset range according to the actual situation, and it is not limited to this. For example, when the preset range is set to 13 cm, when the rear passenger stands up and leaves the seat, and the head extends out of the window by 20 cm, it is determined that the action status of the current rear passenger is a second-level dangerous state.

[0073] Optionally, the corresponding alarm feedback operation is taken according to the evaluation result, including:

[0074] When the passenger action status is a first-level dangerous state, a first alarm feedback operation is taken, and the first alarm feedback operation includes at least an audible and visual alarm operation.

[0075] When the passenger action state is in the secondary danger state, a second alarm feedback operation is taken, and the second alarm feedback operation at least includes an audible and visual alarm operation, a seat vibration alarm operation, and an alarm display operation.

[0076] Among them, the audible and visual alarm operation mainly performs alarm feedback through a buzzer and an LED indicator configured in the vehicle cabin. When the passenger action state is in the primary danger state, a prompt sound is emitted by the buzzer and the LED indicator flashes to remind the passenger. The seat vibration alarm operation mainly performs vibration alarm through a vibration motor integrated on the seat. When the passenger action state is in the secondary danger state, the vibration motor is controlled to vibrate to remind the passenger and guide the passenger to adjust the sitting posture and actions. The alarm display operation is mainly displayed through the human-machine interaction display interface of the vehicle host. In the above audible and visual alarm operation, the sound intensity emitted by the buzzer and the flashing frequency of the LED indicator can be adjusted according to the danger state level.

[0077] Optionally, after taking the corresponding alarm feedback operation according to the evaluation result, it further includes:

[0078] Based on the evaluation result and the alarm feedback operation, record the passenger action state and alarm feedback information, pack and compress the passenger action state and alarm feedback information to obtain a data packet. Encrypt and store the data packet according to the data encryption algorithm; among them, the passenger action state at least includes the state time and the state type; the alarm feedback information at least includes the taken alarm operation and the response time.

[0079] When encrypting and storing the data packet, the data packet can be stored in a large-capacity storage device, such as an SD card (Secure Digital Card) or a solid-state drive. The data encryption algorithm can be the AES (Advanced Encryption Standard) encryption algorithm.

[0080] Optionally, after taking the corresponding alarm feedback operation according to the evaluation result, it further includes:

[0081] Calibrate and calibrate each sensor device according to the cabin environment data after time synchronization and space calibration. For example, the ultrasonic sensor can be calibrated through the ultrasonic data after time synchronization and space calibration, and the infrared sensor device can be calibrated through the temperature data after time synchronization and space calibration.

[0082] Embodiment 2:

[0083] Please refer to Figure 2, this application also proposes a system that adopts the dangerous state monitoring method described in Embodiment 1, mainly including: a data acquisition module, an action recognition module, a risk assessment module, and an alarm module.

[0084] Among them, the data acquisition module is used to acquire passenger image data and cabin environment data in real time. Multiple sensor devices can be used to collect data in the data acquisition module. For example, the cabin environment temperature, passenger position, and passenger status can be collected through an infrared sensor; the distances between the passenger and the seat and the window can be collected through an ultrasonic sensor. The passenger image data can be collected by a high-resolution infrared camera to adapt to low-light environments. The image data and cabin environment data collected by multiple sensor devices can be transmitted to the action recognition module through a low-latency bus.

[0085] The action recognition module is used to monitor the passenger image data in real time based on an action recognition model to obtain the passenger action state. Among them, the action recognition model can mainly be a deep learning module, and the action recognition model is trained by obtaining historical passenger image data in various different scenarios to obtain the final action recognition model. The passenger image data is monitored in real time based on the final action recognition model.

[0086] The risk assessment module is used to perform a dangerous state assessment based on the passenger action state and cabin environment data. Among them, the results of the dangerous state assessment of the passenger action state and cabin environment data at least include a safe state and a dangerous state. The dangerous state at least includes a first-level dangerous state and a second-level dangerous state.

[0087] And, the alarm module is used to take corresponding alarm feedback operations according to the evaluation results. Among them, according to the evaluation results, when the passenger action state is a first-level dangerous state, a first alarm feedback operation is taken, such as an audible and visual alarm operation. When the passenger action state is a second-level dangerous state, a second alarm feedback operation is taken, such as an audible and visual alarm operation, a seat vibration alarm operation, and an alarm display operation.

[0088] Embodiment 3:

[0089] This application also proposes a computer-readable storage medium, and the computer-readable storage medium includes:

[0090] The computer-readable storage medium stores computer-executable instructions.

[0091] When the computer-executable instructions are executed by a control processor, the dangerous state monitoring method described in Embodiment 1 is implemented.

[0092] In the computer-readable storage medium, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)), etc.

[0093] In summary, the present application uses an action recognition model to monitor passenger image data in real time, realizes real-time monitoring and high-precision recognition of the behavior status of rear passengers, accurately captures the dangerous behavior of passengers' body parts protruding out of the window, improves the accuracy of recognizing passengers' dangerous behaviors, and effectively ensures the safety of passengers. Through the dangerous state assessment, it effectively distinguishes between normal and dangerous behaviors of passengers, reduces unnecessary misjudgments, and improves the user experience. By taking corresponding alarm feedback operations based on the assessment results, the alarm feedback is made more flexible, and the interference to passengers can be reduced while ensuring the safety of passengers, improving the comfort of the cabin environment.

[0094] In several embodiments provided in the present application, it can be understood that each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.

[0095] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0096] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of this application. It should be understood that the above are only specific embodiments of this application and are not used to limit the protection scope of this application. In particular, it is pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A dangerous state monitoring method, characterized in that: Obtaining a pre-trained action recognition model, the dangerous state monitoring method includes: Acquire passenger image data and cabin environment data in real time; Based on the action recognition model, the passenger image data is monitored in real time to obtain the passenger action status; A dangerous state assessment is performed based on the passenger action state and cabin environment data, and corresponding alarm feedback operations are taken based on the assessment results.

2. The dangerous state monitoring method according to claim 1, characterized in that: After the real-time acquisition of passenger image data and cabin environment data, the method further includes: Preprocessing the passenger image data and cabin environment data; The cabin environment data is subjected to time synchronization and space calibration operations based on the preprocessed passenger image data to obtain final cabin environment data.

3. The dangerous state monitoring method according to claim 1, characterized in that: The pre-trained action recognition model comprises: Build action recognition models based on deep learning algorithms; Acquiring historical passenger image data to train the action recognition model through the historical passenger image data; The trained action recognition model is optimized based on a preset optimization algorithm to obtain the final action recognition model.

4. The dangerous state monitoring method according to claim 2, characterized in that: The performing of dangerous state assessment according to the passenger action state and cabin environment data includes: Fusing the passenger motion state with the final cabin environment data to obtain fused data; The fused data is subjected to risk assessment according to a preset rule base and a preset risk assessment algorithm to obtain an assessment result.

5. The dangerous state monitoring method according to claim 4, characterized in that: The evaluation result includes at least a dangerous state and a safe state; the dangerous state includes at least a first-level dangerous state and a second-level dangerous state; Performing risk assessment based on the fused data, if it is detected that the passenger has not left the seat and has not extended any body part out of the window, then determining that the passenger's motion state is a safe state; If it is detected that the passenger leaves the seat and / or the body part extends out of the window but does not exceed the preset range, the passenger's action state is determined to be a first-level dangerous state; If it is detected that the passenger has left the seat and the body part is extended out of the window and exceeds the preset range, the passenger's action state is determined to be a secondary dangerous state.

6. The dangerous state monitoring method according to claim 5, characterized in that: The taking of corresponding alarm feedback actions according to the evaluation results includes: When the passenger's motion state is a first-level dangerous state, a first alarm feedback operation is performed, wherein the first alarm feedback operation at least includes an audible and visual alarm operation; When the passenger's motion state is a level-two dangerous state, a second alarm feedback operation is performed, and the second alarm feedback operation at least includes an audible and visual alarm operation, a seat vibration alarm operation, and an alarm display operation.

7. The dangerous state monitoring method according to claim 1, characterized in that: After taking the corresponding alarm feedback operation according to the evaluation result, the method further includes: Based on the evaluation result and the alarm feedback operation record, the passenger action status and the alarm feedback information are packaged and compressed to obtain a data packet; Encrypting and storing the data packet according to a data encryption algorithm; The passenger action status at least includes the status time and the status type; the alarm feedback information at least includes the alarm operation taken and the response time.

8. The dangerous state monitoring method according to claim 2, characterized in that: After taking the corresponding alarm feedback operation according to the evaluation result, the method further includes: Each sensor device is calibrated and calibrated according to the cabin environment data after time synchronization and spatial calibration.

9. A system based on the dangerous state monitoring method according to any one of claims 1 to 8, characterized in that: The system includes: a data acquisition module for acquiring passenger image data and cabin environment data in real time; A motion recognition module, used for monitoring the passenger image data in real time based on a motion recognition model to obtain the passenger's motion status; A risk assessment module, used for performing a risk state assessment based on the passenger action state and cabin environment data; And, an alarm module is used to take corresponding alarm feedback actions according to the evaluation results.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by the control processor, the dangerous state monitoring method as described in any one of claims 1-8 is implemented.

Images