Bus safety warning method, warning device, storage medium and warning system

Abstract

The application relates to a bus safety early warning method, an early warning device, a storage medium and an early warning system. The method comprises the following steps: acquiring video data in a bus in a target period and audio data in the bus in the target period; determining feature information of a safety event in the target period according to the video data in the bus and the audio data in the bus, wherein the feature information at least comprises a type; determining a safety early warning strategy of the target period according to the feature information of the safety event; and executing the safety early warning strategy. The method solves the problem of low reliability of the bus early warning technology in the prior art.

Description

Technical Field

[0001] This application relates to the field of bus safety early warning, and in particular to a bus safety early warning method, early warning device, storage medium and early warning system. Background Technology

[0002] Existing bus safety early warning technologies mainly rely on single onboard video surveillance as the data foundation and visual recognition algorithms to perceive the types of safety events inside the carriage. In scenarios such as dense crowds and sudden changes in lighting, image feature extraction is easily affected by environmental noise, leading to a significant increase in the false detection rate of safety event types. False detections can trigger incorrect safety early warning strategies, seriously weakening the reliability of bus early warning technologies. Summary of the Invention

[0003] This application provides a bus safety early warning method, a bus safety early warning device, a computer-readable storage medium, and a bus safety early warning system to solve the problem of low reliability in existing bus early warning technologies.

[0004] In a first aspect, this application provides a bus safety early warning method, which is applied to an onboard intelligent terminal of the bus. The method includes: acquiring in-bus video data and in-bus audio data for a target time period; determining characteristic information of a safety event during the target time period based on the in-bus video data and in-bus audio data, wherein the characteristic information includes at least a type; determining a safety early warning strategy for the target time period based on the characteristic information of the safety event; and executing the safety early warning strategy.

[0005] Optionally, determining the feature information of the safety event during the target time period based on the in-bus video data and the in-bus audio data includes: inputting the in-bus video data into a visual event recognition model to obtain the visual event type and confidence level of the visual event type during the target time period; inputting the in-bus audio data into an audio event recognition model to obtain the sound event type and confidence level of the sound event type during the target time period; and determining the feature information of the safety event based on the visual event type, the confidence level of the visual event type, the sound event type, and the confidence level of the sound event type during the target time period.

[0006] Optionally, the feature information further includes: a level. The feature information of the security event is determined based on the visual event type, the confidence level of the visual event type, the sound event type, and the confidence level of the sound event type during the target time period. This includes: when the confidence levels of the visual event type and the sound event type during the target time period are both greater than a confidence threshold, querying a preset mapping table based on the visual event type and the sound event type during the target time period to determine the type and level of the security event. The mapping table contains the mapping relationships between visual event type, sound event type, security event type, and security event level.

[0007] Optionally, based on the characteristic information of the safety event, the safety warning strategy for the target time period is determined, including: when the level of the safety event is level 1 and the type of the safety event is passenger conflict, at least the safety warning strategy is determined to include a first preset safety warning strategy, wherein the first preset safety warning strategy is to control the voice playback unit to play a dissuasive voice message; when the level of the safety event is level 2 and the type of the safety event is illegal behavior, at least the safety warning strategy is determined to include a second preset safety warning strategy, wherein the first level is lower than the second level, and the second preset safety warning strategy is to control the sound and light alarm unit to perform a sound and light alarm.

[0008] Optionally, when the level of the safety event is Level 1 and the type of the safety event is passenger conflict, at least the safety warning strategy is determined to include a first preset safety warning strategy, including: determining that the safety warning strategy includes a third preset safety warning strategy, wherein the third preset safety warning strategy is to send the first status data of the safety event to the dispatch cloud platform, and the first status data includes at least: the video data inside the bus, the audio data inside the bus, the type of the safety event, and the level of the safety event.

[0009] Optionally, when the level of the security event is Level 2 and the type of the security event is illegal behavior, at least it is determined that the security warning strategy includes a second preset security warning strategy, including: determining that the security warning strategy includes a fourth preset security warning strategy, wherein the fourth preset security warning strategy is to control the door to close via a door control relay.

[0010] Optionally, when the level of the security incident is Level 2 and the type of the security incident is illegal behavior, at least it is determined that the security warning strategy includes a second preset security warning strategy, including: determining that the security warning strategy includes a fifth preset security warning strategy, wherein the fifth preset security warning strategy is to send the second status data of the security incident to the public security police cloud platform, and the second status data includes at least: the video data inside the bus, the audio data inside the bus, the type of the security incident, the level of the security incident, the license plate image data of the bus, the location information of the bus, and the contact information of the bus driver.

[0011] Secondly, this application provides a bus safety early warning device, which is applied to the on-board intelligent terminal of the bus. The device includes: an acquisition unit for acquiring in-bus video data and in-bus audio data for a target time period; a first determination unit for determining characteristic information of a safety event during the target time period based on the in-bus video data and in-bus audio data, the characteristic information including at least a type; a second determination unit for determining a safety early warning strategy for the target time period based on the characteristic information of the safety event; and an execution unit for executing the safety early warning strategy.

[0012] Thirdly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the bus safety warning methods described above.

[0013] Fourthly, this application also provides a bus safety early warning system, the system comprising: a video acquisition unit located inside the bus for acquiring video data inside the bus; an audio acquisition unit located inside the bus for acquiring audio data inside the bus; a voice playback unit located inside the bus; an audible and visual alarm unit located inside the bus; a door control relay located inside the bus; and an on-board intelligent terminal located inside the bus, the on-board intelligent terminal being used to implement the steps of any of the bus safety early warning methods described above.

[0014] In this embodiment, the characteristic information of a safety event is determined not only based on video data inside the bus, but also by comprehensively considering audio data inside the bus. This ensures the accuracy of the characteristic information of the safety event, and thus ensures the accuracy of the safety warning strategy determined based on the characteristic information of the safety event. This improves the reliability of the bus warning technology and solves the problem of low reliability of the existing bus warning technology. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0018] Figure 1 A flowchart illustrating a bus safety early warning method provided in an embodiment of this application; Figure 2 This is a structural block diagram of a bus safety warning device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0021] To address the technical problem of low reliability in existing bus warning technologies, this application provides a bus safety warning method, a bus safety warning device, a computer-readable storage medium, and a bus safety warning system, which can solve the problem of low reliability in existing bus warning technologies.

[0022] Figure 1This is a flowchart illustrating a bus safety warning method provided in an embodiment of this application. The method is applied to the onboard intelligent terminal of the bus and includes: Step S101: Obtain the video data inside the bus during the target time period and the audio data inside the bus during the target time period. Step S102: Based on the aforementioned video data and audio data inside the bus, determine the characteristic information of the safety event during the target time period. The characteristic information includes at least: type. Specifically, safety incidents can include passenger conflicts, illegal acts, broken windows, smoke in the carriage, and fire in the carriage.

[0023] Step S103: Based on the characteristic information of the aforementioned security events, determine the security early warning strategy for the aforementioned target time period; Step S104: Execute the above-mentioned security warning strategy.

[0024] Through the above embodiments, the characteristic information of safety events is determined not only based on video data inside the bus, but also by comprehensively considering audio data inside the bus. This ensures the accuracy of the characteristic information of safety events, and in turn ensures the accuracy of determining safety warning strategies based on the characteristic information of safety events. This improves the reliability of bus warning technology and solves the problem of low reliability in existing bus warning technologies.

[0025] In an optional embodiment, step S102 above can be implemented as follows: The aforementioned video data from inside the bus is input into the visual event recognition model to obtain the visual event types and confidence levels for the visual event types during the target time period. Specifically, visual event types include: pushing, surrounding, entanglement, tailing, offensive weapons (including knives, blunt objects, and other hard objects with offensive capabilities), smoke, open flames, car doors not closed properly, and broken car windows.

[0026] Specifically, the visual event recognition model is a trained neural network model used to identify visual event types.

[0027] The above-mentioned audio data from inside the bus is input into the audio event recognition model to obtain the sound event types and confidence levels of the sound event types during the target time period. Specifically, sound events include: arguments, cries for help, smoke alarms, vehicle noise, and the sound of breaking glass.

[0028] Specifically, the audio event recognition model is a trained neural network model used to identify the types of sound events.

[0029] Based on the visual event types, confidence levels of the visual event types, sound event types, and confidence levels of the sound event types during the target time period, the characteristic information of the aforementioned security events is determined.

[0030] In this embodiment, firstly, the video data inside the bus is input into the visual event recognition model to obtain the visual event type and confidence level of the visual event type during the target time period. Then, the audio data inside the bus is input into the audio event recognition model to obtain the sound event type and confidence level of the sound event type during the target time period. Finally, based on the visual event type, confidence level, sound event type, and confidence level of the sound event type during the target time period, the characteristic information of the safety event is determined. This approach considers not only the video data inside the bus but also the audio data inside the bus, thereby ensuring the accuracy of the characteristic information of the safety event.

[0031] In an optional embodiment, the aforementioned feature information further includes: a level, which determines the feature information of the security event based on the visual event type of the target time period, the confidence level of the visual event type of the target time period, the sound event type of the target time period, and the confidence level of the sound event type of the target time period, including: When the confidence levels of both the visual event type and the sound event type during the target time period are greater than the confidence threshold, a preset mapping table is queried based on the visual event type and the sound event type during the target time period to determine the type and level of the security event. The mapping table contains the mapping relationships between visual event type, sound event type, security event type, and security event level.

[0032] Specifically, for example, with a confidence threshold of 70%, the visual event type for the target time period is pushing and shoving, and the confidence level of the visual event type for the target time period is 75%. The audio event type for the target time period is arguing, and the confidence level of the audio event type for the target time period is 80%. In this case, the confidence level of the visual event type for the target time period is greater than 70%, and the confidence level of the audio event type for the target time period is also greater than 70%. Therefore, based on the visual event type and the audio event type for the target time period, a preset mapping table is consulted to determine the type of the safety event as passenger conflict, and the level of the safety event is determined to be Level 1. For example, if the confidence threshold is 70%, the visual event type during the target time period is an offensive device, the confidence level of the visual event type during the target time period is 80%, the sound event type during the target time period is a cry for help, and the confidence level of the sound event type during the target time period is 85%, then the confidence level of the visual event type during the target time period is greater than 70%, and the confidence level of the sound event type during the target time period is also greater than 70%. Therefore, based on the visual event type and the sound event type during the target time period, a preset mapping table is consulted to determine that the type of the security event is illegal behavior, and the level of the security event is determined to be the second level.

[0033] Specifically, for example, with a confidence threshold of 75%, the visual event type for the target time period is smoke emanating from the carriage, and the confidence level for this visual event type is 85%. The sound event type for the target time period is a smoke alarm sound, and the confidence level for this sound event type is 80%. In this case, the confidence levels for both the visual and sound event types for the target time period are greater than 75%. Therefore, based on the visual and sound event types for the target time period, a pre-defined mapping table is consulted to determine the type of safety event as smoke emanating from the carriage, and the safety event level is determined to be Level 1. For example, with a confidence threshold of 75%, the visual event type for the target time period is a fire in the carriage, and the confidence level of the visual event type for the target time period is 80%. The sound event type for the target time period is a smoke alarm sound, and the confidence level of the sound event type for the target time period is 82%. At this point, the confidence level of the visual event type for the target time period is greater than 75%, and the confidence level of the sound event type for the target time period is also greater than 75%. Therefore, based on the visual event type and the sound event type for the target time period, a preset mapping table is consulted to determine that the type of safety event is a fire in the carriage, and the level of the safety event is determined to be Level 2.

[0034] In this embodiment, only when the confidence levels of both the visual event type and the sound event type in the target time period are greater than the confidence threshold, a preset mapping table is queried based on the visual event type and the sound event type in the target time period to determine the type and level of the security event. This ensures the accuracy of the determined security event type and the accuracy of the determined security event level.

[0035] In an optional embodiment, step S103 above can be implemented as follows: When the level of the aforementioned safety incident is Level 1 and the type of the aforementioned safety incident is passenger conflict, it is determined that the aforementioned safety warning strategy includes at least a first preset safety warning strategy, wherein the aforementioned first preset safety warning strategy is to control the voice playback unit to play dissuasive voice messages. Specifically, the persuasive voice message could be something like, "Please remain calm and give way to each other in the carriage."

[0036] When the level of the aforementioned security incident is Level 2 and the type of the aforementioned security incident is illegal behavior, it is determined that the aforementioned security warning strategy includes at least a second preset security warning strategy, wherein the aforementioned Level 1 is lower than the aforementioned Level 2, and the aforementioned second preset security warning strategy is to control the audible and visual alarm unit to perform audible and visual alarm.

[0037] In this embodiment, when the safety event is classified as Level 1 and is a passenger conflict, a mild safety warning strategy is adopted, which involves playing a dissuasive voice message through the voice playback unit. This effectively prevents the passenger conflict from escalating further and avoids unnecessary disturbance to other passengers in the carriage due to overreaction. When the safety event is classified as Level 2 and is an illegal act, the sound and light alarm unit is activated to trigger a sound and light alarm. This strong sensory stimulus creates a deterrent effect, achieving the purpose of mandatory intervention and suppressing the escalation of the situation. This method of graded warning based on the level of the safety event avoids the waste of resources and decline in passenger experience caused by overreaction to passenger conflicts, while also avoiding the risk of loss of control due to insufficient response to illegal acts, further improving the reliability of bus warning technology.

[0038] In an optional embodiment, when the level of the aforementioned security event is Level 1 and the type of the aforementioned security event is passenger conflict, determining that the aforementioned security warning strategy includes a first preset security warning strategy can be implemented as follows: The aforementioned safety warning strategy includes a third preset safety warning strategy, which involves sending the first status data of the aforementioned safety event to the dispatch cloud platform. The first status data includes at least: the aforementioned video data inside the bus, the aforementioned audio data inside the bus, the type of the aforementioned safety event, and the level of the aforementioned safety event.

[0039] Specifically, the dispatch cloud platform is located on one side of the bus dispatch center.

[0040] In this embodiment, when the level of the safety incident is Level 1 and the type of the safety incident is passenger conflict, the aforementioned video data inside the bus, the aforementioned audio data inside the bus, the aforementioned type of safety incident, and the aforementioned level of the safety incident are uploaded to the dispatch cloud platform. This allows the dispatchers at the bus dispatch center to intuitively recreate the real scene of the conflict in the carriage. If it is found that the passenger conflict continues to escalate and turns into illegal behavior, the dispatcher can manually upgrade the level of the safety incident to Level 2 through the operation interface.

[0041] In an optional embodiment, when the level of the aforementioned security event is Level 1 and the type of the aforementioned security event is a passenger conflict, after step S104, the method further includes: Obtain video data and audio data inside the bus during the verification period, which is located after the target period. The above-mentioned video data inside the bus is input into the above-mentioned visual event recognition model to obtain the visual event types and confidence levels of the visual event types during the above-mentioned verification period. The above-mentioned audio data from inside the bus is input into the above-mentioned audio event recognition model to obtain the sound event types and the confidence levels of the sound event types during the above-mentioned verification period. If the visual event type during the aforementioned verification period is an offensive device, the confidence level of the visual event type during the aforementioned verification period is greater than the aforementioned confidence threshold, and the sound event type during the aforementioned verification period is a cry for help and the confidence level of the sound event type during the aforementioned verification period is greater than the aforementioned confidence threshold, then the type of the aforementioned security event will be adjusted to the aforementioned illegal behavior and the level of the aforementioned security event will be adjusted to the aforementioned second level.

[0042] In this embodiment, the response window between the initial handling of a security incident and manual confirmation is eliminated. The situation escalation judgment can be completed autonomously without waiting for dispatcher intervention, thus gaining valuable golden time for subsequent mandatory handling (audio and visual alarms, door locking, and police coordination).

[0043] In an optional embodiment, when the level of the aforementioned security event is Level 2 and the type of the aforementioned security event is illegal behavior, determining that the aforementioned security warning strategy includes a second preset security warning strategy can be implemented as follows: The aforementioned safety warning strategy includes a fourth preset safety warning strategy, which involves controlling the door to close via a door control relay.

[0044] In this embodiment, when the level of the security incident is the second level and the type of the security incident is illegal behavior, the door control relay is used to control the door to close, so as to prevent the situation from spreading outside the vehicle or the illegal person from escaping.

[0045] In an optional embodiment, when the security event is classified as Level 2 and its type is illegal activity, it is determined that the security warning strategy includes a second preset security warning strategy, comprising: The aforementioned security early warning strategy includes a fifth preset security early warning strategy, which involves sending the second status data of the aforementioned security event to the public security cloud platform. The second status data includes at least: the video data inside the bus, the audio data inside the bus, the type of the aforementioned security event, the level of the aforementioned security event, the license plate image data of the aforementioned bus, the location information of the aforementioned bus, and the contact information of the driver of the aforementioned bus.

[0046] Specifically, the location information can be GPS positioning information.

[0047] In this embodiment, automated data reporting replaces traditional manual telephone alarms, effectively avoiding the risks of drivers making unclear statements, omitting key information, or delaying the reporting time in tense situations. This ensures the integrity and timeliness of the alarm information, giving the police valuable time to respond quickly. The reported data includes images of the bus license plate, precise location information, and the driver's contact information, enabling the alarm receiving center to immediately identify the vehicle and its specific location, eliminating delays caused by ambiguous location descriptions. It also facilitates nearby patrol officers to respond, achieving precise deployment and efficient dispatch of police resources. The simultaneous reporting of the driver's contact information establishes a direct communication link between the police and the driver, allowing for real-time access to the latest situation during the mission and guiding the driver to cooperate with initial handling. This ensures that upon arrival, the police can quickly connect with the on-site supervisor and efficiently take over control, thus achieving a seamless closed loop in the entire alarm handling process. After the complete audio and video data is simultaneously uploaded to the police cloud platform, it can be directly preserved as electronic evidence, providing legal and compliant original audio-visual materials for subsequent administrative penalties or criminal case filing.

[0048] Figure 2 This application provides a structural block diagram of a bus safety warning device, which is applied to the onboard intelligent terminal of the bus. The device includes: The acquisition unit 10 is used to acquire the video data inside the bus during the target time period and the audio data inside the bus during the target time period. The first determining unit 20 is used to determine the characteristic information of the safety event during the target time period based on the aforementioned video data and audio data inside the bus. The characteristic information includes at least: type. Specifically, safety incidents can include passenger conflicts, illegal acts, broken windows, smoke in the carriage, and fire in the carriage.

[0049] The second determining unit 30 is used to determine the security early warning strategy for the target time period based on the characteristic information of the aforementioned security event. The execution unit 40 is used to execute the aforementioned security warning strategy.

[0050] Through the above embodiments, the characteristic information of safety events is determined not only based on video data inside the bus, but also by comprehensively considering audio data inside the bus. This ensures the accuracy of the characteristic information of safety events, and in turn ensures the accuracy of determining safety warning strategies based on the characteristic information of safety events. This improves the reliability of bus warning technology and solves the problem of low reliability in existing bus warning technologies.

[0051] In an optional embodiment, the first determining unit described above is used for: The aforementioned video data from inside the bus is input into the visual event recognition model to obtain the visual event types and confidence levels for the visual event types during the target time period. Specifically, visual event types include: pushing, surrounding, entanglement, tailing, offensive weapons (including knives, blunt objects, and other hard objects with offensive capabilities), smoke, open flames, car doors not closed properly, and broken car windows.

[0052] Specifically, the visual event recognition model is a trained neural network model used to identify visual event types.

[0053] The above-mentioned audio data from inside the bus is input into the audio event recognition model to obtain the sound event types and confidence levels of the sound event types during the target time period. Specifically, sound events include: arguments, cries for help, smoke alarms, vehicle noise, and the sound of breaking glass.

[0054] Specifically, the audio event recognition model is a trained neural network model used to identify the types of sound events.

[0055] Based on the visual event types, confidence levels of the visual event types, sound event types, and confidence levels of the sound event types during the target time period, the characteristic information of the aforementioned security events is determined.

[0056] In this embodiment, firstly, the video data inside the bus is input into the visual event recognition model to obtain the visual event type and confidence level of the visual event type during the target time period. Then, the audio data inside the bus is input into the audio event recognition model to obtain the sound event type and confidence level of the sound event type during the target time period. Finally, based on the visual event type, confidence level, sound event type, and confidence level of the sound event type during the target time period, the characteristic information of the safety event is determined. This approach considers not only the video data inside the bus but also the audio data inside the bus, thereby ensuring the accuracy of the characteristic information of the safety event.

[0057] In an optional embodiment, the aforementioned feature information further includes: a level, which determines the feature information of the security event based on the visual event type of the target time period, the confidence level of the visual event type of the target time period, the sound event type of the target time period, and the confidence level of the sound event type of the target time period, including: When the confidence levels of both the visual event type and the sound event type during the target time period are greater than the confidence threshold, a preset mapping table is queried based on the visual event type and the sound event type during the target time period to determine the type and level of the security event. The mapping table contains the mapping relationships between visual event type, sound event type, security event type, and security event level.

[0058] Specifically, for example, with a confidence threshold of 70%, the visual event type for the target time period is pushing and shoving, and the confidence level of the visual event type for the target time period is 75%. The audio event type for the target time period is arguing, and the confidence level of the audio event type for the target time period is 80%. In this case, the confidence level of the visual event type for the target time period is greater than 70%, and the confidence level of the audio event type for the target time period is also greater than 70%. Therefore, based on the visual event type and the audio event type for the target time period, a preset mapping table is consulted to determine the type of the safety event as passenger conflict, and the level of the safety event is determined to be Level 1. For example, if the confidence threshold is 70%, the visual event type during the target time period is an offensive device, the confidence level of the visual event type during the target time period is 80%, the sound event type during the target time period is a cry for help, and the confidence level of the sound event type during the target time period is 85%, then the confidence level of the visual event type during the target time period is greater than 70%, and the confidence level of the sound event type during the target time period is also greater than 70%. Therefore, based on the visual event type and the sound event type during the target time period, a preset mapping table is consulted to determine that the type of the security event is illegal behavior, and the level of the security event is determined to be the second level.

[0059] Specifically, for example, with a confidence threshold of 75%, the visual event type for the target time period is smoke emanating from the carriage, and the confidence level for this visual event type is 85%. The sound event type for the target time period is a smoke alarm sound, and the confidence level for this sound event type is 80%. In this case, the confidence levels for both the visual and sound event types for the target time period are greater than 75%. Therefore, based on the visual and sound event types for the target time period, a pre-defined mapping table is consulted to determine the type of safety event as smoke emanating from the carriage, and the safety event level is determined to be Level 1. For example, with a confidence threshold of 75%, the visual event type for the target time period is a fire in the carriage, and the confidence level of the visual event type for the target time period is 80%. The sound event type for the target time period is a smoke alarm sound, and the confidence level of the sound event type for the target time period is 82%. At this point, the confidence level of the visual event type for the target time period is greater than 75%, and the confidence level of the sound event type for the target time period is also greater than 75%. Therefore, based on the visual event type and the sound event type for the target time period, a preset mapping table is consulted to determine that the type of safety event is a fire in the carriage, and the level of the safety event is determined to be Level 2.

[0060] In this embodiment, only when the confidence levels of both the visual event type and the sound event type in the target time period are greater than the confidence threshold, a preset mapping table is queried based on the visual event type and the sound event type in the target time period to determine the type and level of the security event. This ensures the accuracy of the determined security event type and the accuracy of the determined security event level.

[0061] In an optional embodiment, the second determining unit described above is used for: When the level of the aforementioned safety incident is Level 1 and the type of the aforementioned safety incident is passenger conflict, it is determined that the aforementioned safety warning strategy includes at least a first preset safety warning strategy, wherein the aforementioned first preset safety warning strategy is to control the voice playback unit to play dissuasive voice messages. Specifically, the persuasive voice message could be something like, "Please remain calm and give way to each other in the carriage."

[0062] When the level of the aforementioned security incident is Level 2 and the type of the aforementioned security incident is illegal behavior, it is determined that the aforementioned security warning strategy includes at least a second preset security warning strategy, wherein the aforementioned Level 1 is lower than the aforementioned Level 2, and the aforementioned second preset security warning strategy is to control the audible and visual alarm unit to perform audible and visual alarm.

[0063] In this embodiment, when the safety event is classified as Level 1 and is a passenger conflict, a mild safety warning strategy is adopted, which involves playing a dissuasive voice message through the voice playback unit. This effectively prevents the passenger conflict from escalating further and avoids unnecessary disturbance to other passengers in the carriage due to overreaction. When the safety event is classified as Level 2 and is an illegal act, the sound and light alarm unit is activated to trigger a sound and light alarm. This strong sensory stimulus creates a deterrent effect, achieving the purpose of mandatory intervention and suppressing the escalation of the situation. This method of graded warning based on the level of the safety event avoids the waste of resources and decline in passenger experience caused by overreaction to passenger conflicts, while also avoiding the risk of loss of control due to insufficient response to illegal acts, further improving the reliability of bus warning technology.

[0064] In an optional embodiment, when the level of the aforementioned security event is Level 1 and the type of the aforementioned security event is passenger conflict, determining that the aforementioned security warning strategy includes a first preset security warning strategy can be implemented as follows: The aforementioned safety warning strategy includes a third preset safety warning strategy, which involves sending the first status data of the aforementioned safety event to the dispatch cloud platform. The first status data includes at least: the aforementioned video data inside the bus, the aforementioned audio data inside the bus, the type of the aforementioned safety event, and the level of the aforementioned safety event.

[0065] Specifically, the dispatch cloud platform is located on one side of the bus dispatch center.

[0066] In this embodiment, when the level of the safety incident is Level 1 and the type of the safety incident is passenger conflict, the aforementioned video data inside the bus, the aforementioned audio data inside the bus, the aforementioned type of safety incident, and the aforementioned level of the safety incident are uploaded to the dispatch cloud platform. This allows the dispatchers at the bus dispatch center to intuitively recreate the real scene of the conflict in the carriage. If it is found that the passenger conflict continues to escalate and turns into illegal behavior, the dispatcher can manually upgrade the level of the safety incident to Level 2 through the operation interface.

[0067] In an optional embodiment, when the level of the aforementioned security event is Level 1 and the type of the aforementioned security event is a passenger conflict, after step S104, the method further includes: Obtain video data and audio data inside the bus during the verification period, which is located after the target period. The above-mentioned video data inside the bus is input into the above-mentioned visual event recognition model to obtain the visual event types and confidence levels of the visual event types during the above-mentioned verification period. The above-mentioned audio data from inside the bus is input into the above-mentioned audio event recognition model to obtain the sound event types and the confidence levels of the sound event types during the above-mentioned verification period. If the visual event type during the aforementioned verification period is an offensive device, the confidence level of the visual event type during the aforementioned verification period is greater than the aforementioned confidence threshold, and the sound event type during the aforementioned verification period is a cry for help and the confidence level of the sound event type during the aforementioned verification period is greater than the aforementioned confidence threshold, then the type of the aforementioned security event will be adjusted to the aforementioned illegal behavior and the level of the aforementioned security event will be adjusted to the aforementioned second level.

[0068] In this embodiment, the response window between the initial handling of a security incident and manual confirmation is eliminated. The situation escalation judgment can be completed autonomously without waiting for dispatcher intervention, thus gaining valuable golden time for subsequent mandatory handling (audio and visual alarms, door locking, and police coordination).

[0069] In an optional embodiment, when the level of the aforementioned security event is Level 2 and the type of the aforementioned security event is illegal behavior, determining that the aforementioned security warning strategy includes a second preset security warning strategy can be implemented as follows: The aforementioned safety warning strategy includes a fourth preset safety warning strategy, which involves controlling the door to close via a door control relay.

[0070] In this embodiment, when the level of the security incident is the second level and the type of the security incident is illegal behavior, the door control relay is used to control the door to close, so as to prevent the situation from spreading outside the vehicle or the illegal person from escaping.

[0071] In an optional embodiment, when the security event is classified as Level 2 and its type is illegal activity, it is determined that the security warning strategy includes a second preset security warning strategy, comprising: The aforementioned security early warning strategy includes a fifth preset security early warning strategy, which involves sending the second status data of the aforementioned security event to the public security cloud platform. The second status data includes at least: the video data inside the bus, the audio data inside the bus, the type of the aforementioned security event, the level of the aforementioned security event, the license plate image data of the aforementioned bus, the location information of the aforementioned bus, and the contact information of the driver of the aforementioned bus.

[0072] Specifically, the location information can be GPS positioning information.

[0073] In this embodiment, automated data reporting replaces traditional manual telephone alarms, effectively avoiding the risks of drivers making unclear statements, omitting key information, or delaying the reporting time in tense situations. This ensures the integrity and timeliness of the alarm information, giving the police valuable time to respond quickly. The reported data includes images of the bus license plate, precise location information, and the driver's contact information, enabling the alarm receiving center to immediately identify the vehicle and its specific location, eliminating delays caused by ambiguous location descriptions. It also facilitates nearby patrol officers to respond, achieving precise deployment and efficient dispatch of police resources. The simultaneous reporting of the driver's contact information establishes a direct communication link between the police and the driver, allowing for real-time access to the latest situation during the mission and guiding the driver to cooperate with initial handling. This ensures that upon arrival, the police can quickly connect with the on-site supervisor and efficiently take over control, thus achieving a seamless closed loop in the entire alarm handling process. After the complete audio and video data is simultaneously uploaded to the police cloud platform, it can be directly preserved as electronic evidence, providing legal and compliant original audio-visual materials for subsequent administrative penalties or criminal case filing.

[0074] This application provides a bus safety early warning system, the system comprising: The video acquisition unit, located inside the aforementioned bus, is used to collect video data from inside the bus. The audio acquisition unit, located inside the bus, is used to collect audio data inside the bus. The audio playback unit is located inside the aforementioned bus; The audible and visual alarm unit is located inside the aforementioned bus; The door control relay is located inside the aforementioned bus. An in-vehicle intelligent terminal, located inside the aforementioned bus, is used to implement the bus safety warning method provided in any of the foregoing method embodiments, including: Step S101: Obtain the video data inside the bus during the target time period and the audio data inside the bus during the target time period. Step S102: Based on the aforementioned video data and audio data inside the bus, determine the characteristic information of the safety event during the target time period. The characteristic information includes at least: type. Step S103: Based on the characteristic information of the aforementioned security events, determine the security early warning strategy for the aforementioned target time period; Step S104: Execute the above-mentioned security warning strategy.

[0075] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the bus safety warning method provided in any of the foregoing method embodiments.

[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0078] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0079] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for early warning of bus safety, characterized in that, The method is applied to the onboard intelligent terminal of the bus, and the method includes: Acquire video data and audio data inside the bus during the target time period; Based on the video data and audio data inside the bus, characteristic information of the safety event during the target time period is determined, and the characteristic information includes at least: type; Based on the characteristic information of the security event, determine the security early warning strategy for the target time period; Implement the aforementioned security warning strategy.

2. The method according to claim 1, characterized in that, Based on the video data and audio data inside the bus, characteristic information of the safety event during the target time period is determined, including: The video data inside the bus is input into a visual event recognition model to obtain the visual event types and confidence levels of the visual event types during the target time period. The audio data inside the bus is input into an audio event recognition model to obtain the sound event type and the confidence level of the sound event type during the target time period. The characteristic information of the security event is determined based on the visual event type, the confidence level of the visual event type, the sound event type, and the confidence level of the sound event type during the target time period.

3. The method according to claim 2, characterized in that, The feature information further includes: a level, which is determined based on the visual event type, the confidence level of the visual event type, the sound event type, and the confidence level of the sound event type during the target time period, including: When the confidence levels of both the visual event type and the sound event type during the target time period are greater than a confidence threshold, a preset mapping table is queried based on the visual event type and the sound event type during the target time period to determine the type and level of the security event. The mapping table contains the mapping relationships between visual event type, sound event type, security event type, and security event level.

4. The method according to claim 3, characterized in that, Based on the characteristic information of the security event, a security early warning strategy for the target time period is determined, including: When the level of the safety incident is Level 1 and the type of the safety incident is passenger conflict, it is determined that the safety warning strategy includes at least a first preset safety warning strategy, wherein the first preset safety warning strategy is to control the voice playback unit to play dissuasive voice messages. When the level of the security incident is the second level and the type of the security incident is illegal behavior, it is determined that the security warning strategy includes at least a second preset security warning strategy, where the first level is lower than the second level, and the second preset security warning strategy is to control the audible and visual alarm unit to perform an audible and visual alarm.

5. The method according to claim 4, characterized in that, When the level of the safety incident is Level 1 and the type of the safety incident is passenger conflict, it is determined that the safety warning strategy includes at least a first preset safety warning strategy, including: The safety warning strategy is determined to include a third preset safety warning strategy, which is to send the first status data of the safety event to the dispatch cloud platform. The first status data includes at least: the video data inside the bus, the audio data inside the bus, the type of the safety event, and the level of the safety event.

6. The method according to claim 4, characterized in that, When the security incident is classified as Level 2 and its type is illegal activity, the security warning strategy is determined to include at least a second preset security warning strategy, including: The safety warning strategy is determined to include a fourth preset safety warning strategy, which is to control the door to close via a door control relay.

7. The method according to claim 6, characterized in that, When the security incident is classified as Level 2 and its type is illegal activity, the security warning strategy is determined to include at least a second preset security warning strategy, including: The safety warning strategy is determined to include a fifth preset safety warning strategy, which is to send the second status data of the safety event to the public security police cloud platform. The second status data includes at least: the video data inside the bus, the audio data inside the bus, the type of the safety event, the level of the safety event, the license plate image data of the bus, the location information of the bus, and the contact information of the bus driver.

8. A bus safety warning device, characterized in that, The device is applied to the onboard intelligent terminal of the bus, and the device includes: The acquisition unit is used to acquire video data and audio data inside the bus during the target time period. The first determining unit is configured to determine the characteristic information of the safety event during the target time period based on the video data and audio data inside the bus, wherein the characteristic information includes at least: type; The second determining unit is used to determine the security early warning strategy for the target time period based on the characteristic information of the security event. An execution unit is used to execute the security warning strategy.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the bus safety warning method as described in any one of claims 1 to 7.

10. A bus safety early warning system, characterized in that, The system includes: A video acquisition unit, located inside the bus, is used to collect video data inside the bus; An audio acquisition unit, located inside the bus, is used to collect audio data inside the bus; The voice playback unit is located inside the bus; An audible and visual alarm unit is located inside the bus. The door control relay is located inside the bus. An in-vehicle intelligent terminal is located inside the bus and is used to implement the steps of the bus safety early warning method as described in any one of claims 1 to 7.

Images