Accident detection method, apparatus, device, medium, and program product

Abstract

Embodiments of the present application provide an accident detection method, device, equipment, medium and program product, which can be used in the field of intelligent transportation technology. In the method, whether to detect the current road image is determined according to the acquired vehicle speed. When it is determined to detect the current road image, whether the current road scene is an intersection scene or a non-intersection scene is determined according to the acquired vehicle position. Then, the current road image is detected according to the detection strategy corresponding to the current road scene, and a detection result is obtained. According to the scheme, when it is determined to detect the current road image, the current road image is detected according to the detection strategy corresponding to the current road scene, which effectively reduces the repeated recognition rate and improves the detection accuracy.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to an accident detection method, device, equipment, medium, and program product. Background Technology

[0002] In the field of intelligent transportation, accident detection is an indispensable component. By detecting whether an accident has occurred on the road, it not only provides alerts to the vehicle in question to improve driving safety, but also sends the detection results to a management platform, enabling road control and preventing other vehicles from entering the accident area.

[0003] In existing technologies, accident detection typically involves detecting each road image acquired, which leads to a high rate of duplicate detections. Summary of the Invention

[0004] This application provides an accident detection method, apparatus, equipment, medium and program product, electronic device, storage medium and program product to solve the problem of high repetition rate caused by existing accident detection methods that use visual detection algorithms to detect real-time acquired road images.

[0005] In a first aspect, embodiments of this application provide an accident detection method, including:

[0006] Acquire current road images and vehicle information, including vehicle position and speed;

[0007] Based on the vehicle speed, determine whether to perform detection on the current road image;

[0008] If it is determined that the current road image is to be detected, then the current road scene is determined based on the vehicle position, and the current road scene is either an intersection scene or a non-intersection scene.

[0009] According to the detection strategy corresponding to the current road scene, the current road image is detected to obtain the detection result. The detection result includes a first identifier indicating that an accident has occurred and scene description information, or includes a second identifier indicating that no accident has occurred.

[0010] In one possible implementation, determining whether to detect the current road image based on the vehicle speed includes:

[0011] The target detection duration is determined based on the vehicle speed.

[0012] Determine whether the first identifier was detected within the target detection time prior to the acquisition time of the current road image;

[0013] If the first identifier is detected within the target detection time before the acquisition time of the current road image, it is determined that the current road image will not be detected.

[0014] If the first identifier is not detected within the target detection time prior to the acquisition time of the current road image, then it is determined that the current road image will be detected.

[0015] In one possible implementation, if the current road scene is an intersection scene, the step of detecting the current road image according to the detection strategy corresponding to the current road scene to obtain the detection result includes:

[0016] Acquire a preset number of historical road images, wherein the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time interval;

[0017] Target detection is performed on the preset number of historical road images and the current road image to determine whether there are any abnormal targets in the preset number of historical road images and the current road image;

[0018] If at least one of the preset number of historical road images and the current road image contains an abnormal target, the current road image is input into the accident recognition model to obtain the detection result. The accident recognition model is a pre-trained Transform-based visual large language model.

[0019] In one possible implementation, the method further includes:

[0020] If no abnormal targets are found in either the preset number of historical road images or the current road image, a detection result including the second identifier is generated.

[0021] In one possible implementation, if the current road scene is a non-intersection scene, the step of detecting the current road image according to the detection strategy corresponding to the current road scene to obtain the detection result includes:

[0022] Acquire a preset number of historical road images, wherein the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time interval;

[0023] Determine the vehicle speed detection range based on the vehicle's location;

[0024] Based on the preset number of historical road images and the current road image, determine whether there is a stationary target of a preset type in the preset number of historical road images and the current road image;

[0025] If the vehicle speed is within the vehicle speed detection range, and / or, at least one of the preset number of historical road images and the current road image contains a stationary target of a preset type, then the current road image is input into the accident recognition model to obtain the detection result. The accident recognition model is a pre-trained Transform-based visual large language model.

[0026] If the vehicle speed is not within the vehicle speed detection range, and neither the preset number of historical road images nor the current road image contains a stationary target of a preset type, then a detection result including the second identifier is generated.

[0027] In one possible implementation, before acquiring the current road image and vehicle information, the method further includes:

[0028] Acquire multiple sets of training data, each set including training images and training description information;

[0029] Select one set of training data from the multiple sets of training data;

[0030] The training data is input into the initial visual large language model to obtain the training detection results;

[0031] The initial visual large language model is updated based on the training detection results to obtain the trained visual large language model.

[0032] Update training times;

[0033] If the updated number of training iterations equals the preset training iteration threshold, then the trained visual large language model is used as the accident recognition model.

[0034] If the updated number of training iterations is less than the preset training iteration threshold, the trained visual large language model is used as the new initial visual large language model. A new set of training data is selected from the multiple sets of training data and input into the new initial visual large language model to obtain a new training detection result. The new initial visual large model and the number of training iterations are updated to obtain the trained visual large language model. If the updated number of training iterations is less than the preset training iteration threshold, this step is repeated until the updated number of training iterations equals the preset training iteration threshold. The trained visual large language model is then used as the accident recognition model.

[0035] Secondly, embodiments of this application provide an accident detection device, comprising:

[0036] The acquisition module is used to acquire current road images and vehicle information, including vehicle position and speed;

[0037] Processing module, used for:

[0038] Based on the vehicle speed, determine whether to perform detection on the current road image;

[0039] If it is determined that the current road image is to be detected, then the current road scene is determined based on the vehicle position, and the current road scene is either an intersection scene or a non-intersection scene.

[0040] The detection module is used to detect the current road image according to the detection strategy corresponding to the current road scene and obtain the detection result. The detection result includes a first identifier indicating that an accident has occurred and scene description information, or includes a second identifier indicating that no accident has occurred.

[0041] Thirdly, embodiments of this application provide an electronic device, including:

[0042] Processor, memory, communication interface;

[0043] The memory is used to store the executable instructions of the processor;

[0044] The processor is configured to execute the accident detection method according to any one of the first aspects by executing the executable instructions.

[0045] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the accident detection method described in any of the first aspects.

[0046] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the accident detection method described in any of the first aspects.

[0047] The accident detection method, apparatus, equipment, medium, and program products provided in this application determine whether to detect the current road image based on the acquired vehicle speed. When it is determined that the current road image should be detected, the current road scene is determined to be an intersection scene or a non-intersection scene based on the acquired vehicle position. Then, the current road image is detected according to the detection strategy corresponding to the current road scene to obtain the detection result. This solution effectively reduces the duplicate recognition rate and improves the detection accuracy by detecting the current road image according to the detection strategy corresponding to the current road scene when it is determined that the current road image should be detected. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] Figure 1aA flowchart illustrating an embodiment of the accident detection method provided in this application;

[0050] Figure 1b This is a schematic diagram of the vehicle architecture provided in this application;

[0051] Figure 1c Provide a schematic diagram of the pre-set intersection area for this application;

[0052] Figure 2 A flowchart illustrating Embodiment 2 of the accident detection method provided in this application;

[0053] Figure 3 A flowchart illustrating Embodiment 3 of the accident detection method provided in this application;

[0054] Figure 4 A flowchart illustrating Embodiment 4 of the accident detection method provided in this application;

[0055] Figure 5 This is a schematic diagram of the structure of an embodiment of the accident detection device provided in this application;

[0056] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application.

[0057] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0058] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0059] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0060] With the development of technology, intelligent transportation can detect accidents, thereby improving vehicle driving safety.

[0061] In existing technologies, accident detection typically involves acquiring road images in real time using camera equipment, and then detecting each frame of the road image to determine whether an accident has occurred on the current road. This can lead to a high rate of duplicate detections.

[0062] To address the problems existing in the prior art, the inventors, during their research on accident detection methods, discovered that to reduce the rate of duplicate detections, a detection duration can be determined based on vehicle speed. This allows for the assessment of whether an accident has been detected within the detection duration prior to the acquisition of the current road image. Only if no accident is detected is the current road image subjected to detection. Furthermore, by determining the current road scene during detection and applying the corresponding detection strategy, the accuracy of accident detection can be improved. Based on the above inventive concept, the accident detection scheme described in this application was designed.

[0063] The entity executing the accident detection method in this application can be a vehicle, or a server, computer, terminal equipment, etc. This application does not limit it. The following explanation uses a vehicle as an example.

[0064] The following provides examples illustrating the application scenarios of the accident detection method provided in this application.

[0065] For example, in this application scenario, a user is driving a vehicle on the road, and the vehicle is recording video through a camera device, and the vehicle encounters an accident involving another vehicle while driving.

[0066] When a vehicle acquires an image of the current road through a camera device, it also acquires vehicle information, including vehicle position and speed.

[0067] Based on the vehicle's speed, the system determines that the current road image needs to be detected. Then, based on the vehicle's position, it determines the current road scene, which is an intersection scene.

[0068] The vehicle performs detection on the current road image according to the detection strategy corresponding to the current road scene, and obtains the detection result. That is, it first acquires a preset number of historical road images, and the time between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time.

[0069] If an abnormal target is found in a preset number of historical road images and current road images, the current road image is then input into the accident recognition model to obtain the detection results. The detection results include a first identifier indicating that an accident has occurred and scene description information.

[0070] After the vehicle outputs the detection results, the user can input alarm instructions to the vehicle, and the vehicle can send alarm information to the alarm platform. Subsequently, the alarm platform can control the road based on the alarm information.

[0071] It should be noted that the preset quantity can be 1, 2, 3, 10, 50, etc., and the preset duration can be 0.1 seconds, 0.5 seconds, 1 second, 3 seconds, etc. This application embodiment does not limit the preset quantity and preset duration, and can be determined according to the actual situation.

[0072] It should be noted that the above scenario is only an example of an application scenario provided by the embodiments of this application. The embodiments of this application do not limit the actual form of the various devices included in the scenario, nor do they limit the interaction method between devices. In the specific application of the solution, it can be set according to actual needs.

[0073] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0074] Figure 1a This is a flowchart illustrating an embodiment of the accident detection method provided in this application. This embodiment describes how, after a vehicle determines that detection of the current road image is required based on its speed, it performs detection on the current road image according to the detection strategy corresponding to the current road scene. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 1a As shown, the accident detection method specifically includes the following steps:

[0075] S101: Obtain current road images and vehicle information.

[0076] In this step, in order to detect whether an accident has occurred on the road, the vehicle first needs to acquire current road images and vehicle information. Vehicle information includes vehicle position and speed.

[0077] For example, Figure 1b The vehicle architecture diagram provided in this application is as follows: Figure 1b As shown, the vehicle includes an in-vehicle infotainment system, a memory, a camera, a system-on-chip (SoC), a microcontroller unit (MCU), and a vehicle gateway module (VGM). The SoC is connected to the MCU, the in-vehicle infotainment system, the memory, and the camera, while the MCU is connected to the VGM. The SoC acquires current road images through the camera.

[0078] S102: Determine whether to perform detection on the current road image based on the vehicle speed.

[0079] In this step, after the SoC in the vehicle acquires the current road image and vehicle information, in order to reduce the duplicate detection rate, it determines whether to detect the current road image based on the vehicle speed.

[0080] Specifically, the target detection duration is determined based on vehicle speed. First, a vehicle speed range including the vehicle speed is determined. Then, based on the preset correspondence between vehicle speed range and detection duration, the detection duration corresponding to that vehicle speed range is determined, and this detection duration is used as the target detection duration.

[0081] It should be noted that the correspondence between vehicle speed range and detection time can be as follows: a vehicle speed range of [0, 10) in km / h corresponds to a detection time of 20 seconds; a vehicle speed range of [10, 50) in km / h corresponds to a detection time of 10 seconds; a vehicle speed range of [50, 80) in km / h corresponds to a detection time of 8 seconds; and a vehicle speed range of [80, 150) in km / h corresponds to a detection time of 5 seconds. This application does not limit the correspondence between vehicle speed range and detection time; it can be determined according to actual circumstances.

[0082] Then, it is determined whether the first identifier was detected within the target detection time prior to the acquisition time of the current road image. Since the vehicle records the time of obtaining the detection result each time it receives the detection result, which includes the first identifier and scene description information, it is possible to determine whether the first identifier was detected within the target detection time prior to the acquisition time of the current road image. If, among all the recorded times, there is a time that falls within the target detection time prior to the acquisition time of the current road image, then it is determined that the first identifier was detected within the target detection time prior to the acquisition time of the current road image.

[0083] If the first identifier is detected within the target detection time before the current road image is acquired, it means that if the current road image is detected, an accident will still be detected, and it will be the same as the previously detected accident. In order to reduce the duplicate detection rate and reduce repeated prompts to the user, it is determined not to detect the current road image, and thus the detection process ends.

[0084] If the first identifier is not detected within the target detection time before the current road image acquisition time, it means that the vehicle has passed the previously detected accident scene. In order to determine whether an accident has occurred, it is determined to detect the current road image.

[0085] S103: If it is determined that the current road image will be detected, then the current road scene will be determined based on the vehicle position.

[0086] In this step, if the vehicle determines to perform detection on the current road image, since there is a correspondence between the detection strategy and the road scene, it is necessary to determine the current road scene based on the vehicle's position. The current road scene is either an intersection scene or a non-intersection scene.

[0087] The vehicle stores a preset intersection location range. If the vehicle determines that its position falls within the preset intersection location range, the current road scene is determined to be an intersection scene. If the vehicle determines that its position does not fall within the preset intersection location range, the current road scene is determined to be a non-intersection scene.

[0088] For example, Figure 1c This application provides a schematic diagram of the pre-defined intersection area, such as... Figure 1c As shown in the figure, the area within the dashed box is the preset intersection range.

[0089] S104: Based on the detection strategy corresponding to the current road scene, perform detection on the current road image to obtain the detection result.

[0090] In this step, after the vehicle determines the current road scene, it performs detection on the current road image according to the detection strategy corresponding to the current road scene to obtain the detection result.

[0091] The detection results include a first indicator indicating that an accident has occurred and scene description information, or a second indicator indicating that no accident has occurred.

[0092] In both current road scenarios, the vehicle acquires a preset number of historical road images. Based on the current road image and the preset number of historical road images, it determines whether an accident is likely. If an accident is determined to be likely, the current road image is input into the accident recognition model to obtain the detection result. The large-scale traffic accident recognition model is a pre-trained Transform-based visual language model.

[0093] It should be noted that the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than the preset time interval. The preset number can be 1, 2, 3, 10, 50, etc., and the preset time interval can be 0.1 seconds, 0.5 seconds, 1 second, 3 seconds, etc. This application embodiment does not limit the preset number and preset time interval, and can be determined according to the actual situation.

[0094] It should be noted that when the detection results include the first sign indicating that an accident has occurred and scene description information, the vehicle will record the time when the detection results are obtained for use in subsequent accident detection.

[0095] When the vehicle receives detection results that include a first indication of an accident and scene description information, the vehicle can display the detection results on the in-vehicle infotainment system and play a warning sound. After being notified of an accident detection, the user can choose to issue an alarm by inputting an alarm instruction. The vehicle will respond to the user's input alarm instruction by outputting alarm information, which includes the vehicle's location.

[0096] exist Figure 1b Based on this, when the SoC determines that the detection results include the first indicator indicating an accident and scene description information, it transmits the detection results to the vehicle's infotainment system for display and converts the scene description information into voice for playback. After the vehicle's infotainment system receives the alarm indication, it transmits it to the SoC. The SoC transmits the alarm indication to the MCU via the Serial Peripheral Interface (SPI). The MCU then transmits the alarm indication to the VGM via a Controller Area Network (CAN) message, and the VGM outputs the alarm information.

[0097] It should be noted that users can input alarm instructions in various ways, such as by clicking the alarm button on the vehicle's infotainment system, speaking an alarm voice command, or sending an alarm instruction to the vehicle's infotainment system via a terminal device. Vehicles can output alarm information in various ways, such as by sending alarm information to the management platform, sending alarm information to the alarm platform, or sending alarm information to other vehicles. This application does not limit the methods by which users input alarm instructions or vehicles output alarm information; these methods can be determined based on actual circumstances.

[0098] When the vehicle's detection results include a first sign indicating an accident and scene description information, the vehicle can also store video footage of a preset recording duration before and after the moment the detection results were obtained, for later review. The preset recording duration can be 1 minute, 3 minutes, 5 minutes, etc. This application embodiment does not limit the preset recording duration and can determine it according to actual circumstances.

[0099] The accident detection method provided in this embodiment determines whether to detect the current road image based on the acquired vehicle speed and historical detection data. When it is determined that the current road image should be detected, the method determines whether the current road scene is an intersection scene or a non-intersection scene based on the acquired vehicle position. Then, based on the accident recognition model and the detection strategy corresponding to the current road scene, the current road image is detected to obtain the detection result. Compared with the prior art that detects every road image, which can easily lead to multiple detections of the same accident, this solution reduces the duplicate detection rate by only detecting the current road image when it is determined to be detected. Compared with the prior art that uses visual detection algorithms for detection, this solution detects the current road image based on the accident recognition model and the detection strategy corresponding to the current road scene, which effectively improves the detection accuracy.

[0100] Figure 2 This is a flowchart illustrating a second embodiment of the accident detection method provided in this application. Based on the above embodiments, this application describes how, when the current road scene is an intersection, the vehicle detects the current road image according to the detection strategy corresponding to the current road scene, and obtains the detection result. For example... Figure 2 As shown, the accident detection method specifically includes the following steps:

[0101] S201: Obtain a preset number of historical road images.

[0102] In this step, given that the current road scene is an intersection, a preset number of historical road images need to be acquired to improve the accuracy of the detection results.

[0103] The time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than the preset time interval.

[0104] It should be noted that the preset quantity can be 1, 2, 3, 10, 50, etc., and the preset duration can be 0.1 seconds, 0.5 seconds, 1 second, 3 seconds, etc. This application embodiment does not limit the preset quantity and preset duration, and can be determined according to the actual situation.

[0105] S202: Perform target detection on a preset number of historical road images and the current road image to determine whether there are abnormal targets in the preset number of historical road images and the current road image; if there are abnormal targets in at least one of the preset number of historical road images and the current road image, then proceed to step S203; if there are no abnormal targets in either the preset number of historical road images or the current road image, then proceed to step S204.

[0106] In this step, after the vehicle acquires a preset number of historical road images, it can use a target detection method to perform target detection on the preset number of historical road images and the current road image to determine whether there are abnormal targets in the preset number of historical road images and the current road image, so as to determine whether an accident may occur on the current road.

[0107] It should be noted that the object detection method can be any method that uses models such as Fast R-CNN, Faster R-CNN, YOLO, CornerNet, etc. This application does not limit the object detection method, and it can be determined according to the actual situation.

[0108] It should be noted that abnormal targets can be ambulances, fire trucks, police cars, etc. This application embodiment does not limit abnormal targets and can be determined according to the actual situation.

[0109] S203: Input the current road image into the accident recognition model to obtain the detection results.

[0110] In this step, if the vehicle determines that at least one of the preset number of historical road images and the current road image contains an abnormal target, it indicates that an accident is very likely to have occurred on the current road, and further detection is required. The current road image is then input into the accident recognition model to obtain the detection results.

[0111] S204: Generate detection results including the second identifier.

[0112] In this step, if the vehicle determines that there are no abnormal targets in a preset number of historical road images and the current road image, it means that no accident has occurred on the current road, and a detection result including a second identifier indicating that no accident has occurred is generated.

[0113] The accident detection method provided in this embodiment first determines whether there are abnormal targets in the current road image and a preset number of historical road images. If an abnormal target is found in at least one image, the current road image is input into the accident recognition model to obtain the detection result, thus improving detection accuracy. If no abnormal target is found in any of the images, it is determined that no accident has occurred, thus improving detection efficiency.

[0114] Figure 3 This is a flowchart illustrating a third embodiment of the accident detection method provided in this application. Based on the above embodiments, this embodiment describes the situation where, when the current road scene is not an intersection scene, the vehicle detects the current road image according to the detection strategy corresponding to the current road scene, and obtains the detection result. For example... Figure 3 As shown, the accident detection method specifically includes the following steps:

[0115] S301: Obtain a preset number of historical road images.

[0116] In this step, when the current road scene is not an intersection scene, a preset number of historical road images need to be acquired in order to improve the accuracy of the detection results.

[0117] The time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than the preset time interval.

[0118] It should be noted that the preset quantity can be 1, 2, 3, 10, 50, etc., and the preset duration can be 0.1 seconds, 0.5 seconds, 1 second, 3 seconds, etc. This application embodiment does not limit the preset quantity and preset duration, and can be determined according to the actual situation.

[0119] S302: Determine the vehicle speed detection range based on the vehicle's location.

[0120] In the above steps, since vehicles will slow down when they encounter an accident, and the degree of deceleration varies on roads with different speed limits, the vehicle speed detection range is determined based on the vehicle's position.

[0121] The method for determining the vehicle speed detection range based on the vehicle's location can be as follows: First, determine the speed limit range of the road where the vehicle is located based on the vehicle's location. Then, determine the vehicle speed detection range corresponding to the speed limit range of the road where the vehicle is located based on the correspondence between the speed limit range and the vehicle speed detection range.

[0122] For example, the correspondence between speed limit ranges and vehicle speed detection ranges can be as follows: a speed limit range of [0, 30) corresponds to a vehicle speed detection range of [0, 20), in km / h; a speed limit range of [0, 50) corresponds to a vehicle speed detection range of [0, 30), in km / h; a speed limit range of [0, 70) corresponds to a vehicle speed detection range of [0, 50), in km / h; and a speed limit range of [60, 120) corresponds to a vehicle speed detection range of [0, 70), in km / h. This application does not limit the correspondence between speed limit ranges and vehicle speed detection ranges; it can be determined according to actual circumstances.

[0123] Another way to determine the vehicle speed detection range based on the vehicle's location is to first determine the location range to which the vehicle's location belongs, and then determine the vehicle speed detection range corresponding to the location range to which the vehicle's location belongs based on the correspondence between the location range and the vehicle speed detection range.

[0124] For example, the correspondence between location ranges and vehicle speed detection ranges can be as follows: the vehicle speed detection range corresponding to location range A is [0, 20), in km / h; the vehicle speed detection range corresponding to location range B is [0, 30), in km / h; and the vehicle speed detection range corresponding to location range C is [0, 70), in km / h. This embodiment does not limit the correspondence between location ranges and vehicle speed detection ranges; it can be determined according to actual circumstances.

[0125] It should be noted that the execution order of steps S301 and S302 can be: step S301 is executed first, then step S302. Alternatively, step S302 is executed first, then step S301. Or, steps S301 and S302 are executed simultaneously. This embodiment does not limit the execution order of steps S301 and S302; it can be set according to actual circumstances.

[0126] S303: Based on a preset number of historical road images and the current road image, determine whether there is a stationary target of a preset type in the preset number of historical road images and the current road image; if at least one of the preset number of historical road images and the current road image contains a stationary target of a preset type, then proceed to step S305; if neither the preset number of historical road images nor the current road image contains a stationary target of a preset type, then proceed to step S304.

[0127] In this step, after the vehicle acquires the current road image and a preset number of historical road images, a multi-target tracking method is used to process the current road image and the preset number of historical road images to determine whether there are stationary targets of a preset type in the current road image and the preset number of historical road images. Since the current vehicle is not located at an intersection, the preset type of targets on the road are generally not stationary in the absence of an accident. Therefore, the presence of stationary preset type targets in the images can be used to determine whether an accident may have occurred on the road.

[0128] The multi-target tracking method can determine the position of the target in the current road image and a preset number of historical road images, thus determining the target's movement range. If the target's movement range is no greater than a preset range threshold, the target is identified as a stationary target. Then, targets of a preset type are extracted to obtain the stationary targets of that preset type.

[0129] It should be noted that the preset range threshold can be 3 meters, 5 meters, 7 meters, etc., and the preset type can be pedestrians, vehicles, animals, etc., with vehicles including cars, bicycles, motorcycles, electric vehicles, etc. This application embodiment does not limit the preset range threshold and preset type; they can be determined according to the actual situation.

[0130] It should be noted that the multi-target tracking method can be based on tracking methods such as SORT (Simple Online and Realtime Tracking), GNN (Graph Neural Networks), and CenterTrack. This application does not limit the multi-target tracking method, and it can be determined according to the actual situation.

[0131] S304: Determine whether the vehicle speed is within the vehicle speed detection range; if the vehicle speed is within the vehicle speed detection range, proceed to step S305; if the vehicle speed is not within the vehicle speed detection range, proceed to step S306.

[0132] In this step, if the vehicle determines that there are no stationary targets of a preset type in either the preset number of historical road images or the current road image, it is also necessary to determine whether the vehicle speed is within the vehicle speed detection range in order to ensure detection accuracy.

[0133] S305: Input the current road image into the accident recognition model to obtain the detection results.

[0134] In this step, if the vehicle determines that at least one of the preset number of historical road images and the current road image contains a stationary target of a preset type, it indicates that an accident may occur on the road. In order to improve the accuracy of the detection, the current road image is input into the accident recognition model to obtain the detection result.

[0135] If a vehicle's speed is determined to be within the speed detection range, it indicates that an accident may occur on the road. To ensure detection accuracy, the current road image is input into the accident recognition model to obtain the detection result.

[0136] S306: Generate detection results including the second identifier.

[0137] In this step, if the vehicle speed is determined to be outside the vehicle speed detection range, meaning there are no stationary targets of the preset type on the road and the vehicle's current speed is relatively high, it indicates that no accident has occurred on the current road, and a detection result including a second indicator indicating that no accident has occurred is generated.

[0138] In one implementation, the vehicle can first execute step S304. If it is determined that the vehicle speed is not within the vehicle speed detection range, step S303 is executed. Then, if it is determined that there is no stationary target of the preset type in the current road image and the preset number of historical road images, step S306 is executed.

[0139] In another implementation, the vehicle can simultaneously execute steps S303 and S304. If it is determined that the vehicle speed is within the vehicle speed detection range, and / or, at least one of the preset number of historical road images and the current road image contains a stationary target of a preset type, then step S305 is executed. If it is determined that the vehicle speed is not within the vehicle speed detection range, and neither the preset number of historical road images nor the current road image contains a stationary target of a preset type, then step S306 is executed.

[0140] The accident detection method provided in this embodiment first determines whether a stationary target of a preset type exists in the current road image and a preset number of historical road images, and whether the vehicle speed is within the vehicle speed detection range. If at least one image contains a stationary target of the preset type, and / or the vehicle speed is within the vehicle speed detection range, the current road image is input into the accident recognition model to obtain the detection result, thus improving detection accuracy. If no stationary target of the preset type exists in any image, and the vehicle speed is not within the vehicle speed detection range, it is determined that no accident has occurred, thus improving detection efficiency.

[0141] Figure 4 This is a flowchart illustrating Embodiment 4 of the accident detection method provided in this application. Based on the above embodiments, this application describes the process of training an initial visual large language model using training data to obtain an accident recognition model. Figure 4 As shown, the accident detection method specifically includes the following steps:

[0142] S401: Acquire multiple sets of training data.

[0143] In this step, in order to train the accident recognition model, multiple sets of training data are first acquired, each set of training data including training images and training description information.

[0144] S402: Select one set of training data from multiple sets of training data.

[0145] S403: Input the training data into the initial visual large language model to obtain the training detection results.

[0146] S404: Update the initial visual large language model based on the training detection results to obtain the trained visual large language model.

[0147] In the above steps, after obtaining multiple sets of training data, one set of training data is selected and input into the initial visual large language model to obtain the training detection results.

[0148] The initial visual large language model is a Transform-based model. The training detection results include a first identifier indicating that an accident has occurred and scene description information, or a second identifier indicating that no accident has occurred.

[0149] Then, based on the training and detection results, the loss value is calculated and the initial visual large language model is updated to obtain the trained visual large language model.

[0150] S405: Update training iterations.

[0151] In this step, after obtaining the trained visual large language model, the training count is updated, which means incrementing the training count by one.

[0152] S406: Determine whether the updated number of training iterations is equal to the preset training iterations threshold; if the updated number of training iterations is equal to the preset training iterations threshold, proceed to step S407; if the updated number of training iterations is less than the preset training iterations threshold, proceed to step S408.

[0153] In this step, updating the training count requires determining whether the updated training count equals the preset training count threshold to decide whether to stop training.

[0154] It should be noted that the preset training number threshold can be 300, 500, 1000, etc. This application embodiment does not limit the preset training number threshold, which can be determined according to the actual situation.

[0155] S407: Use the trained visual large language model as an accident recognition model.

[0156] In this step, if the updated number of training iterations equals the preset training iteration threshold, it means that the training is complete and no further training is needed. The trained visual large language model will then be used as the accident recognition model.

[0157] S408: Use the trained visual large language model as the new initial visual large language model and execute step S402.

[0158] In this step, if the updated number of training iterations is less than the preset training iteration threshold, it means that the training is not complete and needs to continue. The trained visual large language model is used as the new initial visual large language model. A new set of training data is selected from multiple sets of training data and input into the new initial visual large language model to obtain new training detection results. The new initial visual large model and the number of training iterations are updated to obtain the trained visual large language model. If the updated number of training iterations is less than the preset training iteration threshold, this process is repeated until the updated number of training iterations equals the preset training iteration threshold. The trained visual large language model is then used as the accident recognition model.

[0159] The accident detection method provided in this embodiment trains an initial visual large language model based on training images and training description information. The initial visual large language model can perform low-rank adaptation (LoRA), and encodes and decodes text and images simultaneously through a Transform structure, generating correlation and divergence, which can improve training efficiency and improve the accuracy of the accident recognition model.

[0160] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0161] Figure 5 This is a schematic diagram of the structure of an embodiment of the accident detection device provided in this application. Figure 5 As shown, the accident detection device 50 includes:

[0162] The acquisition module 51 is used to acquire current road images and vehicle information, wherein the vehicle information includes vehicle position and speed;

[0163] Processing module 52 is used for:

[0164] Based on the vehicle speed, determine whether to perform detection on the current road image;

[0165] If it is determined that the current road image is to be detected, then the current road scene is determined based on the vehicle position, and the current road scene is either an intersection scene or a non-intersection scene.

[0166] The detection module 53 is used to detect the current road image according to the detection strategy corresponding to the current road scene and obtain the detection result. The detection result includes a first identifier indicating that an accident has occurred and scene description information, or includes a second identifier indicating that no accident has occurred.

[0167] Furthermore, the processing module 52 is specifically used for:

[0168] The target detection duration is determined based on the vehicle speed.

[0169] Determine whether the first identifier was detected within the target detection time prior to the acquisition time of the current road image;

[0170] If the first identifier is detected within the target detection time before the acquisition time of the current road image, it is determined that the current road image will not be detected.

[0171] If the first identifier is not detected within the target detection time prior to the acquisition time of the current road image, then it is determined that the current road image will be detected.

[0172] Furthermore, if the current road scene is an intersection scene, the detection module 53 is specifically used for:

[0173] Acquire a preset number of historical road images, wherein the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time interval;

[0174] Target detection is performed on the preset number of historical road images and the current road image to determine whether there are any abnormal targets in the preset number of historical road images and the current road image;

[0175] If at least one of the preset number of historical road images and the current road image contains an abnormal target, the current road image is input into the accident recognition model to obtain the detection result. The accident recognition model is a pre-trained Transform-based visual large language model.

[0176] Furthermore, the detection module 53 is also used for:

[0177] If no abnormal targets are found in either the preset number of historical road images or the current road image, a detection result including the second identifier is generated.

[0178] Furthermore, if the current road scenario is not an intersection scenario, the detection module 53 is specifically used for:

[0179] Acquire a preset number of historical road images, wherein the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time interval;

[0180] Determine the vehicle speed detection range based on the vehicle's location;

[0181] Based on the preset number of historical road images and the current road image, determine whether there is a stationary target of a preset type in the preset number of historical road images and the current road image;

[0182] If the vehicle speed is within the vehicle speed detection range, and / or, at least one of the preset number of historical road images and the current road image contains a stationary target of a preset type, then the current road image is input into the accident recognition model to obtain the detection result. The accident recognition model is a pre-trained Transform-based visual large language model.

[0183] If the vehicle speed is not within the vehicle speed detection range, and neither the preset number of historical road images nor the current road image contains a stationary target of a preset type, then a detection result including the second identifier is generated.

[0184] Furthermore, the acquisition module 51 is also used to acquire multiple sets of training data, each set of training data including training images and training description information;

[0185] Furthermore, the processing module 52 is also used for:

[0186] Select one set of training data from the multiple sets of training data;

[0187] The training data is input into the initial visual large language model to obtain the training detection results;

[0188] The initial visual large language model is updated based on the training detection results to obtain the trained visual large language model.

[0189] Update training times;

[0190] If the updated number of training iterations equals the preset training iteration threshold, then the trained visual large language model is used as the accident recognition model.

[0191] If the updated number of training iterations is less than the preset training iteration threshold, the trained visual large language model is used as the new initial visual large language model. A new set of training data is selected from the multiple sets of training data and input into the new initial visual large language model to obtain a new training detection result. The new initial visual large model and the number of training iterations are updated to obtain the trained visual large language model. If the updated number of training iterations is less than the preset training iteration threshold, this step is repeated until the updated number of training iterations equals the preset training iteration threshold. The trained visual large language model is then used as the accident recognition model.

[0192] The accident detection device provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0193] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 6 As shown, the electronic device 60 includes:

[0194] Processor 61, memory 62, and communication interface 63;

[0195] The memory 62 is used to store the executable instructions of the processor 61;

[0196] The processor 61 is configured to execute the technical solutions in any of the foregoing method embodiments by executing the executable instructions.

[0197] Optionally, the memory 62 can be either standalone or integrated with the processor 61.

[0198] Optionally, when the memory 62 is a device independent of the processor 61, the electronic device 60 may further include:

[0199] Bus 64, memory 62 and communication interface 63 are connected to processor 61 through bus 64 and complete communication with each other. Communication interface 63 is used to communicate with other devices.

[0200] Optionally, the communication interface 63 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0201] Bus 64 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0202] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0203] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0204] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the technical solutions provided in any of the foregoing method embodiments.

[0205] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0206] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An accident detection method, characterized in that, include: Acquire current road images and vehicle information, including vehicle position and speed; Based on the vehicle speed, determine whether to perform detection on the current road image; If it is determined that the current road image is to be detected, then the current road scene is determined based on the vehicle position, and the current road scene is either an intersection scene or a non-intersection scene. According to the detection strategy corresponding to the current road scene, the current road image is detected to obtain the detection result. The detection result includes a first identifier indicating that an accident has occurred and scene description information, or includes a second identifier indicating that no accident has occurred. The step of determining whether to perform detection on the current road image based on the vehicle speed includes: The target detection duration is determined based on the vehicle speed. Determine whether the first identifier was detected within the target detection time prior to the acquisition time of the current road image; If the first identifier is detected within the target detection time before the acquisition time of the current road image, it is determined that the current road image will not be detected. If the first identifier is not detected within the target detection time prior to the acquisition time of the current road image, then it is determined that the current road image will be detected.

2. The method according to claim 1, characterized in that, If the current road scene is an intersection scene, the detection of the current road image according to the detection strategy corresponding to the current road scene, and the obtaining of the detection result, includes: Acquire a preset number of historical road images, wherein the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time interval; Target detection is performed on the preset number of historical road images and the current road image to determine whether there are any abnormal targets in the preset number of historical road images and the current road image; If at least one of the preset number of historical road images and the current road image contains an abnormal target, the current road image is input into the accident recognition model to obtain the detection result. The accident recognition model is a pre-trained Transform-based visual large language model.

3. The method according to claim 2, characterized in that, The method further includes: If no abnormal targets are found in either the preset number of historical road images or the current road image, a detection result including the second identifier is generated.

4. The method according to claim 1, characterized in that, If the current road scene is a non-intersection scene, the detection of the current road image according to the detection strategy corresponding to the current road scene, and the obtaining of the detection result, includes: Acquire a preset number of historical road images, wherein the time interval between the acquisition time of each historical road image and the acquisition time of the current road image is less than a preset time interval; Determine the vehicle speed detection range based on the vehicle's location; Based on the preset number of historical road images and the current road image, determine whether there is a stationary target of a preset type in the preset number of historical road images and the current road image; If the vehicle speed is within the vehicle speed detection range, and / or, at least one of the preset number of historical road images and the current road image contains a stationary target of a preset type, then the current road image is input into the accident recognition model to obtain the detection result. The accident recognition model is a pre-trained Transform-based visual large language model. If the vehicle speed is not within the vehicle speed detection range, and neither the preset number of historical road images nor the current road image contains a stationary target of a preset type, then a detection result including the second identifier is generated.

5. The method according to any one of claims 1 to 4, characterized in that, Before acquiring the current road image and vehicle information, the method further includes: Acquire multiple sets of training data, each set including training images and training description information; Select one set of training data from the multiple sets of training data; The training data is input into the initial visual large language model to obtain the training detection results; The initial visual large language model is updated based on the training detection results to obtain the trained visual large language model. Update training times; If the updated number of training iterations equals the preset training iteration threshold, then the trained visual large language model is used as the accident recognition model. If the updated number of training iterations is less than the preset training iteration threshold, the trained visual large language model is used as the new initial visual large language model. A new set of training data is selected from the multiple sets of training data and input into the new initial visual large language model to obtain a new training detection result. The new initial visual large model and the number of training iterations are updated to obtain the trained visual large language model. If the updated number of training iterations is less than the preset training iteration threshold, this step is repeated until the updated number of training iterations equals the preset training iteration threshold. The trained visual large language model is then used as the accident recognition model.

6. An accident detection device, characterized in that, include: The acquisition module is used to acquire current road images and vehicle information, including vehicle position and speed; Processing module, used for: Based on the vehicle speed, determine whether to perform detection on the current road image; If it is determined that the current road image is to be detected, then the current road scene is determined based on the vehicle position, and the current road scene is either an intersection scene or a non-intersection scene. The detection module is used to detect the current road image according to the detection strategy corresponding to the current road scene and obtain the detection result. The detection result includes a first identifier indicating that an accident has occurred and scene description information, or includes a second identifier indicating that no accident has occurred. When the processing module determines whether to perform detection on the current road image based on the vehicle speed, it is specifically used for: The target detection duration is determined based on the vehicle speed. Determine whether the first identifier was detected within the target detection time prior to the acquisition time of the current road image; If the first identifier is detected within the target detection time before the acquisition time of the current road image, it is determined that the current road image will not be detected. If the first identifier is not detected within the target detection time prior to the acquisition time of the current road image, then it is determined that the current road image will be detected.

7. An electronic device, characterized in that, include: Processor, memory, communication interface; The memory is used to store the executable instructions of the processor; The processor is configured to execute the accident detection method according to any one of claims 1 to 5 by executing the executable instructions.

8. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the accident detection method according to any one of claims 1 to 5.

9. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, is used to implement the accident detection method according to any one of claims 1 to 5.

Images