Traffic accident identification method, apparatus, and system, and program product

Abstract

The present application relates to the field of artificial intelligence, and in particular, to a traffic accident identification method, apparatus, and system, and a program product. The method comprises: a vehicle end acquires multi-modal data, performs preliminary traffic accident screening on the basis of the multi-modal data, and acquires information related to a suspected traffic accident; the vehicle end sends the multi-modal data and the information related to the suspected traffic accident to a platform end; on the basis of driver information of the vehicle end, the platform end determines a personal driving habit of a driver, and, on the basis of the location of the suspected traffic accident, determines historical accident information of the location; and the platform end obtains an identification result of the suspected traffic accident on the basis of the personal driving habit of the driver, the historical accident information of the location, and the multi-modal data. The method can combine historical accident information of the location of a suspected traffic accident and a personal driving habit of a driver to perform analysis and identification on multi-modal data, so that a more accurate traffic accident identification result is obtained, thus helping to reduce false alarms and improve the driving efficiency of a fleet of vehicles.

Description

Traffic accident identification methods, devices, systems and procedures Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to methods, devices, systems and program products for traffic accident recognition. Background Technology

[0002] Traffic accident recognition is an important research area in intelligent transportation systems and autonomous driving, playing a crucial role in improving road safety and traffic management. With the increasing complexity of transportation networks and the continuous growth of traffic volume, the importance of intelligent detection and recognition of traffic accidents is constantly rising.

[0003] Traditional accident detection algorithms typically collect data from devices and analyze it in real time to determine if a traffic accident exists at any given moment, and then generate an accident alarm based on the analysis results. However, the accuracy of traffic accident analysis by the device is not high, and false alarms are prone to occur, which can easily affect the driving efficiency of the fleet. Technical issues

[0004] In view of this, embodiments of this application provide a traffic accident identification method, device, system, and program product to solve the problems of low accuracy of traffic accident identification, easy occurrence of false alarms, and easy impact on the driving efficiency of the fleet in the prior art. Technical solutions

[0005] A first aspect of this application provides a traffic accident identification method, the method comprising:

[0006] The vehicle acquires multimodal data and performs preliminary screening of traffic accidents based on the multimodal data to obtain information related to suspected traffic accidents. The multimodal data includes driver information from the vehicle and the information related to suspected traffic accidents includes the location of the suspected traffic accident.

[0007] The vehicle terminal sends the multimodal data and the suspected traffic accident-related information to the platform terminal;

[0008] The platform determines the driver's personalized driving habits based on the driver information on the vehicle, and determines the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident.

[0009] The platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data.

[0010] In conjunction with the first aspect, in a first possible implementation of the first aspect, the platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data, including:

[0011] The platform generates an accident description text based on the suspected traffic accident information, the driver's personalized driving habits, and the location's historical accident information;

[0012] The platform inputs the accident description text into a large language model to obtain the identification result of the suspected traffic accident.

[0013] In conjunction with the first aspect, in a second possible implementation of the first aspect, the vehicle sends the multimodal data and the suspected traffic accident-related information to the platform, and the method further includes:

[0014] After obtaining the information related to the suspected traffic accident, the vehicle continues to acquire multimodal data for a predetermined duration.

[0015] The vehicle terminal will send the multimodal data used to determine information related to suspected traffic accidents, the multimodal data to be acquired, and the traffic accident information to the platform terminal.

[0016] In conjunction with the first aspect, in a third possible implementation of the first aspect, after the platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data, the method further includes:

[0017] The platform determines the accident level based on the vehicle's six-axis acceleration and speed obtained from the sensors, combined with the weighted information corresponding to the terrain at the vehicle's current location.

[0018] The platform determines the priority of vehicle accident handling based on the accident level.

[0019] In conjunction with the first aspect, in the fourth possible implementation of the first aspect, the multimodal data includes vehicle motion data, cockpit in-cabin image data, and cockpit out-of-cabin image data;

[0020] Based on the multimodal data, preliminary screening of traffic accidents is performed to obtain relevant information on suspected traffic accidents, including:

[0021] The vehicle terminal determines at least one of the following based on the vehicle motion data: sudden deceleration of the vehicle, vehicle collision, and vehicle rollover.

[0022] The vehicle terminal determines at least one of the following events based on the image data inside the cockpit: abnormal driver movement and driver panic.

[0023] In a fifth possible implementation of the first aspect, combining any one of the first to fourth possible implementations, the platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data, including:

[0024] The first relevant feature for determining the association of historical accident information and the second relevant feature for determining the association of personalized driving habits were identified.

[0025] The weight of the first relevant feature is determined based on the frequency of historical accident information at the location, and the weight of the second relevant feature is determined based on the driver's personalized driving habits.

[0026] In conjunction with the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, determining the weight of the first relevant feature based on the frequency of historical accident information of the location includes:

[0027] Based on the frequency of historical traffic accidents at the location, the weights of the first relevant features are adjusted according to a direct proportional relationship.

[0028] The weight of the second relevant feature is determined based on the driver's personalized driving habits;

[0029] Based on the degree of the driver's personalized driving habits, the weight of the second relevant feature is adjusted according to an inverse proportional relationship.

[0030] A second aspect of this application provides a traffic accident identification device, the device comprising:

[0031] The data acquisition unit is used to acquire multimodal data from the vehicle and perform preliminary screening of traffic accidents based on the multimodal data to obtain information related to suspected traffic accidents. The multimodal data includes driver information from the vehicle and the information related to suspected traffic accidents includes the location of the suspected traffic accident.

[0032] An information sending unit is used to send the multimodal data and the suspected traffic accident-related information from the vehicle to the platform.

[0033] The information determination unit is used to determine the driver's personalized driving habits based on the driver information of the vehicle, and to determine the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident.

[0034] The accident identification unit is used by the platform to obtain the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data.

[0035] A third aspect of this application provides a traffic accident recognition system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the traffic accident recognition system implements the method as described in any of the first aspects.

[0036] A fourth aspect of this application provides a computer program product that, when run on a computer, causes the computer to execute the methods described in the first aspect or its various implementations.

[0037] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0038] A sixth aspect of this application provides a chip for implementing the methods in the various implementations of the first aspect described above. Specifically, the chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the methods as described in the first aspect or its various implementations. Beneficial effects

[0039] The beneficial effects of this application embodiment compared to the prior art are as follows: This application embodiment acquires multimodal data from the vehicle end, performs preliminary screening of the multimodal data on the vehicle end to obtain information related to suspected traffic accidents, and sends this information to the platform end. The platform end determines the driver's personalized driving habits based on driver information and determines the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident. The platform end performs specific screening and judgment based on the driver's personalized driving habits and the historical accident information of the suspected traffic accident location to obtain the identification result of the suspected traffic accident. Compared with the vehicle end identification method, this method can combine the historical accident information of the suspected traffic accident location and the driver's personalized driving habits to analyze and identify multimodal data, obtaining more accurate traffic accident identification results, which helps to reduce false alarms and improve fleet driving efficiency. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 is a schematic diagram of an application scenario of a traffic accident identification method provided in an embodiment of this application;

[0042] Figure 2 is a schematic diagram of the implementation process of a traffic accident identification method provided in an embodiment of this application;

[0043] Figure 3 is a schematic diagram of the implementation process of a traffic accident identification method provided in an embodiment of this application;

[0044] Figure 4 is a schematic diagram illustrating how to determine the priority of accident handling according to an embodiment of this application;

[0045] Figure 5 is a schematic diagram of a traffic accident recognition device provided in an embodiment of this application;

[0046] Figure 6 is a schematic diagram of a traffic accident recognition system provided in an embodiment of this application. Embodiments of the present invention

[0047] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0048] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0049] The identification of abnormal vehicle states during driving, including traffic accident recognition, can effectively improve emergency response speed, reduce traffic congestion, and enhance road safety. Real-time monitoring and automatic identification of traffic accidents can quickly alert emergency services, shorten response time, and thus save lives in critical moments. Rapid identification and response to traffic accidents can reduce the time spent at the accident scene, thereby reducing resulting traffic congestion. Traffic accident recognition technology can monitor and analyze traffic flow, predict potential traffic accidents, and alert drivers, especially fleet drivers, through early warning systems, thereby reducing the risk of accidents. Therefore, accurate traffic accident identification is of great significance.

[0050] Figure 1 is a schematic diagram of an implementation scenario of a traffic accident recognition method provided in this application embodiment. As shown in Figure 1, the implementation scenario includes a vehicle terminal 1 and a platform terminal 2, wherein the vehicle terminal 1 is used to collect multimodal data and perform preliminary screening. The devices used to collect multimodal data may include a driver monitoring system (DMS) 10, an advanced driver assistance system (ADAS) 11, and a motion sensor 12. The driver monitoring system 10 can collect relevant information about the driver of the vehicle through the DMS camera 101 to determine the driver's actions or expressions, including at least one of abnormal driver movement and driver panic. The driver monitoring system 10 can also use a sound sensor 102 to detect voices in the cockpit, such as abnormal screams by the driver. The advanced driver assistance system 11 can collect the environment during the vehicle's driving process through the ADAS camera to detect whether there are any abnormalities in front of the vehicle, including at least one of vehicle collisions and pedestrian collisions. The motion sensor 12 can be used to collect the vehicle's motion information and determine the vehicle's motion state, including at least one of abnormal deceleration and abnormal vehicle posture. Platform 2 can perform preliminary screening of suspected traffic accidents based on the collected multimodal data, and send the multimodal data of suspected traffic accidents to platform 2. Platform 2 can obtain the identification result of suspected traffic accidents based on the driver's personalized driving habits, historical accident information of the location of the suspected traffic accident, and the multimodal data.

[0051] Figure 2 is a schematic diagram of the implementation process of a traffic accident identification method provided in an embodiment of this application, which is described in detail below:

[0052] In S201, the vehicle acquires multimodal data and performs preliminary screening of traffic accidents based on the multimodal data to obtain information related to suspected traffic accidents.

[0053] The multimodal data includes driver information from the vehicle side, and information related to suspected traffic accidents includes the location of the suspected traffic accident.

[0054] Multimodal data can include external cockpit image data, internal cockpit image data, internal cockpit voice data, and vehicle motion data. Devices for acquiring multimodal data, as shown in Figure 1, include ADAS, DMS, and motion sensors. ADAS can be used to acquire internal cockpit image and audio data. DMS can be used to acquire external cockpit image data. Motion sensors acquire vehicle motion state data.

[0055] When determining information related to suspected traffic accidents based on multimodal data, the vehicle-side system can make separate judgments on suspected traffic accidents based on the collected multimodal data.

[0056] For example, based on the cockpit image data collected by ADAS, at least one of the following can be detected: abnormal driver movement and driver panic.

[0057] Based on the driver's position in the image and the frame rate, the speed of abnormal driver movement can be determined. If both the abnormal speed and amplitude of movement meet preset requirements, a suspected traffic accident based on abnormal driver movement can be identified. For example, the distance and speed of head movement can be detected. If the speed exceeds a predetermined speed threshold and the distance exceeds a predetermined amplitude threshold, a suspected traffic accident due to abnormal driver movement can be identified.

[0058] The driver's facial expressions can be obtained from the collected cockpit image data. When the model identifies the driver's expression as one of fear, a suspected traffic accident can be identified based on the driver's fear.

[0059] Based on the external image data collected by the DMS, it can be used to detect at least one of the following: vehicle collision or pedestrian collision.

[0060] Based on the external image data collected by the DMS camera, the distance between the vehicle and other vehicles, or between the vehicle and pedestrians, can be determined, and the possibility of a vehicle collision or behavioral collision anomaly can be determined based on this distance. Alternatively, the distance between the vehicle and other objects can be determined using distance detection equipment such as LiDAR.

[0061] Motion sensors can collect vehicle motion data to detect whether the vehicle is in an abnormal deceleration state or in a state of suspected traffic accident with abnormal vehicle posture. Abnormal vehicle posture includes situations such as vehicle rollover or abnormal vehicle pitch angle.

[0062] Motion sensors, including gyroscopes or accelerometers, can detect the vehicle's deceleration in the direction of travel. If the deceleration is less than a predetermined speed threshold, a suspected traffic accident based on abnormal deceleration can be identified. Gyroscopes or other six-axis sensors can detect the vehicle's pose, and based on the angle between the vehicle and a standard state, determine whether it is in a suspected traffic accident state with an abnormal vehicle pose, such as a rollover or abnormal pitch angle.

[0063] Not limited to this, a comprehensive judgment can also be made by combining different multimodal data, for example:

[0064] Abnormal driver movement can be combined with facial expression recognition. Using in-cabin image data collected by ADAS (Advanced Driver Assistance Systems), the speed and amplitude of the driver's head movement can be detected. If both the speed and amplitude exceed set thresholds, it can be identified as abnormal movement. Simultaneously, facial expression recognition technology can be used to analyze whether the driver is exhibiting emotions such as fear. If both conditions are met, a suspected traffic accident can be identified.

[0065] Alternatively, forward collision detection can be combined with driver status data. Using exterior image data collected by the DMS (Driver Monitoring System), the risk of a vehicle or pedestrian collision can be detected. Furthermore, combining this with the driver's demeanor (such as abnormal movement or a panicked expression) can further confirm the likelihood of a collision. For example, if a pedestrian is detected ahead and the driver exhibits signs of panic, the situation can be more effectively classified as high-risk.

[0066] Alternatively, combining motion sensors with driver behavior allows for the monitoring of vehicle dynamics (such as deceleration or rollover) by motion sensors, combined with abnormal driver behavior (such as sudden movements or expressions of fear), to comprehensively assess the risk of a traffic accident. For example, if a vehicle decelerates abnormally while the driver displays fear, it can be considered a potential accident risk.

[0067] In S202, the vehicle sends the multimodal data and the suspected traffic accident-related information to the platform.

[0068] Due to limitations in computing power at the vehicle end, the accuracy of identifying suspected traffic accidents based on multimodal features is limited. In this embodiment, after identifying suspected traffic accidents based on multimodal data at the vehicle end, the relevant information of the suspected traffic accident is combined with the corresponding multimodal data and sent to the platform end for further detailed analysis and processing to obtain more accurate traffic accident identification results, reduce false alarm rate, and improve vehicle driving efficiency.

[0069] Once a suspected traffic accident is identified, the vehicle's location information can be obtained as the location of the suspected traffic accident.

[0070] Vehicle location information can be obtained through satellite positioning signals, lane-assisted positioning base stations (such as radio frequency positioning base stations) to determine the location of suspected traffic accidents, vehicle communication modules, or vehicle-to-everything (V2X) communication systems to determine the location of suspected traffic accidents from other vehicles.

[0071] In a possible implementation, after sending suspected traffic accident-related information and multimodal data to the platform, the vehicle can further collect multimodal data for a predetermined duration and send this predetermined duration of multimodal data to the platform, enabling the platform to obtain more accurate analysis results. For example, the vehicle can further collect multimodal data for 1-5 minutes and send it to the platform.

[0072] In S203, the platform determines the driver's personalized driving habits based on the driver information of the vehicle and determines the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident.

[0073] Driver information can include driver feature data such as those from cockpit image data collected by ADAS. For example, the platform can determine the driver's facial image based on the cockpit image data collected by ADAS, find the driver's identifier based on the facial image, and determine the driver's personalized driving habits based on the found driver identifier.

[0074] Driving habit data includes habitual data such as habitually closing one's eyes, habitually braking suddenly, and habitually accelerating suddenly, which are also the second relevant features associated with driving habits.

[0075] Based on the time of the suspected traffic accident detection and the collected location information, historical accident information regarding the suspected accident's location can be determined. This can be done by querying the accident database based on location to identify historical accidents matching the suspected accident's location, such as those located within a predetermined radius.

[0076] The historical accident information searched includes the main accident factors that caused the historical accident, that is, the primary relevant features of the historical accident information, including major accident factors such as road surface factors, visibility factors, and driver operation factors.

[0077] Based on the determined first and second relevant features, the weights of features in the collected multimodal data can be adjusted according to a direct proportional relationship, including the frequency of historical traffic accidents at the location of the suspected traffic accident. That is, the higher the frequency of historical traffic accidents at that location, the higher the weight of the first relevant feature, thus enabling more reliable identification of traffic accidents occurring at that location.

[0078] Based on the degree of a driver's personalized driving habits, the weight of the second relevant feature is adjusted in an inverse proportional relationship. For example, the higher the degree of personalized driving habits, the lower the weight of the second relevant feature, thereby reducing the number of false alarms that are mistaken for traffic accidents due to personalized driving habits, which helps improve recognition accuracy.

[0079] In S204, the platform obtains the identification result of a suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data.

[0080] This application embodiment can include multimodal data uploaded from the platform, including multimodal data within a certain time period before the detection of a suspected traffic accident, or multimodal data within a predetermined time period after the detection of a suspected traffic accident. The certain time period before the detection of a suspected traffic accident and the predetermined time period after the detection of a suspected traffic accident can be the same or different.

[0081] In this embodiment, the platform can generate an accident description text by combining information related to suspected traffic accidents, the driver's personalized driving habits, and historical accident information about the location where the suspected traffic accident was detected. The platform can then identify the accident description text using a large language model to make a more detailed analysis and judgment of the suspected traffic accident and obtain the identification result of the suspected traffic accident.

[0082] Figure 3 illustrates the implementation process of an accident recognition using a large-scale language model according to an embodiment of this application. In this process, the driver's identifier can be found based on the cockpit image data, and the driver's historical driving data can be retrieved based on the driver's identifier. The driver's historical driving data can be input into a trained habit data extraction model to obtain the driver's personalized driving habits. Based on the detected location of a suspected traffic accident, historical traffic accidents with a distance to the suspected accident location that meets a predetermined distance radius (i.e., a distance less than the predetermined distance radius) are searched in the traffic accident database to obtain historical traffic accident information related to the searched historical traffic accidents.

[0083] Based on the uploaded multimodal data, combined with the identified historical traffic accident information and personalized driving habits, textual accident description information can be obtained. This accident description information is then input into a language model to obtain the identification results of suspected traffic accidents based on large-scale language model analysis.

[0084] Because this method is based on historical traffic accident information determined by the location of the suspected traffic accident, and combined with the driver's personalized driving data for analysis and judgment, it is conducive to obtaining more accurate accident identification results, reducing false alarms, and improving fleet driving efficiency.

[0085] In possible implementations, the number of suspected traffic accidents reported by vehicles is relatively large; for example, the platform may receive information related to suspected traffic accidents from N vehicles. To prioritize responses to more urgent suspected traffic accidents, a prioritization method for suspected traffic accidents, as shown in Figure 4, can be adopted, including:

[0086] In S401, the platform determines the accident level based on the vehicle's six-axis acceleration and speed obtained by the sensors, combined with the weighted information corresponding to the terrain at the vehicle's current location.

[0087] This application embodiment can determine the vehicle's six-axis acceleration and speed based on motion data collected by motion sensors at the vehicle end. The six-axis acceleration acquisition device includes a three-axis accelerometer and a three-axis gyroscope. The three-axis accelerometer measures the vehicle's linear acceleration in the X, Y, and Z directions (generally, the vehicle's forward direction is the X direction, and the vertical upward direction is the Z direction), while the three-axis gyroscope measures the vehicle's angular velocities around the X, Y, and Z axes. Specifically: the X-axis gyroscope measures the vehicle's rotational speed around the X-axis, i.e., the vehicle's yaw rate; the Y-axis gyroscope measures the vehicle's rotational speed around the Y-axis, i.e., the vehicle's pitch rate; and the Z-axis gyroscope measures the vehicle's rotational speed around the Z-axis, i.e., the vehicle's roll rate.

[0088] When determining the weighting information based on the terrain at the vehicle's current location, the terrain information can include off-road terrain, unpaved road terrain, and paved road terrain. The lower the road surface smoothness, the lower the weighting of acceleration in the Z direction and rotational speed along the X and Y axes can be, while the weighting of vehicle speed will increase accordingly. The weighting of linear acceleration in the X-axis and rotational speed along the Y-axis is adjusted based on the road surface inclination; for example, the greater the road surface inclination, the smaller the weighting of rotational speed along the Y-axis.

[0089] Based on the detected six-axis acceleration and vehicle speed, and combined with the weights determined according to the terrain, a priority score is obtained for suspected traffic accidents, and the accident level of each suspected traffic accident is determined according to the priority score.

[0090] In S402, the platform determines the priority of vehicle accident handling based on the accident level.

[0091] Based on the determined accident level, higher-level suspected traffic accidents are prioritized for processing, thereby enabling more reliable identification of more serious traffic accidents.

[0092] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0093] Figure 5 is a schematic diagram of a traffic accident recognition device provided in an embodiment of this application. The device includes:

[0094] The data acquisition unit 501 is used to acquire multimodal data from the vehicle and perform preliminary screening of traffic accidents based on the multimodal data to obtain information related to suspected traffic accidents. The multimodal data includes driver information from the vehicle and the information related to suspected traffic accidents includes the location of the suspected traffic accident.

[0095] The information sending unit 502 is used to send the multimodal data and the suspected traffic accident-related information from the vehicle terminal to the platform terminal.

[0096] The information determination unit 503 is used to determine the driver's personalized driving habits based on the driver information of the vehicle, and to determine the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident.

[0097] The accident identification unit 504 is used by the platform to obtain the identification result of a suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data.

[0098] The traffic accident identification device shown in Figure 5 corresponds to the traffic accident identification method shown in Figure 2.

[0099] Figure 6 is a schematic diagram of a traffic accident recognition system provided in an embodiment of this application. As shown in Figure 6, the traffic accident recognition system 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62, such as a traffic accident recognition program, stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the above-described traffic accident recognition method embodiments. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments.

[0100] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 62 in the traffic accident recognition system 6.

[0101] The traffic accident recognition system may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that Figure 6 is merely an example of a traffic accident recognition system 6 and does not constitute a limitation on the traffic accident recognition system 6. It may include more or fewer components than shown, or combine certain components, or use different components. For example, the traffic accident recognition system may also include input / output devices, network access devices, buses, etc.

[0102] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0103] The memory 61 can be an internal storage unit of the traffic accident recognition system 6, such as a hard drive or memory of the traffic accident recognition system 6. The memory 61 can also be an external storage device of the traffic accident recognition system 6, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the traffic accident recognition system 6. Furthermore, the memory 61 can include both internal storage units and external storage devices of the traffic accident recognition system 6. The memory 61 is used to store the computer program and other programs and data required by the traffic accident recognition system. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0111] In addition, this application also provides a computer program product that, when run on a computer, causes the computer to execute the methods in the above-described implementations.

[0112] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying traffic accidents, characterized in that, The method includes: The vehicle acquires multimodal data and performs preliminary screening of traffic accidents based on the multimodal data to obtain information related to suspected traffic accidents. The multimodal data includes driver information from the vehicle and the information related to suspected traffic accidents includes the location of the suspected traffic accident. The vehicle terminal sends the multimodal data and the suspected traffic accident-related information to the platform terminal; The platform determines the driver's personalized driving habits based on the driver information on the vehicle, and determines the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident. The platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data.

2. The method according to claim 1, characterized in that, The platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data, including: The platform generates an accident description text based on the suspected traffic accident information, the driver's personalized driving habits, and the location's historical accident information; The platform inputs the accident description text into a large language model to obtain the identification result of the suspected traffic accident.

3. The method according to claim 1, characterized in that, The vehicle terminal sends the multimodal data and the suspected traffic accident-related information to the platform terminal, and the method further includes: After obtaining the information related to the suspected traffic accident, the vehicle continues to acquire multimodal data for a predetermined duration. The vehicle terminal will send the multimodal data used to determine information related to suspected traffic accidents, the multimodal data to be acquired, and the traffic accident information to the platform terminal.

4. The method according to claim 1, characterized in that, After obtaining the identification result of the suspected traffic accident on the platform based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data, the method further includes: The platform determines the accident level based on the vehicle's six-axis acceleration and speed obtained from the sensors, combined with the weighted information corresponding to the terrain at the vehicle's current location. The platform determines the priority of vehicle accident handling based on the accident level.

5. The method according to claim 1, characterized in that, The multimodal data includes vehicle motion data, cockpit in-cabin image data, and cockpit out-of-cabin image data; Based on the multimodal data, preliminary screening of traffic accidents is performed to obtain relevant information on suspected traffic accidents, including: The vehicle terminal determines at least one of the following based on the vehicle motion data: sudden deceleration of the vehicle, vehicle collision, and vehicle rollover. The vehicle terminal determines at least one of the following events based on the image data inside the cockpit: abnormal driver movement and driver panic.

6. The method according to any one of claims 1-5, characterized in that, The platform obtains the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data, including: The first relevant feature for determining the association of historical accident information and the second relevant feature for determining the association of personalized driving habits were identified. The weight of the first relevant feature is determined based on the frequency of historical accident information at the location, and the weight of the second relevant feature is determined based on the driver's personalized driving habits.

7. The method according to claim 6, characterized in that, Determining the weight of the first relevant feature based on the frequency of historical accident information at the location includes: Based on the frequency of historical traffic accidents at the location, the weights of the first relevant features are adjusted according to a direct proportional relationship. The weight of the second relevant feature is determined based on the driver's personalized driving habits; Based on the degree of the driver's personalized driving habits, the weight of the second relevant feature is adjusted according to an inverse proportional relationship.

8. A traffic accident identification device, characterized in that, The device includes: The data acquisition unit is used to acquire multimodal data from the vehicle and perform preliminary screening of traffic accidents based on the multimodal data to obtain information related to suspected traffic accidents. The multimodal data includes driver information from the vehicle and the information related to suspected traffic accidents includes the location of the suspected traffic accident. An information sending unit is used to send the multimodal data and the suspected traffic accident-related information from the vehicle to the platform. The information determination unit is used to determine the driver's personalized driving habits based on the driver information of the vehicle, and to determine the historical accident information of the suspected traffic accident location based on the location of the suspected traffic accident. The accident identification unit is used by the platform to obtain the identification result of the suspected traffic accident based on the driver's personalized driving habits, the historical accident information of the location, and the multimodal data.

9. A traffic accident recognition system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the traffic accident recognition system to implement the method as described in any one of claims 1-7.

10. A computer program product comprising computer program instructions, characterized in that, When the computer program is run, the method as described in any one of claims 1-7 is performed.

Images