Traffic event perception analysis method and system based on big language model intelligent assistant
By combining an intelligent assistant based on a large language model with computer vision and deep learning technology, the problems of poor human-computer interaction and resource waste in traffic event perception and analysis are solved, and low-cost, intelligent real-time traffic event detection and analysis is achieved.
Patent Information
- Application Number
- CN202510805901.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-19
AI Technical Summary
Existing traffic incident perception and analysis rely on manual retrieval and re-inspection, resulting in problems such as poor human-computer interaction, low intelligence level of event perception and analysis, and waste of human resources.
A traffic incident perception and analysis method based on a large language model intelligent assistant is adopted, combining computer vision technology and deep learning. By collecting traffic data to train the chatGLM model, traffic incident detection in real-time video streams and an intelligent traffic assistant using natural language processing technology are realized to conduct human-computer interaction and intelligent analysis.
It realizes low-cost, intelligent, and real-time traffic incident perception and analysis, reduces the training cost and human resource waste of manual re-inspection, and improves the efficiency of traffic incident analysis.
Smart Images
Figure CN120673316A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of computer vision technology and natural language processing technology, and in particular to a traffic event perception and analysis method and system based on a large language model intelligent assistant. Background Art
[0002] With the development of artificial intelligence (AI) technology and the growing demand for intelligent, automated urban development, the concept of smart cities is expanding across various sectors, with smart transportation becoming a key component. A key challenge in achieving smart transportation lies in intelligent traffic event perception and analysis. Traditional traffic event perception and analysis relies heavily on manual search and analysis. While current computer vision technology can detect traffic events, the output still exhibits errors, requiring re-inspection by professionally trained personnel. This leads to poor human-computer interaction, low intelligence in event perception and analysis, and a waste of human resources.
[0003] With the development of natural language processing technology and the massive growth of data in the information age, large language model algorithms trained on massive amounts of data have emerged. These models and algorithms possess excellent interactivity and, based on prior knowledge, possess considerable analytical and reasoning capabilities, enabling them to understand and discriminate input information. Therefore, they have the potential to enable intelligent traffic event analysis and reduce manual learning costs. Existing traffic event perception methods based on computer vision and deep learning technologies can achieve intelligent traffic event perception, providing a perceptual foundation for intelligent traffic assistants. Based on the above technical background, a traffic event perception and analysis method based on a large language model intelligent assistant would be proposed. This method utilizes a front-end perception and detection algorithm based on computer vision and deep learning, combined with real-time video data streams from road cameras and other real-time video information acquisition devices, to perform intelligent traffic event detection. Simultaneously, an intelligent traffic assistant based on natural language processing technology is applied at the interactive end. This intelligent traffic assistant enables human-computer interaction and leverages its analytical and reasoning capabilities to achieve low-cost, intelligent, and real-time perception and analysis of road traffic events. This effectively reduces the training costs and human resource waste associated with manual re-inspection, enabling efficient intelligent traffic event perception and analysis.
[0004] Based on the above technical background, the present invention proposes a traffic event perception and analysis method based on a large language model intelligent assistant. Summary of the Invention
[0005] In order to solve the above technical problems, the present invention provides a traffic event perception and analysis method based on a large language model intelligent assistant, the method comprising:
[0006] Step S1: Collect traffic data to form a smart transportation vertical domain corpus dataset;
[0007] Step S2: training a chatGLM model using the smart transportation vertical domain corpus dataset to obtain a trained chatGLM model;
[0008] Step S3: collecting real-time video streams and obtaining traffic event perception data in the video streams;
[0009] Step S4: Use a deep learning convolutional neural network to obtain traffic event information from the traffic event perception data, and complete the traffic event perception analysis method based on a large language model intelligent assistant.
[0010] Optionally, in step S1, the smart transportation vertical domain corpus dataset includes traffic rules, road sign information, and vehicle information.
[0011] Optionally, in step S2, the process of training the chatGLM model using the smart transportation vertical domain corpus dataset to obtain the trained chatGLM model specifically includes:
[0012] The chatGLM model is trained using a decoder structure. For the smart transportation vertical domain corpus dataset U={u1,u2,....,u n}, for each input data u0, the neural network calculates the probability distribution through the above:
[0013] h0=UW e +W p
[0014]
[0015] Where U is the context vector of the token, n is the number of layers, and W e is the token encoding matrix, W p is the position embedding matrix, h0 is the initial weight matrix, h l is the matrix after the multi-head attention calculation of the lth layer, P(u) is the activation function, h n is the output of the last layer of attention network, is the transpose of the token encoding matrix, and transformer_block is the multi-head attention calculation function.
[0016] The obtained probability distribution result is used as a training data to calculate the objective function:
[0017]
[0018] Based on the objective function result, the parameters of the chatGLM model are fine-tuned by the gradient descent method to obtain a trained chatGLM model.
[0019] Optionally, based on the objective function result, the parameters of the chatGLM model are fine-tuned by gradient descent method as follows:
[0020] The chatGLM model is fine-tuned by superimposing the objective functions to obtain the final objective function L3:
[0021] L3(C)=L2(C)+λ*L1(C);
[0022] L2 is obtained based on the training data and annotations in the smart transportation vertical domain corpus dataset:
[0023]
[0024] The present invention also discloses a traffic event perception and analysis system based on a large language model intelligent assistant, the system comprising:
[0025] The data collection module is used to collect traffic data to form the smart transportation vertical domain corpus dataset;
[0026] A large language model training module is used to train the chatGLM model using the smart transportation vertical domain corpus dataset to obtain a trained chatGLM model;
[0027] Real-time data acquisition module, used to collect real-time video streams and obtain traffic event perception data in the video streams;
[0028] The traffic perception module is used to use a deep learning convolutional neural network to obtain traffic event information from the traffic event perception data and complete a traffic event perception analysis method based on a large language model intelligent assistant.
[0029] Optionally, in the data acquisition module, the smart transportation vertical domain corpus dataset includes traffic rules, road sign information, and vehicle information.
[0030] Optionally, in the large language model training module, the chatGLM model is trained using the smart transportation vertical domain corpus dataset, and the process of obtaining the trained chatGLM model specifically includes:
[0031] The chatGLM model is trained using a decoder structure. For the smart transportation vertical domain corpus dataset U={u1,u2,....,u n}, for each input data u0, the neural network calculates the probability distribution through the above:
[0032] h0=UW e +W p
[0033]
[0034] Where U is the context vector of the token, n is the number of layers, and W e is the token encoding matrix, W p is the position embedding matrix. h0 is the initial weight matrix, h l is the matrix after the multi-head attention calculation of the lth layer, P(u) is the activation function, h n is the output of the last layer of attention network, is the transpose of the token encoding matrix, and transformer_block is the multi-head attention calculation function;
[0035] The obtained probability distribution result is used as a training data to calculate the objective function:
[0036]
[0037] Based on the objective function result, the parameters of the chatGLM model are fine-tuned by the gradient descent method to obtain a trained chatGLM model.
[0038] Optionally, based on the objective function result, the parameters of the chatGLM model are fine-tuned by gradient descent method as follows:
[0039] The chatGLM model is fine-tuned by superimposing the objective functions to obtain the final objective function L3:
[0040] L3(C)=L2(C)+λ*L1(C);
[0041] L2 is obtained based on the training data and annotations in the smart transportation vertical domain corpus dataset:
[0042]
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] This invention addresses the problems of poor human-computer interaction, low intelligence in event perception and analysis, and waste of human resources in existing traffic event perception and analysis processes, which rely on manual retrieval or re-inspection for event identification, analysis, and reporting. The invention proposes a traffic event perception and analysis method based on a large language model intelligent assistant. By developing a UI interface and integrating it with a back-end visual detection algorithm based on an intelligent assistant algorithm based on a large language model, combined with a traffic event detection algorithm based on computer vision technology as the back-end foundation, this method achieves real-time detection of abnormal road traffic events, human-computer interaction, intelligent analysis, and other functions, providing a technical foundation for intelligent perception and analysis of traffic events, reducing the labor cost of traffic accident retrieval and feedback processing, and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0046] Figure 1 A diagram showing the steps of a traffic incident perception and analysis method based on a large language model intelligent assistant according to an embodiment of the present invention;
[0047] Figure 2 This is a schematic diagram of the intelligent assistant training architecture according to an embodiment of the present invention;
[0048] Figure 3 This is a flowchart of the fine-tuning training of the intelligent assistant according to an embodiment of the present invention;
[0049] Figure 4 A schematic diagram of a UI interface according to an embodiment of the present invention;
[0050] Figure 5 This is a flow chart of the perception algorithm of an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0053] Example 1
[0054] A traffic incident perception and analysis method based on a large language model intelligent assistant, such as Figure 1 As shown, the method includes:
[0055] Step S1: Collect traffic data to form a smart transportation vertical domain corpus dataset.
[0056] The present invention collects text content of prior knowledge in the traffic field, including but not limited to traffic rules, road sign information, vehicle information, and text content of question and answer corpus about traffic event reasoning and analysis. Through the above method, it organizes and produces a smart traffic vertical domain corpus dataset for vertical domain training of large language models.
[0057] During the implementation of this invention, the aforementioned method was used to collect traffic-related information text data and question-and-answer corpus data. After data cleaning, organization, and annotation, a smart transportation vertical domain corpus dataset was constructed. The data collected through retrieval using this method was divided into two parts: prior information and question-and-answer data. The prior information included 10,237 items across five categories, including traffic regulations, vehicle information, and road sign information. The question-and-answer data included four fields, including question, yes / no, and explanation, covering 15 vertical traffic domain categories and totaling 12,076 pairs of data.
[0058] Step S2, use the smart transportation vertical domain corpus dataset to train the chatGLM model, obtain the smart transportation large language model obtained by fine-tuning the chatGLM, and build and implement the subsequent smart transportation vertical domain intelligent assistant based on the interaction and reasoning capabilities of the smart transportation large language model. Vertical domain training generates a smart transportation vertical domain intelligent assistant. The large language model used in the present invention is the language dialogue model algorithm chatGLM developed and open-sourced by Tsinghua University in China. The chatGLM is vertically trained using the constructed smart transportation vertical domain corpus dataset, and the trained language model will be used to implement the smart transportation intelligent assistant. ChatGLM uses a decoder structure for training, and its overall architecture is as follows: Figure 2 As shown in Figure 1. First, the model is pre-trained based on large-scale data.
[0059] The chatGLM model is trained using the smart transportation vertical domain corpus dataset. The process of obtaining the trained chatGLM model includes:
[0060] The chatGLM model is trained using a decoder structure. For the smart transportation vertical domain corpus dataset U={u1,u2,....,u n}, for each input data u0, the neural network calculates the probability distribution through the above:
[0061] h0=UW e +W p
[0062]
[0063] Where U is the context vector of the token, n is the number of layers, and W e is the token encoding matrix, W p is the position embedding matrix. h0 is the initial weight matrix, h l is the matrix after the multi-head attention calculation of the lth layer, P(u) is the activation function, h n is the output result of the last layer of attention network, is the transpose of the token encoding matrix, and transformer_block is the multi-head attention calculation function;
[0064] The obtained probability distribution result is used as a training data to calculate the objective function:
[0065]
[0066] Based on the objective function result, the parameters of the chatGLM model are fine-tuned by the gradient descent method to maximize the objective function and obtain a trained chatGLM model.
[0067] After that, the model is fine-tuned and trained. The fine-tuning training includes the following three characteristics.
[0068] First, as Figure 3 As shown in the figure, the models involved in fine-tuning training include both the pre-trained model in the first stage and the linear model following the pre-trained model. The linear output layer is:
[0069]
[0070] Second, fine-tuning training is similar to other training processes. Based on the training data, an objective function calculated based on the network parameters is maximized, and the network parameters are continuously adjusted through backpropagation to achieve the learning process. The objective function of chatGLM during fine-tuning training is a superposition of two parts:
[0071] L3(C)=L2(C)+λ*L1(C)
[0072] Among them, L3 is the final objective function, and L1 is calculated only based on the training data:
[0073]
[0074] L2 is calculated based on the training data and annotations:
[0075]
[0076] The third characteristic of fine-tuning training is that the input of different tasks needs to be adjusted during the training process.
[0077] The fine-tuned model will have knowledge of the transportation field and will be able to interact with people in the form of dialogue on transportation information exchange, question and answer, and decision support.
[0078] The training environment is the Linux Ubuntu 20.04 operating system, the graphics card used is NVIDIA V100, the division ratio of the training set to the test set is 7:3, the training parameter batch_size is 32, and the initial learning rate is set to 0.0001.
[0079] Step S3: Collect real-time video streams and obtain traffic event perception data in the video streams.
[0080] This embodiment is designed to build a front-end interface based on Gradio, which is used to build a platform for intelligent assistants to interact with people and realize a modular and visual human-computer interaction interface.
[0081] The UI interface sets the dialogue window and question prompt interface, as shown in the following diagram: Figure 4 As shown in the figure, the dialogue window is used to interact with the model. Users can ask questions to obtain information about traffic incidents that occurred within a certain period of time, including the number of incidents, their handling status, and other auxiliary functions such as incident handling decision-making suggestions. All analytical and reasoning functions are provided by the intelligent assistant model, eliminating the need for user professional knowledge training. The question prompt interface contains information on key issues that users may need, helping them to ask questions to obtain relevant information or assist in decision-making.
[0082] Step S4: Use deep learning convolutional neural network to obtain traffic event information from the traffic event perception data, and complete the traffic event perception analysis method based on the large language model intelligent assistant. The algorithm flow chart is as follows: Figure 5 The back-end event detection algorithm based on computer vision technology is the perception module in the design of the present invention.
[0083] The algorithm is mainly divided into three parts. The first part is the basic function of the algorithm. It realizes traffic target detection and tracking through deep learning convolutional neural network, and preliminarily obtains the trajectory information of targets such as pedestrians and vehicles. In this part, it can realize the perception and detection of traffic abnormal events such as road fires and pedestrian intrusions. The abnormal event perception detection implemented in this part is based on the end-to-end target detection neural network, and the perception and judgment of events are realized through the detection of targets such as fires and pedestrians.
[0084] After acquiring information such as the trajectory of moving targets on the road, the second part involves designing logic code for further event discrimination, enabling the detection of traffic events such as wrong-way driving, unusual lane changes, unusual parking, and road congestion. Event perception detection in this part combines target detection with specific logic algorithms for discrimination. For example, the discrimination of wrong-way driving and unusual lane changes involves the association between the detected vehicle target and lane markings, while unusual parking and road congestion involve the results of target detection and tracking.
[0085] The third part of the algorithm is event information reporting. Based on the perception and detection of road traffic incidents, the identified event information is encoded and fed back to the front end. The event time, target information involved, event type and other information are encoded into standard JSON format output information and transmitted to the front-end intelligent assistant for analysis and reasoning.
[0086] Connect the intelligent assistant module and the event perception module interface to realize model interaction.
[0087] First, we developed an interface based on the constructed intelligent vertical transportation assistant, building an open intelligent assistant model to enable it to receive traffic event information output by the back-end event perception algorithm. We also constructed a memory module to give the language model-based intelligent assistant information storage capabilities, enabling it to store event information transmitted by the event perception module.
[0088] The event perception module is connected to hardware devices such as cameras deployed on the road, receives data information such as real-time video streams, perceives traffic events through computer vision technology, and transmits the detected event information in a post manner to the intelligent assistant for analysis, thus completing the construction of the model algorithm.
[0089] Debugging is required based on the specific models and computing power requirements of the server and front-end devices, as well as user test feedback. If necessary, repeat step 2 to make changes.
[0090] Example 2
[0091] A traffic event perception and analysis system based on a large language model intelligent assistant, the system comprising:
[0092] The data acquisition module is used to collect traffic data to form the smart transportation vertical domain corpus dataset.
[0093] The present invention collects text content of prior knowledge in the traffic field, including but not limited to traffic rules, road sign information, vehicle information, and text content of question and answer corpus about traffic event reasoning and analysis. Through the above method, it organizes and produces a smart traffic vertical domain corpus dataset for vertical domain training of large language models.
[0094] During the implementation of this invention, the aforementioned method was used to collect traffic-related information text data and question-and-answer corpus data. After data cleaning, organization, and annotation, a smart transportation vertical domain corpus dataset was constructed. The data collected through retrieval using this method was divided into two parts: prior information and question-and-answer data. The prior information included 10,237 items across five categories, including traffic regulations, vehicle information, and road sign information. The question-and-answer data included four fields, including question, yes / no, and explanation, covering 15 vertical traffic domain categories and totaling 12,076 pairs of data.
[0095] The large language model training module is used to train the chatGLM model using the smart transportation vertical domain corpus dataset to obtain the smart transportation large language model obtained by fine-tuning the chatGLM. Based on the interaction and reasoning capabilities of the smart transportation large language model, the subsequent smart transportation vertical domain intelligent assistant is built and implemented.
[0096] In this embodiment, 1. ChatGLM is an open source large language model. After training with traffic field data, a new large language model for the traffic vertical field is obtained; 2. The role of training is to give the general model vertical field knowledge that it did not previously have; 3. Input data and training data are two concepts. Training data is only used in the training phase. The original training data of chatGLM is not open source. The data used for training in this embodiment belongs to a self-constructed data set. In actual applications, the format of the input data is no different from that of the pre-training model. The large language model interacts in a human-computer dialogue mode, and the input data are all text information; 4. Traffic event perception data is not obtained by the traffic large language model, but by a convolutional neural network model based on deep learning, and is transmitted to the large language model through an interface for reasoning and analysis. The entire system is a combination of a basic perception model and a large language model used for reasoning.
[0097] Vertical domain training generates a smart transportation vertical domain intelligent assistant. The large language model used in this invention is the language dialogue model algorithm chatGLM developed and open-sourced by Tsinghua University in China. The chatGLM is trained using the constructed smart transportation vertical domain corpus dataset. The trained language model will be used to implement the smart transportation intelligent assistant. ChatGLM uses a decoder structure for training, and its overall architecture is as follows: Figure 2 As shown in Figure 1. First, the model is pre-trained based on large-scale data.
[0098] The chatGLM model is trained using the smart transportation vertical domain corpus dataset. The process of obtaining the trained chatGLM model includes:
[0099] The chatGLM model is trained using a decoder structure. For the smart transportation vertical domain corpus dataset U={u1,u2,....,u n}, for each input data u0, the neural network calculates the probability distribution through the above:
[0100] h0=UW e +W p
[0101]
[0102] Where U is the context vector of the token, n is the number of layers, and W e is the token encoding matrix, W p is the position embedding matrix. h0 is the initial weight matrix, h l is the matrix after the multi-head attention calculation of the lth layer, P(u) is the activation function, h n is the output of the last layer of attention network, is the transpose of the token encoding matrix, and transformer_block is the multi-head attention calculation function;
[0103] The obtained probability distribution result is used as a training data to calculate the objective function:
[0104]
[0105] Based on the objective function result, the parameters of the chatGLM model are fine-tuned by the gradient descent method to maximize the objective function and obtain a trained chatGLM model.
[0106] After that, the model is fine-tuned and trained. The fine-tuning training includes the following three characteristics.
[0107] First, as Figure 3 As shown in the figure, the models involved in fine-tuning training include both the pre-trained model in the first stage and the linear model following the pre-trained model. The linear output layer is:
[0108]
[0109] Second, fine-tuning training is similar to other training processes. Based on the training data, an objective function calculated based on the network parameters is maximized, and the network parameters are continuously adjusted through backpropagation to achieve the learning process. The objective function of chatGLM during fine-tuning training is a superposition of two parts:
[0110] L3(C)=L2(C)+λ*L1(C)
[0111] Among them, L3 is the final objective function, and L1 is calculated only based on the training data:
[0112]
[0113] L2 is calculated based on the training data and annotations:
[0114]
[0115] The third characteristic of fine-tuning training is that the input of different tasks needs to be adjusted during the training process.
[0116] The fine-tuned model will have knowledge of the transportation field and will be able to interact with people in the form of dialogue on transportation information exchange, question and answer, and decision support.
[0117] The training environment is the Linux Ubuntu 20.04 operating system, the graphics card used is NVIDIA V100, the division ratio of the training set and the test set is 7:3, the training parameter batch_size is 32, and the initial learning rate is set to 0.0001.
[0118] The real-time data acquisition module is used to collect real-time video streams and obtain traffic event perception data in the video streams.
[0119] This embodiment is designed to build a front-end interface based on Gradio, which is used to build a platform for intelligent assistants to interact with people and realize a modular and visual human-computer interaction interface.
[0120] The UI interface sets the dialogue window and question prompt interface, as shown in the following diagram: Figure 4 As shown in the figure, the dialogue window is used to interact with the model. Users can ask questions to obtain information about traffic incidents that occurred within a certain period of time, including the number of incidents, their handling status, and other auxiliary functions such as incident handling decision-making suggestions. All analytical and reasoning functions are provided by the intelligent assistant model, eliminating the need for user professional knowledge training. The question prompt interface contains information on key issues that users may need, helping them to ask questions to obtain relevant information or assist in decision-making.
[0121] The traffic perception module is used to use deep learning convolutional neural networks to obtain traffic event information from the traffic event perception data and complete the traffic event perception analysis method based on the large language model intelligent assistant. The algorithm flow chart is as follows Figure 5 The back-end event detection algorithm based on computer vision technology is the perception module in the design of the present invention.
[0122] The algorithm is mainly divided into three parts. The first part is the basic function of the algorithm. It realizes traffic target detection and tracking through deep learning convolutional neural network, and preliminarily obtains the trajectory information of targets such as pedestrians and vehicles. In this part, it can realize the perception and detection of traffic abnormal events such as road fires and pedestrian intrusions.
[0123] After obtaining information such as the trajectory of moving targets on the road, the second part designs logic code for further event discrimination to detect traffic events such as vehicle reversing, abnormal lane changes, abnormal parking, and road congestion.
[0124] The third part of the algorithm is event information reporting. Based on the perception and detection of road traffic incidents, the identified event information is encoded and fed back to the front end. The output information includes event time, information about the target involved, event type, and other information for the front-end intelligent assistant to analyze and reason.
[0125] Connect the intelligent assistant module and the event perception module interface to realize model interaction.
[0126] First, we developed an interface based on the vertical domain intelligent assistant for smart transportation and constructed an open intelligent assistant model, enabling it to receive traffic event information output by the backend event perception algorithm. We also constructed a memory module to give the language model-based intelligent assistant information storage capabilities, enabling it to store event information transmitted by the event perception module.
[0127] The event perception module is connected to hardware devices such as cameras deployed on the road, receives data information such as real-time video streams, perceives traffic events through computer vision technology, and transmits the detected event information in a post manner to the intelligent assistant for analysis, thus completing the construction of the model algorithm.
[0128] Debugging is required based on the specific models and computing power requirements of the server and front-end equipment, as well as feedback from user tests. Repeat the operation for improvement if necessary.
[0129] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.
Claims
1. A traffic incident perception and analysis method based on a large language model intelligent assistant, characterized in that: The method comprises: Step S1: Collect traffic data to form a smart transportation vertical domain corpus dataset; Step S2: training a chatGLM model using the smart transportation vertical domain corpus dataset to obtain a trained chatGLM model; Step S3: collecting real-time video streams and obtaining traffic event perception data in the video streams; Step S4: Use a deep learning convolutional neural network to obtain traffic event information from the traffic event perception data, and complete the traffic event perception analysis method based on a large language model intelligent assistant.
2. The traffic event perception analysis method based on a large language model intelligent assistant according to claim 1 is characterized in that: In step S1, the smart transportation vertical domain corpus dataset includes traffic rules, road sign information, and vehicle information.
3. The traffic incident perception and analysis method based on a large language model intelligent assistant according to claim 1 is characterized in that: In step S2, the process of training the chatGLM model using the smart transportation vertical domain corpus dataset to obtain the trained chatGLM model specifically includes: The chatGLM model is trained using a decoder structure. For the smart transportation vertical domain corpus dataset U={u1,u2,....,u n }, for each input data u0, the neural network calculates the probability distribution through the above: h0=UW e +W p Where U is the context vector of the token, n is the number of layers, and W e is the token encoding matrix, W p is the position embedding matrix, h0 is the initial weight matrix, h l is the matrix after the multi-head attention calculation of the lth layer, P(u) is the activation function, h n is the output result of the last layer of attention network, is the transpose of the token encoding matrix, and transformer_block is the multi-head attention calculation function; The obtained probability distribution result is used as a training data to calculate the objective function: Where k is the size of the context window, the conditional probability P is modeled using a neural network with parameters Θ, and all parameters are trained by stochastic gradient descent; Based on the objective function result, the parameters of the chatGLM model are fine-tuned by the gradient descent method to obtain a trained chatGLM model.
4. The traffic event perception analysis method based on a large language model intelligent assistant according to claim 3 is characterized in that: Based on the objective function results, the parameters of the chatGLM model are fine-tuned by the gradient descent method as follows: The chatGLM model is fine-tuned by superimposing the objective functions to obtain the final objective function L3: L3(C)=L2(C)+λ*L1(C); Among them, L2 is obtained based on the training data and annotations in the smart transportation vertical domain corpus dataset:
5. A traffic incident perception and analysis system based on a large language model intelligent assistant, the system being used to implement the traffic incident perception and analysis method based on a large language model intelligent assistant according to any one of claims 1 to 4, characterized in that: The system comprises: The data collection module is used to collect traffic data to form the smart transportation vertical domain corpus dataset; A large language model training module is used to train the chatGLM model using the smart transportation vertical domain corpus dataset to obtain a trained model; Real-time data acquisition module, used to collect real-time video streams and obtain traffic event perception data in the video streams; The traffic perception module is used to use a deep learning convolutional neural network to obtain traffic event information from the traffic event perception data and complete a traffic event perception analysis method based on a large language model intelligent assistant.
6. The traffic event perception and analysis system based on a large language model intelligent assistant according to claim 5 is characterized in that: In the data acquisition module, the smart transportation vertical domain corpus dataset includes traffic rules, road sign information, and vehicle information.
7. The traffic event perception and analysis system based on a large language model intelligent assistant according to claim 6 is characterized in that: In the large language model training module, the chatGLM model is trained using the smart transportation vertical domain corpus dataset to obtain the trained chatGLM model. The process specifically includes: The chatGLM model is trained using a decoder structure. For the smart transportation vertical domain corpus dataset U={u1,u2,....,u n }, for each input data u0, the neural network calculates the probability distribution through the above: h0=UW e +W p Where U is the context vector of the token, n is the number of layers, and W e is the token encoding matrix, W p is the position embedding matrix. h0 is the initial weight matrix, h l is the matrix after the multi-head attention calculation of the lth layer, P(u) is the activation function, h n is the output of the last layer of attention network, is the transpose of the token encoding matrix, and transformer_block is the multi-head attention calculation function; The obtained probability distribution result is used as a training data to calculate the objective function: where k is the size of the context window, and the conditional probability P is modeled using a neural network with parameters Θ, all of which are trained via stochastic gradient descent. Based on the objective function result, the parameters of the chatGLM model are fine-tuned by the gradient descent method to obtain a trained chatGLM model.
8. The traffic event perception and analysis system based on a large language model intelligent assistant according to claim 7 is characterized in that: Based on the objective function results, the parameters of the chatGLM model are fine-tuned by the gradient descent method as follows: The chatGLM model is fine-tuned by superimposing the objective functions to obtain the final objective function L3: L3(C)=L2(C)+λ*L1(C); L2 is obtained based on the training data and annotations in the smart transportation vertical domain corpus dataset: