A method, apparatus, equipment, medium, and product for analyzing traffic anomalies.

Abstract

This application discloses a method, apparatus, device, medium, and product for analyzing traffic anomalies, relating to the field of image processing technology. The method includes: identifying traffic anomaly images from acquired road videos; obtaining a target work order description instruction from a pre-constructed work order description instruction dataset; inputting the traffic anomaly image and the target work order description instruction into a multimodal large language model to obtain event work order information of the traffic anomaly output by the multimodal large language model; inputting the traffic anomaly image and a pre-set false detection analysis instruction into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model; if the authenticity analysis result indicates that the traffic anomaly is a real event, then reporting the event work order information. This application can improve the accuracy of traffic anomaly analysis results.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, equipment, medium, and product for analyzing traffic anomalies. Background Technology

[0002] In the field of traffic management, the timely and accurate handling of traffic anomalies plays a crucial role in ensuring road traffic order, improving traffic safety, and optimizing the allocation of traffic resources. With the increasing complexity of urban traffic networks and the continuous rise in traffic volume, various traffic anomalies, such as traffic accidents, vehicle violations, and road obstructions, occur frequently, posing significant challenges to traffic management.

[0003] Currently, manual processing dominates the handling of traffic incidents. When a traffic incident occurs, relevant staff bear a heavy and meticulous workload. They need to spend a significant amount of time reviewing video footage to analyze the incident's progression, the vehicles and individuals involved, and manually filling out work orders to record detailed information such as incident type, time, location, and characteristics of those involved, providing a basis for subsequent processing and statistical analysis.

[0004] However, this process of analyzing traffic anomalies based on traditional manual methods is inefficient, and manual operation makes it difficult to guarantee the accuracy and comprehensiveness of the processing, resulting in low accuracy of the analysis results. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, equipment, medium, and product for analyzing traffic anomalies, which can improve the accuracy of the analysis results for traffic anomalies.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for analyzing traffic anomalies, including: Traffic anomaly event images are identified from the collected road videos; wherein, the traffic anomaly event image includes an identifier box, road information, timestamp, and anomaly event type, and the identifier box is used to identify the abnormal object of the traffic anomaly event; Obtain a target work order description instruction from a pre-built work order description instruction dataset; wherein, the work order description instruction dataset contains multiple work order description instructions; The traffic anomaly event image and the target work order description instruction are input into a multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model; wherein, the event work order information includes at least the event number, event time, event location, event type, event description, and appearance features of the event object; The traffic anomaly image and the pre-set false detection analysis command are input into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model; If the authenticity analysis results indicate that the traffic anomaly is a real event, then the event work order information will be reported.

[0007] Optionally, the work order description instruction dataset is constructed as follows: Retrieve the pre-built initial work order description instruction; Identify multiple instruction application scenario types; wherein, the instruction application scenario types include at least batch processing type, complex scenario type, interactive type, low-precision model type, and high-precision model type; The initial work order description instruction is optimized based on the various instruction application scenario types to obtain the optimized work order description instruction corresponding to each instruction application scenario type. Construct a work order description instruction dataset that includes the initial work order description instruction and each optimized work order description instruction.

[0008] Optionally, obtaining a target work order description instruction from a pre-built work order description instruction dataset specifically includes: Obtain the current instruction application scenario type corresponding to the multimodal large language model; If there is a target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then the work order description instruction corresponding to the target instruction application scenario type is determined as the target work order description instruction. If there is no target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then a work order description instruction is randomly selected from the work order description instruction dataset as the target work order description instruction.

[0009] Optionally, the step of inputting the traffic anomaly event image and the target work order description instruction into a multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model specifically includes: The traffic anomaly image and the target work order description instruction are input into a multimodal large language model, so that the multimodal large language model can identify the road number and timestamp from the traffic anomaly image based on the target work order description instruction; The road number and the timestamp are used as the event number; The timestamp is determined as the event time; The location of the traffic anomaly is determined based on the road number; The abnormal event type is identified from the traffic abnormal event image using the multimodal large language model; The abnormal event type is determined as the event type of the traffic abnormal event; The abnormal objects in the bounding boxes of the traffic anomaly event image are identified by the multimodal large language model to determine the event description and appearance features of the event objects. The multimodal large language model outputs event work order information containing the event number, event time, event location, event type, event description, and appearance features of the event object.

[0010] Optionally, the step of inputting the traffic anomaly image and pre-set false detection analysis instructions into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model specifically includes: The traffic anomaly image and pre-set false detection analysis instructions are input into the multimodal large language model, so that the multimodal large language model can extract the false detection analysis instructions to obtain multiple false detection analysis sub-instructions; wherein, the false detection analysis sub-instructions include at least image quality analysis sub-instructions, object analysis sub-instructions, type analysis sub-instructions and location analysis sub-instructions; The image quality of the traffic anomaly event image is analyzed using the multimodal large language model based on the image quality analysis sub-instruction, and the image quality analysis result is determined. If the image quality analysis result indicates that the quality of the traffic anomaly image is acceptable, then the abnormal objects in the identifier box of the traffic anomaly image are analyzed by the multimodal large language model based on the object analysis sub-instruction to determine the object analysis result; If the object analysis result indicates that the abnormal object exists in the identification box, then the correctness of the abnormal event type of the traffic abnormal event image is analyzed by the multimodal large language model based on the type analysis sub-instruction to determine the type analysis result; If the type analysis result indicates that the abnormal event type is correct, then the relative positional relationship between the abnormal object and the road in the traffic abnormal event image is analyzed by the multimodal large language model based on the location analysis sub-instruction to obtain the location analysis result; If the location analysis result indicates that the abnormal object is located on the road, then the traffic anomaly event is determined to be a real event as the authenticity analysis result of the traffic anomaly event.

[0011] Optionally, the method further includes: If the image quality analysis result indicates that the quality of the traffic anomaly image is unqualified, then the traffic anomaly is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; If the object analysis result indicates that the abnormal object does not exist in the identifier box, then the traffic anomaly event is determined as a false detection event as the authenticity analysis result of the traffic anomaly event; If the type analysis result indicates that the abnormal event type is incorrect, then the traffic abnormal event is determined to be a false detection event as the authenticity analysis result of the traffic abnormal event; If the location analysis result indicates that the abnormal object is located outside the road, then the traffic anomaly event is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; Furthermore, the method further includes: If the authenticity analysis result indicates that the traffic anomaly is a false detection event, then the false detection type of the traffic anomaly event is determined, and the event work order information is deleted; wherein, the false detection type is used to optimize the identification of traffic anomaly event images from the road video.

[0012] Secondly, this application provides a traffic anomaly event analysis device, comprising: The identification unit is used to identify traffic anomaly event images from the acquired road videos; wherein, the traffic anomaly event image includes an identifier box, road information, a timestamp, and an anomaly event type, and the identifier box is used to identify the abnormal object of the traffic anomaly event; The acquisition unit is used to acquire a target work order description instruction from a pre-built work order description instruction dataset; wherein, the work order description instruction dataset contains multiple work order description instructions; The first input unit is used to input the traffic anomaly event image and the target work order description instruction into the multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model; wherein, the event work order information includes at least the event number, event time, event location, event type, event description, and appearance features of the event object; The second input unit is used to input the traffic anomaly image and the pre-set false detection analysis command into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model; The reporting unit is used to report the event work order information if the authenticity analysis results indicate that the traffic anomaly is a real event.

[0013] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the traffic anomaly event analysis method described in any one of the above.

[0014] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the traffic anomaly analysis method described in any one of the above.

[0015] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the traffic anomaly analysis method described in any one of the above descriptions.

[0016] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run a program or instructions, and the processor executing the program or instructions implementing the steps of the traffic anomaly event analysis method described in any one of the above.

[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method, apparatus, equipment, medium, and product for analyzing traffic anomalies. It identifies and acquires images of traffic anomalies containing various key information from road videos, providing rich material for subsequent analysis. By utilizing a pre-constructed work order description instruction dataset to select target instructions, these instructions are input along with the images into a multimodal large language model, accurately generating event work order information containing multiple elements and comprehensively and accurately describing the event. Simultaneously, inputting the images and false alarm analysis instructions into the model for authenticity analysis effectively distinguishes between genuine and false events. Event work order information is only reported when the event is confirmed to be genuine. This series of processes works in tandem, from information acquisition and work order generation to authenticity judgment, with multiple layers of checks to effectively avoid false alarms and omissions, thereby improving the accuracy of traffic anomaly analysis results and providing a reliable basis for traffic management and decision-making. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a traffic anomaly event analysis method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the functional modules of a traffic anomaly analysis device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] In one exemplary embodiment, such as Figure 1 As shown, a method for analyzing traffic anomalies is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, it includes the following steps 101 to 105. Wherein: Step 101: Identify traffic anomaly images from the collected road videos.

[0023] In this embodiment of the application, the traffic anomaly event image includes an identifier box, road information, timestamp, and anomaly event type. The identifier box is used to identify the abnormal object of the traffic anomaly event.

[0024] In this embodiment of the application, traffic anomaly event images can be identified from the collected road videos by manual annotation, that is, abnormal objects in the traffic anomaly event images can be marked with identification boxes by manual annotation.

[0025] In addition, traffic anomaly event image recognition can be performed on road videos using a pre-built traffic anomaly event recognition model.

[0026] Optionally, road radar data can also be collected using millimeter-wave radar, and road radar data and road video can be fused using a transformation matrix to obtain radar video data. The radar video data can then be input into the traffic anomaly event recognition model so that the traffic anomaly event recognition model can output traffic anomaly event images from the radar video data.

[0027] Step 102: Obtain a target work order description instruction from the pre-built work order description instruction dataset.

[0028] In this embodiment of the application, the work order description instruction dataset contains multiple work order description instructions.

[0029] As an optional implementation method, the work order description instruction dataset can be constructed in the following ways: Retrieve the pre-built initial work order description instruction; Identify multiple instruction application scenario types; wherein, the instruction application scenario types include at least batch processing type, complex scenario type, interactive type, low-precision model type, and high-precision model type; The initial work order description instruction is optimized based on the various instruction application scenario types to obtain the optimized work order description instruction corresponding to each instruction application scenario type. Construct a work order description instruction dataset that includes the initial work order description instruction and each optimized work order description instruction.

[0030] This implementation method, starting with pre-built initial instructions, clarifies various application scenarios for these instructions, enabling precise identification of different practical needs. Optimizing the initial instructions for various scenarios generates highly adaptable optimized instructions, ensuring efficiency in batch processing, accuracy in complex scenarios, and flexibility in interaction, while also accommodating the needs of both low-precision and high-precision models. The final dataset encompasses both initial and optimized instructions, providing rich and comprehensive content to offer diverse and precise instruction support for traffic anomaly event analysis, effectively improving the analysis results and quality.

[0031] In this embodiment, the initial work order description instruction can be: "This is an image of a highway surveillance video, with the road segment and number marked in the upper left corner and the time of the event marked in the lower right corner. Objects exhibiting abnormalities, such as vehicles, pedestrians, non-motorized vehicles, and debris, are marked with red boxes in the image. Within the red boxes, the type of abnormal event triggered by the object is indicated, including parking, driving against traffic, non-motorized vehicle intrusion, pedestrian intrusion, traffic congestion, and debris. Based on the image, summarize the work order information: event number, event time, event location, event type, event description, and appearance characteristics of the event object. The event number is represented by the string 'road segment number_event occurrence time'." For example, because the image lacks background information about the traffic event, the basic instruction specifies all possible event types. The modified prompt for the basic instruction is: "I need a prompt expert. I have a pre-set prompt, and I need your help to improve its diversity. My prompt is: {}", where {} are placeholders for the basic instruction. After obtaining the modified instructions, five instructions are selected to form the work order description instruction dataset.

[0032] The description instruction for batch processing optimization work orders can be: There is one highway monitoring image. The top left corner of the image is labeled with the road segment name and number, and the bottom right corner is labeled with the event occurrence time. Anomalies (including vehicles, pedestrians, non-motorized vehicles, and debris) are selected in red boxes, and the corresponding anomaly type (parking, wrong-way driving, non-motorized vehicle intrusion, pedestrian intrusion, traffic congestion, debris) is labeled within the red box. Please strictly extract and summarize the work order according to the following fields: Event Number (format: Road Segment Number_Event Occurrence Time string), Event Time, Event Location, Event Type, Event Description, and Event Object Appearance Characteristics. No fields are missing.

[0033] This type of instruction is more concise and precise than the original, is suitable for batch processing, and improves efficiency.

[0034] The description instruction for an optimization work order for complex scenarios could be: This is an image captured by real-time monitoring on a highway. The specific road segment name and unique number are clearly marked in the upper left corner of the image, and the precise time of the abnormal event is recorded in the lower right corner. Abnormal objects (vehicles, pedestrians, non-motorized vehicles, or litter) in the image are highlighted with red boxes, and the event type (including parking, driving against traffic, non-motorized vehicle intrusion, pedestrian intrusion, traffic congestion, and litter) is clearly marked within the red boxes. Please summarize the complete work order information based on the image scenario: Event Number (Road Segment Number_Event Occurrence Time String), Event Time, Event Location, Event Type, Event Description (showing the scene association at the time of the anomaly), and the appearance characteristics of the event object (such as vehicle color / model, pedestrian clothing, and litter shape).

[0035] This type of instruction enhances model understanding and adapts to complex scenarios.

[0036] An interactive optimization work order description could be: "This is a highway monitoring map. The top left corner shows the road segment name and number, and the bottom right corner shows the specific time the event occurred. The problematic object is circled in red, which could be a vehicle, pedestrian, non-motorized vehicle, or debris. The red box also specifies the anomaly (e.g., parking, driving against traffic, someone trespassing). Please compile this information into a work order, including the following items: Event Number (written as 'Road Segment Number_Event Time'), Event Time, Event Location, Event Type (written as in the red box), Event Description (briefly explain what happened), and the appearance of the object (e.g., the car is a black sedan, the pedestrian is wearing red clothing, etc.)."

[0037] This type of instruction is adapted for interactive extraction, lowering the barrier to model understanding.

[0038] The description instruction for optimization work orders of low-precision model types can be: Please generate a standardized work order based on the following image information: Image Background: Highway surveillance footage, with the road segment name and number marked in the upper left corner and the event occurrence time (accurate to minutes / seconds) marked in the lower right corner; 2. Information: Abnormal objects are marked in red (only vehicles, pedestrians, non-motorized vehicles, and litter), and the abnormal event type is marked inside the red box (limited to: parking, driving against traffic, non-motorized vehicle intrusion, pedestrian intrusion, traffic congestion, and litter); 3. Fields: Must include event number (mandatory format: road segment number_event occurrence time string, no extra characters), event time (consistent with image annotation), event location (complete road segment name and number), event type (strictly matches the red box annotation), event description (briefly describe the location and status of the abnormality), and appearance characteristics of the event object (as detailed as possible; if there are no clear characteristics, fill in "none").

[0039] This type of instruction is adapted to low-precision models and explicitly states the requirements.

[0040] The optimization work order description instruction for high-precision model types can be: Existing highway monitoring images, with relevant annotations as follows: Basic information: The top left corner is marked with "Road segment name-Road segment number" (e.g., "Beijing-Hong Kong-Macau Expressway-G4-123km"), and the bottom right corner is marked with the time of the event (format: "year-month-day hour:minute:second"). Anomaly labeling: The red box locks the abnormal object (only one or more of vehicles, pedestrians, non-motorized vehicles, and litter), and the red box indicates the abnormal event type (only selectable: parking, driving against the flow of traffic, non-motorized vehicle intrusion, pedestrian intrusion, traffic congestion, litter). Please generate a standardized work order that meets the following requirements: Event number: Strictly generated according to "segment number_event occurrence time string" (remove spaces from the time string, such as "20240520143025"); Event description: must include "abnormal object + location of occurrence (e.g., lane, emergency lane) + abnormal behavior"; Appearance characteristics of the event object: For vehicles, specify color and model (if identifiable); for pedestrians, specify clothing / body type; for scattered objects, specify approximate shape / quantity; if no information is available, fill in "not clear"; The remaining fields (event time, event location, event type) must be exactly the same as the image annotations, and no new or modified information may be added.

[0041] This instruction is adapted for high-precision extraction and supplements edge cases.

[0042] As an optional implementation, step 102, which involves obtaining a target work order description instruction from a pre-built work order description instruction dataset, may include: Obtain the current instruction application scenario type corresponding to the Multi-modal Large Language Model (MLLM); If there is a target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then the work order description instruction corresponding to the target instruction application scenario type is determined as the target work order description instruction. If there is no target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then a work order description instruction is randomly selected from the work order description instruction dataset as the target work order description instruction.

[0043] This implementation method first obtains the current instruction application scenario type of the multimodal large language model. If a matching target type exists in the dataset, the corresponding instruction is precisely selected, ensuring a high degree of adaptation between the instruction and the current scenario. This improves the targeting and accuracy of the analysis, making the generated traffic anomaly event work order information more aligned with actual needs. If no matching type exists, an instruction is randomly selected, ensuring the process continues and avoiding interruptions due to the lack of suitable instructions. This flexible acquisition method balances accuracy and versatility, contributing to improved efficiency and effectiveness of traffic anomaly event analysis.

[0044] In this embodiment, the multimodal large language model can be a commonly used video-text multimodal model (such as VideoBERT, Sora, etc.) or a general-purpose multimodal model (such as GPT-4V, etc.), without structural modifications or additional training. After the work order description instruction dataset is constructed, a GPT-4V-level MLLM can be fine-tuned. MLLM also possesses excellent optical character recognition capabilities and multimodal information processing capabilities. Therefore, it can understand the content of traffic event images, thereby providing accurate descriptions and summaries of the appearance features of abnormal objects.

[0045] Step 103: Input the traffic anomaly event image and the target work order description instruction into the multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model.

[0046] In this embodiment of the application, the event work order information includes at least the event number, event time, event location, event type, event description, and appearance characteristics of the event object. The description of the appearance characteristics of the event object may include object category, color, pedestrian clothing, location of spilled material, etc.

[0047] As an optional implementation, step 103, which involves inputting the traffic anomaly image and the target work order description instruction into a multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model, may include: The traffic anomaly image and the target work order description instruction are input into a multimodal large language model, so that the multimodal large language model can identify the road number and timestamp from the traffic anomaly image based on the target work order description instruction; The road number and the timestamp are used as the event number; The timestamp is determined as the event time; The location of the traffic anomaly is determined based on the road number; The abnormal event type is identified from the traffic abnormal event image using the multimodal large language model; The abnormal event type is determined as the event type of the traffic abnormal event; The abnormal objects in the bounding boxes of the traffic anomaly event image are identified by the multimodal large language model to determine the event description and appearance features of the event objects. The multimodal large language model outputs event work order information containing the event number, event time, event location, event type, event description, and appearance features of the event object.

[0048] This implementation method involves inputting images and target commands into a multimodal large language model. Guided by the commands, the model accurately identifies key information such as road numbers and timestamps from the images, and reasonably determines the event number, time, and location. Simultaneously, the model can accurately identify abnormal event types, conduct detailed analysis of abnormal objects within the identified bounding box, and derive event descriptions and object appearance characteristics. The entire process is logically clear and well-defined, fully utilizing the model's advantages to comprehensively and accurately integrate various types of information, ultimately outputting complete event work order information, providing a reliable and detailed basis for the subsequent handling of traffic anomalies.

[0049] Step 104: Input the traffic anomaly event image and the pre-set false detection analysis command into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly event output by the multimodal large language model.

[0050] In this embodiment, the traffic incident analysis system based on a traditional small detection, recognition, and tracking model may suffer from false detections and false alarms due to the insufficient generalization ability of such models, resulting in low accuracy of alarm events. This increases the cost of manual review of alarm events later. In practical applications, business units have identified four main reasons for false alarms in traffic incident analysis systems based on traditional small models: 1) Image distortion, jitter, stripes, aliasing, blurring, and blockiness may occur due to electromagnetic and electronic signal interference or network problems. This is classified as a low-quality image issue.

[0051] 2) Small models may produce false alarms due to poor generalization, such as misdetecting some shadows as vehicles or pedestrians.

[0052] 3) The small model makes a recognition error, resulting in an incorrect event category judgment, such as misclassifying a pedestrian as a non-motorized vehicle.

[0053] 4) Incomplete filtering of objects outside the road causes false alarms. However, the main focus of high-speed traffic scenarios is on events within the highway.

[0054] The MLLM, fine-tuned using a traffic incident work order description dataset, achieves image-text alignment in the traffic domain while retaining multi-turn dialogue capabilities. Initial incident alarm images are input into the fine-tuned MLLM, which then asks four pre-set questions and requires brief responses. The review conclusion is then derived based on the model's answers.

[0055] The first question: "What is the quality of this image?" For the first type of reason, if MLLM gives a conclusion of "poor" or "low", then the credibility of this alarm event is not high, so low quality alarms can be filtered out.

[0056] Conversely, if a judgment of "high" or "good" is obtained, the second question is asked: "Are there pedestrians, vehicles, non-motorized vehicles, or other objects in the red box?" If MLLM answers "no", the event is classified as a misjudgment. Otherwise, for the third type of reason, ask the third question: "Does the object in the red box match the labeled category?" If MLLM answers "matches", it means that the image quality of the event is good, there is an abnormal object in the red box and the object category is correct, thus ruling out the possibility of the first three types of reasons.

[0057] Finally, when asking the fourth question, "Is the abnormal object in the red box on the highway?", if MLLM answers "yes", then the alarm event is a normal detection result; otherwise, it is a false alarm.

[0058] The order of the four questions is not unique, and negative answers can filter alarm events, thereby enabling automatic review of abnormal events, reducing the manpower requirements of traffic management departments, and helping the system to more accurately identify abnormal traffic events.

[0059] As an optional implementation, step 104, which inputs the traffic anomaly image and a pre-set false detection analysis command into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model, may include: The traffic anomaly image and pre-set false detection analysis instructions are input into the multimodal large language model, so that the multimodal large language model can extract the false detection analysis instructions to obtain multiple false detection analysis sub-instructions; wherein, the false detection analysis sub-instructions include at least image quality analysis sub-instructions, object analysis sub-instructions, type analysis sub-instructions and location analysis sub-instructions; The image quality of the traffic anomaly event image is analyzed using the multimodal large language model based on the image quality analysis sub-instruction, and the image quality analysis result is determined. If the image quality analysis result indicates that the quality of the traffic anomaly image is acceptable, then the abnormal objects in the identifier box of the traffic anomaly image are analyzed by the multimodal large language model based on the object analysis sub-instruction to determine the object analysis result; If the object analysis result indicates that the abnormal object exists in the identification box, then the correctness of the abnormal event type of the traffic abnormal event image is analyzed by the multimodal large language model based on the type analysis sub-instruction to determine the type analysis result; If the type analysis result indicates that the abnormal event type is correct, then the relative positional relationship between the abnormal object and the road in the traffic abnormal event image is analyzed by the multimodal large language model based on the location analysis sub-instruction to obtain the location analysis result; If the location analysis result indicates that the abnormal object is located on the road, then the traffic anomaly event is determined to be a real event as the authenticity analysis result of the traffic anomaly event.

[0060] This implementation method involves inputting images and false positive analysis instructions into the model. Multiple sub-instructions are first extracted, and then analyzed step-by-step according to image quality, object, type, and location. Only when the result of the previous step is satisfactory does the next step proceed, ensuring layered checks and preventing misjudgments due to a single factor. This rigorous analysis process fully utilizes the capabilities of a multimodal large language model to consider traffic anomalies from multiple dimensions, effectively eliminating false positives and significantly improving the accuracy and reliability of the authenticity analysis. This provides a solid guarantee for subsequent reporting and processing of real-world events.

[0061] As an optional implementation, the following steps may also be performed: If the image quality analysis result indicates that the quality of the traffic anomaly image is unqualified, then the traffic anomaly is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; If the object analysis result indicates that the abnormal object does not exist in the identifier box, then the traffic anomaly event is determined as a false detection event as the authenticity analysis result of the traffic anomaly event; If the type analysis result indicates that the abnormal event type is incorrect, then the traffic abnormal event is determined to be a false detection event as the authenticity analysis result of the traffic abnormal event; If the location analysis result indicates that the abnormal object is located outside the road, then the traffic anomaly event is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; Additionally, the following steps can also be performed: If the authenticity analysis result indicates that the traffic anomaly is a false detection event, then the false detection type of the traffic anomaly event is determined, and the event work order information is deleted; wherein, the false detection type is used to optimize the identification of traffic anomaly event images from the road video.

[0062] This implementation method ensures that during the authenticity analysis process, if any aspect—image quality, object, type, or location—failes to meet the standards, it is identified as a false alarm. The rigorous and comprehensive analysis logic effectively avoids false alarms. Furthermore, once a false alarm is identified, the false alarm type is determined, and the event work order information is deleted, preventing interference from invalid information. Simultaneously, using the false alarm type to optimize subsequent steps in identifying traffic anomaly images from road videos forms a feedback loop, helping to continuously improve recognition accuracy, reduce false alarms, and make the entire traffic anomaly analysis process more scientific, efficient, and precise.

[0063] Step 105: If the authenticity analysis result indicates that the traffic anomaly is a real event, then the event work order information is reported.

[0064] Implementing steps 101 to 105 effectively avoids false alarms and missed alarms, thereby improving the accuracy of traffic anomaly analysis results and providing a reliable basis for traffic management and decision-making. Furthermore, this application can provide diversified and precise instruction support for traffic anomaly analysis, effectively improving the analysis effect and quality. In addition, this application balances accuracy and universality, helping to improve the efficiency and effectiveness of traffic anomaly analysis. Moreover, this application can fully utilize the advantages of the model to comprehensively and accurately integrate various types of information, ultimately outputting complete event work order information, providing a reliable and detailed basis for the subsequent handling of traffic anomalies. Furthermore, this application can improve the accuracy and reliability of authenticity analysis, providing a solid guarantee for the subsequent reporting and handling of real events. In addition, this application can continuously improve identification accuracy, reduce false detections, and make the entire traffic anomaly analysis process more scientific, efficient, and accurate.

[0065] The embodiments of this application can be applied to a traffic anomaly detection device based on a multimodal large model, which can detect traffic anomalies.

[0066] Traffic anomaly detection equipment based on multimodal large models may include: Video acquisition module: CD module, VPU decoding module and encoding module, to acquire video data at specific times as the basis for MLLM analysis; Radar module: VLP-16 lidar, the size of which depends on the actual dimensions of the intersection being monitored; Transmission modules: 5G module, Ethernet module and video encoder, for transmitting data and models; Storage modules: SSD module, SD module and the video encoder, storing time-stamped radar data and video data as well as MLLM analysis results; Processing module: includes VPU built-in DSP chip, ARM chip and NPU chip.

[0067] The VPU, NPU, and ARM chips each implement their respective functions through their provided interfaces.

[0068] The high-definition video stream is encoded using H.264 or H.265 and output over the network.

[0069] The VPU, NPU, and ARM chips each implement their respective functions through their provided interfaces.

[0070] Optionally, the signal processing module includes an ARM chip.

[0071] Optionally, the transmission and storage module includes: a transmission unit and a storage unit; The transmission unit includes Ethernet and 5G; The storage unit includes an SSD solid-state drive and an SD card.

[0072] Optional features also include: a video expansion bus and a first bus switch; The video acquisition and input module is connected to the signal processing module through the first bus switch and the video expansion bus; The transmission and storage module is connected to the signal processing module via the video expansion bus.

[0073] The video encoder is connected to the video expansion bus via the second FPGA and the second bus switch; The video encoder is also connected to the video expansion bus via the third bus switch; The video encoder encodes the high-definition video stream using H.264 or H.265 and outputs it over the network.

[0074] Optionally, it may also include: an encryption module; The encryption module is connected to the signal processing module; the encryption module is used to encrypt the high-definition video stream.

[0075] The embodiments of this application can: 1. Reduced false alarm rate: This advantage stems from the four-step intelligent review mechanism designed into the technical solution. Four key judgments are executed sequentially through a multimodal large model (MLLM): (1) Screen image quality and filter out distorted or blurry images caused by electromagnetic interference or network problems; (2) Verify the existence of the target within the red box and eliminate false alarms that mistakenly identify the shadow as a vehicle or pedestrian; (3) Verify the consistency of target categories and correct small model recognition errors (such as misjudging pedestrians as non-motorized vehicles). (4) Confirm that the target is located within the main road area and screen out false alarms in non-road areas.

[0076] 2. Improved work order information completeness: (1) Construct a highway-specific instruction set with 5 variants and enhance instruction diversity through GPT-4o; (2) Fine-tune MLLM to accurately analyze the fine-grained features of the target within the red box (such as the RGB values ​​of vehicle color and the location of the scattered material); (3) The structured output of the work order consists of 6 fields (event number, time, location, type, description, and object appearance) to ensure that no key information is omitted. Its completeness improvement is verified by the BLEU-1 indicator.

[0077] 3. Reduced processing delay: (1) The VPU chip completes the H.265 encoding of the video stream in real time; (2) NPU chip deploys lightweight MLLM model (MiniCPM) to achieve local inference; (3) The ARM chip performs a four-step review logic control.

[0078] 4. Cost of manual review: (1) MLLM automatically generates complete work orders that comply with traffic management regulations; (2) The four-step review mechanism filters out 74.19% of invalid alarms; (3) Manual intervention is triggered only for alarms that are confirmed to be valid.

[0079] 5. Enhanced all-weather operational stability: (1) The VLP-16 lidar stably outputs target coordinates and velocity under low visibility conditions such as rain, fog, and night; (2) The CD video module supplements visual features such as license plates and colors when the lighting is good.

[0080] Based on the same inventive concept, this application also provides a traffic anomaly analysis device for implementing the traffic anomaly analysis method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the traffic anomaly analysis device provided below can be found in the limitations of the traffic anomaly analysis method described above, and will not be repeated here.

[0081] In one exemplary embodiment, such as Figure 2 As shown, a traffic anomaly event analysis device is provided, comprising: The identification unit 201 is used to identify traffic abnormality event images from the collected road videos; wherein, the traffic abnormality event image includes an identification box, road information, timestamp, and abnormality event type, and the identification box is used to identify the abnormal object of the traffic abnormality event; The acquisition unit 202 is used to acquire a target work order description instruction from a pre-built work order description instruction dataset; wherein, the work order description instruction dataset contains multiple work order description instructions; The first input unit 203 is used to input the traffic anomaly event image and the target work order description instruction into the multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model; wherein, the event work order information includes at least the event number, event time, event location, event type, event description and appearance features of the event object; The second input unit 204 is used to input the traffic anomaly event image and the pre-set false detection analysis command into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly event output by the multimodal large language model; The reporting unit 205 is used to report the event work order information if the authenticity analysis result indicates that the traffic abnormality event is a real event.

[0082] As an optional implementation method, the work order description instruction dataset can be constructed in the following ways: Retrieve the pre-built initial work order description instruction; Identify multiple instruction application scenario types; wherein, the instruction application scenario types include at least batch processing type, complex scenario type, interactive type, low-precision model type, and high-precision model type; The initial work order description instruction is optimized based on the various instruction application scenario types to obtain the optimized work order description instruction corresponding to each instruction application scenario type. Construct a work order description instruction dataset that includes the initial work order description instruction and each optimized work order description instruction.

[0083] This implementation method, starting with pre-built initial instructions, clarifies various application scenarios for these instructions, enabling precise identification of different practical needs. Optimizing the initial instructions for various scenarios generates highly adaptable optimized instructions, ensuring efficiency in batch processing, accuracy in complex scenarios, and flexibility in interaction, while also accommodating the needs of both low-precision and high-precision models. The final dataset encompasses both initial and optimized instructions, providing rich and comprehensive content to offer diverse and precise instruction support for traffic anomaly event analysis, effectively improving the analysis results and quality.

[0084] As an optional implementation, the acquisition unit 202 may acquire a target work order description instruction from a pre-built work order description instruction dataset in the following specific ways: Obtain the current instruction application scenario type corresponding to the multimodal large language model; If there is a target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then the work order description instruction corresponding to the target instruction application scenario type is determined as the target work order description instruction. If there is no target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then a work order description instruction is randomly selected from the work order description instruction dataset as the target work order description instruction.

[0085] This implementation method first obtains the current instruction application scenario type of the multimodal large language model. If a matching target type exists in the dataset, the corresponding instruction is precisely selected, ensuring a high degree of adaptation between the instruction and the current scenario. This improves the targeting and accuracy of the analysis, making the generated traffic anomaly event work order information more aligned with actual needs. If no matching type exists, an instruction is randomly selected, ensuring the process continues and avoiding interruptions due to the lack of suitable instructions. This flexible acquisition method balances accuracy and versatility, contributing to improved efficiency and effectiveness of traffic anomaly event analysis.

[0086] As an optional implementation, the first input unit 203 inputs the traffic anomaly event image and the target work order description instruction into the multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model. Specifically, this can be achieved by: The traffic anomaly image and the target work order description instruction are input into a multimodal large language model, so that the multimodal large language model can identify the road number and timestamp from the traffic anomaly image based on the target work order description instruction; The road number and the timestamp are used as the event number; The timestamp is determined as the event time; The location of the traffic anomaly is determined based on the road number; The abnormal event type is identified from the traffic abnormal event image using the multimodal large language model; The abnormal event type is determined as the event type of the traffic abnormal event; The abnormal objects in the bounding boxes of the traffic anomaly event image are identified by the multimodal large language model to determine the event description and appearance features of the event objects. The multimodal large language model outputs event work order information containing the event number, event time, event location, event type, event description, and appearance features of the event object.

[0087] This implementation method involves inputting images and target commands into a multimodal large language model. Guided by the commands, the model accurately identifies key information such as road numbers and timestamps from the images, and reasonably determines the event number, time, and location. Simultaneously, the model can accurately identify abnormal event types, conduct detailed analysis of abnormal objects within the identified bounding box, and derive event descriptions and object appearance characteristics. The entire process is logically clear and well-defined, fully utilizing the model's advantages to comprehensively and accurately integrate various types of information, ultimately outputting complete event work order information, providing a reliable and detailed basis for the subsequent handling of traffic anomalies.

[0088] As an optional implementation, the second input unit 204 inputs the traffic anomaly image and a pre-set false detection analysis command into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model. Specifically, this can be achieved by: The traffic anomaly image and pre-set false detection analysis instructions are input into the multimodal large language model, so that the multimodal large language model can extract the false detection analysis instructions to obtain multiple false detection analysis sub-instructions; wherein, the false detection analysis sub-instructions include at least image quality analysis sub-instructions, object analysis sub-instructions, type analysis sub-instructions and location analysis sub-instructions; The image quality of the traffic anomaly event image is analyzed using the multimodal large language model based on the image quality analysis sub-instruction, and the image quality analysis result is determined. If the image quality analysis result indicates that the quality of the traffic anomaly image is acceptable, then the abnormal objects in the identifier box of the traffic anomaly image are analyzed by the multimodal large language model based on the object analysis sub-instruction to determine the object analysis result; If the object analysis result indicates that the abnormal object exists in the identification box, then the correctness of the abnormal event type of the traffic abnormal event image is analyzed by the multimodal large language model based on the type analysis sub-instruction to determine the type analysis result; If the type analysis result indicates that the abnormal event type is correct, then the relative positional relationship between the abnormal object and the road in the traffic abnormal event image is analyzed by the multimodal large language model based on the location analysis sub-instruction to obtain the location analysis result; If the location analysis result indicates that the abnormal object is located on the road, then the traffic anomaly event is determined to be a real event as the authenticity analysis result of the traffic anomaly event.

[0089] This implementation method involves inputting images and false positive analysis instructions into the model. Multiple sub-instructions are first extracted, and then analyzed step-by-step according to image quality, object, type, and location. Only when the result of the previous step is satisfactory does the next step proceed, ensuring layered checks and preventing misjudgments due to a single factor. This rigorous analysis process fully utilizes the capabilities of a multimodal large language model to consider traffic anomalies from multiple dimensions, effectively eliminating false positives and significantly improving the accuracy and reliability of the authenticity analysis. This provides a solid guarantee for subsequent reporting and processing of real-world events.

[0090] As an optional implementation, the second input unit 204 is further configured to: If the image quality analysis result indicates that the quality of the traffic anomaly image is unqualified, then the traffic anomaly is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; If the object analysis result indicates that the abnormal object does not exist in the identifier box, then the traffic anomaly event is determined as a false detection event as the authenticity analysis result of the traffic anomaly event; If the type analysis result indicates that the abnormal event type is incorrect, then the traffic abnormal event is determined to be a false detection event as the authenticity analysis result of the traffic abnormal event; If the location analysis result indicates that the abnormal object is located outside the road, then the traffic anomaly event is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; In addition, reporting unit 205 is also used for: If the authenticity analysis result indicates that the traffic anomaly is a false detection event, then the false detection type of the traffic anomaly event is determined, and the event work order information is deleted; wherein, the false detection type is used to optimize the identification of traffic anomaly event images from the road video.

[0091] This implementation method ensures that during the authenticity analysis process, if any aspect—image quality, object, type, or location—failes to meet the standards, it is identified as a false alarm. The rigorous and comprehensive analysis logic effectively avoids false alarms. Furthermore, once a false alarm is identified, the false alarm type is determined, and the event work order information is deleted, preventing interference from invalid information. Simultaneously, using the false alarm type to optimize subsequent steps in identifying traffic anomaly images from road videos forms a feedback loop, helping to continuously improve recognition accuracy, reduce false alarms, and make the entire traffic anomaly analysis process more scientific, efficient, and precise.

[0092] Implementing the above-described methods effectively avoids false alarms and missed alarms, thereby improving the accuracy of traffic anomaly analysis results and providing a reliable basis for traffic management and decision-making. Furthermore, this application can provide diversified and precise instruction support for traffic anomaly analysis, effectively improving the analysis results and quality. In addition, this application balances accuracy and versatility, contributing to improved efficiency and effectiveness of traffic anomaly analysis. Moreover, this application fully utilizes the model's advantages to comprehensively and accurately integrate various types of information, ultimately outputting complete event work order information, providing a reliable and detailed basis for subsequent handling of traffic anomalies. Furthermore, this application can improve the accuracy and reliability of realism analysis, providing a solid guarantee for subsequent reporting and processing of real events. Furthermore, this application can continuously improve identification accuracy, reduce false detections, and make the entire traffic anomaly analysis process more scientific, efficient, and accurate.

[0093] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores traffic anomaly event analysis data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a traffic anomaly event analysis method.

[0094] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0095] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0096] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0097] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0098] In one exemplary embodiment, a chip is provided, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps in the above method embodiments and achieve the same technical effect, and will not be described again here to avoid repetition.

[0099] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0100] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0101] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0102] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0103] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A traffic abnormal event analysis method, characterized by, The traffic anomaly event analysis method includes: Traffic anomaly event images are identified from the collected road videos; wherein, the traffic anomaly event image includes an identifier box, road information, timestamp, and anomaly event type, and the identifier box is used to identify the abnormal object of the traffic anomaly event; Obtain a target work order description instruction from a pre-built work order description instruction dataset; wherein, the work order description instruction dataset contains multiple work order description instructions; The traffic anomaly event image and the target work order description instruction are input into a multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model; wherein, the event work order information includes at least the event number, event time, event location, event type, event description, and appearance features of the event object; The traffic anomaly image and the pre-set false detection analysis command are input into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model; If the authenticity analysis results indicate that the traffic anomaly is a real event, then the event work order information will be reported.

2. The traffic abnormal event analysis method according to claim 1, characterized in that, The specific method for constructing the work order description instruction dataset is as follows: Retrieve the pre-built initial work order description instruction; Identify multiple instruction application scenario types; wherein, the instruction application scenario types include at least batch processing type, complex scenario type, interactive type, low-precision model type, and high-precision model type; The initial work order description instruction is optimized based on the various instruction application scenario types to obtain the optimized work order description instruction corresponding to each instruction application scenario type. Construct a work order description instruction dataset that includes the initial work order description instruction and each optimized work order description instruction.

3. The traffic abnormal event analysis method according to claim 2, characterized in that, The step of obtaining a target work order description instruction from a pre-built work order description instruction dataset specifically includes: Obtain the current instruction application scenario type corresponding to the multimodal large language model; If there is a target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then the work order description instruction corresponding to the target instruction application scenario type is determined as the target work order description instruction. If there is no target instruction application scenario type in the pre-built work order description instruction dataset that is the same as the current instruction application scenario type, then a work order description instruction is randomly selected from the work order description instruction dataset as the target work order description instruction.

4. The traffic anomaly event analysis method according to claim 1, characterized in that, The step of inputting the traffic anomaly image and the target work order description instruction into a multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model specifically includes: The traffic anomaly image and the target work order description instruction are input into a multimodal large language model, so that the multimodal large language model can identify the road number and timestamp from the traffic anomaly image based on the target work order description instruction; The road number and the timestamp are used as the event number; The timestamp is determined as the event time; The location of the traffic anomaly is determined based on the road number; The abnormal event type is identified from the traffic abnormal event image using the multimodal large language model; The abnormal event type is determined as the event type of the traffic abnormal event; The abnormal objects in the bounding boxes of the traffic anomaly event image are identified by the multimodal large language model to determine the event description and appearance features of the event objects. The multimodal large language model outputs event work order information containing the event number, event time, event location, event type, event description, and appearance features of the event object.

5. The traffic anomaly event analysis method according to claim 1, characterized in that, The step of inputting the traffic anomaly image and pre-set false detection analysis instructions into the multimodal large language model to obtain the authenticity analysis results of the traffic anomaly output by the multimodal large language model specifically includes: The traffic anomaly image and pre-set false detection analysis instructions are input into the multimodal large language model, so that the multimodal large language model can extract the false detection analysis instructions to obtain multiple false detection analysis sub-instructions; wherein, the false detection analysis sub-instructions include at least image quality analysis sub-instructions, object analysis sub-instructions, type analysis sub-instructions and location analysis sub-instructions; The image quality of the traffic anomaly event image is analyzed using the multimodal large language model based on the image quality analysis sub-instruction, and the image quality analysis result is determined. If the image quality analysis result indicates that the quality of the traffic anomaly image is acceptable, then the abnormal objects in the identifier box of the traffic anomaly image are analyzed by the multimodal large language model based on the object analysis sub-instruction to determine the object analysis result; If the object analysis result indicates that the abnormal object exists in the identification box, then the correctness of the abnormal event type of the traffic abnormal event image is analyzed by the multimodal large language model based on the type analysis sub-instruction to determine the type analysis result; If the type analysis result indicates that the abnormal event type is correct, then the relative positional relationship between the abnormal object and the road in the traffic abnormal event image is analyzed by the multimodal large language model based on the location analysis sub-instruction to obtain the location analysis result; If the location analysis result indicates that the abnormal object is located on the road, then the traffic anomaly event is determined as a real event, which is the result of the traffic anomaly event authenticity analysis.

6. The traffic anomaly event analysis method according to claim 5, characterized in that, The method further includes: If the image quality analysis result indicates that the quality of the traffic anomaly image is unqualified, then the traffic anomaly is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; If the object analysis result indicates that the abnormal object does not exist in the identifier box, then the traffic abnormality event is determined as a false detection event as the authenticity analysis result of the traffic abnormality event; If the type analysis result indicates that the abnormal event type is incorrect, then the traffic abnormal event is determined to be a false detection event as the authenticity analysis result of the traffic abnormal event; If the location analysis result indicates that the abnormal object is located outside the road, then the traffic anomaly event is determined to be a false detection event as the authenticity analysis result of the traffic anomaly event; Furthermore, the method further includes: If the authenticity analysis result indicates that the traffic anomaly is a false detection event, then the false detection type of the traffic anomaly event is determined, and the event work order information is deleted; wherein, the false detection type is used to optimize the identification of traffic anomaly event images from the road video.

7. A traffic anomaly event analysis device, characterized in that, The traffic anomaly analysis device includes: The identification unit is used to identify traffic anomaly event images from the acquired road videos; wherein, the traffic anomaly event image includes an identifier box, road information, a timestamp, and an anomaly event type, and the identifier box is used to identify the abnormal object of the traffic anomaly event; The acquisition unit is used to acquire a target work order description instruction from a pre-built work order description instruction dataset; wherein, the work order description instruction dataset contains multiple work order description instructions; The first input unit is used to input the traffic anomaly event image and the target work order description instruction into the multimodal large language model to obtain the traffic anomaly event work order information output by the multimodal large language model; wherein, the event work order information includes at least the event number, event time, event location, event type, event description, and appearance features of the event object; The second input unit is used to input the traffic anomaly image and the pre-set false detection analysis command into the multimodal large language model to obtain the authenticity analysis result of the traffic anomaly output by the multimodal large language model; The reporting unit is used to report the event work order information if the authenticity analysis results indicate that the traffic anomaly is a real event.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the traffic anomaly analysis method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the traffic anomaly analysis method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the traffic anomaly analysis method according to any one of claims 1-6.

Images