Road construction area dangerous vehicle early warning method based on cooperation of multiple AI large models and related equipment

Abstract

The invention discloses a road construction area dangerous vehicle early warning method and related equipment based on multi-AI large model collaboration, and the method comprises the steps: deploying a construction safety monitoring unmanned aerial vehicle to each region in a road construction area, monitoring the traffic flow condition of the surrounding road surface of the road construction area, carrying out the data preprocessing of a shot video, and carrying out the early warning of a dangerous vehicle in the road construction area. The method comprises the following steps: transmitting a video to a visual AI large model in a ground safety center for analysis and extraction of safety risk information in the video, transmitting the extracted information to a large language model for reasoning and judgment, judging whether a vehicle with a dangerous driving behavior exists or not, evaluating a comprehensive danger level and sorting out structured information; the AI large model further learns the track characteristics of the dangerous vehicle so as to improve the accuracy of subsequent detection of the model. Grading response measures are taken according to different comprehensive danger grades, and constructors are notified to pay attention or avoid. The device is used for improving the safety during road construction and guaranteeing the safety of constructors in the whole construction process.

Description

Technical Field

[0001] The present invention relates to the field of intelligent traffic construction safety technology, and in particular to a dangerous vehicle early warning method and related equipment in road construction areas based on the collaboration of multiple AI large models. Background Art

[0002] With the continued rapid growth of my country's highway mileage and the increasing demand for road maintenance and construction, road construction safety issues face significant challenges. Currently, most road construction projects are conducted in uninterrupted traffic conditions. Traditional road construction safety measures rely primarily on early warning systems such as traffic cones and construction signs, which present numerous drawbacks. On the one hand, traffic cones and construction signs have limited effectiveness. At night, inclement weather, or when visibility is poor or line of sight is obstructed, they can be easily overlooked by drivers driving at high speeds or distracted. On the other hand, the risk of vehicles colliding with traffic cones and construction signs is high, especially in high-speed driving scenarios such as expressways. Furthermore, placing traffic cones and construction signs around construction areas consumes significant manpower and resources, and the placement process itself presents safety risks. Summary of the Invention

[0003] In order to at least solve one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a dangerous vehicle warning method and related equipment in road construction areas based on the collaboration of multiple AI large models.

[0004] The first technical solution adopted by the present invention is:

[0005] A method for early warning dangerous vehicles in road construction zones based on the collaboration of multiple AI large models includes the following steps:

[0006] S1. Deploy construction safety monitoring drones in various areas of the road construction area to obtain real-time video images of road traffic conditions around the road construction area;

[0007] S2. The drone is equipped with a target detection algorithm (such as YOLO) to pre-screen the collected video data and upload only the video clips containing vehicle targets to the ground security center server, and then perform image preprocessing operations;

[0008] S3. Utilize multiple large-scale visual AI models (e.g., llama3.2-vision:90b, llava:34b, and gemma3:27b) to analyze incoming video data in parallel. Guide the large-scale visual AI models through preset prompt words to obtain vehicle traffic and road conditions around the road construction area.

[0009] S4. Use a large language model (e.g., Deepseek) to organize the analyzed vehicle driving conditions and road conditions around the road construction area, determine the comprehensive vehicle hazard level, organize the analysis results into structured information, and create a construction safety information form to store in the system;

[0010] S5. Use a large AI model (e.g., Grok-3) to learn the subsequent trajectory characteristics of vehicles of various comprehensive risk levels. Based on the learned characteristics, optimize the model's judgment criteria to continuously improve the model's accuracy and reliability.

[0011] S6. When a potentially dangerous vehicle is discovered, the ground safety center will take graded response measures based on the different comprehensive danger levels of the vehicle, send warning information to construction workers through wireless communication, and command the drone to send a warning signal at the same time.

[0012] Furthermore, the step S1 includes:

[0013] Before arranging the road construction area, deploy several construction safety monitoring drones to monitor traffic on the construction road. The drones need to conduct reconnaissance along the road construction work area for a sufficient distance upstream to ensure that construction personnel have sufficient reaction time and safe distance when arranging the construction area. The road construction area is divided into a warning zone, an upstream transition zone, a buffer zone, a work zone, and a termination zone. Different conditions are used to determine vehicle danger in different areas:

[0014] 1) For early warning of hazardous vehicles in the warning zone, it is necessary to analyze traffic conditions upstream of the road construction area, including each vehicle's speed, lane position, and possible lane change risks;

[0015] 2) For the early warning of dangerous vehicles in the upstream transition zone, buffer zone, work zone and termination zone, it is only necessary to analyze the speed information of each vehicle and the possible rear-end collision risk.

[0016] Furthermore, the minimum monitoring distance of drones in the warning zone from the upstream buffer zone must be determined based on the design speed of the construction road. This ensures that construction workers have sufficient time to evacuate to a safe location on the roadside after the warning is issued. The drones in the upstream transition zone, buffer zone, work zone, and termination zone must have a field of view that covers the entire construction area, thereby determining the number of drones required.

[0017] Among them, the minimum monitoring distance of the drone needs to be calculated based on the road design speed and meet the following formula:

[0018] L min =v max ×Δt+d s

[0019] Where, Lmin Indicates the minimum monitoring distance of the drone, v max represents the maximum design speed of the road, Δt represents the model response time, d s The minimum safe distance.

[0020] Furthermore, the step S2 includes:

[0021] The lightweight target detection system carried by the drone performs vehicle target detection on the collected video data, captures video clips containing only road video clips of vehicles passing, and uploads them to the ground security center server via the FTP protocol. The image then undergoes image preprocessing operations such as illumination enhancement, denoising, and contrast enhancement to reduce the amount of computation after transmission to the large model.

[0022] Furthermore, the data transmitted to the large model in the ground safety center includes:

[0023] 1) Preprocessed video data of vehicles passing around the road construction area;

[0024] 2) The flight altitude, camera parameters, and camera angle of the drone when shooting the video;

[0025] 3) Real-time GPS positioning data of each drone;

[0026] 4) Location information, speed limit information, and road geometry information of the construction section.

[0027] Furthermore, the step S4 includes:

[0028] The large language model is used to organize and analyze the question results obtained by the large visual AI model, and the comprehensive risk level of the vehicle is divided into low risk level, medium risk level, and high risk level. The indicators used to determine the comprehensive risk level of the vehicle include: weather conditions, road conditions, traffic volume, lane of the vehicle, vehicle speed, and vehicle type;

[0029] The comprehensive risk level of a vehicle is determined according to the following formula:

[0030] D=(ω1×W+ω2×R+ω3×T+ω4×L+ω5×V+ω6×P)

[0031] Where ω1, ω2, ω3, ω4, ω5, and ω6 are the weight coefficients of each influencing factor, R represents the road condition, T represents the traffic volume, W represents the weather condition, L represents the lane the vehicle is in, V represents the vehicle speed, and P represents the vehicle model.

[0032] Furthermore, the step S5 includes:

[0033] The AI ​​large model (such as Grok-3) learns the trajectory characteristics of vehicles of various comprehensive danger levels and subsequently optimizes the model judgment criteria. By recording the trajectory characteristics of vehicles that enter the road construction area and the trajectory characteristics of vehicles that have traffic accidents around the road construction area, and retrieving the weather conditions, road conditions, traffic volume, vehicle lane, vehicle speed, and vehicle model before the accident from the database, the large model conducts feedback learning to continuously correct the risk factor weights of each condition and improve the accuracy of the large model's subsequent judgments.

[0034] Furthermore, the step S6 includes:

[0035] When the comprehensive danger level of a vehicle reaches a high danger level, a command needs to be sent to the drone to warn the dangerous vehicle and notify the construction workers to evacuate to a safe area;

[0036] Among them, the construction safety monitoring drone is equipped with flashing lights and loudspeakers, and the drone can send warning signals to dangerous vehicles.

[0037] The second technical solution adopted by the present invention is:

[0038] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-mentioned dangerous vehicle warning method in road construction areas based on the collaboration of multiple AI large models.

[0039] The third technical solution adopted by the present invention is:

[0040] A computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the above-mentioned method for warning dangerous vehicles in road construction areas based on the collaboration of multiple AI large models.

[0041] The fourth technical solution adopted by the present invention is:

[0042] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to implement the aforementioned method for warning dangerous vehicles in road construction zones based on collaboration between multiple AI large models.

[0043] Compared with the existing technology, the beneficial effects of the present invention are:

[0044] This invention can monitor incoming vehicles upstream of road construction areas in real time, enabling rapid response to potential hazards. It uses a target detection model to analyze the presence of vehicles in video images captured by drones. Only video clips containing vehicles are fed into a large visual AI model for analysis. This hierarchical screening approach improves system efficiency, significantly reduces computational complexity, and enables the entire system to more efficiently process large-scale video data. Furthermore, the invention employs a collaborative analysis approach using multiple AI models, combining the respective strengths of the large visual model and large language model. By employing parallel analysis across multiple models, the system significantly reduces the missed detection rate of a single large model. The large language model also generates and stores structured data output for easy subsequent querying, allowing the model to self-learn and optimize based on historical data, further improving the accuracy of its judgments. The invention equips drones with loudspeakers and flashing lights to more safely and effectively alert vehicles on the road to construction work ahead. Simultaneously, the ground safety center can maintain wireless communication with construction workers and broadcast reports on incoming vehicles, further enhancing safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 This is a flow chart of a dangerous vehicle warning method for road construction areas based on collaboration of multiple AI large models in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the monitoring position of the construction safety drone in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of a dangerous vehicle monitoring interface in a construction area warning zone according to an embodiment of the present invention;

[0049] Figure 4 This is an overall technical flow chart of an embodiment of the present invention;

[0050] Figure 5 It is an illustration in the construction safety information table in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise clearly defined, words such as setting, installing, and connecting should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0053] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0054] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0055] In the description of this application, "and / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0056] In response to existing technical problems, the present invention proposes a dangerous vehicle warning method for road construction areas based on the collaboration of multiple AI large models. By leveraging the characteristics of drones, such as rapid deployment, convenience, flexibility, and a wide field of view, drones equipped with high-definition cameras can be used as real-time monitoring equipment in road construction areas. The video data captured by the drones is analyzed and processed in combination with multiple large visual models, and the analysis results are collated through a large language model to promptly report potential dangers in the road construction area. The large model will also conduct self-learning based on the dangerous vehicles that have been discovered to optimize the accuracy of the next inference. Furthermore, the advantage of drones flying in the air can be used to send voice and light warnings to dangerous vehicles without the risk of collision. At the same time, construction workers can promptly learn about the arrival of dangerous vehicles in the construction area and evacuate to a safe area in a timely manner, greatly improving the safety of road and highway construction workers.

[0057] Example 1

[0058] See also Figure 4 In this embodiment, a dangerous vehicle warning method for road construction areas based on the collaboration of multiple AI large models includes a construction safety monitoring drone, a visual AI large model, a reasoning and judgment AI large model, an adaptive learning AI large model, and a wireless communication module.

[0059] Construction safety monitoring drones are used to collect real-time video data in and around road construction areas, including conditions in the warning zone, upstream transition zone, buffer zone, work zone, and termination zone. The drones are equipped with high-definition cameras, infrared cameras for nighttime or low-visibility conditions, lidar for measuring flight altitude, GPS positioning systems, loudspeakers, and flashing lights for issuing early warning information. Construction safety monitoring drones are deployed and commanded by the ground safety center. Construction safety monitoring drones also have vehicle target detection models that can identify whether there are vehicles passing through in the video.

[0060] The visual AI big model uses multiple AI big models such as llama3.2-vision:90b, llava:34b and gemma3:27b deployed locally in the ground safety center to collaboratively process and analyze the pre-processed video data transmitted to the ground safety center. Different specific big model prompt words are used depending on the road construction area to which the video belongs. The information obtained from the visual AI big model analysis is then passed into the structured semantic generation AI big model.

[0061] The reasoning and judgment AI big model uses the Deepseek big model deployed locally in the ground safety center to integrate the road construction area safety information obtained by the visual AI big model analysis, uses the formula to calculate the comprehensive danger level of each vehicle, and generates a structured voice broadcast.

[0062] The adaptive learning AI big model, after the AI ​​big model determines the comprehensive danger level of each vehicle passing through the road construction area, the adaptive learning AI big model analyzes the subsequent trajectory characteristics and judgment conditions of each vehicle to adjust the weight coefficients of each condition of the inference judgment big model to judge the comprehensive danger level of the vehicle.

[0063] Wireless communication module, construction workers in the road construction area all wear wireless headphones to receive real-time command and dispatch from the ground safety center and receive early warning information in a timely manner.

[0064] like Figure 1 and Figure 4 As shown, an embodiment of the present invention provides a dangerous vehicle warning method for road construction areas based on the collaboration of multiple AI large models, including the following steps:

[0065] Step S1: Deploy construction safety monitoring drones in various areas of the road construction area to obtain real-time video images of road traffic conditions around the road construction area.

[0066] See also Figure 2 In some embodiments, after receiving a construction task, the ground safety center begins to automatically plan the flight route and reconnaissance position of the drone, and deploys different drones into the warning zone, upstream transition zone, buffer zone, work zone and termination zone of the road construction operation. The drone flight control system automatically sends the flight route to the drone, and the construction safety drones enter their respective monitoring points to monitor the situation at each location in the road construction area in real time.

[0067] Deploy construction safety monitoring drones in various areas of the road construction area. The monitoring starting point of the drone in the warning zone of the road construction area needs to meet the minimum monitoring distance L min The number of construction safety monitoring drones required is determined based on the condition that the drone’s monitoring range needs to cover the entire road construction area, ensuring that the drone can comprehensively monitor the overall situation of the entire construction section and monitor the overall traffic flow on the road surface of the construction section in real time.

[0068] Specifically, the starting point for drone monitoring in the warning zone needs to be calculated based on data such as road conditions, road speed limits, and road linearity. It is generally controlled at a distance of 3-5 kilometers from the highway construction area, or a 2-3 minute drive from the construction area. If the road linearity is poor, such as the construction area is located in a downhill position, a bend, or other locations where visibility is affected, the drone's reconnaissance location needs to be further away. The formula for calculating the minimum monitoring distance of drones on generally good road sections is as follows:

[0069] L min =v max ×Δt+d s

[0070] Where, L min Indicates the minimum monitoring distance of the drone, v max represents the maximum design speed of the road, Δt represents the model response time, d s The minimum safe distance.

[0071] Calculate the minimum monitoring distance for drones in the upstream transition zone. Increase the monitoring distance by 20% during nighttime operations, and increase the drone deployment altitude to 1.5 times the standard value in curved road scenarios.

[0072] After the drone arrives at the designated reconnaissance location, construction workers put on wireless communication headsets to connect with the ground safety center and begin setting up warning signs in the construction area, traffic cones in the transition area, and buffer areas.

[0073] Step S2: The drone is equipped with a target detection algorithm (such as YOLO) to pre-screen the collected video data, and only upload the video clips containing vehicle targets to the ground security center server, and then perform image preprocessing operations.

[0074] As an implementation method, the drone's integrated vehicle detection module pre-screens video data, uploading only video clips containing risky targets to a ground server. The captured video clips of vehicles passing through the drone's vehicle detection module are then transferred to a ground-based safety center via FTP. After image preprocessing, the ground-based safety center feeds the data into a large visual AI model and identifies construction area information. The ground-based safety center determines the specific road construction area, including its location on a map, the extent of the impact, and the lanes affected by the construction area.

[0075] The UAV vehicle detection module uses a UAV to perform reconnaissance missions. The vehicle detection system detects in real time whether there are vehicles in the video stream captured by the UAV. It uses a vehicle detection method based on YOLO target detection. The target detection model is relatively lightweight, which is convenient for deployment on the UAV to lower the threshold of vehicle detection and ensure that video clips with fewer vehicles passing through can be screened out.

[0076] Image preprocessing: Many road construction projects are carried out at night when there are fewer vehicles. Since the images taken by drones at night may have unclear target objects or overexposure of lights, which will lead to inaccurate recognition of the large model, it is necessary to perform certain data preprocessing operations on the videos taken by drones before transmitting them to the visual AI large model for analysis. Specific data preprocessing operations include light enhancement, which increases the overall brightness of the image and makes the details in the dark areas clearer; denoising, which reduces noise in the image and improves image clarity; and contrast enhancement, which increases the contrast of the image and makes the target object easier to distinguish from the background.

[0077] Drones in different monitoring areas have different monitoring tasks, and their corresponding videos need to be transmitted to their corresponding AI models. Monitoring drones in the warning zone of a road construction area are responsible for monitoring dangerous vehicles that may enter the construction area. The conditions to be analyzed include weather conditions, road conditions, traffic volume, vehicle lane, vehicle speed, vehicle type, etc.

[0078] For vehicle intrusion analysis in other road construction areas, since there are no bottleneck sections, the risk of vehicles changing lanes is relatively low. It is only necessary to analyze whether the vehicle is speeding and whether a rear-end collision is likely to occur. Since the number of accessible lanes is reduced due to road construction, the impact of rear-end collisions on traffic flow will increase. It is necessary to promptly remind vehicles to maintain safe distance and respond to rear-end collisions in a timely manner.

[0079] The ground safety center also organizes the speed limit information of the road section and the location information of the construction area. The construction area information includes the location of the construction area on the map, the size of the impact of the construction area, and the lane information affected by the construction area.

[0080] Step S3: Use multiple visual AI large models such as llama3.2-vision:90b, llava:34b and gemma3:27b carried by the ground safety center to perform parallel analysis on the data input in step S2. By customizing specific prompt words, ask questions to the multiple visual AI large models to obtain relevant information on the safety of the road construction area.

[0081] In some embodiments, specific question prompts are as follows:

[0082] Question 1: What are the weather conditions in the video? Please rate according to the scoring criteria.

[0083] Question 1: There are several vehicles passing by in the video. Please describe their characteristics using color + small car\medium car\large car.

[0084] Question 2: Is it possible that the vehicle in the video is impacting the construction zone? The visual AI model determines the lane information of the vehicle in the video and combines it with the lane impact data from the construction zone location information to answer the question.

[0085] Question 3: Does the vehicle in the video exceed the speed limit? Based on the known length of the road section in the drone video, or by calculating the length and number of lane lines, analyze the time it takes for the vehicle to travel a known distance in the video. Use the distance-time formula to calculate the time and compare it to the speed limit for the section.

[0086] Question 4: How long will it take for the vehicle in the video to reach the construction area? The large model uses the GPS positioning information of the drone and the distance between the location and the construction area, combined with the average vehicle speed calculated in Question 3, to calculate how long it will take for the vehicle in the video to reach the highway construction area.

[0087] The large model calculates vehicle speed based on the following logic:

[0088] The large model determines the length of the road section traveled by the vehicle in the video based on the number of lane lines on the road, records the vehicle's passing time, and calculates the average speed of the vehicle passing through the road section in the video by dividing the distance by the time.

[0089] Step S4: Use a large language model (such as Deepseek) to organize the analyzed vehicle driving conditions and road conditions around the road construction area, determine the comprehensive vehicle danger level, and organize the analysis results into structured information to create a construction safety information table and store it in the system.

[0090] In this embodiment, the potential dangers of road construction are divided into three levels: low, medium, and high. The large language model DeepSeek is used to analyze the question results obtained by the visual AI large model to determine the comprehensive danger level. The comprehensive danger level is determined as follows:

[0091] D=(ω1×W+ω2×R+ω3×T+ω4×L+ω5×V+ω6×P)

[0092] Where ω1, ω2, ω3, ω4, ω5, and ω6 are the weight coefficients of each influencing factor, T represents the traffic volume, the initial weight coefficients of ω1, ω2, ω4, ω5, and ω6 are 1, and the initial weight coefficient of ω3 is 0.01.

[0093] W represents the weather condition, which is defined as:

[0094]

[0095] R represents the road condition, which is defined as:

[0096]

[0097] Where l represents the number of drivable lanes after excluding lanes closed for construction.

[0098] L represents the lane the vehicle is in and is defined as:

[0099]

[0100] V represents the vehicle speed, which is defined as:

[0101]

[0102] in, Indicates the average speed of vehicles passing by.

[0103] P represents the vehicle type, which is defined as:

[0104]

[0105] According to the above formula, the classification standard of comprehensive hazard level is:

[0106]

[0107] See also Figure 3 A large language model (such as Deepseek) further integrates all construction area-related information, structures the text, creates a construction safety information form (the form format is shown in Table 1 below), stores it in the system, and outputs the voice broadcast content (the broadcast content is shown in Table 2 below). The ground safety center sends this text to the wireless communication headsets worn by highway construction workers via wireless communication.

[0108] Table 1 Road construction safety monitoring data based on Deepseek (example)

[0109]

[0110] Table 2 Structured voice broadcast information

[0111]

[0112] Step S5: Use a large AI model (such as Grok-3) to learn the subsequent trajectory characteristics of vehicles of each comprehensive danger level. Based on the learned characteristics, optimize the model judgment criteria to continuously improve the accuracy and reliability of the model.

[0113] Specifically, a large AI model (such as Grok-3) is used to learn the subsequent trajectory characteristics of vehicles of various comprehensive danger levels. Based on the learned characteristics, the model judgment criteria are optimized to continuously improve the accuracy and reliability of the model. Specifically, the system will record the trajectories of vehicles that enter the road construction area and the trajectories of vehicles that have traffic accidents around the road construction area. At the same time, relevant condition information before the accident is retrieved from the database, including weather conditions, road conditions, traffic volume, vehicle lane, speed and vehicle model, etc. This data is input into large AI models such as Grok-3 for feedback learning. By analyzing the trajectory characteristics and corresponding accident conditions in a large amount of historical data, the model can continuously correct the risk coefficient weights of each condition.

[0114] Step S6: When a potentially dangerous vehicle is discovered, the ground safety center takes graded response measures based on the different comprehensive danger levels of the vehicle, sends warning information to construction workers through wireless communication, and commands the drone to send a warning signal at the same time.

[0115] After identifying potential risks, the ground safety center takes graded response measures based on different comprehensive risk levels. When a medium-risk vehicle is found, a warning message is sent to the construction workers through wireless communication, and at the same time, a command is issued to the drone to send information about the construction ahead to the dangerous vehicle through voice and lights; when a high-risk vehicle is found, the construction workers in the construction area are notified to evacuate.

[0116] For example, if the AI ​​model concludes that a dangerous speeding vehicle is present and determines it to be a high-risk vehicle, the drone control system at the ground safety center will issue a command to the drone, causing it to lower its altitude, turn on its flashing lights and loudspeaker, and announce to the vehicle: "Construction is underway in lane xx ahead, please be careful!" At the same time, a message is sent to the construction workers via the wireless communication system: "There is a vehicle / dangerous vehicle ahead, expected to arrive at the construction area in xx minutes xx seconds, please seek shelter in a safe area."

[0117] Example 2

[0118] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 1 The figure shows a dangerous vehicle warning method for road construction areas based on the collaboration of multiple AI large models.

[0119] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0120] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0121] Since the electronic device is an electronic device corresponding to the dangerous vehicle warning method in the road construction area based on the collaboration of multiple AI large models in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0122] Example 3

[0123] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 The figure shows a dangerous vehicle warning method for road construction areas based on the collaboration of multiple AI large models.

[0124] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0125] Since the storage medium is a storage medium corresponding to a dangerous vehicle warning method for road construction areas based on collaboration of multiple AI large models in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0126] Example 4

[0127] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to execute the steps of the method for warning dangerous vehicles in road construction zones based on the collaboration of multiple AI large models according to various exemplary embodiments of the present application described above in this specification. The executable computer program code or "code" used to execute each embodiment may be written in a high-level programming language such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0128] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0129] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0130] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A dangerous vehicle early warning method for road construction areas based on the collaboration of multiple AI large models, characterized by: The following steps are involved: S1. Deploy construction safety monitoring drones in various areas of the road construction area to obtain real-time video images of road traffic conditions around the road construction area; S2. The drone is equipped with a target detection algorithm to pre-screen the collected video data and upload only the video clips containing vehicle targets to the ground security center server for image pre-processing. S3. Utilize multiple large-scale visual AI models to analyze incoming video data in parallel, guiding them through preset prompts to obtain vehicle traffic and road conditions around the road construction area. S4. Use the large language model to organize the analyzed vehicle driving conditions and road conditions around the road construction area, determine the comprehensive vehicle hazard level, organize the analysis results into structured information, and create a construction safety information form to store in the system; S5. Use the AI ​​large model to learn the subsequent trajectory characteristics of vehicles of various comprehensive danger levels, and optimize the model judgment criteria based on the learned characteristics; S6. When a potentially dangerous vehicle is discovered, the ground safety center will take graded response measures based on the different comprehensive danger levels of the vehicle, send warning information to construction workers, and command the drone to send a warning signal.

2. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 1 is characterized in that: The step S1 comprises: Before arranging the road construction area, deploy several construction safety monitoring drones to monitor traffic on the construction road. The drones need to conduct reconnaissance along the road construction work area for a sufficient distance upstream to ensure that construction personnel have sufficient reaction time and safe distance when arranging the construction area. The road construction area is divided into a warning zone, an upstream transition zone, a buffer zone, a work zone, and a termination zone. Different conditions are used to determine vehicle danger in different areas: 1) For early warning of hazardous vehicles in the warning zone, it is necessary to analyze traffic conditions upstream of the road construction area, including each vehicle's speed, lane position, and possible lane change risks; 2) For the early warning of dangerous vehicles in the upstream transition zone, buffer zone, work zone and termination zone, it is only necessary to analyze the speed information of each vehicle and the possible rear-end collision risk.

3. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 2 is characterized in that: The minimum monitoring distance of drones in the warning zone and upstream buffer zones must be determined based on the design speed of the construction road. This ensures that construction workers have sufficient time to evacuate to a safe location on the roadside after receiving the warning. The drones in the upstream transition zone, buffer zone, work zone, and termination zone must have a field of view that covers the entire construction area, thereby determining the number of drones required. Among them, the minimum monitoring distance of the drone needs to be calculated based on the road design speed and meet the following formula: L min =v max ×Δt+d s Where, L min Indicates the minimum monitoring distance of the drone, v max represents the maximum design speed of the road, Δt represents the model response time, d s The minimum safe distance.

4. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 1 is characterized in that: The step S2 includes: The lightweight target detection system carried by the drone performs vehicle target detection on the collected video data, captures video clips containing only road video clips of vehicles passing, and uploads them to the ground security center server via the FTP protocol. The image then undergoes image preprocessing operations such as illumination enhancement, denoising, and contrast enhancement to reduce the amount of computation after transmission to the large model.

5. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 1 is characterized in that: The data transmitted to the large model in the ground safety center includes: 1) Preprocessed video data of vehicles passing around the road construction area; 2) The flight altitude, camera parameters, and camera angle of the drone when shooting the video; 3) Real-time GPS positioning data of each drone; 4) Location information, speed limit information, and road geometry information of the construction section.

6. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 1 is characterized in that: The step S4 comprises: The large language model is used to organize and analyze the question results obtained by the large visual AI model, and the comprehensive risk level of the vehicle is divided into low risk level, medium risk level, and high risk level. The indicators used to determine the comprehensive risk level of the vehicle include: weather conditions, road conditions, traffic volume, lane of the vehicle, vehicle speed, and vehicle type; The comprehensive risk level of a vehicle is determined according to the following formula: D=(ω1×W+ω2×R+ω3×T+ω4×L+ω5×V+ω6×P) Where ω1, ω2, ω3, ω4, ω5, and ω6 are the weight coefficients of each influencing factor, R represents the road condition, T represents the traffic volume, W represents the weather condition, L represents the lane the vehicle is in, V represents the vehicle speed, and P represents the vehicle model.

7. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 1 is characterized in that: The step S5 comprises: The AI ​​large model learns the trajectory characteristics of vehicles of various comprehensive danger levels, and subsequently optimizes the model judgment criteria. By recording the trajectory characteristics of vehicles that enter the road construction area and the trajectory characteristics of vehicles that have traffic accidents around the road construction area, and retrieving the weather conditions, road conditions, traffic volume, vehicle lane, vehicle speed, and vehicle model before the accident from the database, the large model conducts feedback learning to continuously correct the risk factor weights of each condition and improve the accuracy of the large model's subsequent judgments.

8. The method for early warning of dangerous vehicles in road construction areas based on collaboration of multiple AI large models according to claim 1 is characterized in that: The step S6 comprises: When the comprehensive danger level of a vehicle reaches a high danger level, a command needs to be sent to the drone to warn the dangerous vehicle and notify the construction workers to evacuate to a safe area; Among them, the construction safety monitoring drone is equipped with flashing lights and loudspeakers, and the drone can send warning signals to dangerous vehicles.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

Images