1975 results about "Traffic signal" patented technology

The invention discloses a traffic transportation operation monitoring early warning and decision analysis method based on vehicle infrastructure cooperation, and relates to the technical field of intelligent traffic. According to the method, real-time collection of dynamic traffic elements is realized through a multi-dimensional sensing network of a vehicle end, a road side and an environment and V2X communication, time-space reference unification of multi-source data is ensured, a road-vehicle-environment-event semantic network is constructed, multi-dimensional recognition of abnormal events such as accidents, congestion and severe weather is realized in combination with hierarchical feature extraction, and the method has the advantages of being high in practicability and high in practicability. The method is advantaged in that identification accuracy is improved, abnormal event propagation paths can be predicted, global road network situation prediction capability is realized, differential early warning is generated based on a comprehensive risk index, multi-level responses such as traffic signal adjustment and path planning are triggered, response time is greatly shortened, emergency response efficiency is optimized, a decision execution effect real-time feedback mechanism is established, and the method is suitable for popularization and application. The system performance is continuously optimized along with data accumulation, and the defect that a big data platform lacks an intelligent decision closed loop is avoided.

An intelligent traffic management device for optimizing traffic flow at one or more intersections using reinforcement learning, comprising: a central control unit housed in a weatherproof enclosure and mounted on a structural support near an intersection; a sensor interface operatively connected to a variety of multimodal sensors, including at least radar-based vehicle detectors, inductive loop sensors embedded in the road surface, infrared pedestrian detection detectors, and high-resolution optical imaging modules; a powerful embedded computing unit consisting of a multi-core processor and a GPU configured for on-device inference; a reinforcement learning engine embedded in the computing unit, comprising a deep reinforcement learning model trained to map environmental states defined by traffic-related sensor inputs to optimized action policies for traffic light control; a signal actuation module electrically connected to traffic signal controllers, with action policies outputting signal phase durations and transitions in real time; a communications module supporting wireless connectivity and configured for device-to-device communication and cloud synchronization; a policy synchronization module configured to perform federated learning-based updates of model parameters by exchanging gradient weights or distilled traffic state vectors with neighboring intelligent traffic management devices deployed in a traffic network; where the reinforcement learning engine continuously observes traffic state vectors consisting of vehicle counts, average speeds, queue lengths, and pedestrian activity, and dynamically updates the policy to maximize cumulative traffic throughput and minimize average delay.

The invention discloses a perception intelligent driving complex traffic scene dynamic risk prediction method, and relates to the technical field of risk prediction, and the method comprises the steps: obtaining the multi-source traffic dynamic data of a target region in real time; performing space-time alignment processing on the multi-source traffic dynamic data; inputting the space-time coupling feature matrix into a pre-trained depth space-time prediction model to generate a risk thermodynamic map; calculating a dynamic risk index of each traffic sub-region, and generating a risk level distribution sequence; and triggering a self-adaptive early warning response mechanism according to the risk level distribution sequence, and dynamically adjusting operation parameters of the variable information sign and the traffic signal controller. The technical problems of frequent traffic congestion and high accident risk caused by inaccurate traffic risk prediction and difficulty in dynamic adjustment according to the real-time traffic condition in the prior art are solved, and the technical effects of accurately predicting the traffic risk and improving the safety and traffic efficiency in a complex traffic scene are achieved.

The invention relates to the technical field of traffic management, in particular to a smart city intelligent traffic management system based on big data, which comprises a multi-source data acquisition module, a data fusion module, a dynamic feature generation module, a situation prediction module, a multi-target optimization module and an instruction execution module, the multi-source data acquisition module acquires multi-source data in real time; the data fusion module generates a standardized traffic data flow and outputs a three-dimensional data cube; the dynamic feature generation module outputs a traffic feature matrix; the situation prediction module generates a dynamic situation map; the multi-objective optimization module outputs an optimization instruction set; and the instruction execution module issues a control instruction to the traffic signal controller, the variable information board and the vehicle-mounted terminal. According to the invention, traffic jam can be relieved, road traffic efficiency can be improved, bus priority scheduling can be optimized, vehicle delay can be reduced, and emergency response speed can be improved.

The invention provides an edge computing traffic light intelligent decision-making method integrated with AI video analysis, and the method comprises the steps: collecting a real-time traffic video stream of a target intersection through an edge computing device, and extracting a dynamic traffic feature set in the real-time traffic video stream, calling a pre-trained space-time analysis model to carry out multi-modal fusion processing on the dynamic traffic characteristic set, generating a traffic state vector of the target intersection, matching a candidate control strategy in a preset decision rule base based on the traffic state vector, and carrying out parameter adjustment on the candidate control strategy through a strategy optimization model to obtain a target traffic state vector; generating a target signal lamp control parameter; and issuing the target signal lamp control parameters to a traffic signal control terminal of the target intersection, and monitoring traffic state change data of the target intersection in real time to update weight parameters of the space-time analysis model. According to the invention, the traffic efficiency, pedestrian safety and traffic management intelligence degree of urban intersections can be improved.

The invention provides an edge calculation traffic light emergency control method and device for a sudden traffic event. The method comprises the following steps: acquiring real-time traffic flow information of a target intersection; determining an emergency response area of the target intersection based on the position identifier of the sudden traffic event, and extracting a dynamic offset between a historical traffic flow feature and a current traffic flow feature in the emergency response area through an edge computing node; inputting the dynamic offset into a pre-trained traffic signal adaptive model, generating a traffic signal lamp control parameter set corresponding to the emergency response area, and sending a control instruction sequence to the traffic signal lamp of the target intersection according to the traffic signal lamp control parameter set, and the execution priority of the control instruction sequence is dynamically adjusted based on the real-time change trend of the vehicle density distribution data and the pedestrian movement track data. According to the invention, the intersection passing efficiency and the safety guarantee capability of traffic participants under the sudden traffic event can be improved.

The invention relates to the technical field of intelligent traffic, in particular to an intelligent traffic signal edge computing resource dynamic allocation method and system.The method comprises the steps that firstly, real-time and historical traffic data of multiple intersections are obtained, a traffic prediction model is constructed, and future flow changes are pre-judged; thirdly, forming an edge computing resource demand matrix according to a prediction result and computing resource demands, and dynamically adjusting resource allocation to generate a resource slicing scheme; then, the flow gradient between intersections is analyzed, an edge node cooperative calculation model is established, and calculation task distribution and migration are optimized; and finally, in combination with system operation and traffic control effects, forming closed-loop feedback, continuously optimizing resource allocation and a cooperative strategy, and through traffic flow prediction driven resource demand modeling, realizing active pre-allocation of computing resources, converting resource allocation from passive response to active pre-adaptation, and improving the system resource utilization rate by 40%-60%.

The invention relates to the technical field of traffic flow regulation and control, and discloses an electronic information traffic flow automatic regulation and control system based on a wireless sensor network. The multi-source sensing module collects road network multi-dimensional traffic data through a wireless sensing network, and traffic situation characteristics are generated through a specific data fusion method. The spatio-temporal feature fusion module utilizes a spatio-temporal attention mechanism to associate cross-modal features, and the collaborative decision-making module inputs fusion features into a pre-trained distributed decision-making model to generate a regulation and control instruction set. The dynamic game optimization module constructs a multi-target game optimization model, and traffic signal parameters are optimized by adopting a dynamic game strategy decomposition algorithm. And the hierarchical execution module executes regulation and control instructions in a distributed manner through a three-level control architecture of a central decision-making layer, a regional coordination layer and an intersection execution layer. The system can comprehensively collect and fuse traffic data, realizes scientific decision making and accurate regulation and control, effectively improves the road passing efficiency, balances the road network load, and relieves traffic jam.

Systems and methods for controlling traffic flow along an arterial. The systems comprise a data processing apparatus configured to: monitor traffic light control signals (TLCSs) used to control traffic lights at intersection(s) and traffic delays occurring at intersection(s) at a time when the traffic lights are being controlled by TLCSs; access a hybrid model comprising a first linear term corresponding to an instance of traffic delays, a second linear term corresponding to a concurrent instance of TLCSs, and a nonlinear function defining a nonlinear relationship between the instance of the traffic delays and a previous instance of TLCSs; use the hybrid module to predict traffic delays at intersection(s) based on the traffic control signals and the traffic delays; determine, based on TLCSs and the traffic delays, traffic control signals that cause the predicted traffic delays to decrease; and cause the traffic lights to be controlled using the traffic control signals.

The invention relates to a traffic signal lamp fault diagnosis method and system based on artificial intelligence, and the method comprises the steps: obtaining hardware sensor data, visual monitoring data and communication data of traffic signal lamps, carrying out the integrated label processing, and obtaining a label multi-source data set; performing feature extraction on the label multi-source data set to obtain a hardware state feature set, a display anomaly feature set and a communication fault feature set; performing multi-modal fusion analysis on the hardware state feature set, the display anomaly feature set and the communication fault feature set according to a preset signal fault diagnosis model to obtain a fault diagnosis result and a confidence score; and performing strategy matching on the confidence score and a preset strategy library to obtain an initial control strategy, and performing instruction optimization on the initial control strategy according to a fault diagnosis result to obtain a fault processing control strategy. According to the invention, comprehensive analysis of multi-source data can be realized, and comprehensiveness and accuracy of fault diagnosis are improved.

The invention relates to the technical field of traffic control, in particular to a traffic signal cooperative control method based on multi-agent reinforcement learning, and the method comprises the following steps: modeling each signal lamp intersection in a road network as an agent, and constructing a distributed multi-agent control environment; dividing the whole road network into a plurality of small subnets according to spatial correlation, and sharing and aggregating traffic information in the subnets through a neighborhood information sharing mechanism; utilizing a space-time diagram attention network to extract traffic state characteristics including a space-time dependency relationship in an intersection agent and a neighborhood thereof; and defining a traffic state, a traffic signal control strategy, a reward and punishment function, a network architecture and a target function required by training of the distributed intelligent agent, and carrying out joint training on the distributed intelligent agent until a training target is achieved. According to the invention, the real-time sensing capability of the signal control intelligent agent to the traffic flow dynamic state can be improved, and the collaborative decision-making capability among multiple intelligent agents is enhanced.

The invention discloses a smart city traffic signal lamp management and control system, and relates to the technical field of traffic signal control. The system comprises a region division module, a traffic flow detection sensor, a region traffic flow analysis module, a signal lamp management and control strategy generation module, an inter-region signal lamp cooperative control module and a variable information sign and information processing and generation module. The system can accurately monitor and analyze urban traffic flow conditions in real time by comprehensively applying an advanced region division algorithm, a traffic flow detection technology and an accurate traffic flow state evaluation model, and can intelligently generate a signal lamp management and control strategy according to an analysis result, so that the traffic lamp management and control efficiency is improved. Comprising the steps that a congestion relieving dynamic timing algorithm is adopted in a congestion area, the number of times of signal lamp switching is reduced, a flow balance guiding timing algorithm is applied in an area with small flow, in addition, cooperative control over signal lamps between the areas can be achieved, and efficient connection and balanced distribution of traffic flow between the areas are ensured.

The embodiment of the invention discloses a dynamic traffic signal control method and device based on data fusion and a medium, belongs to the technical field of intelligent traffic, and solves the problem that the existing traffic signal control is difficult to make effective response when an accident occurs. Performing data fusion on first traffic data acquired by a road monitoring and shooting device and second traffic data acquired by a positioning device of the floating car through a multi-head attention mechanism at the edge node to obtain multi-source traffic characteristic data; the cloud end inputs the multi-source traffic characteristic data into a preset signal optimization model every first preset duration; the preset signal optimization model is updated and optimized through a PPO algorithm and a federated average method, and a global traffic model is generated; and through knowledge distillation, compressing the global traffic model, and migrating an edge traffic model obtained after compression to a corresponding edge node so as to carry out traffic signal strategy output on a corresponding road section through the edge traffic model.

The invention discloses a traffic signal and direction-variable lane cooperative control method based on multi-agent learning, and the method comprises the steps: making a decision through the iteration of a traffic signal control module and a lane guiding control module, and carrying out the operation at the same time to control the traffic; the lane guiding control module adopts a dual depth Q network algorithm to enhance a learning model, an intelligent agent selects actions based on traffic flow and traffic signal states to maximize the lane utilization rate, and an award is calculated based on lane queuing length difference and the maximum direction demand traffic flow ratio; the traffic signal control module agent selects an action based on the observed lane density and the current lane state and obtains an award according to the traffic flow satisfaction of the selected phase. According to the method, the lane utilization rate is improved by controlling lane guidance, and the dynamically changing traffic flow is controlled in space and time dimensions by using a reinforcement learning framework; higher stability and efficiency are shown, and the traffic capacity of the intersection is improved.

The invention belongs to the technical field of vehicle control, and particularly relates to a vehicle-mounted image recognition and target detection system based on deep learning, and the system comprises a distributed monitoring module which collects the operation, obstacle and traffic signal information of a target vehicle through multi-modal classification and scene matching, completes the marking of a shielding region and the matching of information through the combination of shared data, and achieves the recognition of the target vehicle. Forming an enhanced monitoring set; the label planning module constructs an enhanced topological space based on the enhanced monitoring set, and adjusts moving tracks in different scenes by combining with vehicle and pedestrian track probability distribution fed back by dynamic intention recognition; the action recognition module predicts trajectory parameters and collision probabilities of non-target vehicles and pedestrians by using Bayesian and multi-modal algorithms; the decision-making module generates a real-time control instruction through particle swarm optimization and fuzzy control, and optimal control parameters are fed back through simulation; according to the invention, intelligent track planning and real-time control in a complex scene are realized, and the detection precision and control robustness of the shielded and label-free area are improved.

According to the method, multi-modal data such as voice, images, texts, GIS and Internet of Things sensing are fused, and deep neural network prediction, reinforcement learning scheduling optimization and rule engine compliance check are combined; the intelligent vehicle and material dispatching method and system are applied to multiple scenes such as emergency material storage depots, fire-fighting emergency command, urban disaster response, traffic accidents and medical first aid. The system is interconnected and intercommunicated with an intelligent emergency material storage cloud platform, a city brain, Beidou navigation, intelligent fire fighting and other external platforms, and supports one-key issuing, path optimization, traffic signal linkage and whole-course return closed loop. Compared with the prior art, the method has the advantages that unification of multi-modal situation awareness, data-driven optimal scheduling and expert knowledge constraints is realized, the response time is remarkably shortened, the resource utilization rate is improved, and compliance safety is ensured.

The invention discloses an intelligent traffic control system and method based on multi-agent near-end strategy optimization, and belongs to the field of intelligent traffic, Internet of Vehicles and deep reinforcement learning. The method comprises the following steps: firstly, constructing a fog-cloud collaborative three-layer architecture, and realizing real-time monitoring and dynamic regulation and control of traffic flow through cloud global decision and local sensing collaboration of a road side unit (RSU); secondly, designing indexes of'road section overlap ratio 'and'road section time overlap ratio', and solving the problem of secondary congestion caused by rerouting; then, a multi-agent near-end strategy optimization (MAPPO) algorithm is adopted, so that the traffic signal lamp is used as an autonomous agent to dynamically adjust the phase, and the limitation of single-point control is broken; and finally, through integrated optimization of rerouting and adaptive signal control, an original multi-objective optimization problem is converted into a layered multi-agent reinforcement learning problem. According to the invention, vehicle driving time and system energy consumption can be effectively reduced, road traffic efficiency is improved, and active avoidance and dynamic alleviation of urban traffic congestion are realized.

The invention, which relates to the technical field of intelligent traffic and illumination control, discloses a street lamp vehicle-road cooperative control system comprising a data acquisition module, a data processing module, a control execution module and a decision management module. The data acquisition module is used for acquiring traffic signal lamp phase, vehicle position and speed, street lamp working state and environment illumination intensity data; according to the invention, through a cross-system collaborative algorithm, the traffic flow direction is predicted in combination with the phase of the traffic signal lamp, the brightness of the street lamp illumination area is adjusted in advance, the energy distribution according to needs is realized, the energy consumption is effectively reduced, and through a vehicle-lamp-cloud three-level decision-making mechanism, the computing power priority is dynamically distributed according to the traffic condition. Traffic signal lamp control and vehicle driving path planning are optimized, the road passing efficiency is improved, traffic congestion is relieved, by pushing the lighting optimization path to the vehicle, the vehicle is driven under the proper lighting condition, the emergency brake risk is reduced, and the road traffic safety level is remarkably improved.

The invention relates to an environmental perception traffic signal control method based on reinforcement learning, and belongs to the technical field of computers. According to the method, traffic signals are intelligently regulated and controlled by sensing environment information and traffic states and combining road states. Firstly, a traffic signal regulation and control strategy is learned in a simulation environment by using a reinforcement learning algorithm, and then the obtained regulation and control strategy is applied to a real scene. The specific implementation comprises the steps of collecting environmental perception data, constructing a simulated traffic environment, configuring a control strategy, setting a typical traffic scene, collecting environmental information, fusing features, setting performance indexes, reinforcing learning optimization, applying a real scene, establishing a feedback mechanism and the like. Through combination of reinforcement learning and environment perception technologies, the method can dynamically adapt to traffic demands in different time and environments, effectively alleviates traffic congestion, improves traffic operation efficiency, and provides an effective scheme for improving urban traffic conditions.

The invention provides a regional boundary traffic signal control method and system based on model predictive control, and belongs to the technical field of intelligent traffic. The method specifically comprises the following steps: partitioning a road network region; collecting traffic parameters of each region of the road network, and fitting an MFD curve of each region based on the collected data; establishing an optimization model by taking minimization of the number of vehicles in a road network as a target; an optimal boundary control value is obtained through optimization solution of an MPC controller with a built-in SLSQP algorithm, so that boundary traffic flow and boundary traffic signal lamp timing of each region of the road network are controlled; a traffic flow is simulated based on SUMO, and from a current time step to a next time step, an MPC controller with a built-in SLSQP algorithm is used for continuous iterative optimization solution to realize rolling optimization. And modeling the traffic state of each region by using MFD to obtain the flow dynamic state of the region. By establishing a network structure to simulate traffic dynamics, the MPC method is realized to control transfer of boundary traffic flow and dynamically control timing of traffic signal lamps, so that traffic congestion is relieved.

The invention discloses a bus and station interactive scheduling method and system based on V2X and deep reinforcement learning decision, a medium and equipment, and the method comprises the steps: collecting bus operation dynamic data, station passenger flow and environment data in real time, and combining V2X network interaction information to construct a full-dimension perception system; a deep reinforcement learning model is adopted for dynamic decision making, intelligent closed-loop control of vehicle scheduling, station service and traffic signal cooperation is achieved, the bus punctuality rate is remarkably increased, the waiting time of passengers is shortened, operation safety is guaranteed through active early warning of abnormal conditions, and intelligent upgrading of a bus system from passive response to active prevention is achieved.

The invention discloses a traffic signal lamp intelligent control method and system, and relates to the technical field of intelligent traffic systems, and the method comprises the steps: collecting traffic operation and environment data through deploying a sensor group in a target traffic area signal lamp; a self-adaptive analysis model including anomaly detection and congestion prediction is obtained through a traffic scheduling platform, traffic conditions are analyzed, and congestion parameters are obtained. And starting an emergency response mechanism according to the parameters, analyzing and determining signal lamp control strategy parameters, and finally realizing adaptive regulation and control of the traffic signal lamp. The technical problems that the real-time response capability of an existing traffic signal lamp control system is insufficient, and traffic abnormity and congestion cannot be effectively predicted are solved, and through real-time traffic data analysis and intelligent prediction, the real-time response capability of the traffic signal lamp system is enhanced, and the traffic safety is improved. And traffic abnormity and congestion can be effectively predicted and dealt with.

The invention relates to the field of smart city management, and provides a smart city operation data management system and method, which adopts a deep learning-based data processing and analysis technology to understand the time sequence change of the traffic flow of an intersection by performing time sequence analysis on the time queue data of the traffic flow of a target management intersection. And the time configuration of the traffic signal lamp is dynamically adjusted based on the time sequence change of the intersection traffic flow. Therefore, the traffic signal lamp time can be scientifically and reasonably configured according to the actual traffic flow, and the operation effect of an urban traffic network can be optimized.

The invention relates to the technical field of intelligent logistics, and discloses an intelligent logistics-oriented path planning method and system, and the method comprises the steps: determining a working area of a logistics vehicle, dividing the logistics vehicle into multiple paths according to traffic lights and guiding arrows, collecting the information of the paths and the vehicle through a detection unit, drawing an electronic map from an overlook angle, and carrying out the planning of the path. The processing unit calculates the vehicle density P of the obtained path; acquiring corresponding vehicle density according to corresponding parameter information, sequencing vehicle density data groups, planning a path by combining with an actual departure place and a destination, predicting a next time period by acquiring current vehicle speed and time period information, and performing combined analysis on three parameters including the vehicle density, an average vehicle speed and a vehicle speed fluctuation value to obtain a vehicle density prediction result. Therefore, the effect of improving the working efficiency is achieved.

The invention discloses a multi-signal lamp linkage control method and system for intelligent urban traffic, and relates to the technical field of intelligent traffic, and the method comprises the steps: constructing an urban multi-level response system which covers the response architecture of three levels of intersections, regions and cities; a sensing acquisition network is accessed to collect weather information and traffic information, and traffic abnormal conditions in the response range of each level are identified; integrating the meteorological data, the traffic data and the traffic abnormal information into an urban multi-level response system according to the association of the response levels, and analyzing the traffic signal lamp strategy of each level so as to formulate the traffic control strategy of each level; and performing overall optimization on the traffic control strategy of each level in combination with the correlation characteristics of each level region, and finally forming a global collaborative control strategy. Therefore, the technical effects of fine control and global optimization, and enhancement of the adaptability and the overall operation benefit of the traffic system are achieved.

The invention discloses an intelligent interpretable traffic signal adaptive control method. The method comprises the following steps: training an intelligent agent through reinforcement learning according to environment state information of an intersection; a timing decision of each phase of the intersection is generated by using the intelligent agent; guiding the first large language model to generate pre-training data by the timing decision and the cue word to perform LoRA fine tuning on the second large language model, and inputting the cue word into the fine-tuned second large language model to enable the second large language model to generate a plurality of reasoning tracks to generate positive samples and negative samples; performing all-parameter fine tuning on the second large language model to obtain a traffic control signal decision model; and inputting the constructed cue word into a traffic control signal decision model to obtain each phase timing scheme of the intersection. According to the method, the defects that an intelligent traffic signal control algorithm based on deep reinforcement learning lacks interpretability and the cross-scene generalization ability is poor are overcome, and the method has important significance in improving the decision credibility and the deployment efficiency.

The invention discloses a traffic flow prediction and signal adjustment method based on machine learning. The method comprises the following steps: data acquisition: acquiring real-time traffic data; data processing: carrying out standardization processing on the traffic flow data and external data; machine learning model training: training the processed multi-modal data through a model; traffic flow prediction: predicting traffic flow change in a period of time in the future according to the current multi-modal data; and signal adjustment: adjusting the traffic signal of each intersection based on the predicted future traffic flow. According to the method, deep learning training is carried out on multi-modal data including traffic flow, weather, emergencies and the like, so that the model can more accurately capture the trend and rule of traffic flow change, and the prediction accuracy is remarkably improved. The system can flexibly adjust the signal period and the green light duration according to the real-time data, so that the traffic signal control has better adaptability, the instantaneous fluctuation of the traffic flow can be effectively dealt with, and the overall efficiency of the traffic system is improved.

The invention discloses a traffic signal timing optimization method and system, a storage medium and a program product, and relates to the field of traffic management data processing systems.The method comprises the steps that a plurality of adjacent intersection nodes are determined based on position identification information of local intersection nodes; collecting vehicle passing state data and vehicle characteristic data of a local intersection node, and generating traffic flow characteristic parameters; identifying a characteristic vehicle type and vehicle position information of the target characteristic vehicle; calculating the traffic influence score of the target characteristic vehicle on the traffic flow of each intersection; constructing a traffic signal optimization model, and calculating based on the traffic signal optimization model to obtain an initial timing scheme; sending the traffic flow output data to the adjacent intersection node, and receiving traffic flow input data sent by the adjacent intersection node; and updating the traffic signal optimization model, generating a final timing scheme, and adjusting signal control parameters of local intersection nodes. According to the invention, the accuracy of traffic signal timing can be improved.

The invention provides a method and a device for determining an end-to-end perception decision regulation and control architecture, and belongs to the technical field of automatic driving. A first multi-modal data set is constructed based on a basic traffic signal image of an automatic driving scene, road map data and a basic driving rule; the second multi-modal data set is constructed based on dynamic traffic element image sequence data of an automatic driving scene, complex environment element image sequence data and complex driving rules, and the third multi-modal data set is constructed based on an automatic driving scene image, an automatic driving target and sensor time sequence data; and performing staged training on the multi-modal large model to obtain an end-to-end perception decision regulation and control architecture suitable for an automatic driving scene. A multi-modal large model is trained through a staged training strategy, basic element recognition, complex element recognition and decision training are gradually realized, the real-time performance of the model is enhanced, and the requirement of an automatic driving task for the real-time performance is met.

In various examples, three-dimensional (3D) object or feature detection and localization for autonomous and semi-autonomous systems and applications is described herein. Systems and methods are disclosed that use different types of sensors, such as an image sensor and a LIDAR sensor, to determine information associated with objects, such as traffic objects (e.g., traffic signs, traffic signals, traffic markings, etc.). To determine the information for an object, image data is processed to determine a bounding shape associated with the object. The bounding shape is then used to determine a 3D shape, such as a frustum, corresponding to the object. Additionally, points data generated using the LIDAR sensor, such as an occupancy map and / or a point cloud, is processed to identify a portion of the points associated with (e.g., located within) the 3D shape. This portion of the points may then be used to determine the information associated with the object.