3229 results about "Deep neural networks" patented technology

The present invention relates to the field of industrial intelligent predictive maintenance, and in particular to an interpretable deep feature fusion network-based industrial intelligent predictive maintenance method, comprising: acquiring gearbox vibration data comprising noise; performing preliminary extraction and noise suppression on features of the acquired data by establishing an interpretable feature extraction module having a physical information constraint; integrating multi-scale features comprising long-distance and local dependencies by means of a dual-branch feature fusion module having global and local feature fusion capabilities; performing dimensionality reduction on a high-dimensional feature and generating an output by means of a classifier to obtain a final fault identification result; and performing interpretability analysis on a diagnosis process of a model. In the present invention, by embedding the signal processing technology having a well-defined physical theory support into a deep neural network, the interpretability and reliability of model inference results are effectively improved while the fault identification accuracy of the model is improved.

The invention relates to an intelligent water service pipe network monitoring method based on Internet of Things fusion, and aims to solve the problems in heterogeneous sensor data accurate acquisition, consistent processing, efficient anomaly recognition and trend prediction. According to the core technical scheme, the method comprises the steps that deployment of multiple types of sensors is optimized, standardized calibration is implemented, efficient collection and local preprocessing of original data are achieved through a wireless communication protocol, and data uniformity and reliability are guaranteed through data normalization, noise suppression and abnormal value elimination; performing historical operation trend and short-term fluctuation feature extraction and conventional trend prediction by adopting space-time mixed feature perception and a deep neural network, and integrating an adaptive anomaly detection and correction mechanism to realize emergency response and cause explanation; and finally, an analysis result is fed back to an early warning and resource scheduling system, and the model is periodically optimized. According to the scheme, the sensing precision, intelligent analysis and abnormal response capability of the operation data of the water service pipe network are remarkably improved.

The invention provides a cellular network flow space-time prediction method and system based on a hybrid neural network, and belongs to the technical field of intelligent communication. The method adopts a layered deep neural network architecture, and comprises a data embedding layer, a space-time coding layer, a feature fusion layer and an output layer. The data embedding layer maps a historical traffic sequence, cross-domain external data and metadata into high-dimensional features; the space-time coding layer is used for respectively fusing one-dimensional causal convolution and a Mama neural network to extract multi-scale time features and densely connecting convolution and a multi-head attention mechanism to capture multi-scale space features through time and space modeling branches; the feature fusion layer realizes adaptive weighted fusion of spatial-temporal features, cross-domain features and metadata features by using a gating fusion mechanism; and the output layer performs linear transformation on the fusion features to generate a final prediction result. According to the method, the spatial-temporal dynamic capture of the service traffic is accurate, the prediction curve is highly fit with the true value, and the accurate prediction of the multi-service traffic of the cellular network is realized.

According to the method, multi-modal data such as voice, images, texts, GIS and Internet of Things sensing are fused, and deep neural network prediction, reinforcement learning scheduling optimization and rule engine compliance check are combined; the intelligent vehicle and material dispatching method and system are applied to multiple scenes such as emergency material storage depots, fire-fighting emergency command, urban disaster response, traffic accidents and medical first aid. The system is interconnected and intercommunicated with an intelligent emergency material storage cloud platform, a city brain, Beidou navigation, intelligent fire fighting and other external platforms, and supports one-key issuing, path optimization, traffic signal linkage and whole-course return closed loop. Compared with the prior art, the method has the advantages that unification of multi-modal situation awareness, data-driven optimal scheduling and expert knowledge constraints is realized, the response time is remarkably shortened, the resource utilization rate is improved, and compliance safety is ensured.

The invention discloses a PINN-based high-precision hydrodynamic numerical simulation method and system, and the method comprises the steps: firstly building a computational domain, setting reasonable geometric parameters and boundary conditions, and constructing a dimensionless Navier-Stokes control equation set; then designing a deep neural network architecture with space-time coordinate input and flow field variable output, and adopting a loss function combining physical constraint and data driving; the core innovation lies in providing a timing sequence sensing RAR-D adaptive sampling strategy, dividing a time domain into a plurality of time frames, performing residual error evaluation in each frame, constructing a probability density function related to residual errors, and balancing priority sampling and overall coverage of a high residual error region; adam and L-BFGS optimizers are adopted to carry out network training, the weight of a loss function is dynamically adjusted, and a sampling point set is periodically updated; and finally, the solution precision is verified through multi-dimensional flow field visualization analysis. Therefore, the prediction precision of the complex flow field is effectively improved, and the calculation efficiency is remarkably improved.

The invention discloses a marine environment real-time monitoring and early warning system based on machine learning, and relates to the technical field of machine learning. Comprising the steps that an ocean multi-source sensing module collects ocean environment data in real time through a sensor and a combined collection scheme; the multi-source feature extraction module performs time domain, change rate and frequency domain feature analysis on the data to construct a unified multi-dimensional feature vector; the multi-model fusion prediction module outputs a marine environment state vector through dynamic weighting and deviation correction based on a parallel learning architecture of a deep neural network, a long-short-term memory network and a one-dimensional convolutional neural network; and the ocean risk identification and early warning module generates graded and classified early warning information through double study and judgment of a sea condition classifier and an abnormal event detector. According to the method, comprehensive acquisition, deep feature mining, high-precision prediction and accurate early warning of marine environment data are realized, the problems of low prediction precision, risk identification lag and the like in the prior art are effectively solved, and reliable guarantee is provided for marine operation safety.

The invention belongs to the technical field of aero-engine state monitoring and intelligent fault diagnosis, and discloses a multi-modal information fusion bearing fault diagnosis method based on self-supervised learning. The method comprises the following steps: firstly, through mask reconstruction self-supervision pre-training, extracting stable feature representation insensitive to mask disturbance from an unlabeled multi-modal signal, and dynamically updating each modal feature reference point by using an index moving average algorithm; in a downstream fault diagnosis task, a multi-modal joint decision model comprising a pre-training encoder, a single-modal classifier and a fusion classifier is constructed, and adaptive weighted fusion of multi-modal decision is realized through contribution degree calculation based on a cooperative game Shapley value in combination with a deviation degree of modal features and a reference point. According to the method, the dependence of the deep neural network on fault labeling data is effectively reduced, the accuracy and robustness of the diagnosis system in a multi-modal signal diagnosis scene are improved through a dynamic fusion mechanism, and the method is suitable for industrial scenes with limited sample label resources.

The invention relates to the technical field of water supply pipeline detection, and discloses an underground water supply pipeline health grade evaluation and risk prediction method, which comprises the following steps: sensor arrangement: arranging a flow sensor, a pressure sensor and a sonic sensor at key positions of an underground water supply pipeline; and data acquisition: acquiring signals of the sensor in real time through a data acquisition module, wherein the signals comprise flow, pressure and sound wave signals. Preprocessing the data: carrying out preprocessing such as denoising and normalization on the collected signals; and multi-source data fusion: inputting flow, pressure and sound wave signals into a deep neural network model, and performing feature extraction and fusion analysis. And health level assessment: assessing the health level of the pipeline based on the output result of the deep neural network model. And risk prediction: predicting abnormal working conditions possibly occurring in the future and risk levels of the abnormal working conditions by analyzing the current pipeline state and historical data. And abnormal positioning: accurately positioning an abnormal position in combination with the propagation time of the sensor signal and a positioning model of the deep neural network.

A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and / or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and / or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and / or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

The invention discloses an automobile seat framework machining defect detection method based on machine vision, and particularly relates to the technical field of defect detection. The method comprises the following steps: constructing a multi-angle image acquisition and edge reflection modeling module aiming at complex defects such as weld joint pseudo soldering, microcracks, hole site deviation and collapse deformation, extracting weld joint continuity, edge integrity and hole site geometric consistency characteristics by using a deep neural network, and generating a structural feature vector; defect type recognition and credibility scoring are completed through small sample anomaly modeling and Gaussian mixture model classification, sub-pixel-level coordinate labeling of defect positions is achieved in combination with a Gaussian fitting algorithm, a defect positioning map is output, traceability analysis and severity grading are conducted based on historical data comparison, and the defect positioning accuracy is improved. The method is suitable for industrial online detection and quality closed-loop control.

A system is disclosed for encoding a data string of a first modality into a hierarchical tokenized representation for processing by a text-based deep neural network (DNN) trained on a second modality. The data string comprises multiple units, each having one or more attributes. Each attribute is represented in the tokenized string as a sequence of hierarchical tokens, with a first hierarchical token encoding one or more most significant bits and a subsequent hierarchical token encoding one or more less significant bits. The DNN processes the data string bidirectionally, across the sequence of units and within the token hierarchy, to select tokens that capture attribute information. The selected hierarchical tokens output by the DNN from a representation of the original data string that preserves attribute detail while enabling cross-modal processing using models trained on text.

An offshore wind turbine integral coupling fatigue calculation method based on artificial intelligence belongs to the technical field of offshore wind power engineering, is used for solving the problem of efficient and high-precision evaluation of the fatigue life of an offshore wind turbine, and is characterized in that key environment parameter vectors influencing structural response and a data set of corresponding fatigue damage labels are acquired; training a deep neural network model by using the data set; the deep neural network model outputs corresponding fatigue damage per unit time in response to a central point parameter of an environment working condition block in an input long-term joint probability distribution diagram of the wind power site; according to the method, the defects that a traditional time domain simulation method is high in calculation cost and long in period are overcome, the calculation efficiency is improved by a plurality of orders of magnitude while the precision is guaranteed, and a powerful tool is provided for rapid design and optimization of the offshore wind turbine.

The invention belongs to the field of bioinformatics, and relates to an improved integrated deep learning cell communication ligand-receptor interaction prediction method. The method comprises the following steps: firstly, carrying out extraction and dimensionality reduction on biological sequence features of a ligand and a receptor, and constructing multi-modal feature input; secondly, constructing an improved deep neural network branch, introducing a batch normalization layer and a Leaky ReLU activation function, solving the problems of gradient disappearance and neuronal necrosis, and improving regularization strength to prevent overfitting; meanwhile, an enhanced heterogeneous graph auto-encoder branch is constructed, the graph embedding dimension is remarkably expanded to improve the feature capacity, and full convergence of the model is ensured by increasing training rounds; thirdly, fusing the improved deep network with the prediction probability of a heterogeneous graph auto-encoder by adopting a weighted integration strategy; and finally, outputting a potential interaction relationship based on the fusion probability. By optimizing the architecture and the strategy, the prediction accuracy and robustness are remarkably improved, and a reliable tool is provided for analyzing a complex cell communication network.

The invention provides a time-varying graph neural network traffic flow prediction method based on a dynamic memory bank, and belongs to the technical field of traffic prediction. The method adopts a layered deep neural network architecture, and comprises a data embedding layer, a space-time coding layer, a memory enhancement layer and a prediction output layer. The data embedding layer preprocesses a traffic flow sequence, associates a collaborative coding time sequence mode with a road network, and synchronously constructs a dynamic graph structure; the space-time coding layer is subjected to space-time stream decoupling extraction, a space branch models multi-scale space dependence through a time delay graph convolution module and a space Mama module, and a time branch extracts multi-granularity time features through a hierarchical time sequence sensing module and a time Mama module; the memory enhancement layer performs pattern matching and reconstruction on the space-time fusion features by means of a dynamic memory bank; and the prediction output layer generates a prediction result by taking the GCRN as a decoder. According to the method, the space-time dependence of the traffic situation is accurately captured, the prediction curve is highly fit with the true value, and the high-precision prediction of the traffic flow is realized.

The invention discloses an automatic driving track generation method and device, equipment and a medium. The method comprises the steps that target state information of a vehicle is obtained, and the target state information comprises vehicle body state information, obstacle information, road structure information and traffic state information; the target state information is input into a trajectory planning model for trajectory generation, a plurality of candidate driving trajectories and trajectory evaluation results corresponding to the candidate driving trajectories are determined, and the trajectory planning model is constructed based on a deep neural network; and determining a target driving trajectory based on the plurality of candidate driving trajectories and the trajectory evaluation result corresponding to each candidate driving trajectory. According to the method, the automatic driving track of the vehicle can be automatically and accurately generated, emergencies in dynamic traffic are covered, the generalization ability during cross-scene migration is improved, the track generation flexibility is improved, and therefore the track precision and safety in complex scenes are guaranteed.

The invention relates to the technical field of intelligent driving, in particular to a vehicle obstacle avoidance and path planning method based on intelligent driving, and provides the following scheme: the method comprises the following steps: collecting environmental data through a multi-modal sensor, constructing a probability space-time confidence field of a dynamic target, and determining the probability space-time confidence field of the dynamic target; and outputting a target category and geometric state probability distribution by using a Bayesian deep neural network, and generating a multi-modal future trajectory prediction set in combination with a historical trajectory. Furthermore, an evolution scene is constructed through trajectory sampling, forward simulation is performed on candidate self-trajectories, an optimal driving path is generated based on conditional value-at-risk assessment, and the optimal driving path is converted into a control instruction to drive the vehicle to run. The method has the uncertainty modeling capability and the high-risk identification capability, and the safety and robustness of path planning are improved.

The invention relates to the technical field of power system power distribution network fault detection, in particular to an arc fault diagnosis and analysis method based on an artificial intelligence algorithm, and the method comprises the four steps: synchronous data collection and topological excitation, deployment of terminals at transformer area nodes, injection of characteristic current, and synchronous collection of response waveforms; signal preprocessing: separating a key frequency band through double-digital band-pass filtering, and calculating energy ratio and other enhanced fault features; performing multi-source feature fusion and intelligent diagnosis, constructing a vector containing statistical features, time domain distortion features and topology identification, and inputting a gradient boosting decision tree and deep neural network hybrid model to obtain fault confidence and type; and based on fault positioning and verification of topology, scheduling multi-node cooperative monitoring, and determining a fault point in combination with a topological relation. The method improves the data reliability and diagnosis precision, achieves the precise positioning of a fault, and guarantees the safe operation of a power distribution network.

The invention discloses a generative odor real-time synthesis method and system based on cross-modal potential space mapping, and belongs to the technical field of artificial intelligence and olfaction calculation. The method comprises the following steps: acquiring a multi-modal input stream of a current scene, and extracting an emotion semantic feature vector by using a deep neural network; mapping the semantic features into target odor chemical feature vectors by using nonlinear projection through a pre-constructed vision-smell joint embedding space; constructing a convex optimization model based on olfactory perception, and calculating a basic liquid optimal mixing proportionality coefficient matrix capable of fitting the target vector; the matrix is converted into a micro-fluidic driving signal, and the target smell is synthesized in situ in the micro-fluidic chip. The invention further discloses a self-adaptive cleaning logic and olfactory fatigue compensation mechanism based on scene mutation detection. The method solves the problems that in the prior art, label matching is dependent, new smell cannot be synthesized, and dynamic transition is lacked, and olfactory replicating and real-time generation of abstract semantic scenes are achieved.

A programmable in-memory computing (IMC) accelerator for low-precision deep neural network inference, also referred to as PIMCA, is provided. Embodiments of the PIMCA integrate a large number of capacitive-coupling-based IMC static random-access memory (SRAM) macros and demonstrate large-scale integration of IMC SRAM macros. For example, a 28 nm prototype integrates 108 capacitive-coupling-based IMC SRAM macros of a total size of 3.4 megabytes (Mb), demonstrating one of the largest IMC hardware to date. In addition, a custom instruction set architecture (ISA) is developed featuring IMC and single-instruction-multiple-data (SIMD) functional units with hardware loop to support a range of deep neural network (DNN) layer types. The 28 nm prototype chip achieves a peak throughput of 4.9 tera operations per second (TOPS) and system-level peak energy-efficiency of 437 TOPS per watt (TOPS / W) at 40 megahertz (MHz) with a 1 volt (V) supply.

The invention discloses a method for estimating plant biomass based on map multi-modal feature extraction and fusion. The method comprises the following steps: acquiring a plant RGB image and a multi-spectral image; inputting the preprocessed RGB image and NIR wave band image into a double-model cooperation segmentation framework to realize image segmentation; binary image features, color features, texture features, reflectivity and the like are calculated, feature splicing is carried out, and high-dimensional features are constructed; carrying out dimension reduction on the high-dimensional features; and training a deep neural network through the effective features and the biomass to realize biomass estimation. According to the method, a zero sample learning-based double-model cooperation segmentation framework is utilized to realize accurate segmentation of a single plant on the premise that a large number of training sets are not needed; multi-modal feature information is extracted based on the segmented single plant image, an improved SHAP model is introduced to reduce the feature space dimension, and the inversion precision and the operation efficiency are improved while the information effectiveness is ensured; through a high-precision deep neural network model, rapid, lossless and accurate biomass acquisition is realized.

Systems and methods are provided for change detection in low-resolution video streams, which can be used for applications such as high resolution video restoration and processing. The techniques effectively detect changes by leveraging a large receptive field and lightweight computation, which are achieved by working with low-resolution images. In particular, the techniques include extracting features from a change detection model and a semantic segmentation model, and integrating the extracted feature outputs from the models to produce a robust change detection map. A pre-processing phase can be employed to optimize the input for each model, ensuring minimal complexity and enhanced performance. The change detection model can be implemented as a deep neural network, and methods are provided for generating ground truth (GT) data, which semantically guides the change detection neural network to perform change detection inpainting during training.

The invention relates to the field of multi-agent control, and provides an unmanned aerial vehicle patrol flight obstacle avoidance method based on deep reinforcement learning. Aiming at the problems of large work amount, incomplete coverage and poor dynamic obstacle response caused by manual route configuration due to dependence on point cloud data path planning in the prior art, the scheme fuses target detection and radar ranging to obtain obstacle types, positions and distance information; constructing a local obstacle avoidance potential field based on an artificial potential field method, calculating an obstacle avoidance direction through a gravitational repulsion gradient, and introducing a dynamic adjustment mechanism to avoid a local minimum value; designing a local dynamic reward function in combination with the target equipment position, and generating a path adjustment signal by integrating the target approaching progress of the unmanned aerial vehicle and the obstacle avoidance demand; a global reward function is constructed to evaluate the task completion degree, the path efficiency and the obstacle avoidance safety, and a macroscopic strategy optimization basis is formed; a deep neural network architecture is adopted to train an autonomous obstacle avoidance strategy, and an autonomous decision in a dynamic environment is realized through experience playback and an attention mechanism optimization model.

The invention relates to the technical field of resource scheduling and enterprise intelligent training, and discloses a cross-modal closed-loop intelligent training optimization method and system. According to the method, multi-platform data of organization management, project execution, enterprise communication and the like are integrated, and a standardized sequence is generated through missing value supplementation and time axis alignment; carrying out multi-modal feature fusion by adopting a deep neural network, and constructing personalized portrait parameters; generating a dynamic training path based on post key item matching, and extracting a capability evaluation index and a trend parameter by executing verification; and finally, closed-loop optimization is formed through differential mapping, and a privacy protection model is generated in combination with adversarial training. According to the invention, the dynamic adaptation between the training scheme and the employee ability is realized, the problems of data islands and insufficient personalization in traditional training are solved, the training effect evaluation accuracy is significantly improved through a closed-loop feedback mechanism, and the data privacy security is ensured by using federal learning.

The invention belongs to the field of computer vision, and discloses a plant disease identification method and system based on a three-branch deep neural network, and the method comprises the steps: firstly extracting global, local and middle-layer multi-scale positioning features through employing Vision Transformer, ResNet50 and YOLO in parallel; secondly, aligning and fusing the three paths of features, carrying out interactive enhancement through Transform block, and then respectively predicting crop types and disease types through a double-end stacking classifier; and finally, by setting a proper loss function combination and a joint training strategy, performing end-to-end optimization on the whole network, so as to improve the recognition capability of difficult-to-judge samples and small-scale lesions. According to the invention, accurate identification of plant diseases in a natural scene can be realized. Experimental results show that the method provided by the invention obtains classification performance equivalent to or even better than that of the most advanced plant disease identification method in the prior art.

The invention relates to the technical field of intelligent driving, and particularly provides a personalized driving behavior identification method based on deep embedded clustering. Extracting a low-dimensional depth feature vector of the driving time sequence segment through a deep neural network auto-encoder, performing end-to-end joint optimization on the auto-encoder and unsupervised clustering by using a deep embedded clustering method, generating a driving style clustering result, and labeling a clustering result label; freezing the trained auto-encoder parameters, constructing a driving style classification model, and establishing a mapping relation between driving time sequence fragments and style labels; and distributing semantic tags for clustering results by extracting interpretable features. According to the end-to-end joint optimization technology based on deep embedded clustering, the convergence speed and generalization ability of the model are remarkably improved, and the driving style identification precision is far higher than that of a traditional machine learning method.

The invention relates to a risk identification method and system. The method comprises the following steps: acquiring communication behaviors, equipment fingerprints and service interaction data in real time through a privacy compliance interface; carrying out parallel preprocessing and safe desensitization on the data; inputting a feature construction engine to extract a communication mode, an equipment behavior and an interactive semantic feature vector in parallel, constructing a dynamic weighted hypergraph communication map, and outputting a social risk feature vector by using a time sequence hypergraph neural network; inputting the four types of features into a privacy perception multi-mode gating attention fusion module to output fusion features; generating a risk score through a deep neural network classifier; and monitoring an abnormal event based on Apache Flink, adaptively adjusting a decision boundary in combination with the dynamic risk entropy, and triggering secondary verification. According to the method, the problems of multi-modal data conflict, privacy disclosure and decision stiffness are solved, and the recognition precision and the real-time performance are improved.

The invention discloses an aero-engine combustion chamber ignition state prediction method and system based on deep learning. The method comprises the steps that a combustion chamber ignition parameter data set is collected; analyzing sample distribution to determine an augmentation amount, generating a new sample based on KNN interpolation, injecting adaptive Gaussian noise, and referring to physical relevance between a fuel-air ratio and a temperature-pressure ratio of a real sample during augmentation of a special working condition; an AttResVGG deep neural network is constructed, gradient disappearance is prevented by adopting residual connection, a multi-head self-attention mechanism is embedded to capture a long-range dependency relationship among parameters, and multi-scale features are integrated through a global feature fusion module; double classifier training is designed, a main classifier adopts category weighted cross entropy loss, an auxiliary classifier adopts label smooth loss, and model parameters are jointly optimized; and inputting working condition parameters for real-time prediction. According to the invention, parameter dependence of a traditional empirical model is broken through, and adaptive prediction of complex working conditions is realized; the calculation time consumption is reduced, and the real-time decision demand is met; and the device adapts to various combustion chamber structures, and re-modeling is not needed.

The invention relates to a multi-target aerodynamic-structure optimization method based on a deep neural network proxy model, and solves the problem that an existing method is insufficient in efficiency, feasibility and credibility, and the method comprises the steps: firstly, carrying out the sampling in a parameterized design target, building an aerodynamic-structure database through CFD / FEA high-fidelity simulation; the method comprises the steps that firstly, a multi-task neural network is used for predicting the lift-drag ratio, quality and stress at the same time, uncertainty is output, then, in NSGA-III multi-target optimization, a proxy model is used for rapidly evaluating and searching out a Pareto leading edge meeting constraints, finally, the precision of the model is gradually improved through high-fidelity checking and incremental updating, and an optimal design scheme is output. According to the method, sub-optimization and repeated iteration caused by'pneumatic first and then structure 'are avoided; the design universality and robustness are improved; a compromise scheme set of the system is directly obtained; and the optimization cost and the optimization period can be obviously reduced.

The invention provides an intelligent decision-making method and system based on adaptive cross-modal attention, and relates to the technical field of data processing, and the method comprises the steps: obtaining multi-modal data; performing preprocessing on the multi-mode data; extracting original semantic features in each mode from the preprocessed multi-mode data through a deep neural network; calculating the cross-modal attention of each original semantic feature; according to the cross-modal attention, performing cross-modal fusion on each original semantic feature to obtain a fusion feature; and according to the fusion features, intelligent decision making is carried out through a full-connection network. According to the cross-modal attention mechanism, information of different modals can be dynamically weighted according to the importance of each modal, and when the data quality of a certain modal is poor or is completely missing, the weight of the modal can be adaptively reduced, so that the interference of irrelevant or repeated information is avoided, the feature fusion effect is enhanced, and the decision accuracy is improved.

A system for learning optimal driving behavior for autonomous vehicles comprises a deep neural network, a first stage training module, and a second stage training module. The deep neural network comprises a feature learning network configured to receive sensor data from a vehicle as input and output spatial temporal feature embeddings and a decision action network configured to receive the spatial temporal feature embeddings as input and output an optimal driving policy for the vehicle. The first training stage module is configured to, during a first training stage, train the feature learning network using object detection loss. The second stage training module is configured to, during a second training stage, train the decision action network using reinforcement learning.