The invention discloses an urban road scene semantic segmentation method based on deep learning. The method comprises the following steps: 1) collecting an image at the front end of a vehicle; 2) input data expansion of the annotated image and the original image: randomly cutting, splicing or adding different types of noise to the image, transforming the image through an image affine matrix, and finally maintaining the original resolution of the image through filling, cutting and other transformation to obtain a data set; 3) training a network by using the image after data expansion and the annotation image, wherein the residual U-net network comprises a down-sampling part, a bridge part, an up-sampling part and a classification part; and 4) modifying the time interval T of the acquisitionmodule, inputting subsequently obtained images into the trained deep learning model, outputting predicted semantic segmentation images, and returning different gray levels in the images to the processor. A small data set is used, too fast gradient descent can be prevented, and it can be ensured that an over-fitting problem does not occur during training.