The invention discloses a method for automatically generating a medical image diagnosis report based on the deep learning method. The method comprises steps that S1, a subject of the diagnosis reportis clustered based on the LDA algorithm, and the diagnosis report is saved separately according to the subject; S2, a subject vector is used as a label of each medical image; S3, CT images and PET images with different sizes are scaled to the same size as training data, subject vectors are used as labels, the VGGNet-19 is used as a network model for training, and a subject vector generation modelis obtained; S4, a text generation model is constructed; and S5, according to the subject vector of each image, the text of the corresponding subject is matched to obtain the diagnosis report of the image. The method is advantaged in that the method can be applied to scenes with images marked with lesions, a doctor has no need to manually summarize training data labels frequently, only the location and the size of the lesion should be marked, and the doctor's work is effectively reduced while the correct rate is improved.