The invention discloses a language recognition system used for the deaf and the dumb. The language recognition system comprises skin sensors, a data acquisition module, a data analysis module, a master control module, a model matching module and a voice module, wherein the skin sensors, the data acquisition module, the data analysis module, the master control module, the model matching module andthe voice module are electrically connected in sequence, the skin sensors comprise an upper lip sensor, a lower lip sensor and a throat sensor, and the language recognition system further comprises akey module, a model library module, a voice recognition module and a display module. The invention further provides a language recognition method. The invention provides the system which can be carried by the deaf and the dumb and realizes direct communication with normal people through language, the information processing conversion can be carried out rapidly, the in-depth learning function is provided, the expression habits of users can be gradually adapted, and certain instantaneity and practicability are also available.