The invention discloses a detecting system and a detecting method for fatigue driving of a driver, and belongs to the technical fields of image processing and pattern recognition. The system comprises a human face localization module, a human eye state judging module, a mouth state judging module and a fatigue determining module, wherein the human face localization module is connected with the human eye state judging module and the mouth state judging module for communication; the determining result is sent to the fatigue determining module. The detecting method comprises the following steps: 1, localizing and detecting the human face; 2, judging the human eye state; 3, judging the mouth state of the driver; 4, judging the mental state of the driver. The detecting system and the detecting method combine two obvious facial characteristics of the driver, namely the human eyes and the human mouth, for complex judging; compared with recognition detection using a single method, the method provided by the invention has a higher recognition accuracy; through determining the mental state of the driver, the driver under fatigue state is reminded to stop the vehicle for rest, so that traffic accidents are effectively reduced, and the safety of life and property is strongly guaranteed.