A worker fatigue detection method and apparatus

Abstract

The application discloses a kind of staff fatigue detection method and device, the method includes the following steps: step one, install visible light imaging module and near-infrared imaging module;Step two, carry out double target definition;Step three, visible light imaging module and near-infrared imaging module synchronous acquisition the face image of the staff to be detected;Step four, respectively to the visible light image and near-infrared image obtained face detection;Step five, according to face detection result, carry out living body detection;Step six, carry out face posture estimation and human eye feature point detection;Step seven, respectively to visible light and near-infrared face image carries out fatigue state identification;Step eight, according to face posture, the fatigue state identification result of visible light image and near-infrared image is weighted average, judges fatigue state, outputs fatigue detection result.In the application, first carry out living body identification, after being determined as non-false face, fatigue state detection is carried out again, improve the effectiveness, accuracy and reliability of fatigue detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing and video monitoring, and particularly relates to a staff fatigue detection method and device. BACKGROUND

[0002] In recent years, with the rapid development of artificial intelligence technology, target automatic detection and recognition using cameras have been widely applied. Among them, the detection of the working state of personnel, especially the detection of fatigue state, has been focused on. The detection of the fatigue state of drivers is widely applied in the field of transportation, and the demand for the detection of the fatigue state of personnel working in other industries requiring attention is also increasing.

[0003] At present, the existing solutions are basically concentrated on the algorithm level of video image processing. For example, a Chinese invention patent with publication number CN105825631A discloses a fatigue detection method and system based on a video intelligent algorithm. The system includes multiple fatigue detection devices, a monitoring center and a monitoring client. The opening and closing degrees of the eyes and mouth of the staff are determined by processing the video images, and the frequency of the opening and closing degrees of the eyes and mouth of the staff reaching a threshold value within a certain time is calculated by weighted average to determine whether the personnel are fatigued. A Chinese invention patent with publication number CN103315754B discloses a fatigue detection method and device. It is proposed to use a self-provided light source to irradiate the face of the measured object, use a camera to obtain a reflection image, intercept an eye image, and then compare the eye image with a template in a database to determine whether the personnel are in a fatigued state. A Chinese invention patent with publication number CN102122357B discloses a fatigue detection method based on the opening and closing state of the human eye. It is proposed to identify the opening and closing state of the human eye from the contour shape of the human eye through image processing, and determine whether the personnel are in a fatigued state from the statistical information of the opening and closing state of the human eye within a unit time. This kind of technology is relatively mature, and can meet the demand for staff fatigue detection in conventional application scenarios. However, in some specific conditions, the staff may use a paper printed human face image to face the camera, or use a tablet computer or mobile phone to play pictures or videos to face the camera to cheat the fatigue detection system, resulting in invalid fatigue detection results. SUMMARY

[0004] To solve the technical problems existing in the prior art, the purpose of the present application is to provide a staff fatigue detection method and device.

[0005] To achieve the above-mentioned purposes and achieve the above-mentioned technical effects, the technical solution adopted by the present application is as follows:

[0006] A staff fatigue detection method, comprising the following steps:

[0007] Step one, according to different application scenarios, visible light imaging module and near-infrared imaging module are installed in the face of the detected person in the form of left and right or up and down, to form a binocular stereo vision system;

[0008] Step two, binocular stereo vision system is calibrated;

[0009] Step three, visible light imaging module and near-infrared imaging module synchronously collect the face image of the detected person, and respectively obtain visible light image and near-infrared image;

[0010] Step four, visible light image and near-infrared image are respectively detected;

[0011] Step five, according to the face detection result obtained in step four, live detection is carried out to judge whether the face in the current image is a false face;

[0012] Step six, face posture estimation and eye feature point detection are carried out;

[0013] Step seven, visible light and near-infrared face image are respectively recognized for fatigue state;

[0014] Step eight, visible light image and near-infrared image fatigue state recognition results are weighted and averaged according to face posture, fatigue state is judged, and worker fatigue detection result is output.

[0015] In the worker fatigue detection method provided by the application, in step one, the visible light imaging module includes at least one visible light camera, and the near-infrared imaging module includes at least one near-infrared camera; when the detected person turns his head left and right due to work needs, the visible light imaging module and the near-infrared imaging module are installed in the face of the detected person in the form of left and right; when the detected person turns his head up and down due to work needs, the visible light imaging module and the near-infrared imaging module are installed in the face of the detected person in the form of up and down.

[0016] In the worker fatigue detection method provided by the application, in step five, the live detection step includes:

[0017] Step 5.1, it is judged whether the face is detected in the visible light image and the near-infrared image in step four, if the face is detected in the visible light image and the near-infrared image, step 5.2 is turned to, otherwise, the detection result is a false face, the fatigue detection process of the image is ended, and the output is no worker detected;

[0018] Step 5.2, the face width is calculated, when the face width meets the set threshold range, step 5.3 is turned to, otherwise, the detection result is a false face, the fatigue detection process of the image is ended, and the output is no worker detected;

[0019] Step 5.3, face liveness detection is performed on the visible light face image and the near-infrared face image, if the detection result is a living body, then go to step six, otherwise, the detection result is a false face, end the fatigue detection processing of the group of images, and output that no worker is detected.

[0020] In the worker fatigue detection method provided by the application, in step 5.2, the step of calculating the face width comprises:

[0021] The face contour edge features are extracted from the regions where the faces are detected in the visible light image and the near-infrared image, the face contour edge features in the two images are stereomatched, the three-dimensional data of the face contour edge are calculated by using the binocular stereovision system in step one and the calibration parameters in step two, and the width of the face can be calculated according to the three-dimensional data.

[0022] The calculation formula of the three-dimensional data is:

[0023]

[0024]

[0025] wherein, is the visible light imaging module image coordinate of a feature point of the face contour edge, is the near-infrared imaging module image coordinate of the same feature point of the face contour edge, M11 is the internal parameter matrix of the visible light imaging module, M12 is the external parameter matrix of the visible light imaging module, M21 is the internal parameter matrix of the near-infrared imaging module, and M22 is the external parameter matrix of the near-infrared imaging module, is the actual three-dimensional coordinate of the point on the face contour edge to be solved, and the least square method is used to solve.

[0026] In the worker fatigue detection method provided by the application, in step 5.3, the step of performing face liveness detection on the visible light face image and the near-infrared face image comprises:

[0027] According to the face detection result obtained in step four, the visible light face sub-image and the near-infrared face sub-image are extracted from the visible light image and the near-infrared image respectively according to the face frame; the visible light face sub-image and the near-infrared face sub-image are normalized respectively and input into the trained convolutional neural network face detection model for liveness detection, and the liveness detection result is output.

[0028] In the staff fatigue detection method provided by the application, the convolutional neural network face detection model comprises two sub-networks, wherein the first sub-network is: performing convolutional processing on the normalized visible light face sub-image to obtain a visible light face feature map, performing convolutional processing on the normalized near-infrared face sub-image to obtain a near-infrared face feature map, and merging the visible light face feature map and the near-infrared face feature map and outputting a merged feature map; and the second sub-network is: performing multi-layer convolutional processing on the merged feature map, and finally adopting a full connection layer or a 1*1 convolutional layer to output the class of the living body detection in the form of a classification network to obtain the final living body detection result.

[0029] In the staff fatigue detection method provided by the application, in step six, the face posture estimation step comprises:

[0030] Three-dimensional data of a face contour feature edge is collected, a plane is fitted to the three-dimensional data of the face contour feature edge, defined as a face plane, and a normal vector of the face plane is solved as a face posture vector.

[0031] In the staff fatigue detection method provided by the application, in step eight, the face posture parameter is defined as an included angle between the face posture vector and the optical axis of the imaging module, and the smaller the included angle, the more the face is directly opposite the visible light imaging module and the near-infrared imaging module; and the posture parameters of the face and the visible light imaging module and the near-infrared imaging module are calculated according to the camera external parameters calibrated in step two.

[0032] The application further provides a staff fatigue detection device, comprising a control module, a processing module, a visible light imaging module and a near-infrared imaging module, wherein the visible light imaging module and the near-infrared imaging module are installed opposite a face of a detected person in a left-right or up-down distribution mode, the control module is connected with the visible light imaging module and the near-infrared imaging module respectively, the control module is used for controlling the visible light imaging module and the near-infrared imaging module to synchronously collect image data, and the processing module is connected with the visible light imaging module and the near-infrared imaging module respectively, the processing module is used for receiving the image data collected by the visible light imaging module and the near-infrared imaging module and performing data processing to obtain a fatigue detection result.

[0033] In the staff fatigue detection device provided by the application, the visible light imaging module comprises at least one visible light camera, and the near-infrared imaging module comprises at least one near-infrared camera; when the head of the detected person is mostly turned left and right due to work needs, the visible light camera and the near-infrared camera are installed opposite the face of the detected person in a left-right distribution mode; and when the head of the detected person is mostly raised and lowered due to work needs, the visible light camera and the near-infrared camera are installed opposite the face of the detected person in an up-down distribution mode.

[0034] Compared with the prior art, the present application has the following beneficial effects:

[0035] 1. By performing living body recognition on the detected personnel first to exclude false working state of the detected personnel, fatigue state detection is performed after determining that the face is not false, the effectiveness, accuracy and reliability of fatigue detection can be improved.

[0036] 2. A binocular stereo vision system is formed by combining a visible light camera with a near-infrared camera, the imaging characteristics of the near-infrared camera, such as the fact that the screen of a tablet computer and a mobile phone cannot be imaged and different materials have different reflectivities, are utilized, and the size of the face is calculated in combination with the binocular stereo vision system, so that the living body detection capability of the camera can be effectively improved under the condition that the cost is not increased much, and the effect of working state detection is ensured.

[0037] 3. The binocular stereo vision system can increase the overall field of view angle of detection, fatigue detection is performed on the face from two directions, the image of the eye opening and closing state has a good shooting angle, the detection rate of fatigue state detection can be improved, and the false alarm rate can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0038] Fig. 1 is a flowchart of the present application;

[0039] Fig. 2 is a structural schematic view of the present application when the visible light camera and the near-infrared camera are separately arranged at the upper front and left and right sides of the face;

[0040] Fig. 3 is a structural schematic view of the present application when the visible light camera and the near-infrared camera are separately arranged at the upper front and left and right sides of the face. DETAILED DESCRIPTION

[0041] The present application will be described in detail below so that the advantages and features of the present application can be more easily understood by those skilled in the art, and the scope of protection of the present application can be more clearly defined.

[0042] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0043] As Figs. 1-3 shown, a working personnel fatigue detection method comprises the following steps:

[0044] Step one, according to different application scenarios, visible light imaging module and near-infrared imaging module are installed in the face of the detected person 3 in the form of left and right or up and down, forming a binocular stereo vision system;

[0045] Among them, the visible light imaging module includes at least one visible light camera 1, and the near-infrared imaging module includes at least one near-infrared camera 2; When the detected person 3 turns his head left and right due to work needs, the visible light imaging module and the near-infrared imaging module are installed in the face of the detected person 3 in the form of left and right; When the detected person turns his head up and down due to work needs, the visible light imaging module and the near-infrared imaging module are installed in the face of the detected person 3 in the form of up and down.

[0046] Step two, double target positioning is performed on the binocular stereo vision system.

[0047] Step three, the visible light imaging module and the near-infrared imaging module synchronously collect the face image of the detected person 3, and respectively obtain a visible light image and a near-infrared image.

[0048] Step four, face detection is respectively performed on the visible light image and the near-infrared image.

[0049] Step five, according to the face detection result obtained in step four, live body detection is performed to judge whether the face in the current image is a false face; wherein, the steps of live body detection include:

[0050] Step 5.1, judge whether the visible light image and the near-infrared image in step four both detect a face, if both detect a face, go to step 5.2; otherwise, the live body detection result is a false face, end the fatigue detection processing of this group of images, and output no detected worker;

[0051] Step 5.2, calculate the face width by using binocular stereo vision method to judge whether it is a live body;

[0052] Step 5.3, the visible light face image and the near-infrared face image are detected by a convolutional neural network face detection model for face live body detection, if the detection result is a live body, go to step six; otherwise, the live body detection result is a false face, end the fatigue detection processing of this group of images, and output no detected worker.

[0053] Step six, face pose estimation and eye feature point detection are performed.

[0054] Step seven, fatigue state recognition is respectively performed on the visible light and near-infrared face images.

[0055] Step eight, the fatigue state recognition results of the visible light image and the near-infrared image are weighted and averaged according to the face pose to judge the fatigue state and output the worker fatigue detection result.

[0056] The application also discloses a staff fatigue detection device, which comprises a master control module 4, a visible light imaging module and a near-infrared imaging module; wherein the visible light imaging module and the near-infrared imaging module are installed opposite to the face of the detected person 3 in a left-right or up-down distribution manner, the master control module 4 comprises a control module and a processing module, the control module is connected with the visible light imaging module and the near-infrared imaging module, the control module is used for controlling the visible light imaging module and the near-infrared imaging module to synchronously collect image data, the processing module is connected with the visible light imaging module and the near-infrared imaging module, the processing module is used for receiving the image data collected by the visible light imaging module and the near-infrared imaging module and performing data processing, and the calibration parameters of the visible light imaging module and the near-infrared imaging module are saved in the processing module. In application, the control module outputs a synchronous trigger signal to the visible light imaging module and the near-infrared imaging module, the visible light imaging module and the near-infrared imaging module simultaneously collect face images and upload the face images to the processing module, the processing module performs face detection, living body detection, face posture estimation, human eye feature point detection and fatigue state detection after obtaining the face images transmitted by the visible light imaging module and the near-infrared imaging module, and finally outputs the fatigue state of the detected person 3.

[0057] The visible light imaging module comprises at least one visible light camera 1, and the near-infrared imaging module comprises at least one near-infrared camera 2. In actual deployment, when the head of the detected person 3 is mostly turned left and right due to work needs, the visible light camera 1 and the near-infrared camera 2 are deployed according to Fig. 2 The visible light camera 1 and the near-infrared camera 2 are separately arranged on the left and right sides above the front of the face and form a binocular stereo vision system in a light axis converging manner, so that when the face is turned left and right, one camera can face the face as much as possible, the human eyes are larger and clearer in the image, and the effectiveness and reliability of fatigue detection are ensured; when the head of the detected person is mostly raised and lowered due to work needs, the visible light camera 1 and the near-infrared camera 2 are deployed according to Fig. 3 The visible light camera 1 and the near-infrared camera 2 are separately arranged on the left and right sides above the front of the face and form a binocular stereo vision system in a light axis converging manner, so that when the face is turned left and right, one camera can face the face as much as possible, the human eyes are larger and clearer in the image, and the effectiveness and reliability of fatigue detection are ensured; when the head of the detected person is mostly raised and lowered due to work needs, the visible light camera 1 and the near-infrared camera 2 are deployed according to

[0058] Embodiment 1

[0059] As shown in Figs. 1-3 A staff fatigue detection method comprises the following steps:

[0060] Step one, visible light camera 1 and near-infrared camera 2 are installed in front of the face of the detected person 3 in the form of left and right or up and down distribution;

[0061] Specifically, when the detected person 3 turns his head left and right due to work needs, the visible light camera 1 and the near-infrared camera 2 are arranged according to Fig. 2 The visible light camera 1 and the near-infrared camera 2 are arranged on the left and right sides of the face, and form a binocular stereo vision system in the form of converging optical axes, so that when the face turns left and right, one camera can face the face as much as possible, and the eye area in the image is larger and clearer, to ensure the effectiveness and reliability of fatigue detection; when the detected person turns his head up and down due to work needs, the visible light camera 1 and the near-infrared camera 2 are arranged according to Fig. 3 The visible light camera 1 and the near-infrared camera 2 are arranged on the left and right sides of the face, and form a binocular stereo vision system in the form of converging optical axes, so that when the face turns left and right, one camera can face the face as much as possible, and the eye area in the image is larger and clearer, to ensure the effectiveness and reliability of fatigue detection; when the detected person turns his head up and down due to work needs, the visible light camera 1 and the near-infrared camera 2 are arranged according to

[0062] Step two, the binocular stereo vision system is calibrated, and the internal and external parameters of the two cameras are obtained. The known methods such as Zhang Zhengyou calibration method can be used to achieve this. The calibration method is as follows: first, use a chessboard plane target, place it in the field of view of the two cameras, preferably in the region where the face of the detected person 3 is located, and change the position and attitude of the target. The two cameras respectively shoot more than 20 chessboard images; extract the corner points of each chessboard image, and the corner point coordinates are sub-pixel values; calculate the homography matrix of each camera according to the image coordinates and the actual world coordinate system coordinates, and obtain the internal parameters of each camera. After iterative correction, the optimal solution is obtained. The external parameters are calculated according to the homography matrix and the internal parameters of the camera.

[0063] Step three, the visible light camera 1 and the near-infrared camera 2 synchronously collect the face image of the detected person 3, and obtain the visible light image and the near-infrared image respectively.

[0064] Step four, the visible light image and the near-infrared image are respectively detected for face detection. The face detection method can use the known methods such as face detection method based on classifier and face detection method based on deep learning convolutional neural network.

[0065] Step five, according to the face detection results of visible light and near-infrared, binocular face living body detection is carried out to judge whether the face in the current image is a false face. The steps of living body detection include:

[0066] Step 5.1, judge whether the visible light image and the near-infrared image in step four detect the face. If both detect the face, go to step 5.2; otherwise, the living body detection result is a false face, end the fatigue detection process of this group of images, and output no detected worker;

[0067] It should be noted that this step mainly uses the near-infrared camera 2 to image the screen of the tablet computer, mobile phone and the like which cannot be imaged by the screen, and the different reflectivity of different materials to form a weak imaging feature of part of the printed material, and to exclude human fraud on the fatigue detection system.

[0068] Step 5.2, calculate the face width by using binocular stereo vision method, and judge whether it is a living body. Specifically, the face contour edge feature of the region where the face is detected in the visible light image and the near-infrared image is extracted, and the face contour edge features in the two images are stereomatched, and the three-dimensional data of the face contour edge is calculated by using the binocular stereo vision system in step one and the calibration parameters in step two, and the calculation formula of the three-dimensional data is:

[0069]

[0070]

[0071] The above formula is the imaging model of the two cameras, wherein, is the image coordinate of a feature point of the face contour edge of the visible light camera 1, is the image coordinate of the same feature point of the face contour edge of the near-infrared camera 2, M11 is the internal parameter matrix of the visible light camera 1, M12 is the external parameter matrix of the visible light camera 1, M21 is the internal parameter matrix of the near-infrared camera 2, and M22 is the external parameter matrix of the near-infrared camera 2, is the actual three-dimensional coordinate of the point on the face contour edge to be solved, and the least square method can be used to solve

[0072] After the three-dimensional coordinates of the points on both sides of the face contour edge are calculated, the distance between the two points is calculated to obtain the face width, and when the face width meets the set threshold range, the process goes to step 5.3; otherwise, the living body detection result is a false face, and the fatigue detection process of the image group is ended, and the working staff is output as not detected;

[0073] Step 5.3, the visible light face image and the near-infrared face image are detected by a convolutional neural network face detection model, and if the detection result is a living body, the process goes to step six; otherwise, the living body detection result is a false face, and the fatigue detection process of the image group is ended, and the working staff is output as not detected;

[0074] Specifically, according to the face detection result of step four, the visible light face sub-image and the near-infrared face sub-image are extracted from the visible light image and the near-infrared image respectively according to the face frame; the visible light face sub-image and the near-infrared face sub-image are normalized and input into the trained convolutional neural network face detection model for living body detection, and the living body detection result is output.

[0075] The convolutional neural network face detection model includes two sub-networks. The first sub-network is: performing convolutional processing on the normalized visible light face sub-image to obtain a visible light face feature map, performing convolutional processing on the normalized near-infrared face sub-image to obtain a near-infrared face feature map, and merging the visible light face feature map and the near-infrared face feature map and outputting a merged feature map. The second sub-network is: performing multi-layer convolutional processing on the merged feature map, and finally outputting the class of the living body detection in the form of a classification network by using a full connection layer or a 1*1 convolutional layer to obtain a final living body detection result. The convolutional neural network face detection model can be implemented by using existing classical CNN convolutional network structure units, such as ResNet, MobileNet, DenseNet, and the like.

[0076] Step six, performing face pose estimation, calculating the relative pose parameters of the face and the visible light camera 1 and the face and the near-infrared camera 2 according to the face pose value, and describing the degree to which the current detected face faces the camera;

[0077] Performing eye feature point detection in the visible light image and the near-infrared image, respectively;

[0078] Among them, the face pose estimation can be implemented in multiple ways. Method one: according to the binocular stereo vision system composed of the visible light camera 1 and the near-infrared camera 2, fitting a plane according to the three-dimensional data of the face contour feature edge extracted in step 5.2, defining the plane as a face plane, and defining the normal vector of the face plane as a face pose vector. Method two: using a deep learning CNN network to implement end-to-end face pose estimation, such as ASMNet, img2pose, 3DDFA_V2, and the like. Method three: selecting a reference coordinate system through the face key points on the two-dimensional image, calculating the transformation matrix of the key points and the reference coordinate system, and then estimating the face pose through an iterative optimization method.

[0079] Among them, the relative pose parameters of the face and the camera are defined as: the included angle between the face pose vector and the camera optical axis. The smaller the included angle, the more the face faces the camera. According to the camera external parameters calibrated in step two, the relative pose parameters of the face and the visible light camera 1 and the near-infrared camera 2 are calculated, respectively.

[0080] Among them, the eye feature point detection can be implemented by using an existing mature deep learning face key point detection network, such as PFLD, PIPNet, HRNet, and the like. The key points of the eyes can be saved from the face key points as feature points.

[0081] Step seven, visible light and near-infrared image fatigue state recognition;

[0082] The fatigue state recognition is respectively performed on the visible light and near-infrared face images, and the fatigue state recognition can be specifically implemented in the following methods: method one, according to the eye feature points detected from the visible light and near-infrared face images in step six, the eye width-height ratio is calculated, and when the width-height ratio is greater than a set threshold, the current frame is determined as the closed-eye fatigue; method two, according to the eye feature point image coordinate values, an eye sub-image is cut out, the eye sub-image is taken as an input, a deep learning CNN classification network is trained, and an end-to-end fatigue state judgment result is output.

[0083] Step eight, the fatigue state recognition results of the visible light and near-infrared images are weighted and averaged according to the face posture, and the fatigue state is judged. Specifically, the weights of the weighted average are defined according to the face and the relative posture parameters of each camera calculated in step six, and when the posture parameter is smaller, that is, the face is more directly facing the camera, the weight is larger, and vice versa, the weight is smaller. The result after the weighted average is taken as the judgment result of the fatigue state of the current frame.

[0084] Preferably, a multi-frame continuous judgment mode can be used, when the fatigue state quantity in the camera video sequence exceeds a set threshold, it is considered that the detected person 3 is in a fatigue state.

[0085] The parts or structures not specifically described in the present application can be realized by using the prior art or existing products, and will not be described here.

[0086] The above description is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A worker fatigue detection method characterized by, The method comprises the following steps: Step one, according to different application scenarios, the visible light imaging module and the near-infrared imaging module are installed on the face of the detected person in a left-right or up-down distribution manner to form a binocular stereo vision system; Step two, the binocular stereo vision system is calibrated; Step three, the visible light imaging module and the near-infrared imaging module synchronously collect the face image of the detected person to obtain a visible light image and a near-infrared image respectively; Step four, the visible light image and the near-infrared image are detected respectively; Step five, according to the face detection result obtained in step four, live body detection is performed to determine whether the face in the current image is a false face; Step six, face posture estimation and eye feature point detection are performed; Step seven, the visible light image and the near-infrared image are detected respectively to identify the fatigue state; Step eight, the visible light image and the near-infrared image fatigue state identification results are weighted and averaged according to the face posture to determine the fatigue state and output the fatigue detection result of the worker; In step one, the visible light imaging module comprises at least one visible light camera, and the near-infrared imaging module comprises at least one near-infrared camera; when the head of the detected person is mostly turned left and right due to work needs, the visible light camera and the near-infrared camera are separately arranged on the left and right sides above the front of the face to form a binocular stereo vision system in a light axis converging manner; when the head of the detected person is mostly raised and lowered due to work needs, the visible light camera and the near-infrared camera are separately arranged on the upper and lower sides in front of the face to form a binocular stereo vision system in a light axis converging manner; In step five, the steps of live body detection comprise: Step 5.1, it is determined whether the visible light image and the near-infrared image both detect the face in step four, if both detect the face, step 5.2 is performed, otherwise, the detection result is a false face, the fatigue detection process of the visible light image and the near-infrared image is ended, and a worker is outputted who is not detected; Step 5.2, the face width is calculated, when the face width meets the set threshold range, step 5.3 is performed, otherwise, the detection result is a false face, the fatigue detection process of the visible light image and the near-infrared image is ended, and a worker is outputted who is not detected; Step 5.3, face live body detection is performed on the visible light face image and the near-infrared face image, if the detection result is a live body, step six is performed, otherwise, the detection result is a false face, the fatigue detection process of the visible light image and the near-infrared image is ended, and a worker is outputted who is not detected; In step 5.2, the steps of calculating the face width comprise: The face contour edge features are extracted in the face detection area in the visible light image and the near-infrared image, the face contour edge features in the two images are stereoscopically matched, the three-dimensional data of the face contour edge are calculated by using the binocular stereo vision system in step one and the calibration parameters in step two, and the face width is calculated according to the three-dimensional data; The calculation formula of the three-dimensional data is: ； wherein, is the image coordinate of a certain feature point of the face profile edge of the visible light imaging module, is the image coordinate of the same feature point of the face profile edge of the near-infrared imaging module, is the internal parameter matrix of the visible light imaging module, is the external parameter matrix of the visible light imaging module, is the internal parameter matrix of the near-infrared imaging module, is the external parameter matrix of the near-infrared imaging module, is the actual three-dimensional coordinate of the point on the face profile edge to be solved, which is obtained by using the least square method. In step 5.3, the steps of performing face live body detection on the visible light face image and the near-infrared face image comprise: According to the face detection result obtained in step four, visible light face sub-images and near-infrared face sub-images are extracted from the visible light image and the near-infrared image respectively according to the face frame; the visible light face sub-images and the near-infrared face sub-images are normalized and input into the trained convolutional neural network face detection model for living body detection, and a living body detection result is output; The convolutional neural network face detection model comprises two sub-networks, wherein the first sub-network is: performing convolutional processing on the normalized visible light face sub-images to obtain visible light face feature maps, performing convolutional processing on the normalized near-infrared face sub-images to obtain near-infrared face feature maps, and merging the visible light face feature maps and the near-infrared face feature maps and outputting a merged feature map; the second sub-network is: performing multi-layer convolutional processing on the merged feature map, and finally outputting the class of the living body detection in the form of a classification network by using a fully connected layer or a 1×1 convolutional layer to obtain the final living body detection result; In step six, the step of performing face pose estimation comprises: Three-dimensional data of the face contour feature edge are collected, planes are fitted according to the three-dimensional data of the face contour feature edge, defined as face planes, and a normal vector of the face plane is solved as a face pose vector; In step eight, the face pose parameter is defined as an angle between the face pose vector and the optical axis of the imaging module, and the smaller the angle, the more the face is directly opposite the visible light imaging module and the near-infrared imaging module; according to the camera external parameters calibrated in step two, the pose parameters of the face and the visible light imaging module and the near-infrared imaging module are calculated.

2. A worker fatigue detection device characterized by, The staff fatigue detection method of claim 1 is executed, comprising a control module, a processing module, a visible light imaging module and a near-infrared imaging module, the visible light imaging module and the near-infrared imaging module are installed opposite the face of the detected person in a left-right or up-down manner, the control module is connected with the visible light imaging module and the near-infrared imaging module respectively, the control module controls the visible light imaging module and the near-infrared imaging module to synchronously collect image data, the processing module is connected with the visible light imaging module and the near-infrared imaging module respectively, the processing module receives the image data collected by the visible light imaging module and the near-infrared imaging module and processes the data to obtain a fatigue detection result.

3. The worker fatigue detection apparatus according to claim 2, characterized by The visible light imaging module comprises at least one visible light camera, and the near-infrared imaging module comprises at least one near-infrared camera; when the head of the detected person is mostly turned left and right due to work needs, the visible light camera and the near-infrared camera are installed opposite the face of the detected person in a left-right manner; when the head of the detected person is mostly raised and lowered due to work needs, the visible light camera and the near-infrared camera are installed opposite the face of the detected person in an up-down manner.

Images