Operating personnel fatigue monitoring method and device based on machine vision

Abstract

The invention provides an operator fatigue monitoring method and device based on machine vision, and the method comprises the steps: carrying out the preprocessing of the obtained operation live data of an operator, and obtaining a preprocessing result; detecting and positioning the eye region by using a gradient weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye region; according to the eye region, extracting the eyelid closing frequency of the operator as a fatigue feature; and performing fatigue evaluation based on the fatigue characteristics, and performing early warning and / or suggestion according to an evaluation result. According to the invention, a non-contact machine vision technology is adopted, fatigue monitoring is carried out on an operator through operation real-time data, the fatigue state of the operator is monitored in real time according to the operation real-time data, especially the eye area with asthenopia is monitored, a de-identification technology is adopted in the monitoring process, personal identity and sensitive information are protected, and the safety of the operator is improved. And meanwhile, personalized early warning and suggestions can be provided according to individual differences, and the universality is good.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue monitoring, and in particular to a method and device for monitoring operator fatigue based on machine vision. Background Art

[0002] Fatigue detection is a technology that evaluates the fatigue state of an individual by analyzing his or her physiological or behavioral characteristics. Fatigue detection is of great significance in the fields of traffic safety, health monitoring and efficiency management.

[0003] In the prior art, fatigue monitoring mostly uses external sensors or devices to monitor the physiological signals or behavioral characteristics of operators, such as heart rate sensors, eye trackers, etc. These external devices not only increase the complexity of the system, but may also cause discomfort to operators. At the same time, since the physiological data or behavioral characteristics of operators need to be collected and processed during the monitoring process, this may cause concerns and resistance among operators, affect their participation and acceptance, and is not conducive to data security and privacy protection. On this basis, in order to improve the problem of low comfort caused by external sensors or devices, some fields use image data for fatigue monitoring, but the scope of application of such methods is often limited and is only applicable to specific types of tasks or working environments. For example, some methods may only be applicable to driver fatigue monitoring, and have limited applicability in other fields such as factory production lines, medical care, etc.

[0004] In summary, the existing fatigue monitoring technology has the problem of poor versatility. Summary of the invention

[0005] The present invention provides a method and device for monitoring fatigue of operators based on machine vision, so as to solve the defect of poor versatility in the prior art and realize fatigue monitoring with good versatility, strong comfort and good safety.

[0006] The present invention provides a method for monitoring operator fatigue based on machine vision, comprising the following steps: Preprocessing the acquired actual operation data of the operators to obtain preprocessing results; Detecting and locating the eye region using a gradient-weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye region; Extracting the number of times the operator's eyelids are closed as a fatigue feature according to the eye region; A fatigue assessment is performed based on the fatigue characteristics, and a warning and / or suggestion is made according to the assessment result.

[0007] According to a method for monitoring fatigue of workers based on machine vision provided by the present invention, the eye area is detected and located using a gradient-weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye area, specifically comprising: Acquire edge points according to the preprocessing results, and determine relative positions of the edge points with respect to the reference points according to a predefined reference point; Obtaining a cumulative matrix according to the relative positions and gradient weights of the edge points; Obtaining a peak value from the accumulation matrix, and obtaining an eye position prediction result according to the peak value; An eye region is obtained according to the eye position prediction result.

[0008] According to a method for monitoring operator fatigue based on machine vision provided by the present invention, the preprocessing includes at least edge detection.

[0009] According to a method for monitoring worker fatigue based on machine vision provided by the present invention, the number of times the worker's eyelids are closed is extracted as a fatigue feature according to the eye region, specifically comprising: Calculating the height and width of the current human eye according to the eye area, and determining the current degree of closure of the human eye according to the ratio of the height to the width; The eye closure degree is calculated based on the current eye closure degree of the human being and the eye closure degree of the human being in the awake state collected in advance; Determine the number of eyelid closures based on the eye closure degree of the current frame and the eye closure degree of the previous frame; The number of blinks per unit time is calculated according to the eyelid closure times to obtain fatigue characteristics.

[0010] According to a method for monitoring operator fatigue based on machine vision provided by the present invention, fatigue assessment is performed based on the fatigue characteristics, and warnings and / or suggestions are made according to the assessment results, specifically including: Calculate the average number of blinks in each time period according to the fatigue characteristics of a plurality of consecutively numbered time periods; A linear regression equation is constructed by taking the number of the time period as an independent variable and the average number of blinks as a dependent variable; When the slope of the linear regression equation is less than 0, it is determined that the operator is in a fatigue state, and an evaluation result is obtained; alarm according to the evaluation result; And / or, recommending the operator to take rest based on the evaluation result.

[0011] According to a method for monitoring operator fatigue based on machine vision provided by the present invention, the actual operation data includes at least one of image data and video data.

[0012] The present invention also provides a worker fatigue monitoring device based on machine vision, comprising the following modules: A preprocessing unit, used to preprocess the acquired actual operation data of the operator to obtain a preprocessing result; A detection and positioning unit, configured to detect and locate the eye region using a gradient-weighted generalized Hough transform algorithm on the preprocessing result to obtain the eye region; A feature extraction unit, configured to extract the number of times the eyelids of the operator are closed as a fatigue feature according to the eye region; An evaluation result unit is used to perform fatigue evaluation based on the fatigue characteristics and to issue warnings and / or suggestions according to the evaluation results.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for monitoring operator fatigue based on machine vision as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for monitoring fatigue of operators based on machine vision as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for monitoring operator fatigue based on machine vision.

[0016] The machine vision-based fatigue monitoring method and device for workers provided by the present invention preprocesses the acquired real-time work data of the workers to obtain a preprocessing result; detects and locates the eye area using the gradient weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye area; extracts the number of eyelid closures of the workers as fatigue features according to the eye area; performs fatigue assessment based on the fatigue features, and gives warnings and / or suggestions based on the assessment results. The present invention uses non-contact machine vision technology to monitor the fatigue of workers through real-time work data without relying on sensors to contact the body or wear equipment, thereby improving the comfort and acceptability of monitoring. The fatigue state of the workers is monitored in real time according to the real-time work data, especially the eye area of ​​visual fatigue. De-identification technology is used in the monitoring process to protect personal identity and sensitive information. At the same time, personalized warnings and suggestions can be provided according to individual differences, and the versatility is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the operator fatigue monitoring method based on machine vision provided by the present invention.

[0019] Figure 2 This is the second flow chart of the operator fatigue monitoring method based on machine vision provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the operator fatigue monitoring device based on machine vision provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Combine the following Figure 1-Figure 2 Describe the operator fatigue monitoring method based on machine vision of the present invention, Figure 1 This is one of the flow charts of the operator fatigue monitoring method based on machine vision provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 110: pre-process the acquired actual operation data of the operator to obtain a pre-processing result.

[0024] It should be emphasized that the application scenarios of the present invention particularly include remote environments such as oil and gas stations, well sites, and offshore drilling platforms. It is understandable that fatigue monitoring of operators is particularly necessary in working environments such as deserts, plateaus, and underwater. Long-term duty of personnel in the aforementioned extreme environments can easily lead to mental health problems, which in turn can lead to misoperation of personnel after taking up their posts, and serious accidents.

[0025] It should be noted that the real-time operation data is obtained based on the visual acquisition device. In some embodiments, the real-time operation data includes at least one of image data and video data.

[0026] After obtaining the actual operation data, in order to enhance the accuracy of subsequent processing, the actual operation data is preprocessed to reduce interference factors and prepare for the processing of subsequent steps. The preprocessing result is the preprocessed actual operation data.

[0027] In some embodiments, the preprocessing includes at least edge detection. Further, the preprocessing may also include image denoising, contrast enhancement and other operations. It should be noted that the purpose of edge detection is to identify the outline or key points of the image subject. Image denoising and contrast enhancement are both for separating the image subject from the background.

[0028] In one embodiment, image denoising, contrast enhancement and edge detection are performed sequentially on the live operation data.

[0029] It should be noted that the present invention does not limit the specific algorithm for performing preprocessing. For example, edge detection or feature point detection technology can be applied to perform edge detection processing, and threshold processing can be used to achieve image denoising.

[0030] Step 120: Detect and locate the eye region using a gradient-weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye region.

[0031] After obtaining the preprocessing results, the gradient weighted generalized HOUGH transform algorithm is used to detect and locate the eye area. Specifically, the eye area is accurately located by detecting and estimating the curves or edges of the image in the preprocessing results.

[0032] It is understandable that the traditional ellipse detection method has limitations in locating the eye area of ​​complex shapes, while the gradient-weighted generalized HOUGH transform method can detect and locate any shape and has stronger applicability and scalability. Furthermore, the gradient-weighted generalized HOUGH transform method greatly reduces the amount of calculation in the detection process by introducing a gradient weight mechanism, thus minimizing the complexity.

[0033] Step 130: extracting the number of times the operator's eyelids are closed as a fatigue feature based on the eye region.

[0034] After locating the eye area, the number of eyelid closures is extracted as a fatigue feature. It should be noted that a number of studies have shown that there is a correlation between the number of eyelid closures and the degree of fatigue. On the one hand, eyelid closure is a direct physiological response to human fatigue. When a person feels tired, the muscle strength required to keep the eyes open will weaken, causing the eyelids to close involuntarily or blink frequently. On the other hand, fatigue can lead to a decrease in attention and concentration, which is reflected in eyelid activity, such as frequent closing of the eyes or yawning, which are manifestations of inattention. The present invention uses the number of eyelid closures as a fatigue feature to conduct fatigue assessment of operators, which ensures the accuracy of the assessment while reducing the amount of calculation.

[0035] The present invention does not limit the method of extracting the number of eyelid closure times, and any method can be used in actual operation.

[0036] In some embodiments, the eye region is first preprocessed by threshold processing, morphological operations, etc. to better separate the eyelids from the background. Then, edge detection or feature point detection technology is applied to identify the contour or key points of the eyelids. In the actual operation process, the changes in the eyelid contour or key points are monitored. It should be noted that when the eyelids are closed, the contour or feature points will change significantly. It is determined whether the eyelids are closed by setting a threshold or using a machine learning classifier. On this basis, eyelid closure detection is performed on each frame, and when the eyelid closure is detected, it is counted.

[0037] Furthermore, in order to reduce false detection, a minimum duration threshold may be set, and the eyelid closure state is counted only when it lasts longer than this threshold.

[0038] Furthermore, the counting results may be smoothed to reduce the influence of accidental factors (such as normal blinking).

[0039] In addition, when performing threshold processing and morphological operations, adjustments need to be made according to different ambient lighting and shooting angles. In actual applications, it may be affected by complex factors such as lighting changes, visual changes, occlusion, etc. The entire process needs to be tested and improved based on feedback to ensure the reliability and accuracy of the system.

[0040] Step 140: Perform fatigue assessment based on the fatigue characteristics, and issue warnings and / or recommendations based on the assessment results.

[0041] The fatigue status of the operator is evaluated according to the fatigue characteristics to obtain an evaluation result. The evaluation according to the fatigue characteristics mentioned in the present invention can be performed in a variety of ways, such as evaluating the fatigue degree according to the number of eyelid closures within a certain period of time, and the more times, the higher the fatigue degree may be.

[0042] At the same time, warnings and / or suggestions are issued based on the evaluation results. Specifically, if the evaluation results show that the operator's fatigue exceeds a certain level, a rest reminder is issued, and suggestions are made to adjust the work intensity, etc. Furthermore, if the evaluation results show that the operator's fatigue has reached a critical value, a fatigue warning is issued to force the operator to stop working.

[0043] It is understandable that although the current safety management, laws and regulations, and leadership attention are relatively complete, safety accidents are still common. In addition to the technical points that have been paid attention to, the human factor is crucial. Before taking up their posts, operators may change their mindsets in an instant due to family quarrels, personal frustrations, and temporary emergencies. The use of the present invention to monitor the eye area of ​​personnel can ensure the mentality of personnel before taking up their posts, and truly achieve the safety concept of going to work happily and returning home safely. The present invention uses actual operation data for real-time monitoring and intelligent early warning. The fatigue feature extraction algorithm has strong anti-interference ability and can accurately locate the eye area and extract fatigue features in complex environments; thereby predicting dangers in advance and conducting intelligent early warnings based on ticket features, reducing personnel misoperation and thereby reducing the accident rate.

[0044] The detection and positioning of the eye region is further described below. In some embodiments, the eye region is detected and positioned using a gradient weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye region, specifically including: Step 210: Acquire edge points according to the preprocessing results, and determine relative positions of the edge points with respect to the reference points according to predefined reference points; Step 220: Obtain a cumulative matrix according to the relative positions and gradient weights of the edge points; Step 230: obtaining a peak value from the accumulation matrix, and obtaining an eye position prediction result according to the peak value; Step 240: Obtain an eye region according to the eye position prediction result.

[0045] Specifically, Figure 2 As shown, in step 210, since the preprocessing result is an image that has been edge detected, edge points can be directly extracted. Afterwards, the relative position of the edge point relative to the reference point is determined based on the predefined reference point. In actual operation, one or more templates are created based on the geometric features of the eye, and these templates should be able to represent the main shape and structural features of the eye. Afterwards, the template is parameterized, the reference point is defined, and the relative position of the edge point relative to the reference point is recorded. In some embodiments, the pupil is selected as the reference point.

[0046] In step 220, after the relative position is obtained, for each edge point, voting is performed in the accumulation matrix according to its relative position. As the voting proceeds, the peak value in the accumulation matrix will correspond to the possible eye position and parameters in the image. In the voting process, different gradient weights are set for each feature of the edge point to ensure that important features have a greater impact on the accumulation result. Among them, each feature of the edge point includes the strength and continuity of the edge.

[0047] In step 230, after the accumulation matrix is ​​obtained, a peak value within a preset range is searched in the accumulation matrix, each peak value represents a possible eye position, and an eye position prediction result is obtained.

[0048] In step 240, the relevant area in the preprocessing result is extracted according to the position in the eye position prediction result, that is, the eye area can be obtained. Further, the eye area is output, and the output content may include information such as the center position of the eyeball and the eyelid contour.

[0049] Furthermore, after obtaining the eye position prediction result, it also includes: verifying whether the detected eye position prediction result meets the expected characteristics according to the peak value in the accumulation matrix, and adjusting the position and parameters when necessary.

[0050] The following further describes the extraction of fatigue features. In some embodiments, extracting the number of eyelid closures of the operator as fatigue features based on the eye region specifically includes: Calculating the height and width of the current human eye according to the eye area, and determining the current degree of closure of the human eye according to the ratio of the height to the width; The eye closure degree is calculated based on the current eye closure degree of the human being and the eye closure degree of the human being in the awake state collected in advance; Determine the number of eyelid closures according to the eye closure degree of the current frame and the eye closure degree of the previous frame; The number of blinks per unit time is calculated according to the eyelid closure times to obtain fatigue characteristics.

[0051] Specifically, the human eye region is cut out based on the eye region, the height and width of the human eye are calculated, and the degree of closure of the human eye is determined according to the ratio of the height to the width. It should be noted that the present invention does not limit the specific algorithm for calculating the height and width of the human eye. In some embodiments, the height and width of the human eye are calculated using grayscale integral projection.

[0052] After obtaining the width W and height H of the human eye, define the current degree of eye closure .

[0053] On this basis, a threshold can be set when When , it is in the closed state, When the eyes are awake, they are open. However, the size and shape of each person's eyes are different, and the state of the eyes is different under different conditions. The degree to which people open their eyes when awake varies, and the results obtained are also different.

[0054] Therefore, based on the above algorithm, we make corrections. First, we collect the height and width ratio of the eyes of a person in a normal awake state. , recorded as the degree of eye closure in the awake state, and then the height and width ratio of the eyes during the operation are collected , that is, the current degree of eye closure.

[0055] Define the degree of eye closure TCOE = , set the threshold Eyes closed more than The eyes are closed when the eye closure degree in the previous frame is greater than the preset threshold. And the eye closure degree of the current frame is less than the preset threshold In this case, it can be considered as one blink, the number of eyelid closures is increased by one, and the number of blinks per unit time can be obtained as a fatigue characteristic.

[0056] In one embodiment, the threshold In this case, if the degree of eye closure in the previous frame is greater than 50%, and the degree of eye closure in the current frame is less than 50%, it is considered a blink.

[0057] The fatigue assessment is further described below. In some embodiments, fatigue assessment is performed based on the fatigue characteristics, and warnings and / or suggestions are made based on the assessment results, specifically including: Calculate the average number of blinks in each time period according to the fatigue characteristics of a plurality of consecutively numbered time periods; A linear regression equation is constructed by taking the number of the time period as an independent variable and the average number of blinks as a dependent variable; When the slope of the linear regression equation is less than 0, it is determined that the operator is in a fatigue state, and an evaluation result is obtained; alarm according to the evaluation result; And / or, recommending the operator to take rest based on the evaluation result.

[0058] Specifically, n time periods are recorded continuously, and the average number of blinks y in each time period is obtained according to the fatigue characteristics of each time period, which can be further formed into a sequence { }, the present invention does not limit the length of each time period, such as 5 minutes for each time period.

[0059] At the same time, the time periods are numbered to obtain the time sequence {1,2…n}.

[0060] After that, the number of the time period (i.e., time series) is used as the independent variable and y as the dependent variable, and the slope K and intercept b are calculated using the linear regression equation.

[0061] Right now: K=

[0062] b=

[0063] If K is less than 0, it means that the blinking frequency is decreasing, and the operator is judged to be in a fatigue state; otherwise, the operator is judged to be in a normal state. The judged state of the operator is taken as the evaluation result.

[0064] Based on the assessment results, an alarm is issued and / or personnel are reminded to take a rest.

[0065] The machine vision-based fatigue monitoring method for workers provided by the present invention pre-processes the acquired real-time work data of the workers to obtain a pre-processing result; detects and locates the eye area using the gradient weighted generalized Hough transform algorithm according to the pre-processing result to obtain the eye area; extracts the number of eyelid closures of the workers as fatigue features according to the eye area; performs fatigue assessment based on the fatigue features, and gives warnings and / or suggestions based on the assessment results. The present invention uses non-contact machine vision technology to monitor the fatigue of workers through real-time work data without relying on sensors to contact the body or wear equipment, thereby improving the comfort and acceptability of monitoring. The fatigue state of the workers is monitored in real time according to the real-time work data, especially the eye area of ​​visual fatigue. De-identification technology is used in the monitoring process to protect personal identity and sensitive information. At the same time, personalized warnings and suggestions can be provided according to individual differences, and the versatility is good.

[0066] The following is a description of the operator fatigue monitoring device based on machine vision provided by the present invention. The operator fatigue monitoring device based on machine vision described below and the operator fatigue monitoring method based on machine vision described above can be referred to each other. Figure 3 As shown, the device includes the following modules: The preprocessing unit 310 is used to preprocess the acquired real-time operation data of the operator to obtain a preprocessing result; A detection and positioning unit 320, configured to detect and locate the eye region using a gradient-weighted generalized Hough transform algorithm on the preprocessing result to obtain the eye region; A feature extraction unit 330, configured to extract the number of eyelid closures of the operator as a fatigue feature according to the eye region; The evaluation result unit 340 is used to perform fatigue evaluation based on the fatigue characteristics, and to issue warnings and / or suggestions according to the evaluation results.

[0067] According to a machine vision-based operator fatigue monitoring device provided by the present invention, the eye area is detected and located using a gradient-weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye area, specifically including: Acquire edge points according to the preprocessing results, and determine relative positions of the edge points with respect to the reference points according to a predefined reference point; Obtaining a cumulative matrix according to the relative positions and gradient weights of the edge points; Obtaining a peak value from the accumulation matrix, and obtaining an eye position prediction result according to the peak value; An eye region is obtained according to the eye position prediction result.

[0068] According to a machine vision-based operator fatigue monitoring device provided by the present invention, the preprocessing at least includes edge detection.

[0069] According to a machine vision-based operator fatigue monitoring device provided by the present invention, the number of times the operator's eyelid closures are taken as fatigue features according to the eye region specifically includes: Calculating the height and width of the current human eye according to the eye area, and determining the current degree of closure of the human eye according to the ratio of the height to the width; The eye closure degree is calculated based on the current eye closure degree of the human being and the eye closure degree of the human being in the awake state collected in advance; Determine the number of eyelid closures according to the eye closure degree of the current frame and the eye closure degree of the previous frame; The number of blinks per unit time is calculated according to the eyelid closure times to obtain fatigue characteristics.

[0070] According to a machine vision-based operator fatigue monitoring device provided by the present invention, fatigue assessment is performed based on the fatigue characteristics, and warnings and / or suggestions are made according to the assessment results, specifically including: Calculate the average number of blinks in each time period according to the fatigue characteristics of a plurality of consecutively numbered time periods; A linear regression equation is constructed by taking the number of the time period as an independent variable and the average number of blinks as a dependent variable; When the slope of the linear regression equation is less than 0, it is determined that the operator is in a fatigue state, and an evaluation result is obtained; alarm according to the evaluation result; And / or, recommending the operator to take rest based on the evaluation result.

[0071] According to a machine vision-based operator fatigue monitoring device provided by the present invention, the actual operation data includes at least one of image data and video data.

[0072] The machine vision-based worker fatigue monitoring device provided by the present invention performs preprocessing on the acquired worker's actual work data to obtain a preprocessing result; detects and locates the eye area using the gradient weighted generalized Hough transform algorithm based on the preprocessing result to obtain the eye area; extracts the worker's eyelid closure times as a fatigue feature based on the eye area; performs fatigue assessment based on the fatigue feature, and provides warnings and / or suggestions based on the assessment results. The present invention uses non-contact machine vision technology to monitor the worker's fatigue through actual work data, without relying on sensors to contact the body or wear equipment, thereby improving the comfort and acceptability of monitoring. The worker's fatigue state is monitored in real time based on the actual work data, especially the eye area of ​​visual fatigue. De-identification technology is used during the monitoring process to protect personal identity and sensitive information. At the same time, personalized warnings and suggestions can be provided based on individual differences, and the versatility is good.

[0073] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a method for monitoring fatigue of workers based on machine vision, the method comprising: preprocessing the acquired real-time working data of the workers to obtain a preprocessing result; detecting and locating the eye area using a gradient weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye area; extracting the eyelid closure times of the workers as fatigue features according to the eye area; performing fatigue assessment based on the fatigue features, and giving warnings and / or suggestions according to the assessment results.

[0074] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0075] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the machine vision-based worker fatigue monitoring method provided by the above-mentioned methods, and the method includes: preprocessing the acquired actual work data of the worker to obtain a preprocessing result; detecting and locating the eye area using a gradient weighted generalized Hough transform algorithm based on the preprocessing result to obtain the eye area; extracting the number of eyelid closures of the worker as a fatigue feature based on the eye area; performing fatigue assessment based on the fatigue feature, and issuing warnings and / or suggestions based on the assessment results.

[0076] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the machine vision-based worker fatigue monitoring method provided by the above-mentioned methods, the method comprising: preprocessing the acquired actual work data of the worker to obtain a preprocessing result; detecting and locating the eye area using a gradient-weighted generalized Hough transform algorithm based on the preprocessing result to obtain the eye area; extracting the number of eyelid closures of the worker as a fatigue feature based on the eye area; performing fatigue assessment based on the fatigue feature, and issuing warnings and / or recommendations based on the assessment results.

[0077] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0078] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring operator fatigue based on machine vision, characterized in that: include: Preprocessing the acquired actual operation data of the operators to obtain preprocessing results; Detecting and locating the eye region using a gradient-weighted generalized Hough transform algorithm according to the preprocessing result to obtain the eye region; Extracting the number of times the operator's eyelids are closed as a fatigue feature according to the eye region; A fatigue assessment is performed based on the fatigue characteristics, and a warning and / or suggestion is made according to the assessment result.

2. The method for monitoring operator fatigue based on machine vision according to claim 1, characterized in that: According to the preprocessing result, the eye region is detected and located using the gradient weighted generalized Hough transform algorithm to obtain the eye region, which specifically includes: Acquire edge points according to the preprocessing results, and determine relative positions of the edge points with respect to the reference points according to a predefined reference point; Obtaining a cumulative matrix according to the relative positions and gradient weights of the edge points; Obtaining a peak value from the accumulation matrix, and obtaining an eye position prediction result according to the peak value; An eye region is obtained according to the eye position prediction result.

3. The method for monitoring operator fatigue based on machine vision according to claim 1, characterized in that: The preprocessing includes at least edge detection.

4. The method for monitoring operator fatigue based on machine vision according to claim 1, characterized in that: Extracting the number of times the operator's eyelids are closed as fatigue features according to the eye region specifically includes: Calculating the height and width of the current human eye according to the eye area, and determining the current degree of closure of the human eye according to the ratio of the height to the width; The eye closure degree is calculated based on the current eye closure degree of the human being and the eye closure degree of the human being in the awake state collected in advance; Determine the number of eyelid closures based on the eye closure degree of the current frame and the eye closure degree of the previous frame; The number of blinks per unit time is calculated according to the eyelid closure times to obtain fatigue characteristics.

5. The method for monitoring operator fatigue based on machine vision according to claim 1, characterized in that: Fatigue assessment is performed based on the fatigue characteristics, and warnings and / or suggestions are made according to the assessment results, including: Calculate the average number of blinks in each time period according to the fatigue characteristics of a plurality of consecutively numbered time periods; A linear regression equation is constructed by taking the number of the time period as an independent variable and the average number of blinks as a dependent variable; When the slope of the linear regression equation is less than 0, it is determined that the operator is in a fatigue state, and an evaluation result is obtained; alarm according to the evaluation result; And / or, recommending the operator to take rest based on the evaluation result.

6. The method for monitoring operator fatigue based on machine vision according to claim 1, characterized in that: The live operation data includes at least one of image data and video data.

7. A worker fatigue monitoring device based on machine vision, characterized in that: include: A preprocessing unit, used to preprocess the acquired actual operation data of the operator to obtain a preprocessing result; A detection and positioning unit, configured to detect and locate the eye region using a gradient-weighted generalized Hough transform algorithm on the preprocessing result to obtain the eye region; A feature extraction unit, configured to extract the number of times the eyelids of the operator are closed as a fatigue feature according to the eye region; An evaluation result unit is used to perform fatigue evaluation based on the fatigue characteristics and to issue warnings and / or suggestions according to the evaluation results.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for monitoring operator fatigue based on machine vision as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring operator fatigue based on machine vision as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for monitoring operator fatigue based on machine vision as described in any one of claims 1 to 6 is implemented.

Images