A facial recognition system based on a camera mounted on a crane vehicle

Abstract

The invention relates to the field of safe driving of a lifting vehicle, in particular to a face recognition system based on a camera mounted on a lifting vehicle, wherein the technical scheme comprises a DSM camera acquisition module, a face recognition module, a safety assessment module and a danger alarm module; wherein a non-uniform B-spline evaluation curve is used to analyze the change of the driver's eye height with driving time to evaluate the driver's fatigue degree, thereby improving the accuracy of the evaluation of the driver's fatigue state and ensuring that the driver is promptly alarmed when he is tired; an image collected by an ADAS camera is converted from an RGB space to a Lab space, and the external light intensity is judged by the numerical distribution of the L dimension in the image, thereby judging the safety of the driver wearing sunglasses and enhancing the flexibility of the evaluation; the driver's distraction degree is judged according to the driver's head turning arc, and a range is set for the driver's head turning arc, thereby effectively avoiding missed judgment and misjudgment of the distracted driving state; and the accurate judgment of the state of the lifting vehicle driver improves the safety of the lifting vehicle operation.

Description

Technical Field

[0001] The invention relates to the field of safe driving of a lifting vehicle, and in particular to a face recognition system based on a camera mounted on a lifting vehicle. Background Art

[0002] Dangerous driving of cranes is the main cause of crane accidents. The driving state of crane drivers has a crucial impact on driving safety. Currently, the facial recognition of crane drivers by on-board cameras can effectively determine the driving state of drivers. However, the driving safety assessment technology still needs to be improved. For example, the accuracy of judging whether crane drivers are driving fatigued needs to be improved. Whether the behavior of crane drivers wearing sunglasses is safe is not effectively judged in combination with the external lighting environment. There is no strict judgment standard for whether crane drivers are distracted.

[0003] In view of this, we propose a facial recognition system based on the lifting vehicle's on-board camera to solve the existing problems. Summary of the invention

[0004] The object of the present invention is to provide a facial recognition system based on a camera mounted on a lifting vehicle to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a facial recognition system based on a camera mounted on a lifting vehicle, comprising:

[0006] DSM camera acquisition module, facial recognition module, safety assessment module, and danger alarm module;

[0007] The facial recognition module includes a mask recognition module, a fatigue driving recognition module, a distracted driving recognition module, a smoking recognition module, a phone call recognition module, and a sunglasses recognition module;

[0008] The safety assessment module includes a mask occlusion assessment module, a fatigue driving assessment module, a distracted driving assessment module, a smoking assessment module, a phone call assessment module, and a sunglasses safety assessment module;

[0009] The fatigue driving evaluation module uses a non-uniform B-spline evaluation curve to analyze the change of the driver's eye height with driving time to evaluate the driver's fatigue level;

[0010] The sunglasses safety assessment module assesses the safety of the driver wearing sunglasses through the brightness of the image collected by the ADAS camera.

[0011] Furthermore, the fatigue driving evaluation module uses driving time as the input value of the non-uniform B-spline evaluation curve and the driver's eye height as the output value of the non-uniform B-spline evaluation curve, thereby constructing a mathematical model of the non-uniform B-spline evaluation curve, and judging the driver's fatigue level through the change law of the control points of the non-uniform B-spline evaluation curve.

[0012] Furthermore, the sunglasses safety assessment module converts the image collected by the ADAS camera from the RGB space to the Lab space, determines the external light intensity through the numerical distribution of the L dimension in the image, and further determines the safety of the driver wearing sunglasses.

[0013] Furthermore, the mask occlusion assessment module judges the safety of the driver wearing the mask according to whether the mask blocks the driver's eyes; draws the circumscribed rectangles of the identified mask and eyes respectively, and if the circumscribed rectangle of the mask and the circumscribed rectangle of the eyes intersect, it is judged that the mask has blocked the eyes; if the circumscribed rectangle of the mask and the circumscribed rectangle of the eyes do not intersect, it is judged that the mask does not block the eyes.

[0014] Furthermore, the distracted driving identification module determines the driver's distraction level according to the driver's head turning arc. If the driver's head turning arc exceeds a set range, it is determined that the driver is in a distracted driving state.

[0015] Furthermore, the facial recognition module uses a Haar cascade classifier to recognize the driver's mask, eyes, head, cigarette, phone, and sunglasses.

[0016] Furthermore, before constructing the mathematical model of the non-uniform B-spline evaluation curve, a non-uniform B-spline reference curve is constructed.

[0017] Furthermore, the input value of the non-uniform B-spline reference curve ranges from [0, 1], the output value is fixed to 1, and the control points of the non-uniform B-spline reference curve are obtained.

[0018] Furthermore, the numerical distribution of the L dimension in the image is the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image.

[0019] Furthermore, if the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image is greater than or equal to the set ratio threshold, the behavior of wearing sunglasses is judged to be a safe driving behavior; if the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image is less than the set ratio threshold, the behavior of wearing sunglasses is judged to be a dangerous driving behavior.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention adopts a non-uniform B-spline evaluation curve to analyze the change of the driver's eye height with driving time to evaluate the driver's fatigue degree, and constructs a non-uniform B-spline reference curve. The control points of the non-uniform B-spline reference curve are used as a standard, and the driver's fatigue degree is judged by the change law of the control points of the non-uniform B-spline evaluation curve, so as to improve the accuracy of the evaluation of the driver's fatigue state and ensure that the driver is promptly warned when he is tired; the image collected by the ADAS camera is converted from RGB space to Lab space, and the external light intensity is judged by the numerical distribution of the L dimension in the image, and then the safety of the driver wearing sunglasses is judged, so as to enhance the flexibility of the evaluation; the driver's distraction degree is judged according to the driver's head turning arc, and the driver's head turning arc is set to a range, so as to effectively avoid missed judgment and misjudgment of distracted driving state; the facial recognition system accurately judges the driving state of the crane driver to improve the safety of the crane operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A structural diagram of a facial recognition system based on a camera mounted on a lifting vehicle according to the present invention;

[0023] Figure 2 A workflow diagram for the safety assessment of mask wearing for a crane vehicle driver of the present invention;

[0024] Figure 3 A flowchart for evaluating the fatigue state of a crane driver according to the present invention;

[0025] Figure 4 A flowchart of the distraction state assessment of a crane driver of the present invention;

[0026] Figure 5 A work flow chart for the smoking assessment of a crane driver of the present invention;

[0027] Figure 6 Workflow diagram for making phone calls and evaluating crane truck drivers of the present invention;

[0028] Figure 7 The present invention is a workflow diagram for the safety assessment of sunglasses wearing for crane vehicle drivers. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0030] Embodiment 1

[0031] like Figure 1 and Figure 2 As shown in the figure, the workflow of the safety assessment of mask wearing for lifting vehicle drivers includes:

[0032] DSM camera acquisition;

[0033] Haar cascade classifier is used to identify masks and eyes;

[0034] Determine whether the mask covers both eyes;

[0035] Whether to issue a danger alert depends on the assessment results.

[0036] The working principle of the safety assessment of the mask wearing of the crane driver based on the first embodiment is:

[0037] Draw the circumscribed rectangles of the identified mask and eyes respectively. If the circumscribed rectangles of the mask and the eyes intersect, it is determined that the mask has covered the eyes and a danger alarm is issued; if the circumscribed rectangles of the mask and the eyes do not intersect, it is determined that the mask does not cover the eyes and no danger alarm is issued for the time being.

[0038] Embodiment 2

[0039] like Figure 1 and Figure 3 As shown in the figure, the workflow of the crane driver fatigue assessment includes:

[0040] DSM camera acquisition;

[0041] Haar cascade classifier is used to identify both eyes;

[0042] Construct a non-uniform B-spline reference curve, set the input value range of the non-uniform B-spline reference curve to [0,1], and the output value to 1, and obtain the control points of the non-uniform B-spline reference curve;

[0043] The driving time is used as the input value of the non-uniform B-spline evaluation curve, and the driver's eye height is used as the output value of the non-uniform B-spline evaluation curve, so as to construct a mathematical model of the non-uniform B-spline evaluation curve and obtain the control points of the non-uniform B-spline evaluation curve;

[0044] The control points of the non-uniform B-spline reference curve are used as the standard, and the driver's fatigue degree is judged by the change law of the control points of the non-uniform B-spline evaluation curve;

[0045] Whether to issue a danger alert depends on the assessment results.

[0046] The working principle of the crane driver fatigue status assessment based on the second embodiment is:

[0047] The mathematical model of the non-uniform B-spline curve is:

[0048]

[0049] Where: is the value of the i-th segment after the n-th non-uniform B-spline curve transformation, where i = 1, 2, 3, ..., m; t is the value before the non-uniform B-spline curve transformation, and for each point of the curve segment, 0 ≤ t ≤ 1 is satisfied, t = 0 represents the beginning node of the i-th segment, and t = 1 represents the end node of the i-th segment; p i+k-1 is the control point of the i-th non-uniform B-spline curve; F k,n (t) is a piecewise mixing function of an n-th degree non-uniform B-spline curve, where 0≤t≤1, k=0,1,2,...,n.

[0050] Using [0,1] as the input value range and 1 as the fixed output value, we can find the control point p of the non-uniform B-spline reference curve. r A collection of .

[0051] For the non-uniform B-spline evaluation curve, the driving time is used as the input value of the non-uniform B-spline evaluation curve, and the driver's eye height is used as the output value of the non-uniform B-spline evaluation curve. The mathematical model of the non-uniform B-spline evaluation curve is constructed to obtain the control point p of the non-uniform B-spline evaluation curve. a The driving time is normalized, so the input value range is [0,1]; the driver's eye height is also normalized, so the driver's eye height is set to 1 when the driver is not tired, and the driver's eye height is the height of the circumscribed rectangle of the driver's eyes.

[0052] According to p r Subtract p a The difference of the values ​​changes in turn to judge the driver's fatigue level. If p r Subtract p a The difference between the values ​​of p and p increases successively, and the driver is judged to be in a fatigue driving state, and a danger alarm is issued; if p r Subtract p a If the difference between the values ​​does not increase successively, it is judged that the driver is not in a fatigue driving state and no danger alarm is issued for the time being.

[0053] Drivers will inevitably blink while driving, so an instantaneous time value is set. The change of eye height from 0 to 1 or from 1 to 0 within the set instantaneous time value is regarded as a blinking behavior. In order to prevent the collision of the evaluation curve node with the blinking moment, which will cause the blinking to interfere with the accuracy of the evaluation curve, a non-uniform B-spline evaluation curve is used to ensure that the blinking moment can be effectively avoided in the process of node selection. In addition, in order to ensure p r With p a One-to-one correspondence is possible. In the process of constructing a non-uniform B-spline curve to obtain control points, the input values ​​of the non-uniform B-spline reference curve and the non-uniform B-spline evaluation curve are consistent and one-to-one correspondence is achieved.

[0054] Embodiment 3

[0055] like Figure 1 and Figure 4 As shown in the figure, the workflow of the crane driver distraction assessment includes:

[0056] DSM camera acquisition;

[0057] Haar cascade classifier is used to identify the head;

[0058] Determine the driver's level of distraction based on the arc of the driver's head turn;

[0059] Whether to issue a danger alert depends on the assessment results.

[0060] The working principle of the crane driver distraction state assessment based on the third embodiment is:

[0061] The driver's head turning arc is classified and set in range through deep learning technology. The driver needs to turn his head to look at the rearview mirror to check the road during driving, so a range needs to be set. If the driver's head turning arc exceeds the set range, it is judged that the driver is in a distracted driving state and a danger alarm is issued; if the driver's head turning arc does not exceed the set range, it is judged that the driver is not in a distracted driving state and no danger alarm is issued for the time being.

[0062] Embodiment 4

[0063] like Figure 1 and Figure 5 As shown in the figure, the workflow of the smoking assessment for crane truck drivers includes:

[0064] DSM camera acquisition;

[0065] Haar cascade classifier is used to identify cigarettes;

[0066] Determine that cigarettes are dangerous items for driving;

[0067] Give a hazard alert.

[0068] Embodiment 5

[0069] like Figure 1 and Figure 6 As shown, the workflow for the crane operator's phone call assessment includes:

[0070] DSM camera acquisition;

[0071] Haar cascade classifier is used to identify telephones;

[0072] Determine that the phone is a dangerous item for driving;

[0073] Give a hazard alert.

[0074] Embodiment 6

[0075] like Figure 1 and Figure 7 As shown in the figure, the workflow of the safety assessment of sunglasses wearing for crane vehicle drivers includes:

[0076] The DSM camera captures the facial image of the crane vehicle driver;

[0077] Use Haar cascade classifier to identify sunglasses;

[0078] The ADAS camera collects images in front of the lifting vehicle;

[0079] Convert the image captured by the ADAS camera from RGB space to Lab space;

[0080] The external light intensity is judged by the numerical distribution of the L dimension in the image;

[0081] Determine the safety of crane truck drivers wearing sunglasses;

[0082] Whether to issue a danger alert depends on the assessment results.

[0083] The working principle of the safety assessment of the crane driver's sunglasses wearing based on the sixth embodiment is:

[0084] If the external light intensity is too high, the driver wearing sunglasses can effectively reduce the impact of sunlight or high beam on the driver's vision; if the external light intensity is not high, the driver wearing sunglasses will dim the vision, causing safety hazards. Therefore, the external light intensity is needed to judge the safety of the driver wearing sunglasses.

[0085] The L in the Lab space represents brightness, and a and b represent chromaticity. The Lab space can distinguish brightness from chromaticity. Compared with the RGB space, the Lab space can more accurately judge the brightness distribution of the image. Therefore, the image captured by the ADAS camera is converted from the RGB space to the Lab space. The numerical distribution of the L dimension in the image can accurately judge the external light intensity.

[0086] The L threshold is set as the critical value of brightness that affects the driver's sight, and the numerical distribution of the L dimension in the image is set as the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image. The ratio threshold is set as the critical value of brightness distribution that affects the driver's sight. If the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image is greater than or equal to the set ratio threshold, it is judged that the behavior of wearing sunglasses is a safe driving behavior, and no danger alarm is temporarily issued; if the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image is less than the set ratio threshold, it is judged that the behavior of wearing sunglasses is a dangerous driving behavior, and a danger alarm is issued.

[0087] The above-mentioned specific embodiments are only several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above-mentioned embodiments, those skilled in the art can make various alternative improvements and combinations to the above-mentioned specific embodiments.

Claims

1. A facial recognition system based on a camera mounted on a lifting vehicle, characterized in that: include: DSM camera acquisition module, facial recognition module, safety assessment module, and danger alarm module; The facial recognition module includes a mask recognition module, a fatigue driving recognition module, a distracted driving recognition module, a smoking recognition module, a phone call recognition module, and a sunglasses recognition module; The safety assessment module includes a mask occlusion assessment module, a fatigue driving assessment module, a distracted driving assessment module, a smoking assessment module, a phone call assessment module, and a sunglasses safety assessment module; The fatigue driving evaluation module uses a non-uniform B-spline evaluation curve to analyze the change of the driver's eye height with driving time to evaluate the driver's fatigue level; The sunglasses safety assessment module assesses the safety of the driver wearing sunglasses through the brightness of the image collected by the ADAS camera.

2. A facial recognition system based on a camera mounted on a lifting vehicle according to claim 1, characterized in that: The fatigue driving evaluation module uses driving time as the input value of the non-uniform B-spline evaluation curve and the driver's eye height as the output value of the non-uniform B-spline evaluation curve to construct a mathematical model of the non-uniform B-spline evaluation curve and judge the driver's fatigue level through the change law of the control points of the non-uniform B-spline evaluation curve.

3. The facial recognition system based on the camera on a crane vehicle according to claim 1, characterized in that: The sunglasses safety assessment module converts the image collected by the ADAS camera from RGB space to Lab space, and determines the external light intensity through the numerical distribution of the L dimension in the image, and further determines the safety of the driver wearing sunglasses.

4. The facial recognition system based on a camera mounted on a lifting vehicle according to claim 1, characterized in that: The mask occlusion assessment module determines the safety of the driver wearing a mask according to whether the mask covers the driver's eyes; the identified mask and eyes are respectively circumscribed with rectangles, and if the circumscribed rectangle of the mask and the circumscribed rectangle of the eyes intersect, it is determined that the mask has covered the eyes; if the circumscribed rectangle of the mask and the circumscribed rectangle of the eyes do not intersect, it is determined that the mask does not cover the eyes.

5. The facial recognition system based on a camera mounted on a lifting vehicle according to claim 1, characterized in that: The distracted driving recognition module determines the driver's distraction level according to the driver's head turning arc. If the driver's head turning arc exceeds a set range, it is determined that the driver is in a distracted driving state.

6. The facial recognition system based on a camera mounted on a lifting vehicle according to claim 1, characterized in that: The facial recognition module uses a Haar cascade classifier to recognize the driver's mask, eyes, head, cigarette, phone, and sunglasses.

7. A facial recognition system based on a camera mounted on a lifting vehicle according to claim 2, characterized in that: Before constructing the mathematical model of the non-uniform B-spline evaluation curve, a non-uniform B-spline reference curve is constructed.

8. The facial recognition system based on the camera on a crane vehicle according to claim 7, characterized in that: The input value range of the non-uniform B-spline reference curve is [0, 1], the output value is fixed to 1, and the control points of the non-uniform B-spline reference curve are obtained.

9. The facial recognition system based on a camera mounted on a lifting vehicle according to claim 3, characterized in that: The numerical distribution of the L dimension in the image is the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image.

10. A facial recognition system based on a camera mounted on a lifting vehicle according to claim 9, characterized in that: If the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image is greater than or equal to the set ratio threshold, it is determined that the behavior of wearing sunglasses is a safe driving behavior; If the ratio of the number of pixels exceeding the set L threshold to the total number of pixels in the image is less than the set ratio threshold, it is determined that the behavior of wearing sunglasses is a dangerous driving behavior.

Images