Intelligent bracelet worn by driver

Abstract

The invention provides a smart bracelet worn by a driver, a visual sensor is located at the side part of a bracelet body, and the smart bracelet further comprises a face image acquisition module for acquiring a face image of the driver acquired by the visual sensor; the eye image analysis module is used for obtaining an eye dynamic analysis instruction or a posture monitoring instruction according to an extraction result according to whether the eye target part of the human face image can be extracted or not; the eye dynamic analysis module is used for respectively obtaining the eyelid closing degree and the blinking frequency in the image of the eye target part when an eye dynamic analysis instruction is obtained, respectively comparing the eyelid closing degree and the blinking frequency with a threshold value, and outputting a current state according to a comparison result; the posture monitoring verification module is used for obtaining the nodding frequency of the driver in the face image through an image dynamic trajectory analysis strategy when a posture monitoring instruction is obtained, and analyzing and outputting the current state according to the nodding frequency; the method has the advantage that the fatigue degree of the driver can be judged by analyzing the eye condition within a certain time.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic service facilities, and more particularly to a smart bracelet worn by a driver. Background Art

[0002] With the continuous development of science and technology, smart bracelets will play a more important role in the future. With the continuous advancement of artificial intelligence technology, smart bracelets will have more powerful intelligent analysis capabilities, and can more accurately predict users' health status and provide personalized suggestions. Its current smart bracelets cover multiple functions such as health monitoring, exercise recording, sleep analysis, and information reminders.

[0003] However, in the field of public transportation, the control of drivers still relies solely on monitoring in the cab. As for driver fatigue monitoring, the current public transportation system does not monitor it or uses manual monitoring. The railway system uses a device in which the driver regularly steps on or presses a certain device to ensure wakefulness. Combining traditional smart bracelets with drivers can only monitor the driver's real-time health status, but cannot monitor fatigue. The driver's fatigue level greatly affects public safety. Therefore, how to effectively monitor the driver's fatigue is a crucial task. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a smart bracelet worn by the driver, which can judge the driver's fatigue level through eye recognition of the driver, further analyze the eye condition within a certain period of time, and verify it through body posture monitoring to improve the accuracy of feedback.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A smart wristband worn by a driver, comprising a wristband body and a plurality of different types of sensors located in the wristband, wherein the sensors include a visual sensor located on the side of the wristband body, and the wristband body also includes

[0007] A face image acquisition module, which acquires the driver's face image acquired by the visual sensor;

[0008] An eye image analysis module, which obtains an eye image by segmenting the face image through a semantic segmentation model, and then determines whether the eye target part can be extracted through a target matching model according to the eye image, and obtains an eye dynamic analysis instruction or a body posture monitoring instruction according to the extraction result of the eye target part;

[0009] The eye dynamic analysis module, when receiving the eye dynamic analysis instruction, obtains the eyelid closure degree and the blinking frequency in the image of the eye target part by using the semantic segmentation strategy and the calculation strategy, and compares the eyelid closure degree and the blinking frequency with the threshold value respectively, and outputs the current state based on the comparison result;

[0010] The posture monitoring verification module, when obtaining the posture monitoring instruction, obtains the driver's nodding frequency in the face image through the image dynamic trajectory analysis strategy, and outputs the current state according to the nodding frequency analysis.

[0011] Furthermore, it also includes an indicator evaluation module, which obtains various monitoring indicators collected by other sensors as dynamic monitoring data, and calculates the dynamic monitoring data, eyelid closure degree and blinking frequency through a weight algorithm to obtain a first risk score. The eyelid closure degree is the ratio of the gap between the upper eyelid and the lower eyelid to the eye width within a certain period of time, and the blinking frequency is the number of times the upper eyelid is closed within a certain period of time.

[0012] Furthermore, the indicator evaluation module calculates the dynamic monitoring data and the nodding frequency through a weighting algorithm to obtain a second risk score, wherein the nodding frequency is the frequency at which the driver's head in the face image moves up and down within a certain period of time.

[0013] Furthermore, it also includes a danger adjustment module, which obtains the driver information stored in the public transportation system, selects the eye information and health information with a large correlation with the eye type from the driver information, adjusts the weight value of the eyelid closure degree and the weight value of the blinking frequency according to the eye information, and adjusts the weight value of the dynamic monitoring data according to the health information, and calculates the updated first danger score again through the weight algorithm;

[0014] When the posture monitoring instruction is obtained, the head information with a high correlation with the head type is screened out from the driver information, the weight value of the nodding frequency is adjusted according to the head information, and the weight value of the dynamic monitoring data is adjusted according to the health information, and the updated second risk score is calculated again through the weight algorithm.

[0015] Furthermore, the semantic segmentation strategy includes a semantic segmentation step and a point comparison step.

[0016] The semantic segmentation step is to draw the eyelid contour line and the lower eyelid line in the image of the eye target part;

[0017] The point comparison step marks the center point of the eyelid contour line and the center point of the lower eyelid line respectively, corrects the positions of the two center points so that the two center points are located on the same vertical line, and then calculates the distance between the two center points as the gap distance. In the continuous eye target part, the minimum value of the gap distance within a cycle is screened out as the target value, and the one cycle reflects the process of the gap distance from the maximum value to the minimum value. The target value is compared with the eye width to obtain the degree of eyelid closure.

[0018] Furthermore, the calculation strategy includes a calculation step, in which an analysis time period is preset, the maximum value of the gap distance in the first cycle is searched as a starting point, the number of cycles of the gap distance passed during the analysis time period is counted as the number of blinks, and the blink frequency is calculated based on the number of blinks and the analysis time period.

[0019] Furthermore, the dynamic trajectory analysis strategy includes a profile analysis step and a displacement calculation step.

[0020] The contour analysis step obtains a plurality of consecutive face images when the vehicle is traveling on a smooth road section, performs image correction on each face image through a reference point so that the driver's head perspective in each face image is the same, and plans the driver's head contour in each face image through a contour algorithm;

[0021] The displacement calculation step establishes a corresponding two-dimensional coordinate system, arbitrarily selects several calculation points in the head contour, calculates the offset of the corresponding calculation point coordinates on the y-axis as the nodding amplitude based on two adjacent face images, identifies the nodding cycle in continuous face images, and calculates the number of nodding cycles within a certain period of time to obtain the nodding frequency.

[0022] Furthermore, dynamic monitoring indicators include heart rate, blood oxygen rate, respiratory rate and electrocardiogram.

[0023] Furthermore, it also includes an information interaction module for obtaining emergency instructions in the public transportation system and displaying a one-touch alarm button on the wristband body.

[0024] Furthermore, the information interaction module obtains the driver's body surface temperature detected by the sensor, and outputs air conditioning control instructions to the public transportation system according to the body surface temperature.

[0025] The beneficial effects of the present invention are as follows: by analyzing the driver's facial image, it is first determined whether a partial image of the eye can be clearly extracted; if so, the driver's fatigue level is analyzed by the degree of eyelid closure and blinking frequency; if not, the driver's fatigue level is analyzed by the nodding frequency, that is, the smart bracelet realizes real-time monitoring and evaluation of the driver's physiological state and body behavior by integrating high-precision sensors and advanced image processing technology, which helps to timely discover and prevent traffic accidents caused by driver fatigue, distraction, etc., and improve road driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the overall flow chart of the present invention;

[0027] Figure 2 It is a partial flow chart of the present invention;

[0028] Figure 3 is a schematic diagram of an eye image in the present invention;

[0029] Figure 4 It is a line graph showing the gap distance changing with time in the present invention. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The same parts are represented by the same reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "bottom surface" and "top surface", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0031] As for the fatigue monitoring of the driver, the current public transportation system does not monitor it or adopts manual monitoring, while the railway system adopts that the driver regularly steps on or presses a certain device to ensure the awake state, and the combination of the traditional smart bracelet and the driver can only monitor the real-time health status of the driver, and cannot achieve fatigue monitoring. The fatigue level of the driver greatly affects public safety. Therefore, the present invention designs such a smart bracelet based on the driver's wear, including a bracelet body and a plurality of different types of sensors located in the bracelet, the sensor includes a visual sensor, and the visual sensor is located on the side of the bracelet body, which is convenient for capturing image data of the driver's face, such as Figure 1 and Figure 2 As shown, the bracelet body also includes:

[0032] The face image acquisition module acquires the driver's face image captured by the visual sensor, which also includes an image processing module. Through the image optimization algorithm, it ensures that the captured image can be corrected and optimized even in the case of light changes or rapid movement, so as to facilitate subsequent analysis and use.

[0033] The eye image analysis module uses a semantic segmentation model trained with a large amount of data in the early stage to perform fine segmentation of the face image, extract the eye area image, and then uses a target matching model trained with a large amount of data in the early stage to determine whether the key parts of the eye image (such as pupils, eyelids, etc.) are clearly visible, so as to confirm that the image is sufficient for subsequent analysis. According to the clarity of the eye image and whether the key parts are clearly visible, the module will decide whether to perform eye dynamic analysis or body monitoring instructions. When the pupils and eyelids in the eye image are relatively clear, eye dynamic analysis will be given priority. When the driver wears sunglasses, the eye image cannot clearly identify the pupils and eyelids, so body monitoring will be given priority.

[0034] Eye dynamic analysis module, when receiving eye dynamic analysis instructions, this module further refines the eye image analysis, synchronously uses semantic segmentation strategy and calculation strategy, accurately measures the degree of eyelid closure and blinking frequency, and compares the measurement results with the preset threshold to determine whether the driver is in a state of fatigue, distraction or normal, and outputs the current status accordingly.

[0035] The posture monitoring verification module, when receiving the posture monitoring instruction, uses the face image dynamic trajectory analysis strategy to analyze the tiny movements of the head in the face image, especially the driver's nodding frequency. When the driver is sleepy, he is prone to nodding behavior, that is, an increase in the nodding frequency may be a manifestation of the driver's sleepiness or inattention. The module outputs the current status accordingly.

[0036] In addition, it also includes an information interaction module, which transmits the current status in the eye dynamic analysis module or the current status in the body posture monitoring and verification module to the public transportation system, which will be evaluated again by the back-end management personnel, or the indicator evaluation module in the bracelet will perform a risk score, and the public transportation system will make records. After the evaluation, if the risk factor is high, the corresponding bracelet will vibrate to remind the sleepy driver.

[0037] The advantages of the present invention are: this smart bracelet realizes real-time monitoring and evaluation of the driver's physiological state and body behavior by integrating high-precision sensors and advanced image processing technology, which helps to timely discover and prevent traffic accidents caused by driver fatigue, distraction, etc., improve road driving safety, and can simultaneously manage and monitor all drivers online.

[0038] In order to more comprehensively reflect the driver's current physiological state and driving risks, Figure 2As shown, it also includes an indicator evaluation module. First, dynamic monitoring data is collected from various sensors in the bracelet. The dynamic monitoring data includes heart rate, blood oxygen rate, respiratory rate and electrocardiogram, etc. Secondly, the collected data is preprocessed, including denoising, filtering, standardization and other steps to ensure the accuracy and comparability of the data, and then the weight score is calculated. According to the preset weight distribution scheme, each feature (including dynamic monitoring data and eye dynamic analysis results) is weighted and summed to calculate the first danger score. The weight distribution is usually based on a large amount of experimental data and expert experience to ensure the accuracy and effectiveness of the score. Assuming that the first danger score is 89 points, the calculated 89 points are output in an intuitive way to the public transportation system for further evaluation or further analysis by the bracelet. Different warnings or suggestions can be given according to different scores, such as reminding the driver to take a rest, adjust the driving state or take other safety measures;

[0039] like Figure 3 As shown, the semantic segmentation strategy includes a semantic segmentation step and a point comparison step.

[0040] The semantic segmentation step accurately identifies and segments the eyelid contour line and the lower eyelid line in the eye image. It uses deep learning (convolutional neural network CNN) or traditional image processing (edge ​​detection, threshold segmentation, etc.) to process the eye image. Through image segmentation technology, the eyelid contour line and the lower eyelid line are separated from the eye image and their boundaries are clearly outlined.

[0041] Point comparison steps,First, mark the center points: mark the center points on the eyelid contour line and the lower eyelid line respectively, these center points are usually calculated from the geometric center of the line;

[0042] Secondly, position correction: to ensure that the two center points are on the same vertical line, the image needs to be rotated, distorted and stretched to eliminate the deviation caused by image shooting;

[0043] Then, calculate the gap distance: calculate the straight-line distance between the two center points, that is, the straight-line distance reflects the size of the gap between the eyelid and the lower eyelid; Figure 3 As shown in A in the figure, the distance between point 1 and point 2 represents the gap distance, and the gap distance is h. Figure 3 As shown in B, point 1 and point 2 can almost be considered to coincide, so the gap distance can be determined to be zero;

[0044] Secondly, filter the target value: in the continuous eye image frames, track the change of the gap distance, and filter out the minimum value of the gap distance in a cycle (from the maximum value to the minimum value) as the target value. This value can reflect the degree of eyelid closure; for example, Figure 3 AB represents a cycle, such as Figure 4As shown in the figure, the horizontal axis n represents the cycle, and the vertical axis represents the gap distance value. When the driver is tired and sleepy, the gap distance will not immediately become zero in the first few cycles, but from the maximum gap distance to the minimum gap distance, and then increase again, which is considered as a cycle. Usually, the minimum value in a cycle is selected as the target value.

[0045] Finally, compare the eyelid closure degree: compare the target value with the eye width (the eye width is equal to the maximum gap distance) to quantify the eyelid closure degree. Figure 3 When the AB process is shown, it means that the eyelids are 100% closed.

[0046] The calculation strategy includes calculation steps. First, preset the analysis time period: determine a fixed time period for analyzing the blinking behavior;

[0047] Secondly: Search for the starting point: The maximum value of the gap distance in the first cycle is taken as the starting point, which marks the beginning of a blink cycle;

[0048] Then: Count the number of cycles: within the preset analysis time period, count the number of complete cycles that the gap distance goes through from the maximum value to the minimum value (and then back to the maximum value). This number of cycles is the number of blinks;

[0049] Finally: Calculate the blink frequency: Divide the number of blinks by the analysis period (converted to seconds or other time units) to get the blink frequency, which can be used to assess eye activity, fatigue, or as a biometric feature.

[0050] As mentioned above, when the driver is driving fatigued and drowsy, under normal circumstances, the blinking frequency is relatively fast and the degree of eyelid closure is changing. If the driver is in a sleeping state, the blinking frequency is relatively low. At this time, it is only necessary to determine whether the degree of eyelid closure is 100%.

[0051] In addition, in order to more comprehensively reflect the driver's current physiological state and driving risk, when the eyelid image recognition is unclear, it is necessary to evaluate the nodding frequency. Figure 2 As shown in FIG. 1 , the index evaluation module also needs to collect dynamic monitoring data from various sensors in the bracelet, assign weights to various features and nodding frequencies of the dynamic monitoring data based on expert experience and experimental data, and sum each feature and nodding frequency according to its weight to obtain a preliminary result of the second hazard score. Assuming that the first hazard score is 85 points, the calculated 85 points will be output in an intuitive way to the public transportation system for further evaluation or further analysis by the bracelet. Different warnings or suggestions can be given according to different scores, such as reminding the driver to take a rest, adjust the driving status, or take other safety measures.

[0052] Specifically, the dynamic trajectory analysis strategy includes a contour analysis step and a displacement calculation step.

[0053] Contour analysis steps: First, when the vehicle is driving steadily, a number of consecutive face images are used to correct each image using image processing technology (such as edge detection, Hough transform, etc.) to ensure that the perspective of the driver's head in all images is consistent. Then, a contour algorithm (such as Canny edge detection) is applied to extract the driver's head contour;

[0054] The displacement calculation step is to establish a corresponding two-dimensional coordinate system. In the two-dimensional coordinate system, several key points on the head contour (such as feature points on the forehead and chin, etc.) are selected. By comparing the changes in the y-axis coordinates of these key points in two adjacent images, the amplitude of the nodding is calculated. Furthermore, the continuous image sequence is analyzed to identify the complete nodding cycle (that is, the process from the head being raised to falling and then returning to the starting position), and the number of nodding cycles within a certain period of time is counted to obtain the nodding frequency.

[0055] Since different drivers have different physiological conditions, the first risk score and the second risk score are relatively rough, so it is necessary to conduct a comprehensive analysis based on the individual conditions of the driver, such as Figure 2 As shown, it also includes a danger adjustment module, which obtains the driver information stored in the public transportation system, and selects the eye type (such as single eyelids, double eyelids, eye size, etc.) and health factors that may affect the degree of eyelid closure (such as eye disease history, myopia, etc.) from the driver information, and adjusts the weight value of the eyelid closure degree and the weight value of the blinking frequency according to the eye information. For example, for drivers with higher myopia, the weight of the blinking frequency may be increased, because they may blink more frequently to relieve eye fatigue. For example, for some drivers who have the habit of blinking frequently, the faster blinking frequency can also be distinguished from fatigue. In addition to eye information, the overall health status of the driver must also be considered, such as cardiovascular disease history, blood pressure status, diabetes, etc., which may affect the driver's physiological response and driving ability. The weight value of the dynamic monitoring data is further adjusted according to the health information. For example, for drivers with a history of cardiovascular disease, the weight of heart rate variability in the assessment may be increased. The adjusted weight is applied and the updated first danger score is calculated again through the weight algorithm;

[0056] When the posture monitoring instruction is obtained, the head information that is highly correlated with the head type (such as head size, neck length, etc.) is screened out from the driver information. This information may affect the accuracy of the nodding frequency assessment. The weight value of the nodding frequency is adjusted according to the head information, and the weight value of the dynamic monitoring data is adjusted according to the health information. For drivers with shorter necks, the nodding action may be more obvious, so the weight needs to be adjusted accordingly. Similarly, the adjusted weight is applied and the updated second hazard score is calculated again through the weight algorithm.

[0057] In addition, since current drivers still need to manually alarm when encountering emergencies, in order to increase the driver's alarm speed, the wristband in the present invention also includes an information interaction module. When the monitoring equipment in the public transportation system detects an emergency in the car, it will send an emergency command to the wristband, and a one-button alarm button will be displayed on the wristband body. Compared with the automatic alarm method of the public transportation system, the present invention quickly gives the driver an alarm button through the wristband, and the decision whether to alarm is made by the driver, which can avoid false alarms and the like.

[0058] In addition, since the temperature inside the bus is controlled by the bus system, and the temperature is often high or low, the information interaction module can obtain the driver's body surface temperature detected by the temperature sensor and output air-conditioning control instructions to the bus system based on the body surface temperature. When the driver's body surface temperature is too high, an instruction to increase the air-conditioning power can be output, otherwise, an instruction to reduce the air-conditioning power can be output.

[0059] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A smart bracelet worn by a driver, comprising a bracelet body and a plurality of different types of sensors located inside the bracelet, wherein the sensors include visual sensors, and characterized in that: The visual sensor is located on the side of the bracelet body, and the bracelet body also includes A face image acquisition module, which acquires the driver's face image acquired by the visual sensor; An eye image analysis module, which obtains an eye image by segmenting the face image through a semantic segmentation model, and then determines whether the eye target part can be extracted through a target matching model according to the eye image, and obtains an eye dynamic analysis instruction or a body posture monitoring instruction according to the extraction result of the eye target part; The eye dynamic analysis module, when receiving the eye dynamic analysis instruction, obtains the eyelid closure degree and the blinking frequency in the image of the eye target part by using the semantic segmentation strategy and the calculation strategy, and compares the eyelid closure degree and the blinking frequency with the threshold value respectively, and outputs the current state based on the comparison result; The posture monitoring verification module, when obtaining the posture monitoring instruction, obtains the driver's nodding frequency in the face image through the image dynamic trajectory analysis strategy, and outputs the current state according to the nodding frequency analysis.

2. According to claim 1, a smart bracelet worn by a driver, characterized in that: It also includes an indicator evaluation module, which obtains various monitoring indicators collected by other sensors as dynamic monitoring data, and calculates the dynamic monitoring data, eyelid closure degree and blinking frequency through a weight algorithm to obtain a first risk score. The eyelid closure degree is the ratio of the gap between the upper eyelid and the lower eyelid to the eye width within a certain period of time, and the blinking frequency is the number of times the upper eyelid is closed within a certain period of time.

3. According to claim 2, a smart bracelet worn by a driver, characterized in that: The index evaluation module calculates the dynamic monitoring data and the nodding frequency through a weight algorithm to obtain a second risk score, wherein the nodding frequency is the frequency at which the driver's head in the face image moves up and down within a certain period of time.

4. According to claim 3, a smart bracelet worn by a driver, characterized in that: It also includes a danger adjustment module, which obtains driver information stored in the public transportation system, selects eye information and health information that are highly correlated with eye type from the driver information, adjusts the weight value of the eyelid closure degree and the weight value of the blinking frequency according to the eye information, and adjusts the weight value of the dynamic monitoring data according to the health information, and calculates the updated first danger score again through the weight algorithm; When the posture monitoring instruction is obtained, the head information with a high correlation with the head type is screened out from the driver information, the weight value of the nodding frequency is adjusted according to the head information, and the weight value of the dynamic monitoring data is adjusted according to the health information, and the updated second risk score is calculated again through the weight algorithm.

5. A smart bracelet worn by a driver according to claim 4, characterized in that: The semantic segmentation strategy includes a semantic segmentation step and a point comparison step. The semantic segmentation step is to draw the eyelid contour line and the lower eyelid line in the image of the eye target part; The point comparison step marks the center point of the eyelid contour line and the center point of the lower eyelid line respectively, corrects the positions of the two center points so that the two center points are located on the same vertical line, and then calculates the distance between the two center points as the gap distance. In the continuous eye target part, the minimum value of the gap distance within a cycle is screened out as the target value, and the one cycle reflects the process of the gap distance from the maximum value to the minimum value. The target value is compared with the eye width to obtain the degree of eyelid closure.

6. According to claim 5, a smart bracelet worn by a driver, characterized in that: The calculation strategy includes a calculation step, wherein the calculation step presets an analysis time period, searches for the maximum value of the gap distance in the first cycle as a starting point, counts the number of cycles that the gap distance has passed during the analysis time period as the number of blinks, and calculates the blink frequency based on the number of blinks and the analysis time period.

7. According to claim 4, a smart bracelet worn by a driver, characterized in that: The dynamic trajectory analysis strategy includes a profile analysis step and a displacement calculation step. The contour analysis step obtains a plurality of consecutive face images when the vehicle is traveling on a smooth road section, performs image correction on each face image through a reference point so that the driver's head perspective in each face image is the same, and plans the driver's head contour in each face image through a contour algorithm; The displacement calculation step establishes a corresponding two-dimensional coordinate system, arbitrarily selects several calculation points in the head contour, calculates the offset of the corresponding calculation point coordinates on the y-axis as the nodding amplitude based on two adjacent face images, identifies the nodding cycle in continuous face images, and calculates the number of nodding cycles within a certain period of time to obtain the nodding frequency.

8. According to claim 4, a smart bracelet worn by a driver, characterized in that: Dynamic monitoring indicators include heart rate, blood oxygen rate, respiratory rate and electrocardiogram.

9. The smart bracelet worn by the driver according to claim 1, characterized in that: It also includes an information interaction module for obtaining emergency instructions in the public transportation system and displaying a one-touch alarm button on the bracelet body.

10. A smart bracelet worn by a driver according to claim 9, characterized in that: The information interaction module obtains the driver's body surface temperature detected by the sensor and outputs an air conditioning control instruction to the public transportation system according to the body surface temperature.

Images