Multifunctional safety helmet and use method thereof

Abstract

The invention discloses a multifunctional safety helmet and a use method thereof, and belongs to the field of safety production, and the use method of the multifunctional safety helmet comprises the following steps: before work starting, each member of a work class wears the multifunctional safety helmet, triggers a pressure sensing module, starts intelligent identification, identifies the RFID with a unique number when entering a work site, and sends the RFID to the intelligent identification module; comparing the work information with the work members on the work ticket stored in the system one by one; the device can assist a worker in charge to confirm whether staff of a working class and the number of staff conform to a work ticket or not in a working site, especially a large-scale construction site, check a to-be-overhauled line or equipment before working, remind the staff to keep a safe distance from a surrounding live line or equipment during working, check the mental state of pole-climbing and tower-climbing workers, and improve the working efficiency. The safety distance is detected and early warned, and after the work is finished, whether all operators evacuate or not can be judged, so that the safety of a working site is improved, safety accidents are avoided, and the functions are diversified.

Description

Technical Field

[0001] The invention relates to the field of safe production, and in particular to a multifunctional safety helmet and a use method thereof. Background Art

[0002] Safety helmets are one of the most important safety items widely used in the production process. They can be applied to many fields such as construction, electricity, petroleum, and transportation. A large amount of development and construction has brought serious safety issues to construction workers.

[0003] Especially in the field of power production, construction or maintenance work near live equipment or live lines involves multiple safety risks such as the risk of electric shock and falling from heights. If work is not carried out in accordance with power safety standards, safety accidents are very likely to occur. At present, in order to avoid safety accidents, on the one hand, we rely on prior investigation, familiarity with the work site environment and construction conditions, as well as the live conditions of surrounding equipment, and the preparation of "three construction measures", and strengthen management and control through organizational measures such as work ticket system, work permit system, work supervision system, work interruption and termination system; on the other hand, through power outages, power testing, grounding, installation of barriers and other technical measures to strengthen on-site management and control, to a certain extent, can avoid the occurrence of safety accidents. However, these basically rely on the operators themselves. Different work shifts face different levels of complexity of the work site, their own safety awareness and skill levels, and different levels of understanding and implementation of safety regulations, which will result in the inability to completely avoid the occurrence of safety accidents. In recent years, safety accidents have also occurred from time to time at home and abroad.

[0004] Although people's lives are becoming more and more intelligent with the rise of the Internet era, the traditional safety helmets available on the market in my country only have the most basic protection functions, with low technological content, simple and single functions, and are limited to traditional protection measures. They are unable to assist the person in charge of the work at the work site, especially at a large construction site, to confirm whether the work shift personnel and the number of personnel are consistent with the work ticket, check the lines or equipment to be inspected before the operation, remind the workers to keep a safe distance from the surrounding live lines or equipment during work, check the mental state of the workers climbing poles and towers, detect and warn of the safe distance, and determine whether all the workers have evacuated after the work is completed. Therefore, they are unable to achieve the goals of improving work efficiency, controlling casualties and accidents, and achieving safe production in the production process. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides a multifunctional safety helmet and a method of using the same to solve the problem that the safety helmet in the prior art has a single function and cannot effectively achieve the goals of controlling casualties in the production process and achieving safe production.

[0006] The present invention adopts the following technical solution.

[0007] A first aspect of the present invention discloses a method for using a multifunctional safety helmet, comprising the following steps:

[0008] Before starting work, each member of the work team wears a multifunctional safety helmet, triggers the pressure sensing module, starts intelligent identification, and enters the work site. The unique RFID number is identified and compared one by one with the work team members on the work ticket saved in the system to confirm that each work team member is present. When non-work team members or other irrelevant personnel enter, an abnormal alarm signal A is issued after identification and comparison to remind the person in charge of the work to promptly notify non-work team members and irrelevant personnel to leave the work site;

[0009] After the start of work, when the work team members approach the line or equipment to be inspected, the multifunctional safety helmet automatically identifies the line or equipment nameplate, and compares the identification result with the line or equipment to be inspected on the work ticket. If there is a discrepancy, an alarm signal B is issued to remind the work team members that the line or equipment is not to be inspected, and they should not continue to approach and should immediately stay away. After the identification results are consistent, a confirmation signal C is issued before the electricity and grounding can be tested;

[0010] After arriving at the inspection location, if there are other nearby lines or equipment, the multifunctional helmet will continue to automatically identify the line or equipment nameplate, record the line or equipment name, voltage level and other information, and remind the nearby live lines or equipment, and send out a reminder signal D;

[0011] Measure the distance of other nearby lines or equipment, and compare the distance measurement result with the safety distance of the line or equipment voltage level. If it is close to or less than the safety distance, the alarm signal E will be issued. As the safety distance gradually approaches the minimum allowable safety distance, the sound and frequency of the alarm signal E will increase. When it leaves 1.5 times the safety distance, the alarm signal E disappears.

[0012] When working at high altitude on the tower, the human body status sensor inside the multifunctional helmet monitors the status of the workers in real time and uploads abnormal status to the host computer. The human body status sensor is used to monitor the temperature, blood oxygen and heart rate of the human body. The multifunctional helmet turns on the automatic face recognition mode between members of different work shifts to determine whether the workers are in a bad mental state or in a fatigued state. The abnormal information is uploaded to the host computer, which makes judgments based on these two abnormal information and reminds the work shift members and the person in charge of the work;

[0013] When encountering danger at work, the work team members touch the alarm button to activate the emergency alarm. After receiving the alarm information, the surrounding work team members and the work supervisor will promptly carry out emergency rescue work and help the work team members in danger out of danger in a timely manner;

[0014] At the end of the workday, the RFID of each work team member is identified when he / she leaves the work site, and is compared one by one with the work team members on the work ticket saved in the system to confirm that each work team member has left. The work team members and the work supervisor who have not left in time will receive an alarm signal E, reminding the work supervisor to confirm whether the work team members who have not left have left the work site. All work team members are not allowed to report power supply before they leave the work site.

[0015] As an improvement, equipment nameplate identification includes the following steps:

[0016] Image preprocessing;

[0017] Get the nameplate recognition model;

[0018] Template matching;

[0019] Feature extraction;

[0020] Impression sampling;

[0021] Reconstruct the image;

[0022] Optical character recognition;

[0023] Information comparison.

[0024] As an improvement, automatic face recognition to determine whether the operator is in a bad mental state or in a fatigued state includes the following steps:

[0025] Build the database,

[0026] We obtained the facial expression pictures of the staff through web crawlers, and then invited professionals with field experience and psychology experts to manually annotate and label the acquired facial expression pictures of the staff. Finally, we performed normalization operations on the facial expression images with the same annotations, including rotation, cropping, normalization, etc., and finally completed the database construction;

[0027] Image preprocessing,

[0028] In the processing of the self-built database data set, the MTCNN algorithm is used to capture the key points of the face, and the translation and linear correlation algorithm is combined to rotate and correct the tilted or irregular pictures. The original coordinates are mapped to the new coordinate axes by scaling and translating the coordinate axes.

[0029] Image enhancement,

[0030] Decompose the image into multiple scales and perform independent illumination compensation for each scale component;

[0031] Perform face detection,

[0032] To locate the human eye,

[0033] The human eye positioning method adopts grayscale integral projection combined with automatic feature point positioning based on constrained local model. First, the human eye position is preliminarily determined, and then the key point positioning problem is transformed into the problem of solving the optimal solution of the function, and then the local model is re-positioned and compared.

[0034] Detect the closed state of human eyes,

[0035] The closed state of human eyes is determined by the human eye area ratio method, and the number of image frames and blinking frequency of closed eye states are counted using multiple sets of test data to obtain the human eye change curve;

[0036] Determine fatigue detection methods,

[0037] The fatigue detection system combines the PERCLOS judgment standard with the human eye change curve to design the overall flow chart of the fatigue work detection system, and conducts multiple groups of experimental comparisons to determine the fatigue work detection coefficient and judge whether the system can perform fatigue work detection in real time and accurately.

[0038] As an improvement, pedestrian re-identification technology is used to determine whether a certain person exists in the video. With the support of computer vision technology, given a pedestrian image, the pedestrian is retrieved from other images to achieve cross-camera search and pedestrian re-identification in video sequences. While extracting the features of a single-frame pedestrian image, the pedestrian's temporal features are extracted through a recursive loop network, thereby determining whether there are still work team members who have not left the area.

[0039] As an improvement, face detection includes the following steps:

[0040] Feature extraction,

[0041] In face detection, the main features include Haar features and LBP features;

[0042] Selection of weak classifiers,

[0043] The weak classifier used is a decision tree classifier based on a single Haar feature;

[0044] Create a cascade structure.

[0045] The AdaBoost algorithm is used in combination with a cascade structure, which consists of multiple strong classifiers. Each strong classifier contains a set of weak classifiers. During the detection process, the image is first quickly screened by the first strong classifier to exclude non-face areas, and then detected and verified by subsequent strong classifiers.

[0046] As an improvement, the eye positioning includes the following steps:

[0047] Grayscale integral projection rough positioning,

[0048] Grayscale integral projection locates the eyes by projecting the image horizontally and vertically based on the distribution information of the peaks and valleys.

[0049] Assume that the sum of the pixels in the yth column of the image is ph(y), then the horizontal grayscale projection is to add the pixel values ​​of each column in the image to obtain a horizontal projection curve, and the vertical grayscale projection is to add the pixel values ​​of each row in the image to obtain a vertical projection curve.

[0050] By analyzing these two projection curves, we can preliminarily find the location of the eyes. The grayscale value of the eye area is significantly different from that of other parts of the face, which will form obvious peaks or troughs on the projection curve.

[0051] The key point positioning problem is transformed into the problem of solving the optimal solution of the function. On the basis of rough positioning, the key points of the eye are further located, and an objective function is defined. The objective function describes the matching degree between the feature points and the corresponding positions in the image and the relative position relationship between the feature points. The objective function includes the error term of feature point matching and the constraint term of the relative position relationship between feature points. The error term is used to measure the difference between the feature points and their corresponding positions in the image. The constraint term of the relative position relationship between feature points is used to ensure that the geometric relationship between feature points meets expectations. By solving the optimal solution of the objective function, the position of the key points of the eye is further found.

[0052] The objective function is:

[0053] in:

[0054] a i is the predicted position of the i-th feature point,

[0055] b i is the true position of the i-th feature point,

[0056] λ is the weight coefficient used to balance the matching error and the constraints of relative position relationship.

[0057] f(a i ,a i+1 ) is a function that describes the relative position relationship between feature points;

[0058] Fine positioning is performed with the help of automatic feature point positioning based on the constrained local model.

[0059] As an improvement, detecting the closed state of human eyes includes the following steps:

[0060] Using the human eye area ratio method,

[0061] After locating the eye key points, the eye area ratio is calculated to determine the closed state of the eyes.

[0062] The formula for calculating the eye area ratio is:

[0063] in:

[0064] EAR is the aspect ratio of the eye, which is the ratio of the height to the width of the eye area.

[0065] h is the height of the eye area, that is, the distance between the upper and lower eyelids,

[0066] w is the width of the eye area, that is, the distance between the left and right eye corners,

[0067] C is a constant factor used to adjust the eye area ratio so that it remains in a relatively constant range when the eyes are open. It can be between 1 and 3.

[0068] Then the eye state is determined, and the eye area ratio is compared with a preset threshold. If the eye area ratio is less than the threshold, it is considered that the eyes are in a closed state, and the counter is incremented by one.

[0069] Count the number of image frames, blink frequency, and blink duration of the closed eye state.

[0070] By continuously capturing and analyzing the eye area ratio in the image frames, the number of image frames in the eye closed state is counted;

[0071] The human eye change curve is obtained.

[0072] Plotting the continuously captured eye area ratios into a curve can intuitively show the changes in the eyes. This curve can be used to monitor the degree of eye fatigue, determine blinking movements, etc.

[0073] As an improvement, counting the number of image frames, blink frequency, and blink duration of a person's closed eyes includes the following steps:

[0074] Count the number of blinks,

[0075] A blink event is identified by detecting a significant drop and subsequent significant rise in the ratio of the eye region;

[0076] Calculate the blink frequency,

[0077] The calculation formula is:

[0078] in:

[0079] B is the number of blinks,

[0080] F is the blinking frequency,

[0081] The time interval is T seconds;

[0082] Calculate the blink duration,

[0083] The calculation formula is

[0084] in:

[0085] Δt is the blink duration.

[0086] As an improvement, in the fatigue detection method, if the PERCLOS value exceeds 80%, it is judged as a fatigue state; if the eye area ratio drops significantly and is less than the eye area ratio threshold, it is judged as an eye closed state, which is further used to calculate the PERCLOS value.

[0087] A second aspect of the present invention discloses a multifunctional safety helmet, comprising:

[0088] Helmet body, main controller, intelligent identification module, memory, pressure sensing module, power module, distance measurement module, RFID module, alarm module, communication module and human body status sensor;

[0089] The main body of the helmet is used to protect the operator's head and carry the main controller, intelligent identification module, memory, pressure sensing module, power module, distance measurement module, RFID module, alarm module, communication module, and human state sensor;

[0090] The main controller is electrically connected to the intelligent recognition module, the memory, the pressure sensing module, the power module, the distance measuring module, the RFID module, the warning module, and the communication module. The main controller is used to control the intelligent recognition module, the memory, the pressure sensing module, the power module, the distance measuring module, the RFID module, the alarm module, the communication module, the human body status sensor, and communicate with the host computer;

[0091] The intelligent identification module is used to intelligently identify the line or nameplate;

[0092] The pressure sensing module is used to determine whether the work team members are wearing multi-functional safety helmets, and activate various functions if they are worn;

[0093] The alarm module is used for sound and light alarm and sends alarm information to the host computer with the help of the communication module. The alarm module is electrically connected to an alarm button.

[0094] The beneficial effect of the present invention lies in that, compared with the prior art, it can assist the person in charge of the work at the work site, especially at a large construction site, to confirm whether the work shift personnel and the number of personnel are consistent with the work ticket, and to check the lines or equipment to be inspected before the operation. When working, it can remind the workers to keep a safe distance from the surrounding live lines or equipment, check the mental state of the workers climbing poles and towers, detect and warn of the safe distance, and after the work is completed, it can determine whether all the workers have evacuated, thereby improving the safety of the work site and avoiding the occurrence of safety accidents. The functions are diverse. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 It is a structural schematic diagram of the present invention;

[0096] Figure 2 It is a schematic diagram of the display system of the present invention. DETAILED DESCRIPTION

[0097] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0098] Example 1

[0099] A method for using a multifunctional safety helmet comprises the following steps:

[0100] Step 1: Before starting work, each member of the work team wears a multifunctional safety helmet, triggers the pressure sensing module, starts intelligent identification, and enters the work site. The unique RFID number is identified and compared with the work team members on the work ticket saved in the system to confirm that each work team member is present. When non-work team members or other unrelated personnel enter, an abnormal alarm signal A is issued after identification and comparison to remind the person in charge of the work to promptly notify non-work team members and unrelated personnel to leave the work site;

[0101] Step 2: After the start of work, when the work team members approach the line or equipment to be inspected, the multifunctional safety helmet automatically identifies the line or equipment nameplate, and compares the identification result with the line or equipment to be inspected on the work ticket. If there is a discrepancy, an alarm signal B is issued to remind the work team members that the line or equipment is not to be inspected, and they should not continue to approach and should immediately stay away. After the identification results are consistent, a confirmation signal C is issued before the electricity test and grounding can be carried out;

[0102] Step 3: After arriving at the inspection location, if there are other nearby lines or equipment, the multifunctional helmet continues to automatically identify the line or equipment nameplate, records the line or equipment name, voltage level and other information, and reminds that there are nearby live lines or equipment, and sends a reminder signal D;

[0103] Step 4: measure the distance of other nearby lines or equipment, and compare the distance measurement result with the safety distance of the voltage level of the line or equipment. If it is close to or less than the safety distance, an alarm signal E is issued. The alarm signal E gradually approaches the minimum allowable safety distance as the safety distance, and the sound and frequency increase accordingly. When it leaves 1.5 times the safety distance, the alarm signal E disappears;

[0104] Step 5: During high-altitude operations on the tower, the human body status sensor inside the multifunctional helmet monitors the status of the worker in real time and uploads abnormal status to the host computer. The human body status sensor is used to monitor the temperature, blood oxygen, and heart rate of the human body. Between members of different work shifts, the multifunctional helmet turns on the automatic face recognition mode to determine whether the worker is in a bad mental state or in a fatigued state. The abnormal information is uploaded to the host computer, which makes a judgment based on these two abnormal information and reminds the work shift members and the person in charge of the work;

[0105] Step 6: When encountering danger at work, the work team members touch the alarm button to start the emergency alarm. After receiving the alarm information, the surrounding work team members and the work supervisor will promptly carry out emergency rescue work and help the work team members in danger out of danger in a timely manner;

[0106] Step 7, when finishing work, the RFID of each work team member is identified when leaving the work site, and compared with the work team members on the work ticket saved in the system one by one to confirm that each work team member has left. The work team members who have not left in time and the work supervisor will receive an alarm signal E to remind the work supervisor to confirm whether the work team members who have not left have left the work site. All work team members are not allowed to report power supply before leaving the work site.

[0107] In a preferred but non-limiting implementation of this embodiment, the equipment nameplate identification in step 2 includes the following steps:

[0108] Step 2.1, image preprocessing;

[0109] Step 2.2, obtaining the nameplate recognition model;

[0110] Step 2.3, template matching;

[0111] Step 2.4, feature extraction;

[0112] Step 2.5, impression sampling;

[0113] Step 2.6, reconstruct the image;

[0114] Step 2.7, optical character recognition;

[0115] Step 2.8, information comparison.

[0116] In a preferred but non-limiting implementation of this embodiment, the step 5 of determining whether the operator is in a poor mental state or in a fatigued state comprises the following steps:

[0117] Step 5.1, build the database,

[0118] We obtained the facial expression pictures of the staff through web crawlers, and then invited professionals with field experience and psychology experts to manually annotate and label the acquired facial expression pictures of the staff. Finally, we performed normalization operations on the facial expression images with the same annotations, including rotation, cropping, normalization, etc., and finally completed the database construction;

[0119] Step 5.2, image preprocessing,

[0120] In the processing of the self-built database data set, the MTCNN algorithm is used to capture the key points of the face, and the tilted or irregular pictures are rotated and corrected by combining the affine transformation. The original coordinates are mapped to the new coordinate axes by scaling and translating the coordinate axes.

[0121] MTCNN consists of three cascaded convolutional neural networks, namely P-Net, R-Net and O-Net.

[0122] P-Net (Proposal Network): Generates face boxes (Proposals). P-Net is a shallow convolutional neural network that scans the input image and generates multiple candidate boxes that cover areas that may contain faces.

[0123] R-Net (Refine Network): refines the face frames output by P-Net and removes low-quality frames. R-Net is a deeper convolutional network used to filter out low-quality candidate frames generated by P-Net, remove false positives through more sophisticated feature extraction and discrimination, and refine the retained candidate frames.

[0124] O-Net (Output Network): further refines the face frame and regresses the five key points of the face (eyes, nose and mouth corners);

[0125] Step 5.3 Image enhancement,

[0126] Decompose the image at multiple scales and perform independent illumination compensation on each scale component, so as to achieve adaptive adjustment for uneven illumination of the image and solve the problem that the image is affected by uneven illumination due to illumination and reflection during image shooting.

[0127] Step 5.4, perform face detection,

[0128] Considering the working environment, the AdaBoost algorithm is used as the core method for face detection.

[0129] Step 5.5, perform eye positioning,

[0130] The human eye positioning method adopts grayscale integral projection combined with automatic feature point positioning based on constrained local model. First, the human eye position is preliminarily determined, and then the key point positioning problem is transformed into the problem of solving the optimal solution of the function. Then, the local model is positioned and compared twice, thus establishing a hybrid positioning method, which not only saves calculation time, but also can achieve real-time monitoring of the system.

[0131] Step 5.6, detect the closed state of the human eye,

[0132] The closed state of human eyes is determined by the human eye area ratio method, and the number of image frames and blinking frequency of closed eye states are counted using multiple sets of test data to obtain the human eye change curve;

[0133] Step 5.7, determine the fatigue detection method,

[0134] The fatigue detection system combines the PERCLOS judgment standard with the human eye change curve to design the overall flow chart of the fatigue work detection system, and conducts multiple groups of experimental comparisons to determine the fatigue work detection coefficient and judge whether the system can perform fatigue work detection in real time and accurately.

[0135] In a preferred but non-restrictive implementation of this embodiment, step 7 adopts pedestrian re-identification technology to determine whether a certain person exists in the video. With the support of computer vision technology, given a pedestrian image, the pedestrian is retrieved from other images to achieve cross-camera search. The purpose is to make up for the visual defects of fixed cameras. It can be combined with target detection and tracking technology to achieve functions such as personnel search. The purpose of pedestrian re-identification in video sequences is to obtain more information, not only relying on the pedestrian information contained in the image, but also relying on the status and motion behavior information of the pedestrian in the entire sequence. While extracting the features of a single-frame pedestrian image, the pedestrian time series features are extracted through a recursive loop network to determine whether there are still members of the work team who have not left the area.

[0136] In a preferred but non-limiting implementation of this embodiment, the face detection in step 5.4 includes the following steps:

[0137] Step 5.4.1, feature extraction,

[0138] In face detection, the main features include Haar features and LBP (local binary pattern) features. These features can capture the local texture and shape information in the image, and quickly calculate Haar features through integral images, thereby accelerating the process of face detection;

[0139] Step 5.4.2, selection of weak classifiers,

[0140] The weak classifiers used are decision tree classifiers based on a single Haar feature. These weak classifiers are simple and easy to obtain, and can be effectively combined into strong classifiers under the AdaBoost framework.

[0141] Step 5.4.3, establish the cascade structure,

[0142] In order to further improve the speed and accuracy of face detection, the AdaBoost algorithm is used in combination with a cascade structure. The cascade structure consists of multiple strong classifiers, each of which contains a set of weak classifiers. During the detection process, the image is first quickly screened by the first strong classifier to exclude a large number of non-face areas, and then more refined detection and verification are performed by subsequent strong classifiers.

[0143] This cascade structure can significantly reduce the amount of calculation and increase the detection speed while maintaining high accuracy.

[0144] In a preferred but non-limiting implementation of this embodiment, the eye positioning in step 5.5 includes the following steps:

[0145] Step 5.5.1: Grayscale integral projection rough positioning.

[0146] Grayscale integral projection locates the eyes by projecting the image horizontally and vertically based on the distribution information of the peaks and valleys.

[0147] Assume that the sum of the pixels in the yth column of the image is ph(y), then the horizontal grayscale projection is to add the pixel values ​​of each column in the image to obtain a horizontal projection curve, and the vertical grayscale projection is to add the pixel values ​​of each row in the image to obtain a vertical projection curve.

[0148] By analyzing these two projection curves, we can preliminarily find the location of the eyes. The grayscale value of the eye area is significantly different from that of other parts of the face, which will form obvious peaks or troughs on the projection curve.

[0149] The key point positioning problem is transformed into the problem of solving the optimal solution to the function. On the basis of rough positioning, the key points of the eye (corners of the eye, pupils) are further located, and an objective function is defined. The objective function describes the matching degree between the feature points and the corresponding positions in the image and the relative position relationship between the feature points. The objective function includes the error term of feature point matching and the constraint term of the relative position relationship between feature points. The error term is used to measure the difference between the feature points and their corresponding positions in the image. The constraint term of the relative position relationship between feature points is used to ensure that the geometric relationship between feature points meets expectations, such as the distance and angle between the corners of the eye. By solving the optimal solution of the objective function, the position of the key points of the eye is further found.

[0150] The objective function is:

[0151] in:

[0152] a i is the predicted position of the i-th feature point,

[0153] b i is the true position of the i-th feature point,

[0154] λ is the weight coefficient used to balance the matching error and the constraints of relative position relationship.

[0155] f(a i ,a i+1 ) is a function that describes the relative position relationship between feature points;

[0156] Step 5.5.2, fine positioning is performed by automatic feature point positioning based on the constrained local model.

[0157] By searching for matches in the neighborhood of each feature point, the best matching position is found, and the position of the feature point is gradually adjusted by iteratively optimizing the objective function. Each feature point is represented as a local model that describes the image features around the feature point. The position of the feature point is further found by comparing the degree of matching between the local model and the corresponding position in the image. At the same time, the relative position relationship between the feature points is considered, and constraints are used to ensure that the arrangement of the feature points conforms to the geometric structure of the face.

[0158] In a preferred but non-limiting implementation of this embodiment, the detection of the closed state of the human eye in step 5.6 comprises the following steps:

[0159] Step 5.6.1, using the human eye area ratio method,

[0160] After locating the eye key points, the eye area ratio is calculated to determine the closed state of the eyes.

[0161] The formula for calculating the eye area ratio is:

[0162] in:

[0163] EAR is the aspect ratio of the eye, which is the ratio of the height to the width of the eye area.

[0164] h is the height of the eye area, that is, the distance between the upper and lower eyelids,

[0165] w is the width of the eye area, that is, the distance between the left and right eye corners,

[0166] C is a constant factor used to adjust the eye area ratio so that it remains in a relatively constant range when the eyes are open. It can be between 1 and 3.

[0167] Then the eye state is determined, and the eye area ratio is compared with a preset threshold. If the eye area ratio is less than the threshold, it is considered that the eyes are in a closed state, and the counter is incremented by one.

[0168] The effectiveness of calculating the eye area ratio lies in that when the eyes are open, the eye area ratio remains in a relatively stable range; when the eyes are closed, the height h will be significantly reduced due to the closeness or contact between the upper and lower eyelids, resulting in a sharp drop in the eye area ratio. Therefore, by setting the eye area ratio threshold, it is possible to determine whether the eyes are in a closed state.

[0169] Step 5.6.2, count the number of image frames, blink frequency, and blink duration of the closed eye state.

[0170] By continuously capturing and analyzing the eye area ratio in the image frames, the number of image frames in the eye closed state is counted;

[0171] Step 5.6.3, obtain the human eye change curve,

[0172] Plotting the continuously captured eye area ratios into a curve can intuitively show the changes in the eyes. This curve can be used to monitor the degree of eye fatigue, determine blinking movements, etc.

[0173] In a preferred but non-limiting implementation of this embodiment, the counting of the number of image frames, blink frequency, and blink duration of the closed eye state in step 5.6.2 includes the following steps:

[0174] Step 5.6.2.1, count the number of blinks,

[0175] A blink event is identified by detecting a significant drop and subsequent significant rise in the ratio of the eye region;

[0176] Step 5.6.2.2, calculate the blink frequency,

[0177] The calculation formula is:

[0178] in:

[0179] B is the number of blinks,

[0180] F is the blinking frequency,

[0181] The time interval is T seconds

[0182] Further analysis of these data can yield parameters such as blinking frequency, which refers to the number of times the eyes close and open per unit time.

[0183] Step 5.6.2.2, calculate the blink duration,

[0184] The calculation formula is

[0185] in:

[0186] Δt is the blink duration.

[0187] In a preferred but non-limiting implementation of this embodiment, in the fatigue detection method determined in step 5.7, if the PERCLOS value exceeds 80%, it is judged to be a fatigue state; if the eye area ratio drops significantly and is less than the eye area ratio threshold, it is judged to be an eye closed state, which is further used to calculate the PERCLOS value.

[0188] Example 2

[0189] A multifunctional safety helmet, comprising:

[0190] Safety helmet body, main controller, intelligent identification module, memory, pressure sensing module, power module, distance measurement module, RFID module, alarm module, communication module and human body status sensor.

[0191] The main body of the helmet is used to protect the operator's head and carry the main controller, intelligent identification module, memory, pressure sensing module, power module, distance measurement module, RFID module, alarm module, communication module, and human state sensor;

[0192] The main controller is electrically connected to the intelligent recognition module, the memory, the pressure sensing module, the power module, the distance measuring module, the RFID module, the warning module, and the communication module. The main controller is used to control the intelligent recognition module, the memory, the pressure sensing module, the power module, the distance measuring module, the RFID module, the alarm module, the communication module, the human body status sensor, and communicate with the host computer;

[0193] The intelligent identification module is used to intelligently identify the line or nameplate;

[0194] The pressure sensing module is used to determine whether the work team members are wearing multi-functional safety helmets, and activate various functions if they are worn;

[0195] The alarm module is used for sound and light alarm and sends alarm information to the upper computer with the help of the communication module. The alarm module is electrically connected to the alarm button;

[0196] The upper computer is electrically connected to a display for viewing various information. A plurality of multifunctional safety helmets are used in conjunction with the upper computer electrically connected to the display to form a display system.

[0197] The beneficial effect of the present invention lies in that, compared with the prior art, it can assist the person in charge of the work at the work site, especially at a large construction site, to confirm whether the work shift personnel and the number of personnel are consistent with the work ticket, and to check the lines or equipment to be inspected before the operation. When working, it can remind the workers to keep a safe distance from the surrounding live lines or equipment, check the mental state of the workers climbing poles and towers, detect and warn of the safe distance, and after the work is completed, it can determine whether all the workers have evacuated, thereby improving the safety of the work site and avoiding the occurrence of safety accidents. The functions are diverse.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for using a multifunctional safety helmet, characterized in that: The following steps are involved: Before starting work, the work team members wear multifunctional safety helmets, trigger the pressure sensing module, start intelligent identification, confirm the work team members' information, and when non-work team members enter, remind the work leader to promptly notify non-work team members to leave the work site; After the start of work, when the work team members approach the line or equipment to be inspected, the multifunctional safety helmet automatically identifies the line or equipment nameplate. If the identification results are inconsistent, the work team members are reminded not to continue approaching. Only after the identification results are consistent can the electricity and grounding be tested; After arriving at the inspection location, if there are other nearby lines or equipment, the multifunctional helmet will continue to automatically identify the line or equipment nameplate, record the line or equipment name, voltage level information, and remind the nearby live lines or equipment; Measure the distance of other nearby lines or equipment, and compare the distance measurement result with the safe distance of the voltage level of the line or equipment. If it is close to or less than the safe distance, an alarm signal E will be issued; When working at high altitude on the tower, the status of the workers is monitored in real time, and the abnormal status is uploaded to the host computer. The human body status sensor is used to monitor the status of the human body. The multifunctional safety helmet turns on the automatic face recognition mode between members of different work shifts to determine whether the workers are in a bad mental state or in a fatigued state. The abnormal information is uploaded to the host computer, which makes judgments based on these two abnormal information and reminds the work shift members and the person in charge of the work; When encountering danger at work, surrounding work team members and work supervisors will carry out emergency rescue work in a timely manner after receiving the alarm information; When finishing work, confirm the information of the work team members again to confirm that each work team member has left. The work team members and the work supervisor who have not left in time will receive an alarm signal E.

2. The method for using a multifunctional safety helmet according to claim 1, characterized in that: Equipment nameplate identification includes the following steps: Image preprocessing; Get the nameplate recognition model; Template matching; Feature extraction; Impression sampling; Reconstruct the image; Optical character recognition; Information comparison.

3. The method for using a multifunctional safety helmet according to claim 1, characterized in that: Automatic face recognition to determine whether an operator is in a bad mental state or fatigued includes the following steps: Build the database, Image preprocessing, In the processing of self-built database data sets, the key points of the face are captured, the tilted or irregular images are corrected, and the original coordinates are mapped to the new coordinate axis; Image enhancement, Decompose the image into multiple scales and perform independent illumination compensation for each scale component; Perform face detection, To locate the human eye, Detect the closed state of human eyes, Determine fatigue detection methods, The fatigue detection system combines the PERCLOS judgment standard with the human eye change curve to design the overall flow chart of the fatigue work detection system, and conducts multiple groups of experimental comparisons to determine the fatigue work detection coefficient and judge whether the system can perform fatigue work detection in real time and accurately.

4. The method for using a multifunctional safety helmet according to claim 1, characterized in that: Pedestrian re-identification technology is used to determine whether a certain person exists in the video. Given a pedestrian image, the pedestrian is retrieved from other images. Pedestrian re-identification in video sequences extracts the features of single-frame pedestrian images and the temporal features of pedestrians at the same time, thereby determining whether there are still members of the work team who have not left the area.

5. The method for using a multifunctional safety helmet according to claim 3, characterized in that: Performing face detection includes the following steps: Feature extraction, In face detection, the main features include Haar features and LBP features; Selection of weak classifiers, The weak classifier adopted is the decision tree classifier; Create a cascade structure. During the detection process, the image is first quickly screened by the first strong classifier to exclude non-face areas, and then detected and verified by subsequent strong classifiers.

6. The method for using a multifunctional safety helmet according to claim 3, characterized in that: Eye positioning includes the following steps: Grayscale integral projection rough positioning, Grayscale integral projection locates the eyes by projecting the image horizontally and vertically based on the distribution information of the peaks and valleys. Assume that the sum of the pixels in the yth column of the image is ph(y), then the horizontal grayscale projection is to add the pixel values ​​of each column in the image to obtain a horizontal projection curve, and the vertical grayscale projection is to add the pixel values ​​of each row in the image to obtain a vertical projection curve. By analyzing these two projection curves, we can preliminarily find the location of the eyes. The key point positioning problem is transformed into the problem of solving the optimal solution of the function. On the basis of rough positioning, the key points of the eye are further located, and an objective function is defined. The objective function describes the matching degree between the feature points and the corresponding positions in the image and the relative position relationship between the feature points. The objective function includes the error term of feature point matching and the constraint term of the relative position relationship between feature points. The error term is used to measure the difference between the feature points and their corresponding positions in the image. The constraint term of the relative position relationship between feature points is used to ensure that the geometric relationship between feature points meets expectations. By solving the optimal solution of the objective function, the position of the key points of the eye is further found. The objective function is: in: a i is the predicted position of the i-th feature point, b i is the true position of the i-th feature point, λ is the weight coefficient, which is used to balance the matching error and the constraints of the relative position relationship. f(a i ,a i+1 ) is a function that describes the relative position relationship between feature points.

7. The method for using a multifunctional safety helmet according to claim 3, characterized in that: Detecting the closed state of human eyes includes the following steps: Using the human eye area ratio method, After locating the eye key points, the eye area ratio is calculated to determine the closed state of the eyes. The formula for calculating the eye area ratio is: in: EAR is the aspect ratio of the eye, which is the ratio of the height to the width of the eye area. h is the height of the eye area, that is, the distance between the upper and lower eyelids, w is the width of the eye area, that is, the distance between the left and right eye corners, C is a constant factor used to adjust the eye area ratio so that it remains in a relatively constant range when the eyes are open. It can be between 1 and 3. Then the eye state is determined, and the eye area ratio is compared with a preset threshold. If the eye area ratio is less than the threshold, it is considered that the eyes are in a closed state, and the counter is incremented by one. Count the number of image frames, blink frequency, and blink duration of the closed eye state. By continuously capturing and analyzing the eye area ratio in the image frames, the number of image frames in the eye closed state is counted; The human eye change curve is obtained. The ratios of the continuously captured eye areas are plotted as a curve.

8. The method for using a multifunctional safety helmet according to claim 7, characterized in that: Counting the number of image frames, blink frequency, and blink duration of a person's closed eyes includes the following steps: Count the number of blinks, A blink event is identified by detecting a significant drop and subsequent significant rise in the ratio of the eye region; Calculate the blink frequency, The calculation formula is: in: B is the number of blinks, F is the blinking frequency, The time interval is T seconds; Calculate the blink duration, The calculation formula is in: Δt is the blink duration.

9. The method for using a multifunctional safety helmet according to claim 3, characterized in that: In the fatigue detection method, if the PERCLOS value exceeds 80%, it is judged as a fatigue state. If the eye area ratio drops significantly and is less than the eye area ratio threshold, it is judged as an eye closed state, which is further used to calculate the PERCLOS value.

10. A multifunctional safety helmet, a method for using a multifunctional safety helmet according to any one of claims 1 to 9, characterized in that: include: Helmet body, main controller, intelligent identification module, memory, pressure sensing module, power module, distance measurement module, RFID module, alarm module, communication module and human body status sensor; The main body of the helmet is used to protect the operator's head and carry the main controller, intelligent identification module, memory, pressure sensing module, power module, distance measurement module, RFID module, alarm module, communication module, and human state sensor; The main controller is electrically connected to the intelligent recognition module, the memory, the pressure sensing module, the power module, the distance measuring module, the RFID module, the warning module, and the communication module. The main controller is used to control the intelligent recognition module, the memory, the pressure sensing module, the power module, the distance measuring module, the RFID module, the alarm module, the communication module, the human body status sensor, and communicate with the host computer; The intelligent identification module is used to intelligently identify the line or nameplate; The pressure sensing module is used to determine whether the work team members are wearing multi-functional safety helmets, and activate various functions if they are worn; The alarm module is used for sound and light alarm and sends alarm information to the host computer with the help of the communication module. The alarm module is electrically connected to an alarm button.

Images