Construction site safety monitoring system based on unmanned aerial vehicle vision

Abstract

The invention discloses a construction site safety monitoring system based on unmanned aerial vehicle vision, relates to the technical field of construction site safety monitoring, and solves the problem that whether related constructors wear safety helmets or not and whether the safety helmets are worn up to the standard or not cannot be identified in a unified manner. According to the method, the unmanned aerial vehicle is adopted to monitor the construction area of the construction site in real time, the monitoring picture is determined in real time, then the face image is selected from the monitoring picture based on the related extraction features, the face image is subjected to initial analysis, and the face image is obtained by locking the center point and the to-be-processed point. According to the method, whether the helmet of the corresponding building constructor is worn abnormally or not can be preliminarily judged by identifying whether the mean values of the included angles of the connecting lines in the contours are consistent or not, related judgment signals are generated and displayed in time, the monitoring mode is more comprehensive, and a better construction site safety supervision effect can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of construction site safety monitoring, and in particular to a construction site safety monitoring system based on drone vision. Background Art

[0002] With the rapid development of science and technology, the identification and prevention of safety hazards have become an indispensable part of various fields. Especially in the field of highway construction, the timely discovery and treatment of safety hazards are of great significance to ensure personnel safety and reduce economic losses. In this context, drone airports, as a fast and flexible inspection tool, have shown their unique advantages and potential in the highway field.

[0003] Through automated control and intelligent management, drone airports can achieve rapid scheduling and periodic inspections, perfectly meeting the requirements for safety inspections during highway construction.

[0004] Equipped with high-definition cameras and advanced sensors, drones are able to capture details that are difficult to see on the ground.

[0005] The application with publication number CN113296427B discloses a construction site safety monitoring system based on the Internet of Things, which relates to the field of construction safety monitoring technology; it includes an environmental analysis module, a database, an environmental safety assessment module and a behavior analysis module; the environmental analysis module is used to analyze environmental parameter data to obtain a first safety state; the environmental safety assessment module is used to obtain the current environmental video information of construction workers in real time; and evaluate the environmental safety in combination with the working environment safety status data in the database to obtain a second safety state; the behavior analysis module is used to identify the actions or movement behaviors of construction workers to obtain identification results of illegal actions or movements; and perform safety analysis based on the identification results to obtain a third safety state; the present invention comprehensively analyzes the environment in which the construction workers are located in combination with the first safety state, the second safety state and the third safety state, and gives reminders based on the analysis results, thereby saving human resource consumption and improving safety.

[0006] Most of the potential safety hazards at construction sites are caused by construction workers not wearing safety helmets correctly. However, the existing safety monitoring system can only determine whether construction workers are wearing safety helmets and issue safety warnings through preset image feature recognition methods. However, this method is relatively one-sided and cannot uniformly identify whether construction workers are wearing safety helmets and whether their safety helmets meet the standards, resulting in poor comprehensiveness of safety monitoring. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a construction site safety monitoring system based on drone vision, which solves the problem of not being able to uniformly identify whether relevant construction workers are wearing safety helmets and whether their safety helmets are worn in compliance with standards.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: A construction site safety monitoring system based on drone vision, comprising:

[0009] The drone vision end controls the drone to fly according to the preset trajectory and takes real-time photos of the construction area. The internal visual end of the drone has its internal shooting angle set in advance before the flight.

[0010] The image initial processing end locks the relevant person appearing in the construction picture based on the set human features, and then extracts the associated image of the facial part of the relevant person based on the preset face template, adjusts the clarity of the associated image to obtain the image to be checked, and transmits it to the image initial analysis end;

[0011] The image initial analysis end identifies the overall relevant contours of the image to be verified based on the processed image to be verified, and by eliminating the face area in the image to be verified, determines whether the person associated with the image to be verified has abnormal wearing. The specific method is as follows:

[0012] Based on the marked face area in the image to be checked, the center point of the face area is locked, a set of vertical lines passing through the center point and perpendicular to the horizontal base plane is constructed, and the horizontal base plane is a preset horizontal plane. The intersection point of the vertical line and the overall outer contour of the image to be checked is locked, and the intersection point is marked as a point to be processed;

[0013] The same plane where the center point and the point to be processed are located is marked as a selected surface, the part of the contour inside the selected surface is marked as a characteristic contour, and the characteristic contour is equidistantly divided into a number of characteristic points;

[0014] The points to be processed divide this feature contour into a left contour and a right contour, and prioritize identifying the associated angle between adjacent points in the left contour and the horizontal plane. Connect the adjacent feature points and translate the line to intersect the horizontal plane. Lock the angle and mark it as J. i , where i represents the angle between the lines of different adjacent points, and the angles J determined by the left contour are i Perform mean processing to determine the left contour mean JZ1;

[0015] Then the right contour is processed in the same way to determine the right contour mean JZ2;

[0016] If JZ1≠JZ2, it means the helmet is worn abnormally or the helmet is not worn, and subsequent related analysis and processing are required;

[0017] If JZ1=JZ2, it means the helmet is worn normally and no further processing is required;

[0018] The secondary adjustment recognition end, for the situation that the person associated with the image to be checked has abnormal wearing, makes relevant adjustments to the points to be processed marked by the characteristic contour in the image to be checked, and identifies whether there is still an abnormality after the relevant adjustment. If there is still an abnormality, the point angle processing end is executed. The specific method is as follows:

[0019] Based on the points to be processed marked in the characteristic contour, the points to be processed are moved on the characteristic contour, and it is recorded whether the left contour mean JZ1 and the right contour mean JZ2 satisfy the following conditions in each different movement process: JZ1 = JZ2:

[0020] If it exists, a signal indicating that the wearing does not meet the standard is generated through the signal generating end, and relevant information is displayed directly;

[0021] If it does not exist, it means that there is still an abnormality in the associated personnel, and the point angle processing end is executed for association analysis;

[0022] The point angle processing end, based on the characteristic contour determined in the image to be checked, identifies several groups of point lines determined in the characteristic contour, and determines the angle parameters between adjacent lines in sequence starting from the edge points of the characteristic contour, and determines the angle sequence, and then evaluates whether the relevant personnel are wearing safety helmets by analyzing the proportion of the same angle in the angle sequence; the specific method is as follows:

[0023] Based on a number of feature points divided within the feature contour, a group of feature points at the edge of the feature contour is selected as the initial point, a line connecting the initial point and its adjacent feature points is determined as the front line, and a group of lines that appear in succession after the front line is used as the back line, and the intersection of the front line and the back line is used as the extension point to extend the front line, and the angle between the extension line and the back line is locked as the first group of angles;

[0024] Then, the rear connection line is used as the front connection line, and the rear connection line of the front connection line is determined, and the second group of angles is locked. Similarly, the angles between subsequent adjacent connection lines are confirmed one by one, and the confirmed angles are sorted according to the confirmation order to confirm the angle sequence;

[0025] Identify the angles with the same value that appear in the angle sequence. When the number of angles with the same value is ≥5, calibrate such angles with the same value as standard angles.

[0026] Identify the total proportion of standard angles in the angle sequence Zb, where Zb = the total number of standard angles ÷ the total number of angles in the angle sequence. If Zb ≥ Y1, a signal that does not meet the wearing standards is generated through the signal generation end, and related display is directly performed, where Y1 is a preset value;

[0027] If Zb<Y1, a no-wearing signal is generated through the signal generating terminal and displayed directly;

[0028] When the number of angles with the same value is less than 5, no relevant calibration of the standard angle is performed.

[0029] The present invention provides a construction site safety monitoring system based on drone vision. Compared with the prior art, it has the following beneficial effects:

[0030] The present invention uses a drone to monitor the construction area of ​​a construction site in real time, determines the monitoring screen in real time, selects a face image from the monitoring screen based on relevant extraction features, and then performs a preliminary analysis on the face image. By locking the center point and the point to be processed, it is identified whether the mean angle of the connecting line in the contour is consistent, and it can preliminarily determine whether the helmet of the corresponding construction worker is abnormally worn, and generate relevant determination signals for timely display. The monitoring method is more comprehensive and can achieve better construction site safety supervision effects.

[0031] In order to more accurately identify whether the relevant personnel are wearing safety helmets correctly, by identifying whether the means of the left and right contours are equal after adjustment, it is assessed whether the next step of analysis is needed, and then the angle sequence is determined by analyzing the relevant angles of the corresponding contours. By analyzing the proportion of the same angle in the angle sequence, it is assessed whether the relevant personnel are wearing safety helmets. The assessment method of sequential analysis and processing can achieve better safety monitoring effects, and can fully identify whether the relevant personnel are wearing safety helmets and whether the wearing of safety helmets meets the standards, which can achieve better safety monitoring effects and fully guarantee the safety of construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the principle framework of the present invention;

[0033] Figure 2 It is a schematic diagram of the division of the external contour of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] See also Figure 1 , the present application provides a construction site safety monitoring system based on drone vision, including a drone vision end, an image initial processing end, an image initial analysis end, a secondary adjustment recognition end, a point angle processing end, and a signal generation end, wherein the drone vision end, the image initial processing end, the image initial analysis end, the secondary adjustment recognition end, the point angle processing end, and the signal generation end are electrically connected in sequence from an output node to an input node;

[0037] Among them, the drone vision end controls the drone to fly according to the preset trajectory based on the preset drone operation trajectory, and shoots the construction scene of the construction area in real time. The internal shooting angle of the drone vision end has been set in advance before the flight, and the real-time construction scene is transmitted to the image initial processing end. Specifically, each different construction area has been marked in advance in the drone setting trajectory. When the relevant drone reaches the corresponding mark, it will shoot the scene of the construction area;

[0038] Among them, the image initial processing end locks the relevant person appearing in the construction picture based on the set human features, and then extracts the associated image of the facial part area of ​​the relevant person based on the preset face template, and obtains the image to be verified after the clarity adjustment processing of the associated image, and transmits it to the image initial analysis end: Specifically, since the technical content of locking the person and the person's face in the image belongs to the existing technology, it will not be described in detail here. When selecting the person, the target detection is generally performed by combining the pre-set features with a classifier such as a support vector machine (SVM); by sliding a window on the image, extracting the features in the window and classifying them, it is determined whether the target person is included, and then using facial recognition technology to extract the facial features of the person in the picture, such as the shape and position of the eyes, nose, mouth, etc., or by using existing facial recognition algorithms and software, such as the facial detection and recognition module in OpenCV, etc., the relevant person in the picture and the relevant face of the corresponding person can be identified and obtained, and the obtained associated image is then subjected to image sharpening and denoising processing (that is, clarity adjustment processing) to obtain the final image to be verified;

[0039] The image initial analysis end identifies the overall relevant contours of the image to be verified based on the processed image to be verified, and locks in whether the person associated with the image to be verified is wearing abnormal clothing by eliminating the face area in the image to be verified. The specific method of locking in whether the associated person is wearing abnormal clothing is as follows:

[0040] Based on the marked face area in the image to be checked, the center point of the face area is locked (the center point can be locked by determining the center point coordinates based on the coordinate point method, for example: a surface is divided into several points, each point has a related coordinate point in the two-dimensional coordinate system, and the corresponding center point can be determined by averaging several coordinate points, and the center point can determine the corresponding center point coordinates, and the center point of the corresponding face area can be locked based on the coordinates), a set of vertical lines passing through the center point and perpendicular to the horizontal base plane is constructed, and the horizontal base plane is a preset horizontal plane (which can be understood as a related plane parallel to the ground), and the intersection of the vertical line and the overall outer contour of the image to be checked is locked, and the intersection is marked as the point to be processed;

[0041] The same plane where the center point and the point to be processed are located is marked as the selected surface, and the part of the contour inside the selected surface is marked as the characteristic contour, and the characteristic contour is equidistantly divided into several characteristic points (because the line is composed of points, the corresponding characteristic contour can be divided into several points);

[0042] The points to be processed divide this feature contour into a left contour and a right contour, and prioritize identifying the associated angle between adjacent points in the left contour and the horizontal plane. Connect the adjacent feature points and translate the line to intersect the horizontal plane. Lock the angle and mark it as J. i , where i represents the angle between the lines of different adjacent points, and the angles J determined by the left contour are i Perform mean processing to determine the left contour mean JZ1;

[0043] Then the right contour is processed in the same way to determine the right contour mean JZ2;

[0044] If JZ1=JZ2, it means the helmet is worn normally and no subsequent processing is required. If JZ1≠JZ2, it means the helmet is worn abnormally or the helmet is not worn, and subsequent relevant analysis and processing are required to determine the specific cause of the abnormality;

[0045] Specifically, when the associated person is not wearing a helmet, the outline generated on the outside is the associated outline generated by the corresponding person's hair. Then such outline is relatively messy, and the generated angle data is definitely inconsistent. If a helmet is worn but not worn standardly, the corresponding mean of the generated angle data is also inconsistent. In order to better conduct identification analysis and determine the specific identification results, subsequent association analysis is required to determine whether such related personnel have safety hazards and conduct comprehensive evaluation.

[0046] Among them, the secondary adjustment recognition end, for the situation that the person associated with the image to be checked has abnormal wearing, makes relevant adjustments to the points to be processed marked by the characteristic contour in the image to be checked, and identifies whether there is still an abnormality after the relevant adjustment. If there is still an abnormality, the point angle processing end is executed, wherein the specific method of identification is:

[0047] Based on the points to be processed calibrated in the characteristic contour, the points to be processed are moved on the characteristic contour, and it is recorded whether the left contour mean JZ1 and the right contour mean JZ2 satisfy: JZ1 = JZ2 in each different movement process. If so, a wearing failure signal is generated through the signal generating end, and the relevant display is directly performed;

[0048] If not, it means that there is still an abnormality in the associated personnel, and the point angle processing end is executed to perform correlation analysis to assess whether the corresponding associated personnel are wearing misaligned helmets or not wearing relevant safety helmets. Based on the specific assessment and analysis results, a comprehensive assessment is carried out in time, and relevant processing signals are generated in time for real-time display to ensure the construction safety of relevant construction personnel in the construction area;

[0049] Specifically, if the helmets worn by the relevant personnel are only offset but not misplaced or in other situations, the two groups of means should be equal by adjusting the position of the treatment point. If the determined means are equal but not equal in the original assessment process, then such situations basically belong to offset in wearing. It is only necessary to warn such personnel in the future to warn them of the correct way to wear the safety helmets without on-site warnings.

[0050] No matter how the processing points are adjusted, the related means between the corresponding adjacent points after the adjustment are not equal. Firstly, the helmet may be misaligned, resulting in a large numerical deviation. When the helmet is misaligned, the arc curvature of each area is inconsistent, so the associated lines will also change, resulting in unequal angles no matter how it is adjusted. Or the relevant helmet is not worn. Therefore, no matter how it is adjusted and processed, the corresponding angle means cannot be equal.

[0051] Example 2

[0052] In the specific implementation process of this embodiment, compared with the above embodiment, this embodiment mainly identifies the numerical overlap between the associated angles during specific processing, and its specific execution end is the point angle processing end;

[0053] Among them, the point angle processing end, based on the characteristic contour determined in the image to be checked, identifies several groups of point lines determined in the characteristic contour, and determines the angle parameters between adjacent lines in sequence starting from the edge points of the characteristic contour, and determines the angle sequence, and then evaluates whether the relevant personnel are wearing safety helmets by analyzing the proportion of the same angle in the angle sequence. The specific method of performing the evaluation is as follows:

[0054] Based on a number of feature points divided within the feature contour, a group of feature points at the edge of the feature contour is selected as the initial point, a line connecting the initial point and its adjacent feature points is determined as the front line, and a group of lines that appear in succession after the front line is used as the back line, and the intersection of the front line and the back line is used as the extension point to extend the front line, and the angle between the extension line and the back line is locked as the first group of angles;

[0055] Then, the rear connection line is used as the front connection line, and the rear connection line of the front connection line is determined, and the second group of angles is locked. Similarly, the angles between subsequent adjacent connection lines are confirmed one by one, and the confirmed angles are sorted according to the confirmation order to confirm the angle sequence;

[0056] Identify the angles with the same value in the angle sequence. When the number of angles with the same value is ≥5, such angles with the same value are calibrated as standard angles. Otherwise, no relevant calibration of standard angles is performed.

[0057] Identify the total proportion of standard angles in the angle sequence Zb, where Zb = the total number of standard angles ÷ the total number of angles in the angle sequence. If Zb ≥ Y1, a signal generating end generates a non-standard wearing signal and directly displays it. Y1 is a preset value, and its specific value is determined by the operator based on experience. If Zb < Y1, a non-wearing signal is generated by the signal generating end and directly displayed, indicating that the relevant personnel are not wearing a safety helmet. The analyzed angle is abnormal, and the relevant personnel need to issue a relevant warning.

[0058] For example: the initial point is A, and the adjacent point is B, then the AB line is the front line. Next, determine the set of lines that appear after the front line as the back line. Assuming the next point is C, then the BC line is the back line. The intersection of the AB line and the BC line is used as the extension point, and the AB line is extended, and the angle between the extension line and the BC line is locked as the first set of angles.

[0059] Then, the BC line is used as the front line, and its back line CD is determined, and the second set of angles is locked. Similarly, the angles between subsequent adjacent lines are confirmed one by one, and the confirmed angles are sorted according to the confirmation order to obtain an angle sequence.

[0060] Assume that in this angle sequence, there are some angles with the same value. For example, there are multiple angles of 60 degrees. If the number of angles with the same value is ≥ 5, for example, there are 6 angles of 60 degrees, then the angle of 60 degrees is marked as the standard angle.

[0061] Next, calculate the total proportion of standard angles in the angle sequence Zb. Assuming there are 20 angles in the angle sequence, and the total number of standard angles (60 degree angles) is 8, then Zb = 8 ÷ 20 = 0.4.

[0062] If the preset value Y1 is 0.35, since Zb(0.4)≥Y1, a signal indicating that the helmet is not worn properly is generated and displayed through the signal generating end, and the words "Helmet wearing not up to standard" may be displayed on the display screen or a specific sound and light prompt may be emitted. If Zb<Y1, a signal indicating that the helmet is not worn properly is generated and displayed, such as "Not wearing a helmet".

[0063] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0064] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. The construction site safety monitoring system based on drone vision is characterized by: include: The drone vision end controls the drone to fly according to the preset trajectory and takes real-time photos of the construction area. The internal visual end of the drone has its internal shooting angle set in advance before the flight. The image initial processing end locks the relevant person appearing in the construction picture based on the set human features, and then extracts the associated image of the facial part of the relevant person based on the preset face template, adjusts the clarity of the associated image to obtain the image to be checked, and transmits it to the image initial analysis end; The image initial analysis end identifies the overall relevant contours of the image to be verified based on the processed image to be verified, and by eliminating the face area in the image to be verified, determines whether the person associated with the image to be verified has abnormal wearing conditions; The secondary adjustment recognition end, in case of abnormal wearing of the person associated with the image to be checked, makes relevant adjustments to the points to be processed marked by the characteristic contour in the image to be checked, and identifies whether there is still abnormality after the relevant adjustment. If there is still abnormality, the point angle processing end is executed; The point angle processing end, based on the characteristic contour determined in the image to be checked, identifies several groups of point lines determined in this characteristic contour, and starting from the edge points of the characteristic contour, determines the angle parameters between adjacent lines in turn, and determines the angle sequence. Then, by analyzing the proportion of the same angle in the angle sequence, it is assessed whether the relevant personnel are wearing safety helmets.

2. The construction site safety monitoring system based on drone vision according to claim 1 is characterized in that: The specific method for the image initial analysis end to identify whether the associated person is wearing abnormal clothing is: Based on the marked face area in the image to be checked, the center point of the face area is locked, a set of vertical lines passing through the center point and perpendicular to the horizontal base plane is constructed, and the horizontal base plane is a preset horizontal plane. The intersection point of the vertical line and the overall outer contour of the image to be checked is locked, and the intersection point is marked as a point to be processed; The same plane where the center point and the point to be processed are located is marked as a selected surface, the part of the contour inside the selected surface is marked as a characteristic contour, and the characteristic contour is equidistantly divided into a number of characteristic points; The points to be processed divide this feature contour into a left contour and a right contour, and prioritize identifying the associated angle between adjacent points in the left contour and the horizontal plane. Connect the adjacent feature points and translate the line to intersect the horizontal plane. Lock the angle and mark it as J. i , where i represents the angle between the lines of different adjacent points, and the angles J determined by the left contour are i Perform mean processing to determine the left contour mean JZ1; Then the right contour is processed in the same way to determine the right contour mean JZ2; If JZ1≠JZ2, it means that the helmet is worn abnormally or the helmet is not worn, and subsequent relevant analysis and processing are required.

3. The construction site safety monitoring system based on drone vision according to claim 2 is characterized in that: If JZ1=JZ2, it means the helmet is worn normally and no further processing is required.

4. The construction site safety monitoring system based on drone vision according to claim 1 is characterized in that: The specific method of the secondary adjustment recognition end for recognizing whether the person associated with the image to be checked still has wearing abnormalities is: Based on the points to be processed marked in the characteristic contour, the points to be processed are moved on the characteristic contour, and it is recorded whether the left contour mean JZ1 and the right contour mean JZ2 satisfy the following conditions in each different movement process: JZ1 = JZ2: If so, a signal indicating that the wearing does not meet the standard is generated through the signal generating end, and the relevant information is displayed directly.

5. The construction site safety monitoring system based on drone vision according to claim 4 is characterized in that: If the left contour mean JZ1 and the right contour mean JZ2 do not satisfy: JZ1 = JZ2, it means that there is still an abnormality in the associated personnel, and the point angle processing end is executed to perform association analysis.

6. The construction site safety monitoring system based on drone vision according to claim 5 is characterized in that: The specific method for evaluating whether the relevant personnel are wearing safety helmets at the point angle processing end is as follows: Based on a number of feature points divided within the feature contour, a group of feature points at the edge of the feature contour is selected as the initial point, a line connecting the initial point and its adjacent feature points is determined as the front line, and a group of lines that appear in succession after the front line is used as the back line, and the intersection of the front line and the back line is used as the extension point to extend the front line, and the angle between the extension line and the back line is locked as the first group of angles; Then, the rear connection line is used as the front connection line, and the rear connection line of the front connection line is determined, and the second group of angles is locked. Similarly, the angles between subsequent adjacent connection lines are confirmed one by one, and the confirmed angles are sorted according to the confirmation order to confirm the angle sequence; Identify the angles with the same value that appear in the angle sequence. When the number of angles with the same value is ≥5, calibrate such angles with the same value as standard angles. The total proportion Zb of the standard angles in the angle sequence is identified, where Zb = the total number of standard angles ÷ the total number of angles in the angle sequence. If Zb≥Y1, a non-compliant wearing signal is generated through the signal generation end, and related display is performed directly, where Y1 is a preset value.

7. The construction site safety monitoring system based on drone vision according to claim 6 is characterized in that: If Zb<Y1, a no-wearing signal is generated through the signal generating end and displayed directly.

8. The construction site safety monitoring system based on drone vision according to claim 6 is characterized in that: When the number of angles with the same value is less than 5, no relevant calibration of the standard angle is performed.

Images