Construction site safety monitoring method and system based on image recognition and distance calculation

Abstract

The present invention discloses a construction site safety monitoring method and system based on image recognition and distance calculation. The method uses a monocular camera to collect a video stream of construction site images; uses BIM to extract construction site image information, combines the camera imaging model to perform camera calibration, and obtains camera calibration data; converts the two-dimensional image transmitted back from the construction site into a three-dimensional spatial relationship, and solves the real distance corresponding to the distance between pixels in the image; uses a target detection algorithm to process the on-site image video stream to obtain the category and position information of workers and target objects; sets a distance threshold between actual objects and a distance threshold between actual distance danger zones, solves the real distance corresponding to the distance between pixels in the image based on the camera calibration data, and marks the current risk frame and issues a risk warning when the real distance is less than the distance threshold. Using only a monocular camera and BIM, the system does not require additional sensors or workers to wear sensing or positioning devices that may affect their work. The system is simple, convenient, and highly accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of construction safety visual monitoring technology, and in particular to a construction site safety monitoring method and system based on image recognition and distance calculation. Background Art

[0002] Construction sites are prone to safety incidents due to the complex and risky interactions between workers and various mobile devices. The resulting delays and additional costs severely hinder the industry's development. Therefore, construction safety management is an essential component of construction management.

[0003] However, both safety training using various virtual reality technologies, such as VR, digital twins, and game engines like Unity, and sensor-based safety monitoring have significant limitations. Safety training cannot guarantee the identification of every dangerous behavior and real-time monitoring, nor can it guarantee that workers will strictly follow the training content and avoid dangerous behaviors and hazardous areas during actual construction. Sensor-based safety monitoring, on the other hand, poses challenges such as increased project costs, misjudgments due to equipment anomalies or improper wear, and the potential for impacting worker behavior and increasing risk.

[0004] The recent development of deep learning algorithms has significantly transformed the application landscape of computer vision, achieving significant breakthroughs in areas such as image classification, object detection, and semantic segmentation. Deep learning algorithms, also known as AI algorithms, use a large number of "neurons" and "activation functions" to form a hierarchical structure that mimics the human brain's thinking process, known as a neural network. These algorithms then use large amounts of data to calibrate input and output, gradually developing a fitting function from input to output. Research has shown that, given sufficient data, deep neural networks generally achieve better fitting results and higher prediction accuracy, giving rise to a new generation of intelligent algorithms. Specifically in the field of image recognition, convolution is often used: applying a convolution kernel to a two-dimensional image to extract deep features. Finally, predictions are made based on the feature maps generated by the multi-layer convolution, known as CNNs. Object detection, a key subtask within image recognition, also applies to the CNN architecture. Its goal is to identify each "object of interest" in an image and pinpoint its specific location within the image. The current mainstream deep learning-based target detection algorithms include the YOLO series, R-CNN series, SSD series, etc. The processing speed and accuracy of these algorithms vary, and different target detection algorithms need to be used according to different application requirements.

[0005] The most direct way to obtain distance information from a camera is with a depth camera. This camera can directly determine the actual distance from each pixel to the camera, but the permitted distance is limited to a few to tens of meters, far from sufficient for construction site scale requirements. Furthermore, the need for long-range ranging also leads to rapidly increasing costs. Furthermore, even if the distance from an object to the camera is determined, determining the distance between objects still requires further algorithmic support, and direct use of this distance data is not possible. Therefore, camera calibration technology is required.

[0006] Camera calibration is an important step for computers to extract three-dimensional information from two-dimensional images. Its applications lie not only in image ranging, but also in image processing fields such as image stitching, three-dimensional calibration, and three-dimensional reconstruction. It plays an important role in computer vision. For monocular cameras, early camera calibration technology required understanding the camera imaging model, knowing the camera's focal length, the physical size of the imaging unit, and the three-dimensional spatial position information. It even required observing the same target plane in multiple orthogonal planes to achieve accurate calibration, which is difficult to achieve in practical applications. After more than a decade of development, the academic community has successively proposed more convenient camera calibration methods. Among them, the most widely used is Zhang Youzheng's calibration method, which only requires the camera to observe a checkerboard at different angles to calculate the camera's internal and external parameter matrix to achieve camera calibration and correct lens distortion.

[0007] In recent years, computer vision-based construction site safety management methods have emerged. Simpler methods directly detect the absence of safety equipment, such as hard hats. More complex safety management methods primarily involve target detection, path prediction or posture recognition, risk detection, and ultimately, the presentation of safety risk warnings. This safety management approach relies heavily on deep learning algorithms. Whether it's path prediction, posture recognition, or risk detection, all these methods are algorithmic predictions. Guaranteeing accuracy with multiple predictions is inherently difficult, and environmental occlusion and variability on site exacerbate this problem, leading to false negatives and false positives, and ultimately, ineffective on-site safety management. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a construction site safety monitoring method and system based on image recognition and distance calculation. It does not require additional equipment such as sensors, nor does it require workers to strictly wear sensing or positioning devices that may affect their work. It can accurately and conveniently realize intelligent monitoring of construction site safety.

[0009] Furthermore, the present invention can achieve that only one camera calibration is required, and no repeated calculation is required before zooming or moving, and the distance calculation only involves simple matrix operations, which does not occupy a large amount of processing resources.

[0010] The technical solution adopted in the present invention is:

[0011] A construction site safety management method based on image recognition and distance calculation includes the following steps:

[0012] S10 uses a monocular camera to capture construction site image video streams;

[0013] S20 calibrates the camera based on any object with adjacent straight edges in the image to obtain camera calibration data;

[0014] The S30 uses camera calibration data to convert the 2D images sent back from the construction site into 3D spatial relationships, solving the real distance corresponding to the distance between pixels in the image;

[0015] S40 uses target detection algorithms or image segmentation technology to process on-site image video streams to obtain the category and location information of workers and target objects;

[0016] S50 sets the actual object distance threshold and the actual distance danger zone distance threshold, solves the real distance corresponding to the distance between pixels in the image based on the camera calibration data, and marks the current risk frame and issues a risk warning when the real distance is less than the distance threshold.

[0017] In the above technical solution, in step S10, the camera is aimed at the area where safety inspection is required, and the camera monitoring screen is connected to the network video stream to collect the construction site image video stream.

[0018] In the above technical solution, in step S10, if it is night or other special environment, an infrared camera is used.

[0019] In the above technical solution, in step S20, the object includes at least one of a crane, an excavator, a roller, a concrete rectangular block, or any other object having adjacent straight edges, wherein two straight edges of the object have a right angle or any angle between them.

[0020] In the above technical solution, in step S20, BIM is used to extract the anchor point information of any object on the construction site image calibration plane, obtain the anchor point pixel coordinates, substitute the anchor point pixel coordinates into the monocular camera imaging model, calculate the camera internal and external parameter matrix, projection matrix and distortion correction coefficient, and complete the camera calibration.

[0021] In the above technical solution, in step S20, camera calibration is performed in combination with the camera imaging model to obtain camera calibration data; or the focal length of the camera and the size parameters and spatial position of the photosensitive plate are directly measured to obtain the internal and external parameter matrix of the camera calibration data.

[0022] In the above technical solution, in step S20, a reference object of known real size on the construction site image calibration plane is obtained to obtain the pixel coordinates of the reference object, and then a spatial coordinate system is established in combination with the real size, and the camera imaging model is introduced to complete the camera calibration.

[0023] In the above technical solution, in step S20, if the adjacent sides of the reference object with known real dimensions on the construction site image calibration plane are not perpendicular, the angle between the two adjacent sides of the reference object is additionally measured to accurately establish a rectangular coordinate system in three-dimensional space, and the pixel coordinates of this reference object are obtained through image processing means.

[0024] In the above technical solution, in step S20, the camera calibration data includes an intrinsic and extrinsic parameter matrix, a projection matrix, and a distortion correction coefficient.

[0025] In the above technical solution, in step S20, BIM is used to extract the three-dimensional coordinates of several key anchor points in the image, and the pixel coordinates of the anchor points are obtained. The corresponding coordinate points of the anchor points are substituted into the monocular camera imaging model, and the camera internal and external parameter matrix, projection matrix and distortion correction coefficient are calculated to complete the camera calibration.

[0026] In the above technical solution, in step S20, a video frame is taken from the on-site image video stream, and the edge line where the anchor point is located is marked with a longer straight line in the BIM software, so that the intersection of different straight lines is the selected anchor point; then, the pixel coordinate expression of the straight line is obtained through color space conversion, color range selection, edge detection, and straight line detection in sequence; finally, the intersection of the straight lines is calculated based on the obtained straight line expression, which is the pixel coordinate.

[0027] In the above technical solution, in step S20, the projection matrix from the three-dimensional coordinates to the pixel coordinates is calculated based on the imaging model of the camera and the correspondence between the real three-dimensional coordinates and a number of anchor points in the image pixel coordinates.

[0028] In the above technical solution, the imaging model of the camera is ;

[0029] in: , are two-dimensional pixel coordinates; , is the actual three-dimensional coordinate; is an arbitrary scale factor related to the distance from the camera to the target plane; is the external parameter matrix, which describes the spatial position of the camera and changes as the camera moves; is the rotation matrix, is a translation vector.

[0030] In the above technical solution, in step S20, is the camera intrinsic parameter matrix, and are the quotient of the focal length and the physical length of a single pixel on the photosensitive plate in two directions, The angle of the photosensitive plate in two directions The cotangent of The product of the opposites of and are the principal point coordinates or the center of the image.

[0031] In the above technical solution, for each anchor point, its distorted pixel coordinates are known. Through the correspondence between the real three-dimensional coordinates and the image pixel coordinates of several anchor points, the distortion coefficient can be obtained using the least squares method according to the distortion formula.

[0032] In the above technical solution, the distortion formula is ;

[0033] in is the distance from the pixel to the principal point, is the ideal undistorted coordinate, is the image coordinate after radial distortion, is the second-order distortion coefficient.

[0034] In the above technical solution, in step S20, after the camera is moved or zoomed, the camera needs to be recalibrated.

[0035] In the above technical solution, in step S30, the real distance corresponding to the distance between pixels in the image is solved through matrix operation.

[0036] In the above technical solution, in step S40, target detection is performed frame by frame on the scene image video stream.

[0037] In the above technical solution, in step S40, the target object includes at least one of a crane, an excavator, a roller, a concrete rectangular block, or any object with adjacent straight edges, wherein two straight edges of the object have a right angle or any angle between them.

[0038] In the above technical solution, in step S40, the target detection algorithm can be at least one of the YOLO algorithm, the R-CNN algorithm, and the SSD algorithm.

[0039] In the above technical solution, the target detection algorithm is set according to the security management platform, hardware conditions and the effect to be achieved.

[0040] In the above technical solution, in step S40, target detection is performed frame by frame on the construction site video stream to obtain the position and category information of each object of interest in the image; the ground where the person and the target object are located is used as the calibration plane, and the bottom center of each bounding box is used as the pixel coordinate of the target.

[0041] In the above technical solution, in step S40, the actual three-dimensional coordinates of the person and the target object are calculated using the obtained projection matrix, and then the actual distance from the person to the target object is calculated based on the coordinates.

[0042] In the above technical solution, in step S40, image segmentation technology is used to directly classify pixels in the image, determine the pixel category and position, and obtain the category and position information of the worker or the target object.

[0043] In the above technical solution, in step S50, if the calculated actual distance is less than the safety threshold, it is considered that there is a safety risk here, the boundary box of the detected danger source is marked with a special color, and a risk warning is issued.

[0044] In the above technical solution, in step S50, different pixel points are selected as target representation points to realize the calculation of the real distance.

[0045] In the above technical solution, in step S50, a safety management platform is also set up to push the image frame where the danger source is located to the safety management platform, issue a risk warning, notify relevant personnel to take corresponding measures, and record the risk situation.

[0046] In the above technical solution, the security management platform includes at least: a storage module for storing images and recording data, and a data processing module for receiving data and images and issuing early warning information.

[0047] The present invention also provides a construction site safety management system or electronic device based on image recognition and distance calculation, or a readable computer storage medium configured to store a computer program, which is used to implement the steps of the above method when the system or electronic device is running or the computer program is executed.

[0048] In summary, the construction site safety management method and system based on image recognition and distance calculation of the present invention uses a monocular camera to collect construction site image video streams; uses BIM to extract construction site image information, combines the camera imaging model to perform camera calibration, and obtains camera calibration data; converts the two-dimensional image transmitted back from the construction site into a three-dimensional spatial relationship, and solves the real distance corresponding to the distance between pixels in the image; uses the target detection algorithm to process the on-site image video stream to obtain the category and position information of the workers and target objects; sets the actual object distance threshold and the actual distance danger zone distance threshold, and solves the real distance corresponding to the distance between pixels in the image according to the camera calibration data, and marks the current risk frame and issues a risk warning when the real distance is less than the distance threshold.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention only uses the on-site video stream captured by a monocular camera and the key anchor point coordinates extracted from BIM as data sources, which to a certain extent solves the problems of high cost and low accuracy of construction site safety management, brings new technologies and methods to the field of construction, accelerates the digitalization process of the construction industry, and has important theoretical and practical significance.

[0051] In comparison, the image technology-based safety management method proposed in the present invention can detect existing safety risks in real time at actual construction sites, mainly including collision warnings between moving objects and safety warnings between people and target objects. It can intelligently monitor and manage construction site safety issues without the need for additional equipment such as sensors.

[0052] Using a monocular camera as the sole source of on-site information avoids the use of expensive and difficult-to-correctly configured sensors. Furthermore, the camera can be placed at a higher location, such as a tower crane, and multiple cameras can be used simultaneously to avoid obstructions. If security management is required at night or in other special environments, infrared cameras or other auxiliary equipment can be used to capture on-site images, without affecting the subsequent processes of this invention.

[0053] Camera calibration technology is used to convert the two-dimensional image data transmitted back from the construction site into three-dimensional spatial relationships, and the actual distance between pixels is calculated. Zhang Youzheng's calibration method requires observing the chessboard from different angles, which is difficult to implement directly at the construction site. Therefore, the present invention does not use special calibration objects, and performs calibration only using the information already available on site. Specifically, the calibration method of the present invention uses BIM to extract the three-dimensional coordinates of several key anchor points, uses image processing to obtain the pixel coordinates of the anchor points, substitutes the corresponding coordinate points of the anchor points into the monocular camera imaging model, calculates the camera's internal and external parameter matrix, projection matrix and distortion correction coefficient, and completes the camera calibration.

[0054] The on-site video stream is processed using an object detection algorithm to determine the category and location of workers and other objects (such as cranes, excavators, and rollers). Safety thresholds, such as object spacing thresholds or distance thresholds to hazardous areas, are set based on safety management needs. The actual distance corresponding to the inter-pixel distance in the image is then calculated based on camera calibration data. When the distance falls below the threshold, the current frame is marked as risky and a risk warning is issued.

[0055] On-site surveillance video streams are continuously monitored and output to the safety management platform. If risks are detected, an alert is issued to the relevant safety management personnel and the risk is recorded. Since camera calibration only needs to be performed once, no repeated calculations are required before zooming or panning, and distance calculations only involve simple matrix operations, which does not consume a lot of processing resources. Users can choose the appropriate target detection algorithm based on their hardware to achieve high-frame-rate video stream output and complete real-time monitoring of construction safety.

[0056] Compared with sensor-based safety management methods, this invention does not require the purchase of additional expensive equipment, nor does it require workers to strictly wear devices that may affect their work. It only requires placing network cameras in appropriate locations to continuously and intelligently identify safety risks. The advantages are more obvious in larger construction sites.

[0057] Compared with computer vision methods based on motion path prediction or posture recognition and behavior recognition, the present invention pre-calibrates the camera and uses actual distance as the judgment standard for risk identification, avoiding the error accumulation caused by multiple predictions. At the same time, it also greatly improves the computing efficiency and realizes the continuous output of detection video stream and risk warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0059] Figure 1 This is a flow chart of the construction safety monitoring method based on the target detection algorithm in image recognition and camera calibration of the present invention.

[0060] Figure 2 This is a rendering of the construction site video stream in Example 2 of the present invention.

[0061] Figure 3 This is the target detection algorithm training result used in Example 2 of the present invention.

[0062] Figure 4 This is an image used for camera calibration after anchor points are marked with red thin lines in Example 2 of the present invention.

[0063] Figure 5 yes Figure 4 A partial enlarged view of .

[0064] Figure 6 This is a diagram showing the effect of target detection and distance calculation in Example 2 of the present invention.

[0065] Figure 7 3 is a schematic diagram of camera calibration performed at a construction site based on a preset calibration object according to Example 3 of the present invention.

[0066] Figure 8 For Figure 7 Schematic diagram of the anchor points marked with red thin lines in calibration object 3.

[0067] Figure 9 This is a diagram showing the effect of target detection and distance calculation in Example 4 of the present invention.

[0068] Figure 10 for Figure 7 A local enlarged effect diagram. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0070] Example 1

[0071] like Figure 1 As shown, the present invention provides a construction safety monitoring and management method based on target detection algorithms and camera calibration in image recognition. This method combines the frame-by-frame target detection results of the on-site video stream with the distance correspondence between the calibrated camera image and the real world to rapidly calculate the true position relationship of relevant objects. This allows for real-time detection of potential safety risks, and simultaneously sends these safety warnings to on-site managers and records them in the safety management platform database, enabling intelligent safety risk detection for construction.

[0072] Step 1: Installation of construction site cameras and preparation of target detection algorithms.

[0073] First, aim the camera at the area to be inspected for safety, placing it at a high position to maximize the unobstructed field of view. Then, connect the monitoring image to the network video stream, completing the acquisition of the construction site video stream. Next, prepare an adaptive target detection algorithm, which should be determined based on the user's hardware and desired performance. Finally, set a distance threshold for risk warning. This completes the preliminary preparations for the present invention.

[0074] Step 2: Calibration of the construction site camera.

[0075] First, let’s introduce the ideal distortion-free imaging model of a monocular camera:

[0076] ;

[0077] in:

[0078] , are two-dimensional pixel coordinates.

[0079] , are the actual three-dimensional coordinates.

[0080] is an arbitrary scale factor related to the distance from the camera to the target plane.

[0081] is the intrinsic parameter matrix, which is not affected by the spatial position of the camera and is only determined by its own physical parameters. Specifically, and are the quotient of the focal length and the physical length of a single pixel on the photosensitive plate in two directions, The angle of the photosensitive plate in two directions The cotangent of The product of the opposite numbers (in ideal condition, due to is a right angle, that is ), and are the coordinates of the principal point (ideally the center of the image).

[0082] is the external parameter matrix, which describes the spatial position of the camera and changes as the camera moves. is the rotation matrix, is a translation vector.

[0083] Assume that the plane where the anchor point is located in the actual three-dimensional coordinates is , then there is a mapping relationship between the two-dimensional pixel coordinates and the three-dimensional space coordinates:

[0084] ;

[0085] ;

[0086] Right now:

[0087] ;

[0088] is a 3×3 projection matrix, for Removal The new vector of direction coordinates.

[0089] According to the imaging model of the monocular camera, it can be seen that under ideal distortion-free conditions, there is a corresponding relationship between the real 3D coordinates and the pixel coordinates (projection matrix ). To get the projection matrix from real 3D coordinates to pixel coordinates , we need the correspondence between the real 3D coordinates and several anchor points in the image pixel coordinates.

[0090] If the pixel coordinates of several anchor points and their corresponding three-dimensional coordinates are known, the projection matrix can be calculated by constructing a set of equations: .

[0091] ;

[0092] ;

[0093] in ; Each set of corresponding anchor point coordinates can provide two equations, because the projection matrix is a homogeneous matrix, so let There are still eight unknowns, so at least four sets of coordinates are needed to get .

[0094] Step 2.1: Extracting the real 3D coordinates of the anchor point

[0095] To obtain the true 3D coordinates of the anchor points, either prefabricated calibration objects can be placed at the construction site to extract the necessary information, or the length and position relationships of known objects can be used for calibration. In this invention, BIM is used to obtain the 3D coordinates of the anchor points and used as a basis for camera calibration.

[0096] Step 2.2: Extracting the pixel coordinates of the anchor image

[0097] In order to accurately extract the coordinates of the anchor points in the corresponding image, due to the problem of excessive noise caused by the environmental complexity of the construction site, the present invention adopts the method of manual marking + image processing, and extracts the coordinates in the form of the intersection of red or other prominently colored marking lines: first, the video frame is taken from the on-site video stream, and the edge line where the anchor point is located is marked with a longer straight line, so that the intersection of different straight lines is the selected anchor point; then the pixel coordinate expression of the straight line is obtained through color space conversion, color range selection, edge detection, and line detection in sequence; finally, the intersection point is calculated based on the obtained straight line expression, which is the pixel coordinate.

[0098] Step 2.3: Calculate the projection matrix from 3D coordinates to pixel coordinates.

[0099] According to the camera imaging model and formula (3) and formula (4), the coordinates obtained in step 2.1 and step 2.2 are substituted into the calculation to obtain matrix .

[0100] Step 2.4 (optional): Solving the camera parameters.

[0101] If you want to further find the intrinsic and extrinsic parameter matrix of the camera, you can perform further camera calibration by following the steps below:

[0102] because is the rotation matrix, so mutually orthogonal, that is and .again ,so , .make , obviously It is a symmetric matrix and is only related to the camera's built-in physical parameters and is not affected by the spatial position.

[0103] make ,have:

[0104] ;

[0105] in, ;

[0106] ;

[0107] From formula (5), we can see that if we want to solve the matrix , then at least six equations are required from at least three images at different positions (different external parameters). If there is only one image, that is, the camera cannot perform operations such as rotation or translation to change the spatial position, the intrinsic parameter matrix can be in And the principal point is the center of the image and then solve If there are two images at different locations, we can just let Matrix Calculation; if there are three images, then we can find The only solution to ; if more images can be obtained, the maximum likelihood estimation is used for the system of equations formed by these images to obtain the optimal solution to the system of equations.

[0108] Solving the Matrix After that, you can Solve the matrix , and then according to Solve , Pick and The outer product of is enough. This completes the solution of the camera's internal and external parameter matrix.

[0109] Step 2.5 (optional): Solve for distortion coefficients.

[0110] For the radial distortion of the camera, there are distortion formulas (6) and (7):

[0111] ;

[0112] in is the distance from the pixel to the principal point, is the ideal undistorted coordinate, is the image coordinate after radial distortion, is the second-order distortion coefficient.

[0113] For each anchor point, its distorted pixel coordinates are known. Then, its ideal coordinates are obtained using formulas (3) and (4). Then, according to formulas (6) and (7), the system of equations is obtained, and the distortion coefficients are obtained using the least squares method. Finally, the relevant parameters are further optimized through maximum likelihood estimation to obtain a more accurate calibration result.

[0114] Step 3: Distance calculation and post-processing:

[0115] Step 3.1: Target detection and pixel extraction.

[0116] Use the object detection algorithm developed in Step 1 to perform frame-by-frame object detection on the construction site video stream, obtaining the location and category information of each object of interest in the image. Since people and machines are supposed to be located on the ground, i.e., the calibration plane, the bottom center of each bounding box is used as the pixel coordinate of the object.

[0117] Step 3.2: Three-dimensional coordinate transformation and actual distance calculation.

[0118] The projection matrix obtained in step 2 Calculate the actual three-dimensional coordinates of the worker and the machine, and then calculate the actual distance from the worker to the machine based on the coordinates.

[0119] Assume that the pixel coordinates of the object obtained in step 3.1 are , then the real coordinates of this object are , eliminating the scale parameter get If the real coordinates of another object are The distance between the two objects .

[0120] Step 3.3: Security risk detection and image post-processing.

[0121] If the actual distance calculated in step 3.2 is less than the safety threshold, it is considered that there is a safety risk here. The bounding box of the detected hazard source is marked with a special color to further improve the readability of the safety detection. This frame is pushed to the security management platform, a risk warning is issued, and relevant personnel are notified to take corresponding measures.

[0122] Of course, for ease of use, the present invention is also integrated into a system or electronic device, or set as a computer program based on the above method. When the system or electronic device is running, or the program is executed, it is used to implement the steps of the above method.

[0123] In addition, as an alternative 1: in step 1, a non-deep learning target detection method is used to detect different objects on the construction site, such as workers wearing specific equipment to achieve positioning.

[0124] In addition, as an alternative 2: the target detection method in step 1 does not use the target detection algorithm, but adopts image segmentation technology to directly classify the pixels in the image to achieve the purpose of determining the pixel category and position.

[0125] In addition, as an alternative 3: the camera calibration method in step 2 can directly measure the physical parameters and spatial position of the camera, such as the focal length and the photosensitive plate size, to directly obtain the intrinsic and extrinsic parameter matrix of the camera.

[0126] In addition, as an alternative solution 4: in step three (step 3.1), different pixel points are selected as target representation points to realize distance calculation.

[0127] Other uses:

[0128] Other uses 1: Calculate the horizontal distance and size information between different components in a plane, determine whether operations are performed according to standards, and monitor more safety indicators.

[0129] Other Use 2: The calculated camera parameters are used in BIM software to achieve viewing angle positioning, overlay and match model screenshots with on-site screenshots, and highlight current construction components.

[0130] Other Uses 3: Calculate the activities of people or equipment in a specific area for intelligent detection of major construction areas and progress monitoring of construction tasks.

[0131] Other uses 4: Calculate the distance between a worker and the boundary of a specific area to detect people in dangerous areas.

[0132] Example 2

[0133] Based on Example 1, this example selects a construction site for actual verification.

[0134] Step 1: Install a camera at the actual construction site to capture the video stream. Figure 2 The experiment uses the YOLO series algorithm (yolo11x) with faster processing speed. The training results are shown in Figure 3 shown.

[0135] Figure 3 The horizontal axis of each subgraph is the number of training rounds (epochs). The first subgraph in the first column is the border loss, category loss and segmentation loss during training, and the last two graphs are the precision and recall rate; the first subgraph in the second column is the border loss, category loss and segmentation loss during verification, and the last two graphs are the mAP of the model on the verification set. 50 Metrics and mAP 50-95 Indicators. Lower loss values indicate a better fit of the model to the training set, while higher precision, recall, and mAP indicators indicate better model performance. As can be seen during training and validation, each loss value gradually decreases and stabilizes, while precision, recall, and mAP also gradually increase until convergence, indicating that the model has been trained and has achieved good results based on these indicators.

[0136] In the target detection algorithm, this embodiment uses yolo11x as the basis and uses a self-configured dataset for training to obtain a worker and tower crane target detection algorithm. This algorithm takes a two-dimensional image as input and outputs the target location, category, and confidence level in the image. The confidence level includes the prediction box confidence and the category confidence level: the prediction box confidence level is an indicator of the confidence level of whether the prediction box contains the target object. The higher the value, the greater the possibility that the algorithm believes the object exists here; the category confidence level is the confidence level of the object belonging to a certain category. The final confidence level is calculated by multiplying the two.

[0137] Step 2: Data graph used for camera calibration and calibration results.

[0138] exist Figure 4 In the image, select the four corners of the crane base as the camera calibration anchor points. Use different color segments to mark the edges of the crane base, as shown in the following figure. Figure 4 shown.

[0139] In order to ensure the accuracy of edge detection, the marking line cannot be too thick, otherwise redundant lines will be detected, affecting the accuracy. The actual effect is as follows Figure 4 shown.

[0140] To show the marking details, Figure 5 The area is magnified and the lines are thickened. The four selected edge lines are numbered 1-4. After marking, HSV color space conversion is performed to select the red channel to filter out interference from other colors. A bitwise AND operation and line segment detection are then performed to obtain the marked line segment equation. Finally, the pixel coordinates of the anchor points are solved based on the detected line segment equations.

[0141] From the BIM measurement, we know that the size of the bottom of the tower crane is 6 meters × 6 meters, so the four corner points of the bottom of the tower crane are used as anchor points, and the coordinates are set as 、 、 、 .

[0142] The pixel coordinates of the anchor points extracted using image processing methods are:

[0143] ; ; ; ;

[0144] Calculate the camera calibration parameters:

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] .

[0151] Step 3: Target detection and distance calculation. According to the camera calibration parameters, the specific distance between workers is obtained as follows: Figure 6 As shown in the figure, this demonstrates the use of surveillance video streams to perform real-time target detection in a specific area and calculate the actual distance between objects, laying the foundation for distance-based risk detection.

[0152] Example 3

[0153] A variety of methods can be used in the camera calibration part. In addition to using BIM to measure the real scale of objects already in the picture in Example 2, calibration can also be completed by setting reference objects on site as calibration objects as in this embodiment.

[0154] Specifically, a reference object of known size can be set on the target plane, such as Figure 8 As shown in the middle mark. When setting up the calibration object, please note that the size information should be the actual distance on the calibration plane, that is, the bottom length of the calibration object. If the adjacent sides of the calibration object are not perpendicular, their angles should be additionally measured to accurately establish a rectangular coordinate system in three-dimensional space. Similar to the tower crane base, after placing the calibration object, the pixel coordinates of the calibration object are obtained through image processing. Then, based on its actual dimensions, a spatial coordinate system is established and incorporated into the camera imaging model to complete the camera calibration.

[0155] like Figure 7 and 8 As shown, the camera calibration process is performed based on calibration object 3:

[0156] Step 1:

[0157] On-site measurement to obtain the anchor point coordinates 、 、 、 ; The unit is centimeters.

[0158] Step 2: Image processing method extracts pixel coordinates, and the pixel coordinates of the four points can be obtained as follows: 、 、 、 .

[0159] Step 3: Calculate the camera parameters:

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] ;

[0165] .

[0166] Example 4

[0167] On the basis of the unchanged camera calibration parameters in Example 2, this example selects workers and cranes to detect workers and cranes and calculate the distance between them, such as Figure 9 and Figure 10 shown.

[0168] Regarding the target detection algorithm, this example uses yolo11x as the basis and uses a self-configured dataset for training to develop a worker and crane target detection algorithm. This algorithm takes a two-dimensional image as input and outputs the location, category, and confidence of the target in the image. The confidence level includes the prediction box confidence and the category confidence level. The prediction box confidence level measures the confidence level of the prediction box in determining whether the target object is contained within the prediction box. The higher the value, the greater the likelihood that the object exists within the prediction box. The category confidence level indicates the confidence level of the object belonging to a certain category. The final confidence level is calculated by multiplying the two.

[0169] In the risk detection section, set the distance threshold for risk warning to 5 meters.

[0170] Attachment Figure 9 This is the scene where the original calibration objects (the four corners of the tower crane base serve as camera calibration anchor points) disappear after camera calibration in Example 2. However, since the calibration has been completed, the disappearance of the calibration objects does not affect the distance calculation.

[0171] Attachment Figure 9 One crane (confidence level 0.98) and two workers (confidence levels 0.95 and 0.75) were detected in operation. Based on the calibration parameters obtained in step 2, the distances between the workers and the edge of the crane were calculated to be 3.06 meters and 4.16 meters, respectively. These distances are below the danger threshold, so a risk warning is issued.

[0172] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A construction site safety monitoring method based on image recognition and distance calculation, characterized in that The steps include: S10 uses a monocular camera to capture construction site image video streams; S20 calibrates the camera based on any object with adjacent straight edges in the image to obtain camera calibration data; extracts video frames from the on-site image video stream, marks the edge lines where the anchor points are located with longer straight lines in the BIM, and makes the intersection points of different straight lines the anchor points of the selected objects; uses BIM to extract the three-dimensional coordinate information of the anchor points of any object on the calibration plane of the construction site image, obtains the pixel coordinates, and obtains the camera calibration data by combining with the camera imaging model; S30 uses camera calibration data to convert the two-dimensional images sent back from the construction site into three-dimensional spatial relationships, and solves the real distance corresponding to the distance between pixels in the image. The real distance includes the actual distance between objects and the actual distance between people and target objects. S40 uses target detection algorithms or image segmentation technology to process on-site image video streams to obtain the category and location information of workers and target objects; S50 sets the actual object distance threshold and the actual distance danger zone distance threshold, and solves the real distance corresponding to the distance between pixels in the image based on the camera calibration data. When the actual distance between objects is less than the actual object distance threshold, or when the actual distance from a person to the target object is less than the actual distance danger zone distance threshold, the current risk frame is marked and a risk warning is issued.

2. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S20 , an angle exists between two straight edges of any object with adjacent straight edges.

3. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S20, the camera calibration is replaced by using BIM to extract any object anchor point on the construction site image calibration plane: according to the reference object of known real size on the construction site image calibration plane, the pixel coordinates of the reference object are obtained, and then the spatial coordinate system is established in combination with the real size, and the camera imaging model is brought into it to complete the camera calibration.

4. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S20, after selecting the anchor point, the pixel coordinate expression of the straight line is obtained through color space conversion, color range selection, edge detection, and line detection in sequence; finally, the intersection of the straight lines is calculated as the pixel coordinates based on the obtained straight line expression.

5. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S20 , a projection matrix from the three-dimensional coordinates to the pixel coordinates is calculated based on the imaging model of the camera and the correspondence between the real three-dimensional coordinates and a number of anchor points in the image pixel coordinates.

6. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S20, after the camera is moved or zoomed, the camera needs to be recalibrated.

7. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S30, the real distance corresponding to the distance between pixels in the image is solved by matrix operation; or different pixel points are selected as target representation points to realize the calculation of the real distance.

8. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S40, target detection is performed frame by frame on the construction site video stream to obtain the position and category information of each object of interest in the image; the ground where the person and the target object are located is used as the calibration plane, and the bottom center of each bounding box is used as the pixel coordinate of the target.

9. The construction site safety monitoring method based on image recognition and distance calculation according to claim 1 is characterized in that In step S40, the actual three-dimensional coordinates of the person and the target object are calculated using the obtained projection matrix, and then the actual distance from the person to the target object is calculated based on the coordinates.

10. A construction site safety monitoring system based on image recognition and distance calculation, used to implement the construction site safety monitoring method based on image recognition and distance calculation as described in any one of claims 1 to 9.

Images