Construction site inspection method, inspection system, equipment and nonvolatile storage medium

Abstract

The invention discloses a construction site inspection method, inspection system and equipment and a nonvolatile storage medium, and relates to the technical field of intelligent construction, and the method specifically comprises the steps: carrying out the inspection planning according to the inspection requirements of a construction site; based on the inspection plan, executing an inspection task, and obtaining inspection data of a target area; and analyzing the inspection data by constructing a deep learning algorithm model, and carrying out risk event classification based on an analysis result. According to the invention, a closed-loop processing mechanism of data in a platform-nest-unmanned aerial vehicle-platform is driven through control of the unmanned aerial vehicle cruise by the intelligent construction transportation and management platform for construction engineering, and a mobile supervision mode for engineering quality and safety is innovated.

Description

Technical Field

[0001] The present invention relates to the field of intelligent construction technology, and in particular to a construction site inspection method, inspection system, equipment and non-volatile storage medium. Background Art

[0002] The maturity of drone technology has begun to effectively empower other traditional industries, such as efficient inspections and assisted professional operations in agriculture and water conservancy. In the construction industry, drones have been widely used to take aerial photos of projects, tracking and recording project progress in different time periods. At present, due to the diversity and complexity of construction projects, drones equipped with AI algorithms are rarely used for intelligent assisted construction operations on construction sites. Due to the complexity of construction conditions on construction sites and the diversity of AI recognition in mobile environments, the accuracy of algorithms such as drone mobile target recognition and image segmentation is low. The main reasons that affect the accuracy of mobile image recognition at construction sites are lighting changes and complex background interference. Changes in lighting conditions (such as strong light, shadows, and night) will affect image quality. Existing algorithms are difficult to optimize pictures with strong sunlight, which may cause overexposure of the target area in the image, resulting in loss of details and affecting the recognition effect. Related algorithms cannot optimize blur and noise problems under low-light conditions, resulting in a decrease in the performance of the recognition algorithm. Complex background interference is the interference of cluttered background and similar objects on the construction site, which makes it difficult to identify the target object. The current algorithm mainly relies on a large amount of data training to distinguish between identification targets and influencing factors for the identification of similar items, but there is still a phenomenon of incorrect identification, and an effective mechanism is needed to cooperate with the algorithm to distinguish between identification items.

[0003] By increasing the complexity and auxiliary elements of the deep learning algorithm, the environmental compatibility of the UAV mobile identification is improved, and the algorithm optimizes the impact of lighting changes. Through the effective classification mechanism of the recognition results, the deep learning algorithm is assisted, the impact of complex background interference on the UAV mobile identification algorithm is optimized, and the accuracy of event judgment is improved. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that traditional manual inspections are inefficient and prone to omissions in construction site environments, especially in complex construction site environments, where manual inspections are often not comprehensive and accurate, and there are major safety hazards. In addition, the construction site environment is often affected by factors such as light and weather, and traditional monitoring systems are difficult to adapt to these changes and cannot identify potential risks in real time. At the same time, there is a large computational burden in the processing of a large amount of inspection data, and traditional methods cannot efficiently process and analyze massive amounts of data, resulting in delayed feedback and affecting the timeliness of decision-making. Existing data analysis methods fail to effectively integrate multi-dimensional data and are unable to comprehensively evaluate and respond to potential risks in a timely manner. A construction site inspection method, inspection system, equipment, and non-volatile storage medium are provided, which meet the needs of high-altitude mobile intelligent identification of construction projects through a management and control platform equipped with a drone control program + AI algorithm model, and can effectively supervise the safety and time-space of personnel in complex engineering environments and intelligently identify the quality of engineering construction. This method achieves the accuracy of engineering event identification with an effective logical architecture and algorithm-assisted identification system.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions, a construction site inspection method, comprising:

[0007] Carry out inspection planning according to the inspection requirements of the construction site; based on the inspection plan, execute inspection tasks and obtain inspection data of the target area; analyze the inspection data by building a deep learning algorithm model, and classify risk events based on the analysis results.

[0008] As a preferred solution of the construction site inspection method described in the present invention, wherein: the inspection requirements include safety requirements and quality requirements; and the initial inspection path is determined based on the inspection requirements.

[0009] As a preferred solution of the construction site inspection method described in the present invention, the execution of the inspection task includes determining the inspection time and inspection mode, and starting the inspection task along the initial path.

[0010] As a preferred solution of the construction site inspection method of the present invention, wherein: during the execution of the inspection task, the construction site inspection area is divided into a plurality of grid units, and the boundary and position of each unit are defined;

[0011] In the target area, path planning is carried out to determine the starting point, end point and turning point of the inspection, and the optimal inspection path is determined through the path optimization algorithm; the inspection task is executed according to the optimal inspection path; during the execution of the inspection task, the location of obstacles is detected and the inspection path is adjusted.

[0012] As a preferred solution of the construction site inspection method described in the present invention, the deep learning algorithm model is a YOLO v8 model, which includes a backbone network, a feature fusion module and a detection head.

[0013] As a preferred solution of the construction site inspection method described in the present invention, before analyzing the inspection data, it also includes enhancing the inspection data of the target area, generating samples under different lighting conditions through image transformation technology, and expanding the data set; extracting illumination invariant features of the image, and generating a feature map that is insensitive to illumination changes by calculating the binary pattern of the local neighborhood of each pixel; using source domain data and target domain data for domain adaptive training, and optimizing the deep learning algorithm model by minimizing the distribution difference between the source domain and the target domain.

[0014] As a preferred solution of the construction site inspection method described in the present invention, the analysis of the inspection data includes using a backbone network to extract multi-scale features of an input image; using a feature fusion module to fuse the multi-scale features to generate a multi-level feature map; inputting the fused feature map into a detection head for target classification and bounding box regression; based on the classification confidence and bounding box parameters, using a non-maximum suppression algorithm to remove redundant prediction boxes, and outputting target categories, bounding box coordinates and confidence.

[0015] As a preferred solution of the construction site inspection method described in the present invention, after the detection result is generated, the illumination of the detection result is adjusted, and the illumination value is corrected using the illumination correction coefficient to compensate for the influence of the illumination on the result.

[0016] As a preferred solution of the construction site inspection method described in the present invention, the risk event classification includes classifying events based on the confidence of event identification results; mapping confidence values ​​to different event categories according to predetermined classification standards; and executing corresponding response measures according to the event categories.

[0017] Another object of the present invention is to provide a construction site inspection system, which reduces the time and labor costs required for manual inspections through automated inspections by drones, uses deep learning algorithms to analyze real-time on-site videos, quickly identifies problems such as insufficient personnel safety protection, equipment abnormalities, and construction defects, and improves the utilization rate of drones and other resources and reduces project costs through effective inspections and real-time data return.

[0018] The construction site inspection method of the present invention can be easily implemented by an automated construction site inspection system, as follows:

[0019] As a preferred solution of the construction site inspection system described in the present invention, wherein: the system is an intelligent construction operation management platform;

[0020] The drone nest receives instructions from the intelligent construction and operation management platform to realize unmanned or auxiliary flight personnel scheduling, synchronizes operation information to the micro-UAV in the nest, and acts as an RTK base station to provide signals for the micro-UAV.

[0021] Based on the set cruising time and cruising route, the drone nest provides RTK base station information services for the micro-UAV, performs spatial information positioning of the UAV, and realizes the positioning information needs of the micro-UAV. At the same time, the drone nest provides wireless charging services for the micro-UAV to meet the needs of the drone's time-space inspection.

[0022] The drone receives the signal from the machine nest and completes the inspection at the specified time and on the specified route. The inspection process includes the inspection route and stop point information. The drone flies according to the pre-set route and arrives at the set stop points in sequence. During the flight and docking of the drone, the onboard camera collects images of the designated area of ​​the construction site and transmits the data to the video control center of the intelligent construction operation and management platform in real time until the inspection mission is completed.

[0023] As a preferred solution of the construction site inspection system described in the present invention, during the operation, the operation management platform receives the real-time images of the drone and links with the intelligent edge gateway, which provides computing power and algorithm resources. Through the deep learning algorithm pre-set by the scheduling task, the data frames in the video stream are subjected to target recognition, image classification, and confidence analysis.

[0024] A computer device includes a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of a construction site inspection method when executing the computer program.

[0025] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of a construction site inspection method when executed by a processor.

[0026] Beneficial effects of the invention: The intelligent construction and operation management platform of the construction project of the invention controls the cruise of drones, drives the closed-loop processing mechanism of data in the platform-machine nest-drone-platform, and innovates the mobile supervision mode for project quality and safety. The use of the deepened AI algorithm model to perform real-time event recognition on the video stream can effectively warn of project quality and safety events, ensure recognition accuracy, and improve event recognition efficiency.

[0027] By combining drones with deep learning algorithms, the automation and intelligence of construction site inspections have been achieved, greatly improving the efficiency and safety of inspections. The deep learning model can effectively identify construction site risks, reduce errors in manual inspections, and has stronger environmental adaptability and good robustness to factors such as lighting changes. The YOLO v8 model is combined with the non-maximum suppression algorithm to improve the accuracy of target recognition, while reducing redundant calculations and improving data processing efficiency. Through multi-dimensional risk assessment and classification, the present invention can accurately determine the risk level of events, ensure that high-risk events are responded to in a timely manner, and reduce the probability of accidents. In addition, path planning and dynamic obstacle avoidance optimize inspection paths and resource scheduling, further improving the efficiency and safety of inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0029] Figure 1 A schematic flow chart of a construction site inspection method provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0032] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0033] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0034] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0035] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0036] Example 1

[0037] Reference Figure 1 , which is the first embodiment of the present invention, provides a construction site inspection method, comprising:

[0038] Step S1: Carry out inspection planning according to the inspection requirements of the construction site;

[0039] In step S1, the embodiment of the present invention includes the inspection requirements into safety requirements and quality requirements; and determines an initial inspection path based on the inspection requirements.

[0040] It should be noted that safety requirements include identification of construction site workers not wearing safety helmets, identification of workers not wearing reflective clothing, identification of gatherings of people, identification of area intrusions, etc. Quality requirements include identification of cracks on the exterior facade of the main structure, identification of steel structure welding, etc.

[0041] Step S2: Based on the inspection plan, perform the inspection task and obtain inspection data of the target area;

[0042] In step S2, the cruise time of the embodiment of the present invention supports two modes of setting, one is to inspect at the current time, and the other is to inspect at a specified time node; the route selection supports two modes of setting, one is to set the actual route using the spatial position provided by the project BIM model, and the other is to set the remote control route of the drone by personnel.

[0043] In an optional embodiment of the present invention, an inspection task is performed by a drone to obtain inspection data of the target area.

[0044] Specifically, engineering management and control personnel operate the intelligent construction and operation management platform and use the drone module to send scheduling instructions to the drone nest. The scheduling instructions include inspection time, established routes and inspection scheduling algorithms.

[0045] Specifically, the inspection area of ​​the construction site is divided into multiple grid units through the inspection scheduling algorithm, and the boundary and position of each unit are defined; in the target area, path planning is carried out to determine the starting point, end point and turning point of the inspection, and the optimal inspection path is determined through the path optimization algorithm; the inspection task is executed according to the optimal inspection path; during the execution of the inspection task, the location of obstacles is detected and the inspection path is adjusted.

[0046] Specifically, the input parameters of the inspection scheduling algorithm include the target area (usually expressed as polygons or grids), drone performance parameters (such as flight speed, aerial photography interval, maximum flight time, battery life, shooting angle, etc.), data collection requirements (including coverage, image overlap rate, resolution, etc.), obstacles (such as buildings, trees, etc. that need to be avoided). Output variables include flight route (drone flight path, including starting point, end point, turning point, etc.), flight altitude (to ensure data collection quality, flight altitude may need to be adjusted), speed (to ensure that the drone completes the task within a given time), aerial photography interval (to ensure that image coverage and overlap meet requirements). Mathematical models include coverage problem modeling, path optimization, flight altitude and speed, and obstacle avoidance.

[0047] Furthermore, the coverage problem modeling can decompose the target area into multiple grid cells. The goal of the model is to ensure that all grid cells are covered. Let the target area be A and the grid cells be (i=1,2,…,n), then the constraints are:

[0048]

[0049] The path optimization problem can be modeled as the traveling salesman problem (TSP) or its variants, the goal is to find the shortest path so that the drone can cover all target areas. Let the starting point of the drone be S, the end point be T, and the path be P, then the optimization goal is to minimize the path length, expressed as:

[0050]

[0051] in, is a point on the path, For point and The Euclidean distance between .

[0052] The flight height h and speed v need to be adjusted to meet the requirements of image resolution and overlap rate. Assuming the image resolution is r and the overlap rate is o, the relationship between the flight height and speed can be expressed as:

[0053]

[0054]

[0055]

[0056] Where f is the focal length of the camera, d is the width or height of the resized image, is the width or height of the initial image, Distance to be covered is the distance to be covered, and Time available is the available time.

[0057] When modeling, the location of obstacles also needs to be considered. Obstacles can be avoided by setting avoidance areas or using path planning algorithms (such as Dijkstra's algorithm) to ensure that the drone can complete the task without hitting obstacles.

[0058] Furthermore, the drone nest receives instructions from the intelligent construction and operation management platform to realize unmanned or assist flight personnel scheduling, synchronize the operation information to the micro-drone in the nest, and provide accurate signals for the micro-drone as an RTK base station. With the established cruise time and cruise route, the nest provides RTK base station information services for the micro-drone, effectively locates the drone's spatial information, meets the needs of micro-drone positioning information, and reduces the deviation of the route. At the same time, the drone nest provides wireless charging services for the micro-drone to meet the needs of drone time and space inspection. The drone receives the signal from the nest and completes the inspection at the specified time and designated route. The inspection process includes the inspection route and stop point information. The drone flies according to the previous settings and arrives at the set stop point in sequence. During the flight and docking of the drone, the onboard camera collects images of the designated area of ​​the construction site and transmits the data to the video control center of the intelligent construction and operation management platform in real time until the inspection task is completed. The operation management platform receives real-time images from the drone and links with the intelligent edge gateway. The gateway provides computing power and algorithm resources to avoid redundant applications of AI servers. It performs target recognition, image classification, and confidence analysis on the data frames in the video stream through the deep learning algorithm pre-set by the scheduling task.

[0059] In an optional embodiment of the present invention, the data points collected by the micro-UAV are set to , extract features from the data collected by the drone And construct dynamic modeling of state characteristics, perform high-dimensional constraints, and conduct initial risk assessment of events, which can be expressed as:

[0060]

[0061]

[0062] in, As a result of the risk assessment, is the state feature at the current moment, is the risk attenuation factor, Indicates a continuously changing state value, is a parameter representing periodic changes, is the risk transfer function, is the state variable, is a constant exponent, is the base of natural logarithms.

[0063] Through multi-layer network analysis, the category probability of the event is defined and expressed as:

[0064]

[0065] in, For a given risk Under the conditions of category The probability of is the activation function, is the weight matrix, is a constant, is the bias term, is the change in the risk function from 0 to RISK.

[0066] When the risk assessment is less than And the category probability is not less than When , there is a low risk and it is judged as a risk event type I; when No greater than the risk assessment Less than And the category probability is not less than When the risk assessment is not less than And the class probability is greater than or equal to When the event is detected, there is a high risk and it is judged as a Class III risk event.

[0067] It should be noted that deep learning models usually require a lot of computation, especially when processing large-scale data sets. Without risk assessment screening, the algorithm in this embodiment would have to calculate all the data, increasing the computational complexity and time overhead. By processing only those events with higher risks, the computational burden can be significantly reduced.

[0068] If the algorithm of this embodiment directly inputs all data into the deep learning model without screening, low-confidence data may affect the training and decision-making accuracy of the model. Low-confidence data may come from noise, erroneous sensor data, or interference from environmental factors. Directly inputting this data into the model may lead to misjudgment and misclassification. By conducting risk assessment on the data in advance, it is ensured that only high-confidence data enters the model, avoiding the interference of low-quality data on the model.

[0069] When the quality of the data set is guaranteed, the accuracy and effectiveness of the deep learning model will be greatly improved. Pre-screening high-quality data can not only reduce misidentification, but also improve overall reliability, making the classification results more accurate, and helping to respond to actual problems in site inspections in a timely manner.

[0070] By using a risk assessment screening mechanism, the decision-making process becomes more efficient. Only when the risk event reaches the preset standard will computing resources be further invested for analysis, thus avoiding the waste of a large amount of low-risk data and enabling faster response and handling of emergency situations.

[0071] Step S3: Analyze the inspection data by building a deep learning algorithm model, and classify risk events based on the analysis results.

[0072] In step S3, the embodiment of the present invention uses the YOLOv8 model, which has higher accuracy in target detection than other CNN algorithms for target detection and recognition. In order to support real-time applications and scenarios with limited resources, high accuracy is maintained while achieving faster processing speed. The Darknet-53 architecture is used for better feature extraction and representation. The detected targets can be classified more accurately and in detail.

[0073] Specifically, the changes in the backbone network and Neck in the YOLOv8 structural design are that the convolution kernel of the first convolution layer is changed from 6*6 to 3*3, and all C3 are changed to C2f, which adds a lot of jump connections. In addition, two convolution layers in Neck are removed, and the detected targets can be classified more accurately and in detail.

[0074] It should be noted that when improving the robustness of the YOLOv8 model to illumination changes, a mathematical model framework for illumination conditions is provided, including data enhancement, illumination invariant feature extraction, model training strategies, and post-technical processing.

[0075] Specifically, data enhancement involves brightness and contrast adjustment, which can be achieved through gamma correction, which is used to adjust the brightness of the image to adapt it to different lighting conditions. The mathematical model of gamma correction is:

[0076]

[0077] Where r is the gamma factor (e.g. r>1 increases brightness, r<1 decreases brightness).

[0078] Specifically, illumination invariant feature extraction can be achieved by calculating illumination invariant features. In order to extract features that are insensitive to illumination, a local binary pattern (LBP) can be used. , calculate its local neighborhood The binary pattern of :

[0079]

[0080] Where N is the set of neighborhood pixels and k is the position of the pixel.

[0081] Specifically, the model training strategy can be implemented through domain adaptive training. In domain adaptive training, the goal is to improve the generalization ability of the model by minimizing the distribution difference between the source domain and the target domain. Suppose the source domain sample is , the target domain sample is The objective function optimized during training can be expressed as:

[0082]

[0083] in is the classification loss, is the regression loss, is the maximum mean difference (MMD) between the source and target domains, and is the weight coefficient.

[0084] Specifically, the post-processing technology can be implemented through illumination compensation, which is used to adjust the illumination conditions of the detection results. Suppose the illumination correction coefficient of the detection result is and , adjust the lighting of the target area:

[0085] .

[0086] Specifically, an image is input to the backbone network, and image features are extracted layer by layer using a multi-layer convolution module to output a multi-scale feature map; the multi-scale feature map output by the backbone network is input to a feature fusion module, and feature fusion is performed through upsampling operations and feature splicing to generate a multi-scale fused feature map; the fused feature map is input to a detection head, and target classification and bounding box regression features are extracted using a convolution operation, and the category probability and bounding box parameters of each target are output; the bounding box parameters are optimized through a loss function, and a non-maximum suppression algorithm is applied according to a target detection confidence threshold to remove duplicate prediction boxes; and the detection results are output, including the target category, bounding box coordinates, and confidence.

[0087] Furthermore, classifying risk events includes classifying events based on the confidence of the event identification results; mapping the confidence values ​​to different event categories according to predetermined classification standards; and executing corresponding response measures according to the event categories.

[0088] Furthermore, the confidence level determined after event recognition is expressed as:

[0089]

[0090] in, is the confidence level, is the Sigmoid function, is the total dimension of the feature, is the weight coefficient of the jth feature, is the value of the jth feature, For the time interval arrive Between, a dynamic feature The integral of is used to calculate the cumulative impact of a feature during this time period. is the weight coefficient of the jth feature.

[0091] In this embodiment, in order to improve the event recognition accuracy of the mobile AI inspection of drones, an event classification mechanism based on confidence is adopted. Specifically, the events are classified by the normalized confidence value, the confidence value range is 0 to 1, and the classification criteria are as follows: when the confidence value is less than 0.2, it is judged as a first-level event; when the confidence value is between 0.2 and 0.4, it is judged as a second-level event; when the confidence value is between 0.4 and 0.6, it is judged as a third-level event; when the confidence value is between 0.6 and 0.8, it is judged as a fourth-level event; when the confidence value is greater than or equal to 0.8, it is judged as a fifth-level event.

[0092] After the event classification, in order to improve the overall accuracy of identification, processing strategies for events of different levels are adopted. Specifically, the first-level events may have a high probability of misidentification due to their low confidence, so all of them need to be manually reviewed to ensure the accuracy of event classification. Although the second-level events have a certain degree of confidence, there may still be certain errors. For this reason, manual spot checks are adopted for review. The setting of the spot check ratio and the number of corrections should be adjusted according to the actual situation to improve the accuracy of identification. Through the classification and processing mechanism, it is possible to effectively filter out erroneous data with low confidence, ensure that the final output risk event category has a higher accuracy, thereby optimizing the inspection results and reducing the risk of misjudgment. If the event level is level three to level five, the identification event is identified.

[0093] In one embodiment, the YOLOv8 model is a deep learning algorithm for target detection, which has three parts: a backbone network (Backbone), a feature fusion module (Neck) and a detection head (Head).

[0094] The backbone network is responsible for extracting multi-scale features of the input image. Its structure consists of multi-layer convolution modules and feature aggregation modules, including five layers of feature extraction from P1 to P5. The input image is subjected to a convolution layer with a convolution kernel size of 3×3 and a pooling operation with a stride of 2, which gradually extracts high-level semantic information and reduces the spatial resolution of the feature map. The size of the feature map output by each layer is halved in turn, from 640×640 to 20×20, and the feature dimension is gradually increased. In order to improve the efficiency and feature extraction ability of the model, the traditional C3 module is improved to the C2f module. The C2f module effectively reduces the loss of gradient information through multi-jump connections and improves the feature extraction quality of the model. In the final stage of feature extraction, the spatial pyramid pooling (SPPF) module is added. This module aggregates spatial information at multiple feature scales through parallel multi-scale pooling operations (the convolution kernel sizes are 3×3, 5×5, 9×9, and 13×13, respectively), enhancing the ability to extract target edge features and semantic background. The multi-level feature output of the backbone network provides multi-scale input for the feature fusion module, which not only retains the spatial details of the low-level but also extracts the semantic information of the high-level, thus providing a high-quality feature foundation for subsequent modules.

[0095] The core function of the feature fusion module is to integrate the multi-scale features output by the backbone network. The model adopts the structural idea of ​​the feature pyramid network (FPN) to fuse low-resolution high-level semantic features with high-resolution underlying detail features through upsampling and feature concatenation operations. During the fusion process, the model further uses the C2f module to optimize the expression ability of the fused features to ensure that the loss of semantic information is reduced during the information flow transmission process. For small target detection, the feature fusion module can enhance the capture of local details; for large target detection, the semantically enhanced features are used to improve the classification and positioning capabilities of the target. The output of the feature fusion module provides the detection head with three feature maps of different scales (P3, P4, P5), which are used to detect targets of different sizes.

[0096] The detection head is the final output part of the entire model, and its main function is to complete the classification and bounding box prediction of the target. Through multiple convolutional layers, the detection head outputs the category probability and bounding box parameters of each target, including the center position, width and height of the target. The YOLOv8 detection head uses the CIoU loss function to optimize the regression performance of the bounding box, comprehensively considering the overlapping area, center point distance and aspect ratio of the bounding box, and significantly improves the accuracy of target positioning. In addition, to solve the problem of category imbalance, the model introduces the DFL (Distribution Focal Loss) loss function, which ensures the accuracy of common category detection while enhancing the detection performance of rare categories. The detection head uses the non-maximum suppression (NMS) algorithm to remove overlapping prediction boxes, retain only the targets with the highest confidence, and output the detection results as categories, bounding box coordinates and confidence.

[0097] The backbone network significantly improves the efficiency of feature extraction by introducing the C2f module to replace the traditional C3 module. The C2f module enhances the transmission of gradient flow through more jump connections while reducing the number of parameters. In addition, the multi-scale pooling design of the SPPF module ensures the model's efficient modeling of small targets and complex background scenes. Compared with the traditional backbone network, the YOLOv8 backbone network significantly reduces the computational complexity and improves the real-time performance while ensuring detection accuracy.

[0098] The feature fusion module removes the traditional complex convolution stacking operation, adopts simple upsampling and feature splicing, and introduces the C2f module again in the fused feature map for optimization. This design enhances the detection performance of small targets, especially in scenes with dense targets or complex backgrounds, and significantly improves the robustness of the model.

[0099] The detection head optimizes classification and regression performance. The introduction of the CIoU loss function allows the model to consider more geometric constraints when locating targets, especially in target overlapping scenarios, which greatly improves the positioning accuracy. The DFL loss function effectively solves the category imbalance problem in category classification and improves the detection performance of low-confidence categories.

[0100] The backbone network provides a high-quality feature extraction foundation, the feature fusion module realizes the efficient integration of multi-scale features, and the detection head performs classification and regression processing on the feature map, and finally outputs the detection results. The entire model structure is closely coordinated, which not only ensures efficient computing performance, but also realizes high-precision target detection in complex scenes.

[0101] Through the above improvements, the YOLOv8 model surpasses traditional methods in terms of accuracy, real-time and adaptability of target detection. In scenes with large changes in lighting conditions, dense small targets and complex target categories, the model ensures the integrity and stability of feature extraction through the design of the C2f module and the SPPF module. The efficient fusion mechanism of the feature fusion module significantly improves the performance of the model in small target detection. The CIoU and DFL optimization strategies of the detection head ensure the accuracy of the model in bounding box positioning and target classification. The design of the entire YOLOv8 model combines the optimization schemes of the backbone network, feature fusion module and detection head, which is not only suitable for target detection tasks in construction site inspection scenarios, but also provides reliable technical support for real-time detection and intelligent management in complex environments.

[0102] Example 2

[0103] This is the second embodiment of the present invention, which provides a construction site inspection system, wherein the system is an intelligent construction operation management platform;

[0104] The drone nest receives instructions from the intelligent construction and operation management platform to realize unmanned or assist flight personnel scheduling, synchronizes operation information to the micro-drones in the nest, and acts as an RTK base station to provide signals for the micro-drones;

[0105] Based on the set cruising time and cruising route, the drone nest provides RTK base station information services for the micro-UAV, effectively positioning the drone in space, meeting the micro-UAV's positioning information needs and reducing route deviations. At the same time, the drone nest provides wireless charging services for the micro-UAV to meet the drone's time-space inspection needs.

[0106] The drone receives the signal from the machine nest and completes the inspection at the specified time and on the specified route. The inspection process includes the inspection route and stop point information. The drone flies according to the pre-set route and arrives at the set stop points in sequence. During the flight and docking of the drone, the onboard camera collects images of the designated area of ​​the construction site and transmits the data to the video control center of the intelligent construction operation and management platform in real time until the inspection mission is completed.

[0107] The system includes that during the operation, the operation management platform receives the real-time images of the drone and links with the intelligent edge gateway, which provides computing power and algorithm resources. Through the deep learning algorithm pre-set by the scheduling task, the data frames in the video stream are subjected to target recognition, image classification, and confidence analysis.

[0108] Example 3

[0109] The third embodiment of the present invention provides a computer device dedicated to inspection. In this embodiment, the various processing functions involved can be completed collaboratively by the core processor of the computer system or the external expansion processor. These processors include but are not limited to CPU, FPGA (field programmable gate array), ASIC (application-specific integrated circuit) and edge computing devices. Specifically, the CPU can be used to control the core logic process and execute basic algorithms; FPGA is suitable for real-time processing of complex inspection data, and supports dynamic adjustment of algorithms to adapt to lighting changes and image processing requirements in the construction site environment; ASIC can be specially optimized for deep learning algorithms (such as YOLOv8) to provide efficient target detection and classification capabilities; edge computing devices (such as smart gateways) are embedded with GPU or TPU, which is dedicated to deep neural network analysis of video streams transmitted by drones.

[0110] During the inspection process, the camera on the drone collects the site images in real time, and the video stream is transmitted to the intelligent construction and operation management platform through the 5G or Wi-Fi module. To ensure data quality, the drone and the machine nest system work together. The machine nest serves as the infrastructure of the drone, receiving platform instructions and providing RTK positioning signals to optimize the flight path, while supporting real-time data processing and wireless charging functions. The drone performs tasks according to the preset time, route and inspection rules, and its path planning and obstacle avoidance can be calculated in real time through embedded FPGA or ASIC.

[0111] In addition, in order to improve the processing effect of video stream data under complex lighting conditions, the system uses pre-processing techniques such as gamma correction to adjust the image brightness and contrast, and extracts lighting invariant features in combination with the local binary pattern (LBP) algorithm. Subsequently, the images transmitted to the platform are analyzed by deep learning algorithms (such as the improved YOLOv8 model). The YOLOv8 model uses an optimized Darknet-53 backbone network architecture, and improves the recognition robustness and generalization ability in a changing environment through feature extraction enhancement and domain adaptive training strategies. After event recognition, the results are dynamically classified and graded according to the confidence of the model output, where the higher the confidence, the higher the reliability of the recognition result.

[0112] At the same time, in order to cope with the limitations of mobile AI recognition, the system has designed a closed-loop processing mechanism in the intelligent construction and operation management platform: the drone will send the collected images back to the platform in real time, and after the intelligent gateway completes the deep learning analysis, the processing results will be fed back to the platform. The platform classifies events according to the confidence classification rules, and divides them into five levels from low risk to high risk. High-risk events require manual review to further improve the overall recognition accuracy.

[0113] The system's software programs (including inspection task planning, AI algorithm models, and hierarchical processing coordination mechanism algorithms) can be pre-stored in non-volatile storage media such as ROM, CD-ROM, DVD-ROM, or USB memory, or dynamically deployed by downloading from external devices over the network. All processing functions can be completed by a single type of processor (such as CPU, FPGA), or by a combination of multiple processors (such as GPU and ASIC) to fully utilize hardware resources and improve efficiency.

[0114] Example 4

[0115] The fourth embodiment of the present invention provides a non-volatile computer-readable storage medium. In this embodiment, the non-volatile computer-readable storage medium is a key component for storing software programs, data and AI algorithm models required to implement various functions of the inspection system. These storage media ensure that data and programs can still be completely preserved after the equipment is powered off, thereby ensuring the stable operation and efficient performance of the system. According to functional requirements, this embodiment uses various types of non-volatile storage media, including ROM, flash memory, optical storage media, solid-state drives (SSDs) and hard disk drives (HDDs) to meet the different application scenarios of drone systems, machine nest systems and intelligent construction and operation management platforms.

[0116] Among them, ROM is mainly used to store the core logic programs of the system, such as the basic inspection algorithm and communication protocol module of the drone, to ensure the stable execution of the inspection tasks; flash memory (such as USB memory and SD card) stores dynamic configuration files, deep learning models (such as YOLOv8 optimized weight parameters) and data generated during the task execution; optical storage media (such as CD-ROM and DVD-ROM) are mostly used to distribute system installation packages or store static training data sets; SSD and HDD are mainly deployed in the intelligent construction and operation management platform to store large-scale video data, historical records and AI algorithm model training data to support real-time processing and subsequent analysis of complex tasks.

[0117] The content stored in the storage medium of this embodiment covers a variety of data types, including: AI algorithm model files (such as optimized YOLOv8 weight parameters, lighting robustness models and event classifier models), which support efficient detection and classification of targets in complex construction site environments; program code, used to implement the core functions of inspection task scheduling, path planning, image preprocessing (such as gamma correction) and event classification; system configuration files, which store drone performance parameters (such as flight speed and battery life), route information (such as RTK positioning correction parameters) and event classification thresholds, and support personalized task settings; historical records and operation data, including images, video data and recognition results collected during the inspection process, to provide a basis for subsequent analysis and optimization; and upgrade and expansion modules, used to support the iteration and expansion of system functions.

[0118] These data are accessed in three ways: the pre-loading method is suitable for fixed programs (such as basic inspection logic and path planning algorithms), which are written into the ROM during the production stage to ensure that they can run immediately after the system starts, reducing initialization delays; the dynamic loading method is mainly used to dynamically load AI algorithm models, configuration files and inspection task data from flash memory or SSD, supporting real-time updates and task adjustments; interactive access is achieved through the interactive interface of the intelligent construction and operation management platform. Users can upload or download data to facilitate personalized configuration of inspection tasks and data management.

[0119] The use of non-volatile storage media in this embodiment brings significant technical advantages. The first is high data reliability. Even if the device is powered off, the data can still be fully saved to ensure stable operation of the system; the second is high efficiency. The pre-loading method can quickly run key programs, reduce dependence on external network connections, and significantly improve the system response speed; in addition, these storage media are also flexible. Training data and system update packages are stored through external media (such as USB memory or optical disks), which facilitates the rapid upgrade and expansion of system functions; the last is scalability. Especially in the intelligent construction and operation management platform, high-capacity SSDs and HDDs can support long-term video storage and multi-version deep learning model management, providing strong data support for complex tasks.

[0120] Specifically, the drone nest has a small non-volatile storage medium (such as ROM or flash memory) built in to store core algorithms and scheduling instructions to ensure that the drone can quickly complete on-site tasks; while the intelligent construction and operation management platform is equipped with a high-capacity SSD or HDD, which is mainly used to store task data, model parameters and historical video records to meet large-scale data storage needs. In addition, portable storage media (such as USB memory) can also be used for temporary storage and transfer of drone on-site data to improve the flexibility and convenience of data management.

[0121] Example 5

[0122] As an embodiment of the present invention, a construction site inspection method is provided. In order to verify the beneficial effects of the present invention, a scientific demonstration is carried out through simulation experiments.

[0123] The experimental environment was selected from a construction site in actual construction, with a site size of about 200 meters × 300 meters. The experimental area is divided into three parts: the first part is the main building area, including the exterior walls, steel structures, and floors under construction; the second part is the construction equipment area, where steel bars, cement, formwork, and mechanical equipment (such as excavators and cranes) are stacked; the third part is the personnel activity area, including workers under construction and material transportation channels. The lighting conditions are set to three states: strong light at noon, shadow, and low light at night to ensure that the experiment covers a variety of real scenes.

[0124] In the experimental equipment, the method group of the present invention adopts the intelligent construction and operation management platform of the drone nest linkage, configures the optimized YOLOv8 model, and supports real-time video stream processing and event recognition. The drone is equipped with a high-definition camera with a resolution of 1920×1080, a cruising speed of 2m / s, and an adjustable flight altitude of 10-30 meters. The traditional method group adopts the existing YOLOv5 model, and other parameters are consistent with the method group of the present invention. The traditional method does not set the optimization algorithm and confidence classification mechanism, and directly outputs the target detection results.

[0125] The experimental design is divided into two groups. The method group of the present invention applies the YOLOv8 model to the video stream collected by the drone, adopts optimized brightness adjustment, feature extraction, illumination invariance optimization, confidence classification and other mechanisms, and classifies the events according to the confidence threshold after event recognition. Low-confidence events are confirmed by manual review. The traditional method group uses the YOLOv5 model to directly identify the target category and location without setting up further optimization mechanisms.

[0126] The experimental steps are as follows. The drone flies in the experimental area according to the preset path, and collects video streams under strong light at noon, shadow and low light at night. The video stream is transmitted back to the platform in real time. The traditional method group directly detects the target on the video frame and outputs the category and position of the detected target. After the video stream is transmitted back, the method group of the present invention starts the YOLOv8 model for detection. The model performs gamma correction, local binary pattern calculation and domain adaptive training on the image data. After detecting the target, it assigns confidence and classifies it according to the confidence. Low-confidence events enter the manual review module, and the remaining events generate classification reports. The statistical results of the experimental data are shown in Table 1.

[0127] Table 1 Comparison of experimental results

[0128] Experiments show that the target detection experimental results of the method of the present invention under different lighting conditions show that its performance in strong light, shadow and low light environments is better than that of traditional methods. Under strong light conditions, the present invention significantly alleviates the overexposure problem and completely retains the detail information of the target area through the coordinated optimization of gamma correction and feature extraction; under shadow conditions, the deep feature fusion capability of the model enables the dark details to be fully displayed; and under low light conditions, the illumination invariant feature extraction technology and domain adaptive training strategy further enhance the model's adaptability to low light scenes. These improvements are derived from the optimized design of the model structure for illumination changes, which ensures the ability to accurately detect target objects in actual construction site environments. In contrast, due to the lack of illumination optimization mechanism, the performance of traditional methods in complex lighting environments is greatly limited, and they cannot effectively deal with problems such as loss of details or blurred targets caused by drastic changes in illumination.

[0129] In order to further verify the adaptability of the model to complex backgrounds, the detection performance of the two groups of methods in the main building area, construction equipment area and personnel activity area were statistically analyzed. The experimental results are shown in Table 2.

[0130] Table 2 Comparison of experimental results

[0131] The experimental results of target detection under complex background interference conditions of the method of the present invention show that its detection capabilities in the main building area, construction equipment storage area and personnel intensive area are significantly better than traditional methods. In the main building area, the method of the present invention achieves accurate identification of facade cracks and steel structure welding defects through the jump connection design and deep feature extraction capabilities of the C2f module, effectively reducing the false detection rate. In the construction equipment storage area, the multi-layer feature fusion mechanism enhances the anti-interference ability of scenes with high background complexity and realizes accurate distinction of equipment classification. In the personnel intensive area, combined with the dynamic adjustment of confidence grading and manual review strategy, the method of the present invention effectively reduces the false detection problems caused by dynamic background changes and interference from similar objects. This shows that the present invention significantly improves the detection accuracy and reliability in complex scenarios on the construction site through an improved detection model, an optimized classification mechanism and an event grading strategy.

[0132] In summary, the experimental results show that the method of the present invention has significant technical advantages over traditional methods in terms of target detection accuracy, anti-interference ability and illumination adaptability. Through multi-level structural optimization, combined with illumination optimization technology and confidence classification strategy, the present invention shows good robustness and efficiency in scenes with drastic illumination changes and complex background interference.

[0133] Example 6

[0134] The sixth embodiment of the present invention is different from the first five embodiments in that:

[0135] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0136] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0140] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A construction site inspection method, characterized in that: include, Carry out inspection planning according to the inspection requirements of the construction site; Based on the inspection plan, perform inspection tasks and obtain inspection data of the target area; Analyze the inspection data by building a deep learning algorithm model, and classify risk events based on the analysis results; Before analyzing the inspection data, the method further includes: Enhance the inspection data of the target area, generate samples under different lighting conditions through image transformation technology, and expand the data set; Perform illumination invariant feature extraction on the image, and generate a feature map that is insensitive to illumination changes by calculating the binary pattern of the local neighborhood of each pixel; Domain adaptation training is performed using source domain data and target domain data to optimize the deep learning algorithm model by minimizing the distribution difference between the source domain and the target domain.

2. The construction site inspection method according to claim 1, characterized in that: The inspection requirements include: Safety requirements and quality requirements; Determine the initial inspection path based on inspection requirements.

3. The construction site inspection method according to claim 1 or 2, characterized in that: The inspection task is performed, include, Determine the inspection time and inspection mode, and start the inspection task along the initial path.

4. The construction site inspection method according to claim 3, characterized in that: During the inspection task, the inspection area of ​​the construction site is divided into a plurality of grid units, and the boundaries and positions of each unit are defined; In the target area, path planning is performed to determine the starting point, end point and turning point of the inspection, and the optimal inspection path is determined through the path optimization algorithm; Perform inspection tasks according to the optimal inspection path; During the inspection task execution, the obstacle location is detected and the inspection path is adjusted.

5. The construction site inspection method according to any one of claims 1, 2 or 4, characterized in that: The deep learning algorithm model is a YOLO v8 model, which includes a backbone network, a feature fusion module and a detection head.

6. The construction site inspection method according to claim 5, characterized in that: The analyzing the inspection data includes: Use the backbone network to extract multi-scale features of the input image; The feature fusion module is used to fuse multi-scale features and generate multi-level feature maps; The fused feature map is input into the detection head for target classification and bounding box regression; Based on the classification confidence and bounding box parameters, the non-maximum suppression algorithm is used to remove redundant prediction boxes and output the target category, bounding box coordinates and confidence.

7. The construction site inspection method according to any one of claims 1, 4 or 6, characterized in that: After the detection results are generated, the illumination of the detection results is adjusted, and the illumination value is corrected using the illumination correction coefficient to compensate for the influence of illumination on the results.

8. The construction site inspection method according to any one of claims 1, 4 or 6, characterized in that: The risk event classification includes: Classify events based on the confidence of event recognition results; According to the predetermined classification criteria, the confidence values ​​are mapped to different event categories; Execute corresponding response measures according to the event category.

9. A construction site inspection system, characterized in that: The system is an intelligent construction operation and management platform; The drone nest receives instructions from the intelligent construction and operation management platform to realize unmanned or assist flight personnel scheduling, synchronizes operation information to the micro-drones in the nest, and acts as an RTK base station to provide signals for the micro-drones; The drone receives the signal from the machine nest and completes the inspection at the specified time and on the specified route. The inspection process includes the inspection route and stop point information. The drone flies according to the pre-set route and arrives at the set stop points in sequence. During the flight and docking of the drone, the onboard camera collects images of the designated area of ​​the construction site and transmits the data to the video control center of the intelligent construction operation and management platform in real time until the inspection mission is completed.

10. The construction site inspection system according to claim 9, characterized in that: The system includes that during the operation, the operation management platform receives the real-time images of the drone and links with the intelligent edge gateway, and the gateway provides computing power and algorithm resources.

11. A computer device dedicated to inspection, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: The processor includes but is not limited to CPU, FPGA, ASIC and edge computing device; the CPU is used to control the core logic process and execute the basic algorithm; the FPGA is used to process complex inspection data in real time and support dynamic adjustment of the algorithm to adapt to the lighting changes and image processing requirements in the construction site environment; the ASIC is specially optimized for the AI ​​algorithm; the edge computing device has an embedded GPU or TPU, which is dedicated to deep neural network analysis of the video stream transmitted by the drone; At the same time, the processor implements the steps of any one of the methods of claims 1 to 8 when executing.

12. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: The storage medium includes a small non-volatile storage medium built into the drone nest for storing core algorithms and scheduling instructions; the intelligent construction operation management platform is configured with an SSD or HDD for storing mission data, model parameters and historical video records, and a portable storage medium for temporary storage and transfer of drone field data; When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images