Construction safety hazard analysis system and method based on image recognition

Abstract

The application provides a construction safety hazard analysis system and method based on image recognition, and relates to the technical field of safety hazard analysis.The method comprises the following steps: obtaining a digging video of excavated earthwork and a three-dimensional point cloud model of unexcavated earthwork; determining a plurality of initial safe digging points based on the three-dimensional point cloud model of the unexcavated earthwork; determining a plurality of preliminary digging routes based on the plurality of initial safe digging points and the three-dimensional point cloud model of the unexcavated earthwork, and each preliminary digging route passes through each initial safe digging point; generating a simulated digging video of each preliminary digging route based on the preliminary digging route, the three-dimensional point cloud model of the unexcavated earthwork and the digging video of the excavated earthwork; and determining a target digging route based on the simulated digging video of each preliminary digging route.The method can accurately determine the target digging route of earthwork excavation operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of safety hazard analysis, and particularly relates to a construction safety hazard analysis system and method based on image recognition. BACKGROUND

[0002] In the field of building construction, with the increasing complexity of projects, the safety hazard analysis of earth excavation operations and construction efficiency are facing severe challenges. Traditional construction safety management mainly relies on manual site reconnaissance and experience-based judgment, which is difficult to accurately quantify the stability of the geological structure of unexcavated earth, and lacks real-time monitoring means for the dynamic changes of the soil in the excavated area. At the same time, when manually planning the excavation route, it is difficult to efficiently utilize three-dimensional geological data for global safety assessment, resulting in long design cycle and high risk of excavation schemes, especially in deep foundation pits and complex rock layers, etc. scenes, prone to accidents such as collapse and landslide due to delayed safety hazard investigation. In the prior art, it is difficult to realize the full-process digital modeling of the construction scene through single sensor monitoring or simple algorithm analysis, and lacks intelligent closed-loop management capabilities from data collection, risk analysis to scheme optimization, resulting in low efficiency of construction safety control and failing to meet the needs of modern engineering for refined and intelligent management.

[0003] Therefore, how to accurately determine the target excavation route of earth excavation operation is a problem to be solved at present. SUMMARY

[0004] The technical problem solved by the present application is how to accurately determine the target excavation route of earth excavation operation.

[0005] According to a first aspect, the present application provides a construction safety hazard analysis method based on image recognition, comprising: acquiring an excavation video of excavated earth and a three-dimensional point cloud model of unexcavated earth; determining a plurality of initial safe excavation points based on the three-dimensional point cloud model of unexcavated earth; determining a plurality of preliminary excavation routes based on the plurality of initial safe excavation points and the three-dimensional point cloud model of unexcavated earth, each preliminary excavation route passing through each initial safe excavation point; generating a simulated excavation video of each preliminary excavation route based on the preliminary excavation route, the three-dimensional point cloud model of unexcavated earth, and the excavation video of excavated earth; and determining a target excavation route based on the simulated excavation video of each preliminary excavation route.

[0006] In a possible implementation, the determining the target excavation line based on the simulated excavation video of each preliminary excavation line comprises: constructing a knowledge graph, the knowledge graph comprising a plurality of preliminary excavation line nodes and a plurality of edges between the preliminary excavation line nodes, the node features of each preliminary excavation line node comprising the simulated excavation video of the preliminary excavation line, and the edges between the preliminary excavation line nodes being position relationships between different lines; processing the knowledge graph based on a graph neural network to determine a plurality of target safe excavation points; determining a plurality of preferred excavation lines based on the plurality of target safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, each preferred excavation line passing through each target safe excavation point; generating a simulated excavation video of each preferred excavation line based on the plurality of preferred excavation lines, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork; determining excavation information of each preferred excavation line based on the simulated excavation video of each preferred excavation line using the excavation processing model; and determining the target excavation line based on the excavation information of each preferred excavation line.

[0007] In a possible implementation, the generating the simulated excavation video of each preliminary excavation line based on the preliminary excavation line, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork comprises: generating the simulated excavation video of each preliminary excavation line based on the preliminary excavation line, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork using a generative adversarial network.

[0008] In a possible implementation, the excavation processing model is a Transformer model.

[0009] According to a second aspect, the present application provides a construction safety hazard analysis system based on image recognition, comprising: an acquisition module configured to acquire an excavation video of excavated earthwork and a three-dimensional point cloud model of unexcavated earthwork; a safety point determination module configured to determine a plurality of initial safe excavation points based on the three-dimensional point cloud model of the unexcavated earthwork; a line generation module configured to determine a plurality of preliminary excavation lines based on the plurality of initial safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, each preliminary excavation line passing through each initial safe excavation point; a simulation generation module configured to generate a simulated excavation video of each preliminary excavation line based on the preliminary excavation line, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork; and a target determination module configured to determine a target excavation line based on the simulated excavation video of each preliminary excavation line.

[0010] In a possible implementation, the target determination module is further configured to: construct a knowledge graph, the knowledge graph comprising a plurality of preliminary mining route nodes and a plurality of edges between the preliminary mining route nodes, a node feature of each preliminary mining route node comprising a simulated mining video of a preliminary mining route, and the edges between the preliminary mining route nodes being positional relationships between different routes; determine a plurality of target safe mining points based on processing of the knowledge graph by using a graph neural network; determine a plurality of preferred mining routes based on the plurality of target safe mining points and the three-dimensional point cloud model of the unmined earthwork, each preferred mining route passing through each target safe mining point; generate a simulated mining video of each preferred mining route based on the plurality of preferred mining routes, the three-dimensional point cloud model of the unmined earthwork, and the mining video of the mined earthwork; determine mining information of each preferred mining route based on the simulated mining video of each preferred mining route by using a mining processing model; and determine a target mining route based on the mining information of each preferred mining route.

[0011] In a possible implementation, the simulation generation module is specifically configured to: generate a simulated mining video of each preliminary mining route based on the preliminary mining route, the three-dimensional point cloud model of the unmined earthwork, and the mining video of the mined earthwork by using a generative adversarial network.

[0012] In a possible implementation, the mining processing model is a Transformer model.

[0013] According to a third aspect, embodiments of the present application provide an electronic device, comprising: a processor; a memory; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement a method as described above, the method comprising: obtaining a mining video of a mined earthwork and a three-dimensional point cloud model of an unmined earthwork; determining a plurality of initial safe mining points based on the three-dimensional point cloud model of the unmined earthwork; determining a plurality of preliminary mining routes based on the plurality of initial safe mining points and the three-dimensional point cloud model of the unmined earthwork, each preliminary mining route passing through each initial safe mining point; generating a simulated mining video of each preliminary mining route based on the preliminary mining route, the three-dimensional point cloud model of the unmined earthwork, and the mining video of the mined earthwork; and determining a target mining route based on the simulated mining video of each preliminary mining route.

[0014] According to a fourth aspect, the embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image recognition-based construction safety hazard analysis method provided in the foregoing aspects, and the method comprises the following steps: acquiring a digging video of a dug earthwork and a three-dimensional point cloud model of an undug earthwork; determining a plurality of initial safe digging points based on the three-dimensional point cloud model of the undug earthwork; determining a plurality of preliminary digging routes based on the plurality of initial safe digging points and the three-dimensional point cloud model of the undug earthwork, and each preliminary digging route passes through each initial safe digging point; generating a simulated digging video of each preliminary digging route based on the preliminary digging route, the three-dimensional point cloud model of the undug earthwork and the digging video of the dug earthwork; and determining a target digging route based on the simulated digging video of each preliminary digging route.

[0015] The image recognition-based construction safety hazard analysis system and method provided by the embodiment can accurately determine a target digging route of earthwork digging operation. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of the image recognition-based construction safety hazard analysis method provided by the embodiment;

[0017] Figure 2 A flowchart of the method for determining a target digging route provided by the embodiment;

[0018] Figure 3 A schematic diagram of the image recognition-based construction safety hazard analysis system provided by the embodiment;

[0019] Figure 4 A schematic diagram of an electronic device provided by the embodiment; DETAILED DESCRIPTION

[0020] The application will be described in further detail below with specific reference being made to the drawings. Like elements are referred to with like reference numerals throughout the various figures in the drawings. In the following description, numerous specific details are described to provide a thorough understanding of the application. However, it will be apparent to one skilled in the art that the application can be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order not to unnecessarily obscure the application. In the following description, numerous specific details are described to provide a thorough understanding of the application. However, it will be apparent to one skilled in the art that the application can be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order not to unnecessarily obscure the application.

[0021] In the embodiments of the present application, there is provided an image recognition-based construction safety hazard analysis method as shown in Figure 1 The image recognition-based construction safety hazard analysis method comprises steps S1-S5.

[0022] In step S1, a digging video of a dug earthwork and a three-dimensional point cloud model of an undug earthwork are obtained.

[0023] The dug earthwork is an earthwork region that has completed the digging operation in the construction process.

[0024] The undug earthwork is an original earthwork region that has not implemented digging within the construction range.

[0025] The digging video of the dug earthwork is a video for recording the construction process of the dug earthwork region by a camera.

[0026] The three-dimensional point cloud model of the undug earthwork is a discrete point set of the undug earthwork region obtained by a laser scanning technology, each point of which contains accurate X, Y, Z coordinates and reflectivity attributes. The three-dimensional point cloud model can completely reconstruct the surface morphology and spatial structure, and can accurately restore the original appearance and geological structure detail information of the undug earthwork.

[0027] In step S2, a plurality of initial safe digging points are determined based on the three-dimensional point cloud model of the undug earthwork.

[0028] In some embodiments, the plurality of initial safe digging points can be determined based on the three-dimensional point cloud model of the undug earthwork using a safe point determination model, the safe point determination model being a convolutional neural network model, an input of the safe point determination model being the three-dimensional point cloud model of the undug earthwork, and an output of the safe point determination model being the plurality of initial safe digging points.

[0029] The convolutional neural network model includes a convolutional neural network (CNN), which is a model architecture designed for grid-like data in the field of deep learning. The core of the convolutional neural network is composed of a convolutional layer, a pooling layer, an activation function, and a fully connected layer. The convolutional layer slides over the data with a learnable convolutional kernel to extract local features while sharing weights to reduce the number of parameters; the pooling layer can reduce the data dimension to enhance the robustness of the model to data deformation; the activation function can give the model nonlinear expression ability; and the fully connected layer can complete the final classification or regression task based on the extracted features. The convolutional neural network can be used to process complex data types such as three-dimensional point cloud models, and its advantage lies in its ability to automatically learn feature patterns in the data and reduce the cost and error of manually designed features.

[0030] The initial safe excavation point is a basic safe point position in the unexcavated earthwork determined by the safety point determination model output suitable for excavation operation. The initial safe excavation point meets the basic conditions of safety constraints such as geological stability and construction accessibility.

[0031] The three-dimensional point cloud model of unexcavated earthwork is a digital expression of the construction site terrain, which contains a large number of discrete points, each containing precise three-dimensional coordinate information. Through these points, the details of the earthwork slope, concave-convex shape, geological faults, etc. can be completely presented, thereby providing intuitive and accurate basic data for construction planning. Three-dimensional point cloud model data can be converted into structured data that can be processed by a convolutional neural network through multi-view projection processing. The convolutional neural network can accurately capture the geometric features related to excavation safety such as earthwork slope and loose area through its local feature extraction capability, then analyze the local geometric information of the point cloud and output the safety probability of each point. Points higher than the threshold value will be marked, and finally the convolutional neural network can filter and optimize the initial safe excavation points in combination with construction machinery operation requirements and geological conditions.

[0032] In some embodiments, the safety point determination model includes a terrain analysis layer, a region evaluation layer, and a safety point determination layer. The terrain analysis layer, the region evaluation layer, and the safety point determination layer all include a convolutional neural network structure. The input of the terrain analysis layer is the three-dimensional point cloud model of the unexcavated earthwork, and the output of the terrain analysis layer is a terrain slope map, fault position markers, and surface concave-convex region distribution. The input of the region evaluation layer is the terrain slope map, fault position markers, and surface concave-convex region distribution, and the output of the region evaluation layer is a plurality of segmented regions, a slope safety score for each region, a fault risk level for each region, and a mechanical access score for each region. The input of the safety point determination layer is the plurality of segmented regions, the slope safety score for each region, the fault risk level for each region, and the mechanical access score for each region, and the output of the safety point determination layer is a plurality of initial safe excavation points.

[0033] The terrain slope map is a visual result of the slope distribution of the unexcavated earthwork terrain generated by calculating the three-dimensional point cloud model data. The terrain slope map labels the slope size of different regions (such as flat areas and steep areas), and can directly reflect the degree of fluctuation of the unexcavated earthwork terrain.

[0034] The fault position marker is a geological fault region position marker identified in the three-dimensional point cloud model data. It marks the specific coordinate range of the fault zone and can prompt the high-risk geological region of construction.

[0035] The surface concave-convex region distribution is a label of the distribution of the surface convex and concave regions of the unexcavated earthwork terrain. This label can be used to distinguish the spatial positions of different terrain forms, thereby assisting in judging the difficulty of mechanical access.

[0036] The multiple segmentation regions are used to refine the terrain safety analysis. The multiple segmentation regions are irregular block regions divided by the model according to terrain features (such as slope change, fault distribution, and surface concave-convex form). Each region has independent terrain attributes to facilitate targeted calculation of safety indicators.

[0037] The slope safety score of each region is a quantitative score of the maximum slope value of each segmentation region. The higher the score, the flatter the slope and the higher the construction safety.

[0038] The fault risk level of each region is a numerical value of the fault risk level of the segmentation region evaluated according to the distance from the fault zone. The higher the level, the greater the risk near the fault zone.

[0039] The mechanical access score of each region is a score based on the terrain flatness in the segmentation region, such as the presence of convexities and concavities, to judge the difficulty of smooth mechanical access. The mechanical access score is used to evaluate the construction accessibility.

[0040] Different layers are responsible for different levels of information processing. The terrain analysis layer is responsible for extracting intuitive terrain features from the three-dimensional point cloud model. The region evaluation layer is responsible for converting terrain features into each segmentation region for refined analysis and quantifiable region safety indicators. The safety point determination layer is responsible for selecting initial safe excavation points based on region safety indicators. Through such layered processing, complex terrain safety analysis can be broken down into a modular process of feature extraction, region scoring, and point determination. Each layer can focus on a single task, improving processing efficiency and facilitating targeted optimization of accuracy in each link, ultimately ensuring the reliability and reasonableness of the initial safe excavation points.

[0041] Step S3, based on the multiple initial safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, determine multiple preliminary excavation routes, each of which passes through each initial safe excavation point.

[0042] In some embodiments, a plurality of preliminary excavation routes can be determined based on the plurality of initial safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork using a route planning model, the route planning model being a deep neural network model, an input of the route planning model being the plurality of initial safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, and an output of the route planning model being the plurality of preliminary excavation routes.

[0043] The deep neural network model includes a deep neural network (DNN), which is a machine learning model with multiple layers of neurons. It simulates the working mechanism of brain neurons to learn and process complex data. The deep neural network is composed of an input layer, multiple hidden layers, and an output layer, and the neurons between layers are connected by weights. Data enters the input layer and then undergoes activation operations and feature extraction of neurons in the hidden layer, and finally produces results in the output layer. With the increase of network layers, the deep neural network can automatically learn the deep and abstract feature representation in the data.

[0044] The plurality of preliminary excavation routes are a plurality of possible excavation paths determined by the route planning model, and each preliminary excavation route passes through each initial safe excavation point.

[0045] The deep neural network can extract features from complex spatial data of the three-dimensional point cloud model, thereby identifying key information such as slope and obstacle distribution in the earthwork terrain. The deep neural network can also simulate a variety of route combinations that meet construction requirements by learning the positional relationship of the initial safe excavation points and combining terrain features. In addition, the deep neural network can filter out a plurality of reasonable preliminary excavation routes from a large number of potential paths by incorporating safety, efficiency, and other constraints of engineering construction as optimization objectives, thereby providing a rich candidate path for the formulation of a construction plan.

[0046] In step S4, a simulated excavation video of each preliminary excavation route is generated based on the preliminary excavation route, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork.

[0047] In some embodiments, a simulated excavation video of each preliminary excavation route can be generated based on the preliminary excavation route, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork using a generative adversarial network.

[0048] A generative adversarial network (GAN) is a deep learning architecture composed of two neural networks: a generator and a discriminator. The generator is responsible for generating data, while the discriminator attempts to distinguish between the generated data and real data. Through adversarial training, the generator and discriminator continually optimize, ultimately enabling the GAN to generate highly realistic data that conforms to a specific pattern.

[0049] The simulated excavation video is a virtual video output by the generative adversarial network for simulating the earthwork excavation process along the preliminary excavation route.

[0050] The generative adversarial network has strong feature learning and adversarial optimization capabilities. The generator extracts terrain features from the three-dimensional point cloud model of the unexcavated earthwork, converts the preliminary excavation route into a spatiotemporal trajectory, and then learns the dynamics of mechanical action from the excavation video of the excavated earthwork, thereby encoding this information into a video frame sequence. The discriminator can evaluate the authenticity of the generated results based on the visual features of the real excavation video, and then guide the generator to optimize the details through gradient feedback. Through this collaborative training process, the generator and discriminator can make the final generated simulated excavation video conform to the construction logic and approach the real scene in terms of visual quality and temporal coherence.

[0051] Step S5, determining a target excavation route based on the simulated excavation video of each preliminary excavation route.

[0052] In some embodiments, Figure 2 A flowchart for determining a target excavation route is provided for the embodiments of the present application, which includes steps S21-S26:

[0053] Step S21, constructing a knowledge graph, the knowledge graph including a plurality of preliminary excavation route nodes and a plurality of edges between the preliminary excavation route nodes, the node features of each preliminary excavation route node including the simulated excavation video of the preliminary excavation route, and the edges between the preliminary excavation route nodes being the positional relationships between different routes.

[0054] A knowledge graph is a data structure composed of nodes and edges, with nodes representing various entities and edges representing relationships between entities. In some embodiments, each node can represent each preliminary excavation route, and the edges can be used to describe the positional relationships, sequence, etc. between different preliminary excavation routes. Each node feature also has attribute information, such as the simulated excavation video of the preliminary excavation route. Through this structured representation, the knowledge graph can integrate complex and scattered information into a knowledge network system that is interconnected, thereby clearly presenting the attributes and spatial relationships of each route.

[0055] Step S22, determine a plurality of target safe excavation points based on the knowledge graph processed by the graph neural network.

[0056] The graph neural network (GNN) is a deep learning model that can directly operate on the knowledge graph. It constructs an information transmission mechanism between nodes, so that each node can aggregate the feature information of adjacent nodes and itself. After multiple layers of iterative updates, the graph neural network can learn the feature representation and complex relationship pattern of the entire knowledge graph, thereby realizing tasks such as feature extraction and classification prediction of the knowledge graph. The input of the graph neural network is the knowledge graph, and the output of the graph neural network is a plurality of target safe excavation points.

[0057] The plurality of target safe excavation points are optimized excavation point positions with high safety and strong construction feasibility determined by analyzing the knowledge graph using the graph neural network. The selection of target safe excavation points combines dynamic construction performance and global collaboration, while eliminating conflict points and optimizing point positions with strong connectivity to better meet actual construction needs.

[0058] The knowledge graph provides a rich information base for graph neural network processing. The nodes of the knowledge graph represent preliminary excavation routes, and the nodes constitute the basic unit of analysis. The node features contain simulated excavation videos and intuitively display the dynamic details of the construction process of each route, thereby helping the graph neural network learn the safety elements and potential risks in the excavation operation. The edges represent the positional relationship between the routes, and the edges allow the graph neural network to perceive the spatial correlation between the routes, such as whether there is a risk of cross-operation, whether an efficient connection path can be formed, etc.

[0059] The graph neural network learns the node features and edge relationships and conducts information transmission and aggregation on the knowledge graph, and then evaluates the safety and feasibility of each preliminary excavation route node, thereby screening out target safe excavation points with high safety and convenient construction.

[0060] Step S23, determine a plurality of preferred excavation routes based on the plurality of target safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, each preferred excavation route passing through each target safe excavation point.

[0061] In some embodiments, a plurality of preferred excavation routes can be determined based on the plurality of target safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork using a preferred route planning model, the preferred route planning model being a deep neural network model, the input of the preferred route planning model being the plurality of target safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, and the output of the preferred route planning model being a plurality of preferred excavation routes.

[0062] The plurality of preferred excavation routes are determined by a preferred route planning model based on the plurality of target safe excavation points and the three-dimensional point cloud model of the unexcavated earthwork, and taking into account construction safety, mechanical operation efficiency, construction period cost and other factors. The plurality of preferred excavation routes can provide a variety of reliable candidate schemes for construction planning, which facilitates comparison and evaluation from multiple dimensions such as safety (avoiding easy collapse areas), feasibility (adapting to mechanical operation), and economy (reducing route detours), so as to help determine the most suitable excavation route for actual construction needs.

[0063] The deep neural network can directly map the position information of the target safe excavation point and the terrain data of the three-dimensional point cloud model to the excavation route by virtue of its powerful end-to-end learning ability, and can quickly determine the plurality of preferred excavation routes that take into account safety and construction efficiency by automatically learning the spatial topological relationship and construction constraint conditions through multiple layers of neurons.

[0064] Step S24, based on the plurality of preferred excavation routes, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork, a simulated excavation video of each preferred excavation route is generated.

[0065] In some embodiments, a simulated excavation video of each preferred excavation route can be generated based on the plurality of preferred excavation routes, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork using an excavation deduction model, wherein the excavation deduction model is a generative adversarial network, the input of the excavation deduction model is the plurality of preferred excavation routes, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork, and the output of the excavation deduction model is the simulated excavation video of each preferred excavation route.

[0066] The simulated excavation video of the preferred excavation route is a virtual construction process video simulated by the excavation deduction model, which can intuitively present dynamic details such as earthwork deformation, mechanical action and environmental changes when the machine operates along the preferred route.

[0067] Step S25, based on the simulated excavation video of each preferred excavation route, an excavation processing model is used to determine the excavation information of each preferred excavation route.

[0068] The excavation processing model is a Transformer model, the input of the excavation processing is the simulated excavation video of each preferred excavation route, and the output of the excavation processing is the excavation information of each preferred excavation route.

[0069] The Transformer model is a deep learning model based on a self-attention mechanism, which can capture long-distance dependencies in sequence data through multi-head attention mechanism. The core of the Transformer model consists of an encoder and a decoder. The encoder learns the context representation of the input sequence through self-attention, and the decoder can generate the target sequence based on the encoder output.

[0070] The excavation information of the preferred excavation route is preferably a set of structured construction indicators extracted by analyzing the simulated excavation video of the preferred excavation route through the excavation processing model. The core of the excavation information is to convert the dynamic visual information in the simulated excavation video into quantifiable and comparable key indicators, including terrain adaptation data, time sequence operation data, safety risk indicators, construction efficiency parameters, etc. For example, from a simulated excavation video of a certain preferred excavation route, data such as "the line needs to pass through an area with a slope > 25°, and the soil slip warning value reaches 75%", "the fuel consumption rate of the machinery in this line is 15% higher than the baseline value" are extracted. The excavation information of each preferred excavation route can make the safety and efficiency of different lines quantifiable and sortable, thereby providing a basis for the selection of the target excavation route.

[0071] The terrain adaptation data includes the terrain slope extreme value, obstacle bypass distance, and mechanical operation space margin of the excavation route coverage area.

[0072] The time sequence operation data includes the total length of the single line excavation, the time consumption proportion of process connection, and the idle time proportion of machinery.

[0073] The safety risk indicators include soil slip warning value, mechanical collision warning probability, and environmental compliance score (such as dust and noise level).

[0074] The construction efficiency parameters include unit length of earthwork, equipment fuel consumption rate, and capacity utilization rate (actual output / theoretical capacity).

[0075] The simulated excavation video contains complete spatio-temporal information of the construction process, and the rich visual information contained can be analyzed into structured excavation information. The three-dimensional terrain changes in the video frame reflect the earthwork excavation progress and morphology, the mechanical action trajectory reflects the operation path and efficiency, and the time sequence change records the construction time consumption and process connection. The model analyzes the pixel-level features (such as mechanical position, soil color change) and motion features (such as mechanical arm action speed, material transportation frequency) in the video, thereby quantifying the excavation depth, earthwork volume, and equipment load. The environmental interaction information in the video (such as dust diffusion range, mechanical collision risk) can also be used to evaluate safety risks.

[0076] The Transformer model has a self-attention mechanism and a high-efficiency modeling capability for simulating the spatiotemporal characteristics of a video. The Transformer model can convert a video frame sequence into a computable image block sequence and capture inter-frame temporal dependencies and intra-frame spatial relationships through the self-attention mechanism, without the need for convolution or recursive structures to process long video sequences and thus avoid the problem of gradient disappearance. The encoder of the Transformer model can also support multi-modal input, and the encoder can fuse video visual features and three-dimensional point cloud spatial data to enhance feature complementarity. For simulating the excavation video, the Transformer model can accurately identify key information such as the start and end of a process, a mechanical trajectory, and earth deformation, and can calculate risk and efficiency indicators in different regions in parallel, thereby quickly outputting structured excavation information.

[0077] In step S26, a target excavation route is determined based on the excavation information of each preferred excavation route.

[0078] In some embodiments, the target excavation route can be determined based on the excavation information of each preferred excavation route using a target route decision model, the target route decision model being a support vector machine, the input of the target route decision model being the excavation information of each preferred excavation route, and the output of the target route decision model being the target excavation route.

[0079] A support vector machine (SVM) is a supervised learning model based on statistical learning theory, and its core idea is to realize classification and regression of data by constructing an optimal hyperplane. The support vector machine maps linearly inseparable data in a low-dimensional space to a high-dimensional space through a kernel function, and then finds a linear classification boundary. In some embodiments, the support vector machine can take the excavation information of the preferred excavation route as input, and then learn the decision rules of safety, efficiency, etc. through training, thereby outputting the classification results (such as feasible / infeasible) and ranking scores of the route, to realize the automatic decision from the excavation information of the preferred excavation route to the target excavation route.

[0080] The target excavation route is the final construction execution path selected by the target route decision model based on the excavation information of the preferred excavation route, and is the route with the best effect that meets the requirements of high safety, high construction efficiency, controllable cost, and environmental compliance. The target excavation route has clear three-dimensional spatial coordinates, a time sequence operation plan, and a device configuration scheme.

[0081] The support vector machine is suitable for analyzing a target mining route meeting construction requirements from structured mining information. The support vector machine can efficiently classify and sort high-dimensional mining information (such as safety risk values and construction efficiency parameters) under limited samples through a structured risk minimization theory, then map a low-dimensional index space to a high-dimensional space by using a kernel function, so that a nonlinear relationship between different routes in safety, efficiency, cost and other dimensions can be captured, and an optimal decision boundary can be constructed. For multi-dimensional evaluation data of the preferred mining route, the support vector machine can not only quickly filter out obviously infeasible schemes through binary classification, but also sort routes meeting conditions through multi-classification strategies, and finally output a target route with the best comprehensive performance.

[0082] Based on the same inventive concept, Figure 3 The construction safety hazard analysis system based on image recognition provided by the embodiment of the present application is shown in the figure, which comprises:

[0083] The acquisition module 31 is configured to acquire a mining video of the mined earthwork and a three-dimensional point cloud model of the unmined earthwork.

[0084] The safety point determination module 32 is configured to determine a plurality of initial safe mining points based on the three-dimensional point cloud model of the unmined earthwork.

[0085] The route generation module 33 is configured to determine a plurality of preliminary mining routes based on the plurality of initial safe mining points and the three-dimensional point cloud model of the unmined earthwork, each of which passes through each initial safe mining point.

[0086] The simulation generation module 34 is configured to generate a simulation mining video of each preliminary mining route based on the preliminary mining route, the three-dimensional point cloud model of the unmined earthwork and the mining video of the mined earthwork.

[0087] The target determination module 35 is configured to determine a target mining route based on the simulation mining video of each preliminary mining route.

[0088] Based on the same inventive concept, the embodiment of the present application provides an electronic device, such as Figure 4As shown, it comprises: a processor 41; a memory 42; and a computer program; wherein the computer program is stored in the memory 42 and is configured to be executed by the processor 41 to realize the image recognition-based construction safety hazard analysis method provided in the foregoing, which comprises: acquiring a digging video of excavated earthwork and a three-dimensional point cloud model of unexcavated earthwork; determining a plurality of initial safe digging points based on the three-dimensional point cloud model of unexcavated earthwork; determining a plurality of preliminary digging routes based on the plurality of initial safe digging points and the three-dimensional point cloud model of unexcavated earthwork, each preliminary digging route passing through each initial safe digging point; generating a simulated digging video of each preliminary digging route based on the preliminary digging route, the three-dimensional point cloud model of unexcavated earthwork and the digging video of excavated earthwork; and determining a target digging route based on the simulated digging video of each preliminary digging route.

[0089] Based on the same inventive concept, the embodiment provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor 41, realizes the image recognition-based construction safety hazard analysis method provided in the foregoing, which comprises: acquiring a digging video of excavated earthwork and a three-dimensional point cloud model of unexcavated earthwork; determining a plurality of initial safe digging points based on the three-dimensional point cloud model of unexcavated earthwork; determining a plurality of preliminary digging routes based on the plurality of initial safe digging points and the three-dimensional point cloud model of unexcavated earthwork, each preliminary digging route passing through each initial safe digging point; generating a simulated digging video of each preliminary digging route based on the preliminary digging route, the three-dimensional point cloud model of unexcavated earthwork and the digging video of excavated earthwork; and determining a target digging route based on the simulated digging video of each preliminary digging route.

[0090] The image recognition-based construction safety hazard analysis method provided by the embodiment of the present application can be applied to terminal devices (such as mobile phones), tablet computers, notebook computers, ultra-mobile personal computers (UMPC), handheld computers, netbooks, personal digital assistants (PDA), wearable devices (such as smart watches, smart glasses or smart helmets, etc.), augmented reality (AR) \ virtual reality (VR) devices, smart home devices, vehicle-mounted computers and other electronic devices, and the present application does not make any limitation in this regard.

[0091] Having now described the basic concept of the application, and having demonstrated by the detailed description and examples only the preferred embodiments thereof, it will be apparent to those of ordinary skill in the art that various modifications and improvements and equivalents can be made thereto without departing from the spirit and scope of the application. Accordingly, the application is not to be limited as by that which has been particularly shown and described.

[0092] Also, the use of "an" or "one" to describe the present application shall not be construed to mean there is only one of the features or one article. Rather, such phrases shall indicate that there is at least one of the features or one article present. As such, "an object" or "a member" of the application shall be construed to mean that there is at least one of those features or one article present.

[0093] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. One having ordinary skill in the art will understand that information and signals can be represented using any of a variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0094] It should be noted that while the above detailed description has been described in the context of a fully functional application, the application also can be implemented as a computer program product. The computer program product can include a computer readable storage medium having instructions stored on it. The computer readable storage medium can be a hard disk drive, floppy diskette, optical disk, memory, or other storage device. The computer readable storage medium can be a hard disk drive, floppy diskette, optical disk, memory, or other storage device.

[0095] Finally, the described embodiments are to be considered in all respects as illustrative and not restrictive. Other embodiments will readily occur to those skilled in the art. Accordingly, the application is not to be limited as by that which has been particularly shown and described.

Claims

1. A construction safety hazard analysis method based on image recognition, characterized by, The method comprises the following steps: obtaining a digging video of excavated earthwork and a three-dimensional point cloud model of unexcavated earthwork; determining a plurality of initial safe digging points based on the three-dimensional point cloud model of unexcavated earthwork, which comprises determining a plurality of initial safe digging points based on the three-dimensional point cloud model of unexcavated earthwork using a safe point determination model, wherein the safe point determination model is a convolutional neural network model, and the safe point determination model comprises a terrain analysis layer, a region evaluation layer and a safe point determination layer, all of which comprise a convolutional neural network structure, the input of the terrain analysis layer is the three-dimensional point cloud model of unexcavated earthwork, the output of the terrain analysis layer is a terrain slope map, a fault position mark and a surface concave-convex region distribution, the input of the region evaluation layer is the terrain slope map, the fault position mark and the surface concave-convex region distribution, the output of the region evaluation layer is a plurality of segmented regions, a slope safety score of each region, a fault risk level of each region and a mechanical access score of each region, the input of the safe point determination layer is the plurality of segmented regions, the slope safety score of each region, the fault risk level of each region and the mechanical access score of each region, and the output of the safe point determination layer is the plurality of initial safe digging points; determining a plurality of preliminary digging routes based on the plurality of initial safe digging points and the three-dimensional point cloud model of unexcavated earthwork, each preliminary digging route passing through each initial safe digging point; generating a simulated digging video of each preliminary digging route based on the preliminary digging route, the three-dimensional point cloud model of unexcavated earthwork and the digging video of excavated earthwork; determining a target digging route based on the simulated digging video of each preliminary digging route, which comprises: constructing a knowledge graph, wherein the knowledge graph comprises a plurality of preliminary digging route nodes and a plurality of edges between the preliminary digging route nodes, and the node features of each preliminary digging route node comprise the simulated digging video of the preliminary digging route, and the edges between the preliminary digging route nodes are position relationships between different routes; processing the knowledge graph based on a graph neural network to determine a plurality of target safe digging points; determining a plurality of preferred digging routes based on the plurality of target safe digging points and the three-dimensional point cloud model of unexcavated earthwork, each preferred digging route passing through each target safe digging point; generating a simulated digging video of each preferred digging route based on the plurality of preferred digging routes, the three-dimensional point cloud model of unexcavated earthwork and the digging video of excavated earthwork; determining digging information of each preferred digging route based on the simulated digging video of each preferred digging route using a digging processing model; determining a target digging route based on the digging information of each preferred digging route.

2. The image recognition-based construction safety hazard analysis method of claim 1, wherein, The method for generating a simulated digging video of each preliminary digging route based on the preliminary digging route, the three-dimensional point cloud model of unexcavated earthwork and the digging video of excavated earthwork comprises: The simulation excavation video of each preliminary excavation line is generated based on the preliminary excavation line, the three-dimensional point cloud model of the unexcavated earthwork and the excavation video of the excavated earthwork using a generative adversarial network. 3.The image recognition-based construction safety hazard analysis method of claim 1, wherein, The excavation processing model is a Transformer model.

4. The construction safety hazard analysis system based on image recognition, characterized by, The method comprises the following steps: an acquisition module configured to acquire an excavation video of an excavated earthwork and a three-dimensional point cloud model of an unexcavated earthwork; a safety point determination module configured to determine a plurality of initial safety excavation points based on the three-dimensional point cloud model of the unexcavated earthwork, wherein the determination of the plurality of initial safety excavation points based on the three-dimensional point cloud model of the unexcavated earthwork comprises determining the plurality of initial safety excavation points based on the three-dimensional point cloud model of the unexcavated earthwork using a safety point determination model, wherein the safety point determination model is a convolutional neural network model, and the safety point determination model comprises a terrain analysis layer, a region evaluation layer and a safety point determination layer, all of which comprise a convolutional neural network structure, the input of the terrain analysis layer is the three-dimensional point cloud model of the unexcavated earthwork, the output of the terrain analysis layer is a terrain slope graph, a fault position marker and a surface concave-convex region distribution, the input of the region evaluation layer is the terrain slope graph, the fault position marker and the surface concave-convex region distribution, the output of the region evaluation layer is a plurality of segmented regions, a slope safety score of each region, a fault risk level of each region and a mechanical passage score of each region, the input of the safety point determination layer is the plurality of segmented regions, the slope safety score of each region, the fault risk level of each region and the mechanical passage score of each region, and the output of the safety point determination layer is the plurality of initial safety excavation points; a line generation module configured to determine a plurality of preliminary excavation lines based on the plurality of initial safety excavation points and the three-dimensional point cloud model of the unexcavated earthwork, wherein each preliminary excavation line passes through each initial safety excavation point; a simulation generation module configured to generate a simulation excavation video of each preliminary excavation line based on the preliminary excavation line, the three-dimensional point cloud model of the unexcavated earthwork and the excavation video of the excavated earthwork; a target determination module configured to determine a target excavation line based on the simulation excavation video of each preliminary excavation line, and further configured to: construct a knowledge graph, wherein the knowledge graph comprises a plurality of preliminary excavation line nodes and a plurality of edges between the preliminary excavation line nodes, and the node features of each preliminary excavation line node comprise the simulation excavation video of the preliminary excavation line, and the edges between the preliminary excavation line nodes are position relationships between different lines; determine a plurality of target safety excavation points by processing the knowledge graph based on a graph neural network; determine a plurality of preferred excavation lines based on the plurality of target safety excavation points and the three-dimensional point cloud model of the unexcavated earthwork, wherein each preferred excavation line passes through each target safety excavation point; generate a simulation excavation video of each preferred excavation line based on the plurality of preferred excavation lines, the three-dimensional point cloud model of the unexcavated earthwork and the excavation video of the excavated earthwork; determine excavation information of each preferred excavation line based on the simulation excavation video of each preferred excavation line using an excavation processing model; and determine the excavation information of each preferred excavation line based on the simulation excavation video of each preferred excavation line using an excavation processing model. determine a target excavation route based on the excavation information of each preferred excavation route.

5. The image recognition-based construction safety hazard analysis system of claim 4, wherein, The simulation generation module is specifically configured to: generate a simulation excavation video of each preliminary excavation route based on the preliminary excavation route, the three-dimensional point cloud model of the unexcavated earthwork, and the excavation video of the excavated earthwork using a generative adversarial network. 6.The image recognition-based construction safety hazard analysis system of claim 4, wherein, The excavation processing model is a Transformer model.

7. An electronic device, comprising: comprise: a processor; a memory; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the construction safety hazard analysis method based on image recognition according to any one of claims 1 to 3.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the construction safety hazard analysis method based on image recognition according to any one of claims 1 to 3.

Images