Project potential safety hazard prediction and early warning method based on image recognition

Abstract

The invention relates to an engineering potential safety hazard prediction technology, is used for solving the problems that data acquisition equipment is large in installation and maintenance difficulty, high in cost, easy to damage, large in manual monitoring workload and large in subjective judgment error, and particularly relates to an engineering potential safety hazard prediction and early warning method based on image recognition. According to the method, the risk prediction model based on the graph network is constructed, comprehensive analysis is carried out in combination with the time sequence features and the spatial features, the risk prediction accuracy is improved, the risk prediction model can more accurately predict the possibility and severity of potential safety hazards through multiple technical means and a large amount of data training, and the risk prediction accuracy is improved. The model learns the feature change rule of the image under different weather conditions, and the influence of severe weather on the image quality is reduced; valuable information can be mined from existing data by the model, dependence on continuous and stable operation of equipment is reduced, and the risk that potential safety hazards cannot be accurately monitored due to equipment problems is reduced.

Description

Technical Field

[0001] The invention relates to a technology for predicting engineering safety hazards, and in particular to an engineering safety hazard prediction and early warning method based on image recognition. Background Art

[0002] Reservoirs play a huge role in flood prevention and disaster reduction, food security and ecological maintenance by smoothing flood processes, delaying peak times and providing water for irrigation.

[0003] In the prior art, reservoir sites are usually located in remote areas with harsh climatic conditions. Bad weather can affect the normal operation of camera equipment, resulting in reduced image acquisition quality. The installation and maintenance of data acquisition equipment in remote areas is difficult, costly, and the equipment is easily damaged. It is difficult to detect safety hazards in the early stages of monitoring with camera equipment alone. Traditional manual monitoring methods have errors caused by large workloads and subjective judgments, resulting in untimely and incomplete monitoring and inaccurate judgment of hazards.

[0004] In view of the above technical problems, this application proposes a solution. Summary of the invention

[0005] The purpose of the present invention is to improve the accuracy of risk prediction by constructing a risk prediction model based on a graph network and conducting a comprehensive analysis in combination with time series features and spatial features. The risk prediction model can more accurately predict the possibility and severity of safety hazards through a variety of technical means and a large amount of data training. The model learns the characteristic change patterns of images under different weather conditions to reduce the impact of bad weather on image quality. The model can mine valuable information from existing data, reduce dependence on continuous and stable operation of equipment, reduce the risk of inability to accurately monitor safety hazards due to equipment problems, and solve the problems of difficult and high cost installation and maintenance of data acquisition equipment, easy damage of equipment, large workload of manual monitoring, and large subjective judgment errors. An image recognition engineering safety hazard prediction and early warning method is proposed.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A method for predicting and warning engineering safety hazards based on image recognition, the method comprising the following steps:

[0008] Step 1: Collect reservoir project site data through high-definition cameras in different scenarios. The collected image data includes views from three angles: front, side, and back, as well as reservoir conditions under different lighting conditions and backgrounds, to build a reservoir project site database;

[0009] Step 2: Data preprocessing: Mark out the engineering safety hazards in each image, and perform preprocessing and feature extraction; perform preprocessing operations on the image data and convert the preprocessed images into a standard format that can be processed by the deep learning algorithm;

[0010] Step 3: Use the time series feature extraction network to extract the time series features of the initial image of the reservoir project safety hazard; use the convolutional neural network to extract the spatial features of the reservoir project safety hazard image data within the range; build a risk prediction model for reservoir project safety hazards based on the graph network; use the constructed risk prediction model to predict the reservoir project risk and simulate the safety hazards under different scenarios; determine the safety hazard area based on the prediction results and issue an alarm.

[0011] As a preferred embodiment of the present invention, an engineering safety component is provided inside the high-definition camera, and the engineering safety component includes an image recognition module, a hidden danger prediction module and an early warning module. The data processing steps of the image recognition module are as follows:

[0012] Step 1: The color image data collected on the dam surface, spillway and gate positions are converted into grayscale images through the formula GRAY(x, y) = {0.299 × R(i, j), 0.587 × G(i, j), 0.114 × B(i, j)}, and then the grayscale image after grayscale processing is processed through the image normalization processing formula to scale the image pixel value to the range of [0, 1] or [-1, 1], and then adjust the image to a uniform size. R(i, j), G(i, j) and B(i, j) represent the pixel values ​​of the three color channels R, G and B at the i-th row and j-th column position of the image respectively;

[0013] Step 2: Remove the noise from the grayscale image after the uniform size through three steps of discrete Fourier transform, filter processing and Fourier inverse transform, and then complete the image standardization operation by adjusting the image pixel values ​​to a distribution with a mean of 0 and a standard deviation of 1;

[0014] Step 3: Sort the processed grayscale images by acquisition time. The time interval of the acquisition time is the same. The image acquired at time t is TX t , the image collected at time t-1 is TX t-1 , get the grayscale difference image CF;

[0015] Step 4: Sort the grayscale difference images by time and calculate the average pixel value change rate of a certain area in several consecutive grayscale difference images; if for the mth difference image CF t-m The area R contains n pixels, and its pixel value is CF t-m (x, y), then the average pixel value of region R is A t-m(R), the pixel change rate of area R in the image is r t (R);

[0016] Step 5: If the pixel change threshold r of region R t (R) max <r t (R), it is determined that there are no safety hazards in area R.

[0017] As a preferred embodiment of the present invention, the hidden danger prediction module processes the data of dam cracks in the following steps:

[0018] Step 1: Have personnel mark the engineering safety hazards in each picture, and record the type and location information of the hazards pointed out by the personnel; grayscale the collected image data, and use the grayscale threshold HD max Distinguish the image data after grayscale processing, and mark the area with grayscale value less than the grayscale threshold as a potential risk area;

[0019] Step 2: Divide the image data into n blocks of the same size, count the number of blocks with hidden danger areas, the number is m, and then calculate the percentage data ZB of the hidden danger areas marked on each image x =m / n, x is the number of each picture; the total proportion data ZB in the picture data obtained for the dam surface, spillway and gate position ZH1 , ZB ZH2 , ZB ZH3 Calculate separately, if there is a preset percentage data threshold ZB max <ZB ZHa , a=1,2,3, then it is determined that the dam has safety hazards; otherwise, the total proportion data corresponding to the dam surface, spillway and gate are substituted into the formula, and the hazard value YH=ω1*ZB ZH1 +ω2*ZB ZH2 +ω3*ZB ZH3 , ω1, ω2 and ω3 are specific gravity coefficients, if YH max ≤YH, the dam is still considered to have safety hazards, and the dam is considered to have safety hazards according to YH-YH max / YH max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively.

[0020] As a preferred embodiment of the present invention, the steps of data processing of dam slope collapse and landslide by the hidden danger prediction module are as follows:

[0021] Step 1: Divide the grayscale image into several small image blocks, calculate the change of the grayscale value of the pixel in the corresponding small image block to approximate the slope value of the dam slope corresponding to the small image block, and construct a surface according to the grayscale value f(x, y) along the horizontal and vertical pixel coordinates;

[0022] Step 2: Calculate the slope value by using the difference method, and get the horizontal slope value as PD x , the vertical slope value PD y .

[0023] As a preferred embodiment of the present invention, the hidden danger prediction module determines the dam slope collapse and landslide in the following steps:

[0024] Step 1: The image block corresponds to the slope value of the dam If the slope threshold PD max ≤PD, the dam slope is judged to have the risk of collapse; otherwise, the change of slope value over time is judged. If the change of slope value ΔPD≥ΔPD during the time period from time t to time t+q max , and the dam slope is also judged to have the risk of collapse, ΔPD max is the preset slope value change threshold, t is the starting time point, and q is the extended time period for judging the change;

[0025] Step 2: Predict the elapsed time T = PD max -PD t+q / ΔPD after the dam slope will collapse, with a unit time T t As standard, if T / T t <1, it is determined that the collapse of the dam slope is about to occur, an emergency evacuation signal is generated, and the emergency evacuation signal is transmitted to the early warning module; if 1≤T / T t <2, the dam slope is judged to be at risk of collapse, a risk avoidance signal is generated, and the risk avoidance signal is transmitted to the early warning module; if T / T t ≥2, the dam slope is judged to have the potential safety hazard of collapse, and a maintenance warning signal is generated and transmitted to the early warning module.

[0026] As a preferred embodiment of the present invention, the steps of data processing of leakage damage by the hidden danger prediction module are as follows:

[0027] Step 1: Construct the gray-level co-occurrence matrix with parameter 1 pixel as distance and parameters 0°, 45°, 90° and 135° as directions;

[0028] Step 2: If the grayscale level of the grayscale image f(x, y) is N, when the distance is d and the direction is θ, the probability that a pixel with a grayscale value of i and a pixel with a grayscale value of j appear at the same time is P(d, θ)(i, j).

[0029] As a preferred embodiment of the present invention, the steps of determining leakage damage by the hidden danger prediction module are as follows:

[0030] Step 1: Mark the contrast data as DB. When there is leakage in the dam, the materials around the leakage area will be eroded, discoloration and water stains will appear, resulting in increased contrast. If the preset contrast threshold DB max ≤DB, it is determined that there is leakage damage at the dam detection location, and according to DB-DB max / DB max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively;

[0031] Step 2: The energy data is E. When there is leakage damage in the dam, the leakage area will cause uneven texture and reduce the energy E. If the preset energy threshold E min ≥E, it is determined that there is leakage damage at the dam detection location, and according to E min -E / E min The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively;

[0032] Step 3: If E>E min And DB <DB max , then the leakage damage risk of the dam is determined, and the contrast DB and energy E are normalized to obtain DB GY and E GY ; After analyzing a large number of normal and leaking dam images, the weight of contrast in leakage judgment is obtained as ω DB , the energy weight is ω E , then the risk of leakage and damage to the dam FX = ω DB *DB GY +ω E *E GY , if the preset risk threshold FX max ≤FX, the dam is judged to have the risk of leakage and damage, and the FX-FX max / FX max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively.

[0033] As a preferred embodiment of the present invention, the warning processing steps of the early warning module are as follows:

[0034] Step 1: The warning signals transmitted by the hidden danger prediction module are distinguished according to their levels. When the signal is transmitted, it is counted and recorded as level 1 YJ, level 2 EJ and level 3 SJ; if the transmitted warning signal is one of the three risks of dam cracks, dam slope collapse and collapse or leakage damage, the number of signals of the corresponding level is accumulated. The level signal satisfies the rule of 3YJ=EJ, 3EJ=SJ. After conversion according to this rule, the first-level warning signal count is counted and recorded as YJ Z , if 3*YJ max ≤YJ Z , then the third-level warning signal is used for warning; if 2*YJ max ≤YJ Z <3*YJ max , then the secondary warning signal will be used for warning; if YJ Z <2*YJ max , then a warning will be issued according to the first-level warning signal; YJ max is a preset single counting threshold;

[0035] Step 2: If the transmitted warning signals are multiple of the three risks of dam cracks, dam slope collapse, and collapse or leakage damage, they are classified according to the risk type. The total number of first-level warning signals for dam crack risk is LF Z The total number of first-level warning signals for dam slope collapse and landslide risk is BT Z The total number of first-level warning signals for leakage damage risk is SL Z , then according to 6*BT Z +3*SL Z +LF Z The calculated value of / 10 is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively;

[0036] Step 3: When the first-level warning signal is triggered, a warning signal of "there are safety hazards in the dam" is sent to the dam staff via email; when the second-level warning signal is triggered, the sound and light alarm emits a sound with a duration of 1s and a frequency of 500Hz; when the third-level warning signal is triggered, the sound and light alarm emits light and sound with a duration of 3s and a frequency of 1000Hz.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The accuracy of risk prediction is improved by constructing a risk prediction model based on graph networks and combining temporal and spatial features for comprehensive analysis. The risk prediction model can more accurately predict the possibility and severity of potential safety hazards through a variety of technical means and a large amount of data training;

[0039] 2. Use risk prediction models to simulate safety hazards in different scenarios and predict results. The model is trained on a large amount of image data of different quality to learn the characteristic change patterns of images under different weather conditions, reduce the impact of bad weather on image quality, and improve the ability to identify safety hazards. The model can mine valuable information from existing data, reduce dependence on the continuous and stable operation of equipment, and reduce the risk of inability to accurately monitor safety hazards due to equipment problems.

[0040] 3. By collecting data in different scenarios, various situations where potential safety hazards may occur are covered, thus improving the comprehensiveness of monitoring; in the process of image recognition, graying, normalization, denoising and standardization are used to improve the quality and processability of data; different levels of warning signals are issued according to the different degrees of safety hazards. The graded warning enables staff to promptly understand the severity of safety hazards and take corresponding measures in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0042] Figure 1 It is a system flow chart of the present invention;

[0043] Figure 2 A network structure diagram for extracting the timing characteristics of the present invention;

[0044] Figure 3 A spatial feature structure diagram of image data within the range of the convolutional neural network extraction of the present invention;

[0045] Figure 4 This is a structural diagram of the prediction model structure based on graph network of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Example:

[0048] See also Figure 1-Figure 4 As shown, an image recognition engineering safety hazard prediction and early warning method includes a high-definition camera, an engineering safety component is arranged inside the high-definition camera, and the engineering safety component includes an image recognition module, a hazard prediction module and an early warning module;

[0049] The reservoir engineering site data includes image data of various types of engineering hidden dangers such as dam cracks, dam slope collapse, leakage damage, and collapse; the collected image data are preprocessed, including grayscale processing, image normalization processing, and image scaling processing;

[0050] The image recognition module performs grayscale processing on the engineering hidden danger image data, and then processes the grayscale value after grayscale processing by weighted average method. The grayscale value is the weighted average result of the pixel values ​​of the three channels R, G, and B; GRAY(x, y) = {0.299×R(i, j), 0.587×G(i, j), 0.114×B(i, j)}, and all weighted parameters are made 1 / 3 by the weighted average method to obtain a more reasonable grayscale image. R(i, j), G(i, j) and B(i, j) respectively represent the pixel values ​​of the three color channels R, G, and B at the i-th row and j-th column position of the image;

[0051] The image pixel values ​​after grayscale processing are processed through the image normalization processing formula, and the image pixel values ​​are scaled to the range of [0,1] or [-1,1] to improve the stability and speed of training; the pixel value range is 0-255, with 256 levels, and the normalized pixel value XS gy =XS0 / 255, XS0 is the original pixel value;

[0052] After normalization, the processed image data is scaled to adjust the image to a uniform size for batch processing and training. The coordinates (x0, y0) in the new image after image scaling = (KD0*x / KD, GD0*y / GD), where (x, y) is the coordinate in the original image, KD and GD are the width and height data of the original image, and KD0 and GD0 are the width and height data of the new image, respectively.

[0053] Remove noise from the image through frequency domain filtering to improve image quality; the filtering process of frequency domain filtering:

[0054]

[0055] Where, f(x, y) is the original image, DFT is discrete Fourier transform, F(x, y) is the frequency domain function obtained after Fourier transform of f(x, y), H(u, v) is the filter, G(u, v) is the function obtained after low-frequency filtering, IDFT is inverse Fourier transform, and g(x, y) is the image output after inverse Fourier transform;

[0056] After denoising, the image is standardized to adjust the image pixel values ​​to a distribution with a mean of 0 and a standard deviation of 1 to eliminate the effects of brightness and contrast. is the mean value of pixel values, σ is the standard deviation of pixel values;

[0057] The temporal feature extraction network consists of an embedding layer, three graph convolution layers, and a pooling layer; there are four temporal feature extraction networks, which are respectively used to extract the temporal features of people or objects in the image, extract the temporal features of the movement speed of people or objects in the image, extract the temporal features of the impact area of ​​the movement of people or objects in the image, and extract the movement trajectory features of people or objects in the image; the temporal feature process of the initial image is: reconstruct the four temporal sequences into topological graphs, and then send them to four temporal feature extraction networks respectively, and the embedding layer converts each node into a 1×256 feature vector; then update the feature vector of each node through three graph convolution layers. For the temporal topological graph, each graph convolution layer updates the node feature vector as follows:

[0058] F gi =L3{ReLU[0.6*L1(f i )+0.4*L2(f i-1 )]}

[0059] L1, L2, and L3 are three fully connected layers of size 256×256; finally, a global maximum pooling layer is used to obtain the global feature vector; through the feature extraction network, each initial sequence is converted into a feature vector of size 1×256. For a construction site event, there are a total of four time series feature vectors (f1, f2, f3, f4). The process of reconstructing the four time series into a topological graph is as follows: the construction process status value of a person or object is used as a node, and the nodes are connected one-way in sequence according to the time sequence, with the later node pointing to the newer node, and f i and f i-1 They are the feature vectors of the i-th node and the i-1-th node before the current graph convolution layer is updated;

[0060] The convolutional neural network is used to extract the spatial features of the engineering safety hazard image data within the range, and the deep learning target recognition algorithm is used to train the target recognition model based on the target sample library constructed in the previous stage; in view of the complex and changeable background of the engineering safety hazard target, and the uncertain factors such as the size and posture of the target to be detected, considering the requirements for the accuracy and time of the model algorithm, a single-stage target detection algorithm detection and recognition model can obtain better detection speed and accuracy; the whole picture is input into the network, and the convolutional neural network (CNN) is applied to the whole image, the image is divided into a series of grids and the class probability and bounding box corresponding to each grid are predicted, and finally the category information of the target to be detected is directly output and position information, achieving end-to-end processing; first, the entire input image is normalized and adjusted to a fixed size, and then the fixed size is evenly divided into S×S small cells. If a small cell contains the center of an object, the cell is responsible for predicting the target; each cell is responsible for predicting the position of the target bounding box and the category information to which the target belongs; the category confidence score is obtained by the probability and confidence of the category to which the target belongs, and then the confidence score is compared with the set threshold, the target box greater than the threshold is retained, and the target box less than the threshold is discarded, and all the retained prediction boxes are subjected to non-maximum suppression processing to ensure that each target category retains a prediction box, and finally the position and category information of the target prediction bounding box are output;

[0061] The engineering safety hazard image prediction model based on graph network is composed of an encoder and a decoder; the encoder part is composed of two adaptive graph convolution layers for updating the construction site data features and environmental data features; the decoder part is composed of four double-layer fully connected networks, which respectively decode and predict the four types of construction data; the adaptive graph convolution layer is characterized by the introduction of a learnable graph adjacency matrix JZ lj , the two adaptive graph convolutional layers share the same adjacency matrix JZ lj ; Combine the four eigenvectors obtained into the feature matrix F n =(f1, f2, f3, f4) T , updated by the encoder module to;

[0062] F m =JZ lj ×ReLU(JZ lj ×F n ×G1)×G2

[0063] G1 and G2 are two learnable parameter matrices of different sizes, which transform the updated construction image data features (f g1 , f g2 , f g3 , f g4 ) is sent to four corresponding decoders to obtain a single-step prediction result of the construction image data;

[0064] The method of training and testing the overall model is: taking engineering safety hazard events as the main body, 80% of the construction events in the database are divided into training sets, and the remaining 20% ​​are divided into test sets. The initial engineering safety hazard events and surrounding environment data are input into the neural network, and RMSE is used as the loss function for training. The RMSE calculation formula is as follows;

[0065]

[0066] N is the number of samples, y i is the true value of the i-th sample, yc i is the predicted value of the i-th sample; the training is divided into two stages: single-step prediction and multi-step prediction. First, the whole model parameters are optimized by single-step prediction, and then the model parameters are fine-tuned by multi-step prediction. In single-step prediction, the temporal feature extraction network extracts the initial image sequence {x1, x2, …, x t} extracts the temporal feature f = F(x) and updates it to f through the encoder g =E(f), the temporal feature f extracted from the initial image sequence by the temporal feature extraction network is input into the encoder E for processing to obtain the updated feature f g , and then get the prediction result yc for the next moment through the decoder t+1 =D(f g ), the feature f updated by the encoder g Input to the decoder D, the decoder D according to the input feature f g Perform decoding prediction and finally get the prediction result yc for the next moment t+1 ; In multi-step prediction, it is only necessary to extract the time series features once, and then iterate continuously to make predictions for l steps {yc t+1 ,yc t+2 ,…,yc t+l}; The overall loss function is;

[0067]

[0068] Loss = γ1*L1+γ2*L2+γ3*L3

[0069] Among them, yc t+1 and t+1 are the predicted value and true value of the sample at time t+1, L1 and L2 are the losses in the single-step prediction stage, y is the true value of the sample at time t, D[F(x)] is the prediction result of the next moment obtained by the decoder through the corresponding extracted temporal features, L1 is the single-step prediction loss, L2 is the temporal feature extraction network loss, L3 is the multi-step prediction stage loss, γ1=0.4, γ2=0.4, γ3=0.2;

[0070] The hidden danger prediction module processes and analyzes three situations: dam cracks, dam slope collapse and collapse, and leakage damage;

[0071] The collected image data is grayed out and the gray threshold HD max Distinguish the image data after grayscale processing, and mark the area with grayscale value less than the grayscale threshold as the potential hazard area; divide the image data into n blocks of the same size, count the number of blocks marked with potential hazard areas, the number is m, and then calculate the proportion data ZB of the potential hazard areas marked on each image x =m / n, x is a positive integer; the total proportion data ZB in the image data obtained at the dam surface, spillway and gate position ZH1 , ZB ZH2 , ZB ZH3 Calculate separately, if there is a preset percentage data threshold ZB max <ZB ZHa , a=1,2,3, then it is determined that the dam has safety hazards; otherwise, the total proportion data corresponding to the dam surface, spillway and gate are substituted into the formula, and the hazard value YH=ω1*ZB ZH1 +ω2*ZB ZH2 +ω3*ZB ZH3 , ω1, ω2 and ω3 are specific gravity coefficients, if YH max ≤YH, the dam is still considered to have safety hazards, and the dam is considered to have safety hazards according to YH-YH max / YH max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively;

[0072] The grayscale image is divided into several small image blocks. The slope value of the dam slope corresponding to the image block is approximated by calculating the change of the pixel grayscale value in the corresponding image block. A surface is constructed according to the grayscale value f(x, y) along the horizontal and vertical pixel coordinates, and the slope value is calculated by a simple difference method. The slope value PD in the horizontal direction x =f(x+Δx, y)-f(x, y) / Δx, the vertical slope value PD y =f(x, y+Δy)-f(x, y) / Δy, Δx and Δy are the pixel spacings in the horizontal and vertical directions of adjacent image blocks respectively; the image block corresponds to the slope value of the dam If the slope threshold PD max ≤PD, the dam slope is judged to have the risk of collapse; otherwise, the change of slope value over time is judged. If the change of slope value ΔPD≥ΔPD during the time period from time t to time t+q max, and the dam slope is also judged to have the risk of collapse, ΔPD max is the preset slope value change threshold, t is the starting time point, q is the extended time period for judging the change; the predicted elapsed time T = PD max -PD t+q / ΔPD after the dam slope will collapse, with a unit time T t As standard, if T / T t <1, it is determined that the collapse of the dam slope is about to occur, an emergency evacuation signal is generated, and the emergency evacuation signal is transmitted to the early warning module; if 1≤T / T t <2, the dam slope is judged to be at risk of collapse, a risk avoidance signal is generated, and the risk avoidance signal is transmitted to the early warning module; if T / T t ≥2, the dam slope is judged to have the potential safety hazard of collapse, and a maintenance warning signal is generated and transmitted to the early warning module;

[0073] When analyzing the risk of leakage damage, the pixel relationship with a smaller distance can capture the subtle texture changes on the dam surface; the grayscale co-occurrence matrix is ​​constructed with the parameter 1 pixel as the distance and the parameters 0°, 45°, 90° and 135° as the direction; if the grayscale level of the grayscale image f(x, y) is N, when the distance is d and the direction is θ, traverse each pixel (x, y) in the image and find the adjacent pixels (x) at the distance d and direction θ. # ,y # ), the statistical gray value I(x, y) = i and the gray value I(x # ,y # )=j The number of times the pixel pair appears is CS ij , the probability that a pixel with gray value i and a pixel with gray value j appear at the same time Contrast P(i, j) is the probability that a pixel with gray value i and a pixel with gray value j appear at the same time in the four directions. When the dam is leaking, the material around the leaking area will be eroded, discolored, and water-stained, which will intensify the local texture change and increase the contrast. If the preset contrast threshold DB max ≤DB, it is determined that there is leakage damage at the dam detection location, and according to DB-DB max / DB max The ratio of is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively; energy When there is leakage damage in the dam, the leakage area will cause uneven texture and the energy E will decrease. If the preset energy threshold E min≥E, it is determined that there is leakage damage at the dam detection location, and according to E min -E / E min The ratio of is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively; if E>E min And DB <DB max , then the leakage damage risk of the dam is determined, and the contrast DB and energy E are normalized to obtain DB GY and E GY ; After analyzing a large number of normal and leaking dam images, the weight of contrast in leakage judgment is obtained as ω DB , the energy weight is ω E , then the risk of leakage and damage to the dam FX = ω DB *DB GY +ω E *E GY , if the preset risk threshold FX max ≤FX, the dam is judged to have the risk of leakage and damage, and the FX-FX max / FX max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively;

[0074] The warning signals transmitted by the hidden danger prediction module are differentiated according to their levels. When the signal is transmitted, it is counted and recorded as level 1 YJ, level 2 EJ and level 3 SJ. If the transmitted warning signal is one of the three risks of dam cracks, dam slope collapse and collapse or leakage damage, the number of signals of the corresponding level is accumulated. The level signal satisfies the rule of 3YJ=EJ, 3EJ=SJ. After conversion according to this rule, the first-level warning signal count is counted and recorded as YJ. Z , if 3*YJ max ≤YJ Z , then the third-level warning signal is used for warning; if 2*YJ max ≤YJ Z <3*YJ max , then the secondary warning signal will be used for warning; if YJ Z <2*YJ max , then a warning will be issued according to the first-level warning signal; YJ max is a preset single counting threshold; if the transmitted warning signal is more than one of the three risks of dam crack, dam slope collapse and collapse or leakage damage, it is distinguished according to the risk type, and the total count of the first-level warning signal of dam crack risk is LF Z The total number of first-level warning signals for dam slope collapse and landslide risk is BT Z The total number of first-level warning signals for leakage damage risk is SLZ , then according to 6*BT Z +3*SL Z +LF Z The calculated value of / 10 is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively; when the first level warning signal is triggered, a warning signal of "there are safety hazards in the dam" is sent to the dam staff by email; when the second level warning signal is triggered, the sound and light alarm sounds, the sound duration is 1s, and the frequency is 500Hz; when the third level warning signal is triggered, the sound and light alarm emits light and sound, the sound duration is 3s, and the frequency is 1000Hz.

[0075] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for predicting and warning engineering safety hazards based on image recognition, characterized in that: The project safety hazard prediction and early warning method includes the following steps: Step 1: Collect reservoir project site data through high-definition cameras in different scenarios. The collected image data includes views from three angles: front, side, and back, as well as reservoir conditions under different lighting conditions and backgrounds, to build a reservoir project site database; Step 2: Data preprocessing: Mark out the engineering safety hazards in each image, and perform preprocessing and feature extraction; perform preprocessing operations on the image data and convert the preprocessed images into a standard format that can be processed by the deep learning algorithm; Step 3: Use the time series feature extraction network to extract the time series features of the initial image of the reservoir project safety hazard; use the convolutional neural network to extract the spatial features of the reservoir project safety hazard image data within the range; build a risk prediction model for reservoir project safety hazards based on the graph network; use the constructed risk prediction model to predict the reservoir project risk and simulate the safety hazards under different scenarios; determine the safety hazard area based on the prediction results and issue an alarm.

2. The method for predicting and warning engineering safety hazards based on image recognition according to claim 1, characterized in that: The HD camera is equipped with an engineering safety component, which includes an image recognition module, a hidden danger prediction module, and an early warning module. The data processing steps of the image recognition module are as follows: Step 1: The color image data collected on the dam surface, spillway and gate positions are converted into grayscale images through the formula GRAY(x, y) = {0.299 × R(i, j), 0.587 × G(i, j), 0.114 × B(i, j)}, and then the grayscale image after grayscale processing is processed through the image normalization processing formula to scale the image pixel value to the range of [0, 1] or [-1, 1], and then adjust the image to a uniform size. R(i, j), G(i, j) and B(i, j) represent the pixel values ​​of the three color channels R, G and B at the i-th row and j-th column position of the image respectively; Step 2: Remove the noise from the grayscale image after the uniform size through three steps of discrete Fourier transform, filter processing and Fourier inverse transform, and then complete the image standardization operation by adjusting the image pixel values ​​to a distribution with a mean of 0 and a standard deviation of 1; Step 3: Sort the processed grayscale images by acquisition time. The time interval of the acquisition time is the same. The image acquired at time t is TX t , the image collected at time t-1 is TX t-1 , get the grayscale difference image CF; Step 4: Sort the grayscale difference images by time and calculate the average pixel value change rate of a certain area in several consecutive grayscale difference images; if for the mth difference image CF t-m The area R contains n pixels, and its pixel value is CF t-m (x, ty), then the average pixel value of region R is A t-m (R), the pixel change rate of area R in the image is r t (R); Step 5: If the pixel change threshold r of region R t (R) max <r t (R), it is determined that there are no safety hazards in area R.

3. The method for predicting and warning engineering safety hazards based on image recognition according to claim 2, characterized in that: The steps for data processing of dam cracks in the hidden danger prediction module are as follows: Step 1: Have personnel mark the engineering safety hazards in each picture, and record the type and location information of the hazards pointed out by the personnel; grayscale the collected image data, and use the grayscale threshold HD max Distinguish the image data after grayscale processing, and mark the area with grayscale value less than the grayscale threshold as a potential risk area; Step 2: Divide the image data into n blocks of the same size, count the number of blocks with hidden danger areas, the number is m, and then calculate the percentage data ZB of the hidden danger areas marked on each image x =m / n, x is the number of each picture; the total proportion data ZB in the picture data obtained for the dam surface, spillway and gate position ZH1 , ZB ZH2 , ZB ZH3 Calculate separately, if there is a preset percentage data threshold ZB max <ZB ZHa , a=1,2,3, then it is determined that the dam has safety hazards; On the contrary, the total proportion data corresponding to the dam surface, spillway and gate are substituted into the formula, and the hidden danger value YH=ω1*ZB ZH1 +ω2*ZB ZH2 +ω3*ZB ZH3 , ω1, ω2 and ω3 are specific gravity coefficients, if YH max ≤YH, the dam is still considered to have safety hazards, and the dam is considered to have safety hazards according to YH-YH max / YH max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively.

4. The method for predicting and warning engineering safety hazards based on image recognition according to claim 2, characterized in that: The steps for data processing of dam slope collapse and landslide in the hidden danger prediction module are as follows: Step 1: Divide the grayscale image into several small image blocks, calculate the change of the grayscale value of the pixel in the corresponding small image block to approximate the slope value of the dam slope corresponding to the small image block, and construct a surface according to the grayscale value f(x, y) along the horizontal and vertical pixel coordinates; Step 2: Calculate the slope value by using the difference method, and get the horizontal slope value as PD x , the vertical slope value PD y .

5. The method for predicting and warning engineering safety hazards based on image recognition according to claim 4, characterized in that: The steps for the hidden danger prediction module to determine dam slope collapse and landslide are as follows: Step 1: The image block corresponds to the slope value of the dam If the slope threshold PD max ≤PD, the dam slope is judged to have the risk of collapse; otherwise, the change of slope value over time is judged. If the change of slope value ΔPD ≥ ΔPD during the time period from time t to time t+q max , and the dam slope is also judged to have the risk of collapse, ΔPD max is the preset slope value change threshold, t is the starting time point, and q is the extended time period for judging the change; Step 2: Predict the elapsed time T = PD max -PD t+q / ΔPD after the dam slope will collapse, with a unit time T t As standard, if T / T t <1, it is determined that the collapse of the dam slope is about to occur, an emergency evacuation signal is generated, and the emergency evacuation signal is transmitted to the early warning module; if 1≤T / T t <2, the dam slope is judged to be at risk of collapse, a risk avoidance signal is generated, and the risk avoidance signal is transmitted to the early warning module; if T / T t ≥2, the dam slope is judged to have the potential safety hazard of collapse, and a maintenance warning signal is generated and transmitted to the early warning module.

6. The method for predicting and warning engineering safety hazards based on image recognition according to claim 2, characterized in that: The steps for the hidden danger prediction module to process the leakage damage data are as follows: Step 1: Construct the gray-level co-occurrence matrix with parameter 1 pixel as distance and parameters 0°, 45°, 90° and 135° as directions; Step 2: If the grayscale level of the grayscale image f(x, y) is N, when the distance is d and the direction is θ, the probability that a pixel with a grayscale value of i and a pixel with a grayscale value of j appear at the same time is P(d, θ)(i, j).

7. The method for predicting and warning engineering safety hazards based on image recognition according to claim 1, characterized in that: The steps for the hidden danger prediction module to determine leakage damage are as follows: Step 1: Mark the contrast data as DB. If the preset contrast threshold DB max ≤DB, it is determined that there is leakage damage at the dam detection location, and according to DB-DB max / DB max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively; Step 2: The energy data is E. If the preset energy threshold E min ≥E, it is determined that there is leakage damage at the dam detection location, and according to E min -E / E min The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively; Step 3: If E>E min And DB <DB max , then the leakage damage risk of the dam is determined, and the contrast DB and energy E are normalized to obtain DB GY and E GY ; After analyzing the normal and leaking dam images, the weight of contrast in leakage judgment is obtained as ω DB , the energy weight is ω E , then the risk of leakage and damage to the dam FX = ω DB *DB GY +ω E *E GY , if the preset risk threshold FX max ≤FX, the dam is judged to have the risk of leakage and damage, and the FX-FX max / FX max The ratio is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively.

8. The method for predicting and warning engineering safety hazards based on image recognition according to claim 1, characterized in that: The warning processing steps of the early warning module are as follows: Step 1: The warning signals transmitted by the hidden danger prediction module are distinguished according to their levels. When the signal is transmitted, it is counted and recorded as level 1 YJ, level 2 EJ and level 3 SJ; if the transmitted warning signal is one of the three risks of dam cracks, dam slope collapse and collapse or leakage damage, the number of signals of the corresponding level is accumulated. The level signal satisfies the rule of 3YJ=EJ, 3EJ=SJ. After conversion according to this rule, the first-level warning signal count is counted and recorded as YJ Z , if 3*YJ max ≤YJ Z , then the third-level warning signal is used for warning; if 2*YJ max ≤YJ Z <3*YJ max , then the secondary warning signal will be used for warning; if YJ Z <2*YJ max , then a warning will be issued according to the first-level warning signal; YJ max is a preset single counting threshold; Step 2: If the transmitted warning signals are multiple of the three risks of dam cracks, dam slope collapse, and collapse or leakage damage, they are classified according to the risk type. The total number of first-level warning signals for dam crack risk is LF Z The total number of first-level warning signals for dam slope collapse and landslide risk is BT Z The total number of first-level warning signals for leakage damage risk is SL Z , then according to 6*BT Z +3*SL Z +LF Z The calculated value of / 10 is compared with the data of 0.25, 0.5 and 0.75, and the first, second and third level warning signals are transmitted respectively; Step 3: When the first-level warning signal is triggered, a warning signal of "there are safety hazards in the dam" is sent to the dam staff by email; when the second-level warning signal is triggered, the sound and light alarm emits a sound with a duration of 1s and a frequency of 500Hz; when the third-level warning signal is triggered, the sound and light alarm emits light and sound with a duration of 3s and a frequency of 1000Hz.

Images