Engineering construction early warning method and system based on deep learning

Abstract

The invention relates to an engineering construction early warning method and system based on deep learning. The method comprises the steps of obtaining a monitoring image in an engineering construction process; performing illumination balance on the monitoring image to obtain a monitoring image after illumination balance; evaluating the monitoring image after illumination balance by using an index to obtain an evaluation result; screening out the monitoring image which accords with a preset evaluation result and is subjected to illumination balance as a training sample; engineering construction behaviors in the training samples are marked to distinguish whether engineering construction is safe or not; inputting the marked training sample into a deep learning network for training to obtain an engineering construction early warning model; and the engineering construction early warning model is utilized to complete identification of unsafe behaviors in the engineering construction process, and early warning is given out. By using the deep learning technology, unsafe behaviors of a construction site can be automatically identified without human intervention, and early warning information can be timely sent out. The efficiency of construction safety management is improved, and omissions caused by manual inspection are reduced.

Description

Technical Field

[0001] The present invention relates to the field of engineering construction early warning technology, and in particular to an engineering construction early warning method and system based on deep learning. Background Art

[0002] With the rapid development of the construction industry, the scale of construction projects is expanding, and the complexity and uncertainty of construction sites are also increasing. In this context, the safety issues of construction projects are becoming increasingly prominent, and construction accidents are frequent, posing a major threat to the safety of people's lives and property and social stability. In order to effectively reduce construction risks and improve construction safety, various early warning systems have emerged.

[0003] Traditional construction safety management methods mainly rely on personnel's experience and post-analysis, which often makes it difficult to achieve real-time monitoring and rapid response. This method is not only inefficient, but also has defects such as strong subjectivity and delayed response, resulting in many potential risks that cannot be identified and handled in a timely manner. Summary of the invention

[0004] To solve the above problems, an embodiment of the present invention aims to provide a construction early warning method and system based on deep learning.

[0005] A construction early warning method based on deep learning, comprising:

[0006] Step 1: Obtain monitoring images during the construction process;

[0007] Step 2: Performing illumination balance on the monitoring image to obtain an illumination balanced monitoring image;

[0008] Step 3: Use indicators to evaluate the monitoring image after the light balance to obtain an evaluation result;

[0009] Step 4: Filter out the surveillance images after illumination balance that meet the preset evaluation results as training samples;

[0010] Step 5: Label the construction behaviors in the training samples to distinguish whether the construction is safe;

[0011] Step 6: Input the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model;

[0012] Step 7: Use the engineering construction early warning model to identify unsafe behaviors during the engineering construction process and issue an early warning.

[0013] Preferably, the step 2: performing illumination balancing on the monitoring image to obtain the illumination-balanced monitoring image comprises:

[0014] Step 2.1: Convert the monitoring image into HSV space and perform linear contrast stretching enhancement to form a color image;

[0015] Step 2.2: construct a balance window from the color image with any point as the midpoint;

[0016] Step 2.3: Calculate the mean value of the pixels in the balance window;

[0017] Step 2.4: construct a balancing coefficient based on the mean of the pixels in the balancing window;

[0018] Step 2.5: Performing illumination balance on the monitoring image using the balance coefficient to obtain an illumination-balanced monitoring image.

[0019] Preferably, in step 2.3, the balance coefficient is:

[0020]

[0021] Among them, R(x,y) is the balance coefficient, n×n is the size of the balance window W with point (i,j) as the midpoint, and v(i,j) is the brightness value of point (i,j). is the average brightness value under the balanced window W, s(i,j) is the saturation of point (i,j), is the average value of saturation under the equilibrium window W, δ v (x,y) is the brightness value variance, δ s (x,y) is the saturation variance.

[0022] Preferably, the step 2.5: using the balance coefficient to perform light balance on the monitoring image to obtain a light balanced monitoring image comprises:

[0023] Step 2.5.1: Set the adjustable coefficient;

[0024] Step 2.5.2: constructing a light balance formula based on the set adjustable coefficient, and using the light balance formula to normalize the saturation of the monitoring image to obtain a standardized monitoring image; wherein the normalization formula is:

[0025] s′(x,y)=s(x,y)+t(v′(x,y)-v(x,y))×R(x,y)

[0026] Among them, s′(x,y) is the saturation of the standardized processing, and t is the balance coefficient;

[0027] Step 2.5.3: De-noising the standardized monitoring image to obtain a monitoring image after light balance.

[0028] Preferably, the step 2.5.3: performing denoising on the standardized monitoring image to obtain a monitoring image after light balance includes:

[0029] The standardized monitoring image is denoised based on a Gaussian function to obtain a monitoring image after light balance; wherein the denoising formula is:

[0030]

[0031] Where I′(i,j) is the monitoring image after illumination balance, H(i,j,σ) is the Gaussian kernel function, I(i,j) is the standardized monitoring image, σ is the standard deviation of the Gaussian kernel function, and the size of the Gaussian kernel is (2p+1)(2p+1).

[0032] Preferably, the step 4: screening out surveillance images after illumination balance that meet preset evaluation results as training samples includes:

[0033] Step 4.1: Using the evaluation index to evaluate the monitoring image after the illumination balance to obtain an evaluation result;

[0034] Step 4.2: When the evaluation index is not within the preset range, the adjustable coefficient is adjusted and the monitoring image is re-processed for light balance; wherein the calculation formula of the evaluation index is:

[0035]

[0036] Among them, δ is the image color difference index, G is the image clarity index, W is the length of the image, H is the width of the image, x(i,j) is the pixel value of point (i,j) on the image, and μ is the mean value of the pixels on the image.

[0037] Preferably, the step 6: inputting the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model includes:

[0038] Step 6.1: Input the sample into the deep learning network for training;

[0039] Step 6.2: Use the loss function to optimize the deep learning network. When the value of the loss function is less than a preset value, output the engineering construction early warning model; wherein the loss function is:

[0040]

[0041] Among them, Loss is the value of the loss function, n is the number of samples, y i is the actual label of the i-th sample, The labels predicted by the deep learning network.

[0042] The present invention also provides an engineering construction early warning system based on deep learning, comprising:

[0043] A monitoring image acquisition module is used to acquire monitoring images during the construction process;

[0044] A light balancing module, used to perform light balancing on the monitoring image to obtain a light-balanced monitoring image;

[0045] An evaluation module, used to evaluate the monitoring image after the illumination balance using indicators to obtain an evaluation result;

[0046] A sample screening module is used to screen out surveillance images after illumination balance that meet preset evaluation results as training samples;

[0047] The labeling module is used to label the engineering construction behaviors in the training samples to distinguish whether the engineering construction is safe;

[0048] The training module is used to input the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model;

[0049] The early warning module is used to use the engineering construction early warning model to complete the identification of unsafe behaviors during the engineering construction process and issue early warnings.

[0050] The present invention also provides an electronic device, comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, and wherein the computer program, when executed by the processor, implements the steps in the above-mentioned engineering construction early warning method based on deep learning.

[0051] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps in the above-mentioned engineering construction early warning method based on deep learning are implemented.

[0052] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0053] The present invention relates to a construction early warning method and system based on deep learning. Compared with the prior art, the present invention can automatically identify unsafe behaviors at the construction site without human intervention and issue early warning information in a timely manner by utilizing deep learning technology. This improves the efficiency of construction safety management and reduces omissions in manual inspections.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A flowchart of a method for early warning of engineering construction based on deep learning in an embodiment provided by the present invention;

[0057] Figure 2 This is a schematic diagram of a deep learning-based engineering construction early warning system in an embodiment provided by the present invention. DETAILED DESCRIPTION

[0058] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0059] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0060] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] See also Figure 1 , an engineering construction early warning method based on deep learning, comprising:

[0062] Step 1: Obtain monitoring images during the construction process;

[0063] Step 2: Performing illumination balance on the monitoring image to obtain an illumination balanced monitoring image;

[0064] Furthermore, step 2 includes:

[0065] Step 2.1: Convert the monitoring image into HSV space and perform linear contrast stretching enhancement to form a color image;

[0066] Since the vividness of color in HSV space is determined by the color and brightness, the change of brightness value (V component) will affect the color sense of the image, because there is a relative relationship between brightness and saturation (S component). When the brightness increases, if the saturation is not adjusted adaptively, it means that the same color is mixed with more brightness, and the richness of the color will be relatively reduced. Therefore, after the brightness V is enhanced, if the saturation S component is not further adjusted accordingly, the saturation of the image will be reduced to a certain extent. In order to maintain the color balance of the image and the naturalness of the visual effect, after the brightness value of the image V component is processed, in order to ensure the natural transition of the color, it is usually necessary to adaptively adjust the saturation (S component).

[0067] The process is as follows:

[0068] Step 2.2: construct a balance window from the color image with any point as the midpoint;

[0069] Step 2.3: Calculate the mean value of the pixels in the balance window;

[0070] In step 2.3, the balance coefficient is:

[0071]

[0072] Among them, R(x,y) is the balance coefficient, n×n is the size of the balance window W with point (i,j) as the midpoint, and v(i,j) is the brightness value of point (i,j). is the average brightness value under the balanced window W, s(i,j) is the saturation of point (i,j), is the average value of saturation under the equilibrium window W, δ v (x,y) is the brightness value variance, δ s (x,y) is the saturation variance.

[0073] Step 2.4: construct a balancing coefficient based on the mean of the pixels in the balancing window;

[0074] Step 2.5: performing illumination balance on the monitoring image using the balance coefficient to obtain an illumination-balanced monitoring image;

[0075] Furthermore, step 2.5 includes:

[0076] Step 2.5.1: Set the adjustable coefficient;

[0077] Step 2.5.2: constructing a light balance formula based on the set adjustable coefficient, and using the light balance formula to normalize the saturation of the monitoring image to obtain a standardized monitoring image; wherein the normalization formula is:

[0078] s′(x,y)=s(x,y)+t(v′(x,y)-v(x,y))×R(x,y)

[0079] Among them, s′(x,y) is the saturation of the standardized processing, and t is the balance coefficient;

[0080] By adjusting the saturation appropriately, you can ensure that the color transitions between multiple brightness levels are more natural and avoid color distortion caused by lighting changes.

[0081] Step 2.5.3: performing denoising processing on the standardized monitoring image to obtain a monitoring image after light balance;

[0082] Some noise points on the enhanced surveillance image may be amplified. In order to improve the visual quality of the image, it is necessary to suppress the noise of the image.

[0083] In step 2.5.3, the present invention can perform denoising on the standardized monitoring image based on a Gaussian function to obtain a monitoring image after light balance; wherein the denoising formula is:

[0084]

[0085] Where I′(i,j) is the monitoring image after illumination balance, H(i,j,σ) is the Gaussian kernel function, I(i,j) is the standardized monitoring image, σ is the standard deviation of the Gaussian kernel function, and the size of the Gaussian kernel is (2p+1)(2p+1).

[0086] The present invention uses the weight of the Gaussian function to smooth the image, thereby reducing the impact of noise. By traversing each pixel of the image in turn, the neighborhood pixels are weighted averaged, so that each pixel value of the image is replaced by the weighted average of itself and the surrounding pixels, and the pixel weight farther from the center pixel is smaller. The purpose of this is to retain the main features of the image and weaken the impact of noise.

[0087] Step 3: Use indicators to evaluate the monitoring image after the light balance to obtain an evaluation result;

[0088] Step 4: Filter out the surveillance images after illumination balance that meet the preset evaluation results as training samples;

[0089] Furthermore, step 4 includes:

[0090] Step 4.1: Using the evaluation index to evaluate the monitoring image after the illumination balance to obtain an evaluation result;

[0091] Step 4.2: When the evaluation index is not within the preset range, the adjustable coefficient is adjusted and the monitoring image is re-processed for light balance; wherein the calculation formula of the evaluation index is:

[0092]

[0093] Among them, δ is the image color difference index, G is the image clarity index, W is the length of the image, H is the width of the image, x(i,j) is the pixel value of point (i,j) on the image, and μ is the mean value of the pixels on the image.

[0094] Step 5: Label the construction behaviors in the training samples to distinguish whether the construction is safe;

[0095] Step 6: Input the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model;

[0096] Further, step 6 includes:

[0097] Step 6.1: Input the sample into the deep learning network for training;

[0098] Step 6.2: Use the loss function to optimize the deep learning network. When the value of the loss function is less than a preset value, output the engineering construction early warning model; wherein the loss function is:

[0099]

[0100] Among them, Loss is the value of the loss function, n is the number of samples, y i is the actual label of the i-th sample, The labels predicted by the deep learning network.

[0101] Step 7: Use the engineering construction early warning model to identify unsafe behaviors during the engineering construction process and issue an early warning.

[0102] By using deep learning technology, the present invention can automatically identify unsafe behaviors at the construction site without human intervention and issue early warning information in a timely manner, which improves the efficiency of construction safety management and reduces omissions in human inspections.

[0103] See also Figure 2The present invention also provides an engineering construction early warning system based on deep learning, comprising:

[0104] A monitoring image acquisition module is used to acquire monitoring images during the construction process;

[0105] A light balancing module, used to perform light balancing on the monitoring image to obtain a light-balanced monitoring image;

[0106] An evaluation module, used to evaluate the monitoring image after the illumination balance using indicators to obtain an evaluation result;

[0107] A sample screening module is used to screen out surveillance images after illumination balance that meet preset evaluation results as training samples;

[0108] The labeling module is used to label the engineering construction behaviors in the training samples to distinguish whether the engineering construction is safe;

[0109] The training module is used to input the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model;

[0110] The early warning module is used to use the engineering construction early warning model to complete the identification of unsafe behaviors during the engineering construction process and issue early warnings.

[0111] Compared with the prior art, the beneficial effects of the engineering construction early warning system based on deep learning provided by the present invention are the same as the beneficial effects of the engineering construction early warning method based on deep learning described in the above technical scheme, which will not be repeated here.

[0112] The present invention also provides an electronic device, comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, and is characterized in that when the computer program is executed by the processor, the steps in the above-mentioned method for early warning of engineering construction based on deep learning are implemented. Compared with the prior art, the beneficial effects of the electronic device provided by the present invention are the same as the beneficial effects of the method for early warning of engineering construction based on deep learning described in the above-mentioned technical solution, which will not be elaborated here.

[0113] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps in the above-mentioned engineering construction early warning method based on deep learning are implemented. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as the beneficial effects of the engineering construction early warning method based on deep learning described in the above-mentioned technical scheme, which will not be repeated here.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technical solution that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A construction early warning method based on deep learning, characterized in that: include: Step 1: Obtain monitoring images during the construction process; Step 2: Performing illumination balance on the monitoring image to obtain an illumination balanced monitoring image; Step 3: Use indicators to evaluate the monitoring image after the light balance to obtain an evaluation result; Step 4: Filter out the surveillance images after illumination balance that meet the preset evaluation results as training samples; Step 5: Label the construction behaviors in the training samples to distinguish whether the construction is safe; Step 6: Input the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model; Step 7: Use the engineering construction early warning model to identify unsafe behaviors during the engineering construction process and issue an early warning.

2. According to the deep learning-based engineering construction early warning method of claim 1, it is characterized in that: The step 2: performing illumination balancing on the monitoring image to obtain an illumination-balanced monitoring image comprises: Step 2.1: Convert the monitoring image into HSV space and perform linear contrast stretching enhancement to form a color image; Step 2.2: construct a balance window from the color image with any point as the midpoint; Step 2.3: Calculate the mean value of the pixels in the balance window; Step 2.4: construct a balancing coefficient based on the mean of the pixels in the balancing window; Step 2.5: Performing illumination balance on the monitoring image using the balance coefficient to obtain an illumination-balanced monitoring image.

3. According to the deep learning-based engineering construction early warning method of claim 2, it is characterized in that: In step 2.3, the balance coefficient is: Among them, R(x,y) is the balance coefficient, n×n is the size of the balance window W with point (i,j) as the midpoint, and v(i,j) is the brightness value of point (i,j). is the average brightness value under the balanced window W, s(i,j) is the saturation of point (i,j), is the average value of saturation under the equilibrium window W, δ v (x,y) is the brightness value variance, δ s (x,y) is the saturation variance.

4. The engineering construction early warning method based on deep learning according to claim 3 is characterized in that: The step 2.5: performing illumination balance on the monitoring image using the balance coefficient to obtain an illumination-balanced monitoring image comprises: Step 2.5.1: Set the adjustable coefficient; Step 2.5.2: constructing a light balance formula based on the set adjustable coefficient, and using the light balance formula to normalize the saturation of the monitoring image to obtain a standardized monitoring image; wherein the normalization formula is: s′(x,y)=s(x,y)+t(v′(x,y)-v(x,y))×R(x,y) Among them, s′(x,y) is the saturation of the standardized processing, and t is the balance coefficient; Step 2.5.3: De-noising the standardized monitoring image to obtain a monitoring image after light balance.

5. The engineering construction early warning method based on deep learning according to claim 4 is characterized in that: The step 2.5.3: performing denoising on the standardized monitoring image to obtain a monitoring image after light balance, comprises: The standardized monitoring image is denoised based on a Gaussian function to obtain a monitoring image after light balance; wherein the denoising formula is: Where I′(i,j) is the monitoring image after illumination balance, H(i,j,σ) is the Gaussian kernel function, I(i,j) is the standardized monitoring image, σ is the standard deviation of the Gaussian kernel function, and the size of the Gaussian kernel is (2p+1)(2p+1).

6. According to the deep learning-based engineering construction early warning method of claim 5, it is characterized in that: The step 4: selecting the surveillance images after illumination balance that meet the preset evaluation results as training samples, includes: Step 4.1: Using the evaluation index to evaluate the monitoring image after the illumination balance to obtain an evaluation result; Step 4.2: When the evaluation index is not within the preset range, the adjustable coefficient is adjusted and the monitoring image is re-processed for light balance; wherein the calculation formula of the evaluation index is: Among them, δ is the image color difference index, G is the image clarity index, W is the length of the image, H is the width of the image, x(i,j) is the pixel value of point (i,j) on the image, and μ is the mean value of the pixels on the image.

7. The engineering construction early warning method based on deep learning according to claim 6 is characterized in that: Step 6: inputting the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model, including: Step 6.1: Input the sample into the deep learning network for training; Step 6.2: Use the loss function to optimize the deep learning network. When the value of the loss function is less than a preset value, output the engineering construction early warning model; wherein the loss function is: Among them, Loss is the value of the loss function, n is the number of samples, y i is the actual label of the i-th sample, The labels predicted by the deep learning network.

8. A construction early warning system based on deep learning, characterized in that: include: A monitoring image acquisition module is used to acquire monitoring images during the construction process; A light balancing module, used to perform light balancing on the monitoring image to obtain a light-balanced monitoring image; An evaluation module, used to evaluate the monitoring image after the illumination balance using indicators to obtain an evaluation result; A sample screening module is used to screen out surveillance images after illumination balance that meet preset evaluation results as training samples; The labeling module is used to label the engineering construction behaviors in the training samples to distinguish whether the engineering construction is safe; The training module is used to input the labeled training samples into the deep learning network for training to obtain the engineering construction early warning model; The early warning module is used to use the engineering construction early warning model to complete the identification of unsafe behaviors during the engineering construction process and issue early warnings.

9. An electronic device, comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, wherein: When the computer program is executed by the processor, the steps of the engineering construction early warning method based on deep learning as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the engineering construction early warning method based on deep learning as described in any one of claims 1 to 8 are implemented.

Images