Natural speckle enhancement method for large-scale slope DIC monitoring
Through the neural network method based on CNN and GAN, the natural speckle characteristics of large-scale slopes are extracted and enhanced, and the speckle characteristics are enhanced on the slopes through the drone spraying equipment, the problem of insufficient natural speckle in DIC monitoring of large-scale slopes is solved, the monitoring accuracy is improved and the efficient and low-cost monitoring effect is achieved.
Patent Information
- Application Number
- CN202411986269.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-03-21
AI Technical Summary
The prior art is difficult to effectively use natural speckle in large-scale slope DIC monitoring, resulting in insufficient monitoring accuracy.
Using a neural network method based on CNN and GAN, the natural speckle characteristics of slope images are extracted and enhanced, and the speckle characteristics are enhanced on large-scale slopes through drone spraying equipment.
The accuracy of large-scale slope DIC monitoring is improved, the advantages of natural characteristics are fully utilized, and efficient and low-cost monitoring effect is achieved.
Smart Images
Figure CN119671883A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of DIC monitoring speckles in civil engineering, and in particular to a natural speckle enhancement method for large-scale slope DIC monitoring. Background Art
[0002] Digital image correlation (DIC) is a non-contact optical measurement method that quantitatively measures the deformation, displacement and strain of an object by analyzing the changes in speckle on the surface of the object. It is widely used in the fields of material science and engineering testing. During the construction and operation of civil engineering projects, DIC's non-contact measurement method can be used to monitor roadbed slopes, foundation pit slopes, etc., and measure the deformation and displacement of the slopes, thereby providing reliable data for engineering and research.
[0003] The quality of DIC speckle pattern has a significant impact on the measurement results. At present, for small-scale materials, the speckle used in DIC measurement is a high-contrast random grayscale distribution image, but this method is not suitable for complex large-scale scenes such as slopes. Large-area random grayscale distribution images are difficult to prepare and expensive. The natural speckle characteristics of slope images are often difficult to meet the speckle standards of DIC monitoring. Summary of the invention
[0004] The present invention aims to solve the shortcomings of the prior art and provide a natural speckle enhancement method for large-scale slope DIC monitoring. The natural speckle characteristics of the slope are enhanced by a neural network method based on CNN and GAN, and the natural speckle characteristics of the large-scale slope area are enhanced by projection and drone spraying, thereby improving the accuracy of large-scale slope DIC monitoring.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A natural speckle enhancement method for DIC monitoring of large-scale slopes is disclosed. The method comprises the following steps: sampling an image of a monitored large-scale slope in a DIC camera, preprocessing the image, extracting natural speckle features of the slope image using a speckle feature network based on a convolutional neural network (CNN), and enhancing features of the slope image using a speckle enhancement network based on a generative adversarial network (GAN). The speckle features of the image are enhanced by generating partial pixels through the network, and a laser is used to project the enhanced image contour onto the large-scale slope. An unmanned aerial vehicle spraying device is used to spray the area corresponding to the generated pixels, thereby enhancing the natural speckle features of the large-scale slope area and improving the DIC monitoring accuracy of the large-scale slope.
[0007] The training and operation steps of the speckle feature network and speckle enhancement network are as follows:
[0008] S1. Dataset creation:
[0009] Artificially designed high-contrast speckle images with random grayscale distribution and speckle size of 3-5 pixels were collected as learning targets and recorded as Dataset 1 to train the speckle feature network; real slope images were collected and classified into two types according to the quality of natural speckles. The part with high contrast and speckle size of 3-5 pixels was recorded as Dataset 2, and the remaining natural speckles were recorded as Dataset 3 to train the speckle enhancement network; the above three data sets were preprocessed; high contrast means: speckle density is 50%, and the area of bright and dark pixels is the same;
[0010] S2. Speckle feature network design and training:
[0011] The image in the dataset 1 is annotated, and the annotated value is the weighted value of the experimental measurement error, signal-to-noise ratio, and contrast; the feature network adopts a deep learning network based on the convolutional neural network CNN, and the network consists of an input layer, a feature extraction layer, a fully connected layer, and an output layer; the loss function is the difference between the network prediction value and the annotated value, and back propagation is used to adjust the network weight;
[0012] S3. Speckle enhancement network design:
[0013] The speckle enhancement network is based on the generative adversarial network GAN. The image in dataset 3 is input to the generator, which generates some pixels to obtain the enhanced image. The discriminator distinguishes the enhanced image from the image in dataset 2. The discriminator uses the binary cross entropy loss function, and the generator is trained through a composite model.
[0014] S4. Speckle enhancement network training:
[0015] The speckle enhancement network is trained, while considering the enhancement of pixels and features. The parameters of the trained speckle feature network are frozen. The loss function consists of adversarial loss and feature conditional loss. At the same time, a pixel regularization term is added to the loss function.
[0016] S5. Operation:
[0017] The trained network is run forward to operate on the collected images to obtain the slope image with enhanced natural speckle features. Further post-processing is performed to identify the contour of the enhanced area as the spraying area.
[0018] The beneficial effects of the present invention are as follows: the present invention effectively targets large-scale DIC monitoring scenes, proposes an intelligent method for enhancing natural speckle features to improve monitoring accuracy, gives full play to the advantages of natural features of large scenes, is efficient and low in cost; effectively learns high-quality speckle features through a neural network, and simultaneously enhances natural features at two scales, pixel and feature, to become high-quality speckle; the present invention is not only suitable for natural speckle enhancement of DIC monitoring of large-scale slopes, but also suitable for other large-scale DIC monitoring scenes and scenes where speckle features need to be enhanced, has a wide range of applicability, and is easy to promote. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a working principle diagram of the present invention;
[0020] Figure 2 This is a diagram showing the relationship between the DIC camera and the large-scale slope arrangement;
[0021] Figure 3 This is a schematic diagram of the structure of the UAV spraying equipment;
[0022] In the picture: 1-large-scale slope; 2-DIC camera; 3-drone spraying equipment;
[0023] 21-mobile computer; 22-camera body; 23-lens; 24-laser projector; 25-bracket; 26-data cable;
[0024] 31-UAV; 32-Spray agent storage unit; 33-Spraying system; 34-Visual guidance system; 35-Remote controller;
[0025] The following is a detailed description of the embodiments of the present invention with reference to the accompanying drawings. DETAILED DESCRIPTION
[0026] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The embodiments are only used to explain the present invention and are not used to limit the scope of the present invention. The present invention is described in more detail by way of example with reference to the accompanying drawings in the following paragraphs. The advantages and features of the present invention will become clearer according to the following description. It should be noted that the accompanying drawings are all in a very simplified form and are not in precise proportions, and are only used to facilitate and clearly assist in explaining the purpose of the embodiments of the present invention.
[0027] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may also be a component centered. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may also be a component centered. When a component is considered to be "set on" another component, it may be directly set on the other component or there may also be a component centered. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0029] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0030] like Figures 1 to 3 As shown, a natural speckle enhancement method for DIC monitoring of large-scale slopes is used. The image sampling is performed on the area of the monitored large-scale slope 1 in the DIC camera 2, and the image is preprocessed. The natural speckle features of the slope image are extracted by a speckle feature network based on a convolutional neural network (CNN). The features of the slope image are enhanced by a speckle enhancement network based on a generative adversarial network (GAN). The speckle features of the image are enhanced by generating some pixels through the network. The enhanced image contour is projected onto the large-scale slope 1 by a laser. The drone spraying equipment 3 is used to spray the area corresponding to the generated pixels, thereby enhancing the natural speckle features of the large-scale slope 1 area and improving the DIC monitoring accuracy of the large-scale slope 1.
[0031] The training and operation steps of the speckle feature network and speckle enhancement network are as follows:
[0032] S1. Dataset creation:
[0033] Artificially designed high-contrast speckle images with random grayscale distribution and speckle size of 3-5 pixels were collected as learning targets and recorded as Dataset 1 to train the speckle feature network; real slope images were collected and classified into two types according to the quality of natural speckles. The part with high contrast and speckle size of 3-5 pixels was recorded as Dataset 2, and the remaining natural speckles were recorded as Dataset 3 to train the speckle enhancement network; the above three data sets were preprocessed; high contrast means: speckle density is 50%, and the area of bright and dark pixels is the same;
[0034] S2. Speckle feature network design and training:
[0035] The image in the dataset 1 is annotated, and the annotated value is the weighted value of the experimental measurement error, signal-to-noise ratio, and contrast; the feature network adopts a deep learning network based on the convolutional neural network CNN, and the network consists of an input layer, a feature extraction layer, a fully connected layer, and an output layer; the loss function is the difference between the network prediction value and the annotated value, and back propagation is used to adjust the network weight;
[0036] S3. Speckle enhancement network design:
[0037] The speckle enhancement network is based on the generative adversarial network GAN. The image in dataset 3 is input to the generator, which generates some pixels to obtain the enhanced image. The discriminator distinguishes the enhanced image from the image in dataset 2. The discriminator uses the binary cross entropy loss function, and the generator is trained through a composite model.
[0038] S4. Speckle enhancement network training:
[0039] The speckle enhancement network is trained, and the enhancement of pixels and features is considered at the same time. The parameters of the trained speckle feature network are frozen. The loss function consists of adversarial loss and feature conditional loss. At the same time, a pixel regularization term is added to the loss function, so that the enhanced image is close to the real image with excellent speckle features at both the pixel and feature levels, and the image pixel change is encouraged to be minimal.
[0040] S5. Operation:
[0041] The trained network is run forward to operate on the collected images to obtain the slope image with enhanced natural speckle features. Further post-processing is performed to identify the contour of the enhanced area as the spraying area.
[0042] The DIC camera 2 includes a mobile computer 21, a camera body 22, a lens 23, a laser projector 24, and a bracket 25; the camera body 22 and the laser projector 24 are installed on the bracket 25, the laser projector 24 is located below the camera body 22, the lens 23 is set on one side of the camera body 22 and aimed at the large-scale slope 1, and the camera body 22 and the laser projector 24 are connected to the mobile computer 21 through a data cable 26.
[0043] The mobile computer 21 is equipped with a data acquisition system, analysis software, a trained speckle feature network and a speckle enhancement network.
[0044] The camera body 22 transmits the image to the mobile computer 21 via the data line 26. The mobile computer 21 transmits the signal to the laser projector 24 via the data line 26. The laser projector 24 converts the digital signal of the mobile computer 21 into a projection image and projects it onto the large-scale slope 1. Before projection, the laser projector 24 needs to be aligned with the field of view of the camera body 22 during observation.
[0045] The drone spraying equipment 3 includes a drone 31, a spray agent storage unit 32, a spraying system 33, a visual guidance system 34, and a remote control 35. The spray agent storage unit 32 is used to store the paint or spray agent to be sprayed. The spraying system 33 is responsible for evenly spraying the paint or spray agent in the spray agent storage unit 32 onto the target surface. The visual guidance system 34 uses a camera and image processing technology to identify the target area and guide the drone 31 to perform precise spraying. The remote control 35 is used to remotely control the flight and spraying operations of the drone 31. Specific embodiment 1:
[0047] The natural speckle enhancement is performed on the DIC monitoring of a large-scale slope 1, such as Figures 1 to 3 As shown:
[0048] Through on-site shooting and web crawling, high-contrast random grayscale distribution speckle images with excellent features, real slope images, etc. are collected as data sets, and the data sets are preprocessed and labeled; the speckle feature network and speckle enhancement network are trained.
[0049] At the project site, the area of the large-scale slope 1 to be monitored is confirmed, and a DIC camera 2 is set up to transmit the captured image to the speckle enhancement network to obtain an enhanced image. The trained speckle feature network is used to compare and verify the enhancement effect. The enhanced image is post-processed to identify the contour, and the processed image is projected to the area of the large-scale slope 1 through the laser projector 24 under the camera body 22. The drone 31 carries a spraying system 33 to spray the area corresponding to the generated pixels, thereby enhancing the natural speckle characteristics of the large-scale slope 1 area and improving the DIC monitoring accuracy of the large-scale slope 1.
[0050] The present invention effectively targets large-scale DIC monitoring scenes, proposes an intelligent method for enhancing natural speckle features to improve monitoring accuracy, gives full play to the advantages of natural features of large scenes, is efficient and low in cost; effectively learns high-quality speckle features through a neural network, and simultaneously enhances natural features at two scales, pixel and feature, to turn them into high-quality speckle; the present invention is not only suitable for natural speckle enhancement of DIC monitoring of large-scale slopes 1, but also suitable for other large-scale DIC monitoring scenes and scenes where speckle features need to be enhanced, has a wide range of applicability, and is easy to promote.
[0051] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various improvements are made using the method concept and technical solution of the present invention, or are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.
Claims
1. A natural speckle enhancement method for large-scale slope DIC monitoring, characterized in that: The image of the monitored large-scale slope (1) in the DIC camera (2) is sampled, and the image is preprocessed. A speckle feature network based on a convolutional neural network (CNN) is used to extract the natural speckle features of the slope image. A speckle enhancement network based on a generative adversarial network (GAN) is used to enhance the features of the slope image. The speckle features of the image are enhanced by generating some pixels through the network. The enhanced image contour is projected onto the large-scale slope (1) using a laser. The area corresponding to the generated pixels is sprayed using an unmanned aerial vehicle (UAV) spraying device (3), thereby enhancing the natural speckle features of the large-scale slope (1) area and improving the DIC monitoring accuracy of the large-scale slope (1).
2. The natural speckle enhancement method for large-scale slope DIC monitoring according to claim 1 is characterized in that: The training and operation steps of the speckle feature network and speckle enhancement network are as follows: S1. Dataset creation: Artificially designed high-contrast speckle images with random grayscale distribution and speckle size of 3-5 pixels were collected as learning targets and recorded as Dataset 1 to train the speckle feature network; real slope images were collected and classified into two types according to the quality of natural speckles. The part with high contrast and speckle size of 3-5 pixels was recorded as Dataset 2, and the remaining natural speckles were recorded as Dataset 3 to train the speckle enhancement network; the above three data sets were preprocessed; high contrast means: speckle density is 50%, and the area of bright and dark pixels is the same; S2. Speckle feature network design and training: The image in the dataset 1 is annotated, and the annotated value is the weighted value of the experimental measurement error, signal-to-noise ratio, and contrast; the feature network adopts a deep learning network based on the convolutional neural network CNN, and the network consists of an input layer, a feature extraction layer, a fully connected layer, and an output layer; the loss function is the difference between the network prediction value and the annotated value, and back propagation is used to adjust the network weight; S3. Speckle enhancement network design: The speckle enhancement network is based on the generative adversarial network GAN. The image in dataset 3 is input to the generator, which generates some pixels to obtain the enhanced image. The discriminator distinguishes the enhanced image from the image in dataset 2. The discriminator uses a binary cross entropy loss function, and the generator is trained using a composite model; S4. Speckle enhancement network training: The speckle enhancement network is trained, while considering the enhancement of pixels and features. The parameters of the trained speckle feature network are frozen. The loss function consists of adversarial loss and feature conditional loss. At the same time, a pixel regularization term is added to the loss function. S5. Operation: The trained network is run forward to operate on the collected images to obtain the slope image with enhanced natural speckle features. Further post-processing is performed to identify the contour of the enhanced area as the spraying area.
3. The natural speckle enhancement method for large-scale slope DIC monitoring according to claim 2 is characterized in that: The DIC camera (2) comprises a mobile computer (21), a camera body (22), a lens (23), a laser projector (24), and a bracket (25); the camera body (22) and the laser projector (24) are mounted on the bracket (25), the laser projector (24) is located below the camera body (22), the lens (23) is arranged on one side of the camera body (22) and is aimed at the large-scale slope (1), and the camera body (22) and the laser projector (24) are connected to the mobile computer (21) via a data cable (26).
4. The natural speckle enhancement method for large-scale slope DIC monitoring according to claim 3 is characterized in that: The mobile computer (21) is equipped with a data acquisition system, analysis software, a trained speckle feature network and a speckle enhancement network.
5. The natural speckle enhancement method for large-scale slope DIC monitoring according to claim 4, characterized in that: The camera body (22) transmits the image to the mobile computer (21) via the data line (26), and the mobile computer (21) transmits the signal to the laser projector (24) via the data line (26). The laser projector (24) converts the digital signal of the mobile computer (21) into a projection image and projects it onto the large-scale slope (1). Before projection, the laser projector (24) needs to be aligned with the field of view of the camera body (22) during observation.
6. The natural speckle enhancement method for large-scale slope DIC monitoring according to claim 5, characterized in that: The unmanned aerial vehicle spraying equipment (3) comprises an unmanned aerial vehicle (31), a spray agent storage unit (32), a spraying system (33), a visual guidance system (34), and a remote controller (35). The spray agent storage unit (32) is used to store paint or spray agent to be sprayed. The spraying system (33) is responsible for uniformly spraying the paint or spray agent in the spray agent storage unit (32) onto a target surface. The visual guidance system (34) uses a camera and image processing technology to identify a target area and guide the unmanned aerial vehicle (31) to perform precise spraying. The remote controller (35) is used to remotely control the flight and spraying operations of the unmanned aerial vehicle (31).