An engineering activity mining sample image labeling and identifying method

Abstract

The present application relates to the technical field of image recognition processing, and particularly relates to a kind of engineering activity exploitation sample image labeling recognition method, comprising the following steps: obtaining optical remote sensing image is preprocessed, and feature data is extracted;Wherein, the feature data includes map patch color data, texture variation degree data, figure size data, unique feature data;The feature data is based on U-Net deep neural network model training and labeled recognition engineering activity type is obtained.The feature of optical remote sensing image data is extracted to train labeling recognition model, the rapid labeling recognition of engineering activity map patch is realized, without manual on-site visit to understand construction type, and the efficiency is high.

Description

Technical Field

[0001] This invention relates to the field of image recognition and processing technology, and in particular to a method for labeling and recognizing mining sample images from engineering activities. Background Technology

[0002] Construction projects play a vital role in urban modernization, infrastructure improvement, and economic development. Currently, monitoring of construction activities involving excavation, filling, and stripping is primarily conducted through manual on-site visits to understand the construction type, which is inefficient. Summary of the Invention

[0003] The purpose of this invention is to provide a method for labeling and identifying mining sample images of engineering activities. By extracting and labeling multiple features from optical remote sensing image data and training a U-Net deep neural network model, engineering activities can be quickly labeled and identified with high efficiency.

[0004] To achieve the above objectives, the present invention provides a method for annotating and recognizing mining sample images from engineering activities, comprising the following steps:

[0005] Optical remote sensing images are acquired, preprocessed, and feature data is extracted; wherein, the feature data includes patch color data, texture variation data, image size data, and unique feature data;

[0006] The feature data is obtained by training and labeling the U-Net deep neural network model to identify the types of engineering activities.

[0007] In one embodiment, the feature data is obtained by training and labeling an engineering activity type using a U-Net deep neural network model. Specific steps include:

[0008] If the detected patch is reddish-brown and the texture change is not obvious, it is identified as a soil breaking category.

[0009] In one embodiment, the feature data is obtained by training and labeling a U-Net deep neural network model to identify the type of engineering activity. Specific steps further include:

[0010] If a brownish-yellow patch is detected with obvious texture changes and regular features inside, it is identified as a construction site.

[0011] In one embodiment, the feature data is obtained by training and labeling a U-Net deep neural network model to identify the type of engineering activity. Specific steps further include:

[0012] If the detected patch is yellowish-white with extremely obvious texture changes, has steps and step shadows inside, and the size of the patch is within a large preset range, then it is labeled and identified as a mining type.

[0013] In one embodiment, the feature data is obtained by training and labeling a U-Net deep neural network model to identify the type of engineering activity. Specific steps further include:

[0014] If the detected patch is grayish-white or grayish-yellow with no obvious texture changes, the size of the patch is within a small preset area, and there are connected roads around the patch, then it is identified as waste and gravel.

[0015] In one embodiment, the detection of connected roads around a patch includes the following steps:

[0016] Detect at least two lines connected to the edge of the patch, calculate the average of the distance difference between different positions of adjacent lines to obtain the target distance difference, select the two lines with the smallest target distance difference as a group, and determine whether the width of the two lines in each group at different positions is within the preset spacing.

[0017] If it is within the preset spacing, it will be marked as a road.

[0018] In one embodiment, the acquisition of optical remote sensing images is preprocessed, and the specific steps include:

[0019] Select the image containing line textures and determine whether it is a complete engineering texture image to be identified;

[0020] If the line texture connects to the selected border, it is determined to be an incomplete engineering texture image to be identified;

[0021] If the line texture is not connected to the selected border, it is judged as a complete engineering texture image to be identified.

[0022] In one embodiment, after determining that the image to be identified is an incomplete engineering texture image, the method further includes:

[0023] Starting from a preset distance between the lines and textures in the incomplete image of the engineering texture to be identified that are not connected to the selected border, select the image until it becomes a complete image of the engineering texture to be identified.

[0024] This invention discloses a method for annotating and identifying mining sample images of engineering activities. The method involves acquiring optical remote sensing images, preprocessing them, and extracting color data, texture variation data, image size data, and unique feature data of the resulting patches. The acquired feature data is then used to train a U-Net deep neural network model for annotation and identification of engineering activity types. By extracting features from optical remote sensing image data and training the annotation and identification model, rapid annotation and identification of engineering activity patches can be achieved without the need for manual on-site visits, resulting in high efficiency. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0026] Figure 1 This is a flowchart illustrating a method for annotating and recognizing mining sample images in engineering activities provided by the present invention.

[0027] Figure 2 This is a schematic diagram illustrating the specific process of an image annotation and recognition method for mining samples in engineering activities provided by the present invention;

[0028] Figure 3 This is a schematic diagram of an optical remote sensing image of a soil breaking formation.

[0029] Figure 4 This is a schematic diagram of optical remote sensing imagery related to construction.

[0030] Figure 5 This is a schematic diagram of an optical remote sensing image for mining applications.

[0031] Figure 6 This is a schematic diagram of an optical remote sensing image of waste and crushed stone. Detailed Implementation

[0032] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0033] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for labeling and recognizing mining sample images from engineering activities, provided by the present invention. Specifically, the method for labeling and recognizing mining sample images from engineering activities may include the following steps:

[0034] S101. Acquire optical remote sensing images, perform preprocessing, and extract feature data;

[0035] In this embodiment of the invention, image data is acquired using the multispectral camera on the Gaofen-2 satellite. The multispectral camera on the Gaofen-2 satellite possesses sub-meter spatial resolution, high positioning accuracy, and rapid attitude maneuverability, effectively improving the satellite's overall observation efficiency and data clarity. The acquired optical remote sensing images have a resolution of 0.8m and include four channels: R, G, B, and NIR (near-infrared), in TIFF format. The feature data includes patch color data, texture variation data, graphic size data, and unique feature data. The unique feature data includes visible regular features within the patch, mining steps and their shadows within the patch, and connected roads around the patch. An image containing line textures is selected and cropped to determine if it is a complete image of the engineering texture to be identified. If the line texture is connected to the selected border, it is determined to be an incomplete image of the engineering texture to be identified; if the line texture is not connected to the selected border, it is determined to be a complete image of the engineering texture to be identified. For example, an optical remote sensing image might contain two construction projects. One project is crescent-shaped, and the other is candy-shaped. The candy-shaped project's edges are entirely within the captured image, allowing direct feature extraction. The crescent-shaped project, however, has two corners within the image but its central curve is not. Therefore, the crescent-shaped project is incomplete, requiring a re-capture. This avoids erroneous feature extraction and improves annotation accuracy. The re-capture involves starting from a preset distance between the unconnected lines in the incomplete image and the bounding box, and then capturing the entire image until a complete image is obtained. Specifically, the crescent shape is captured completely, with no other incomplete images, and the edges of the crescent are a certain distance from the captured image's edges. This facilitates feature extraction of the area surrounding the crescent-shaped construction site, aiding in subsequent project classification and improving annotation accuracy.

[0036] S102. Obtain the feature data and train and label it based on the U-Net deep neural network model to identify the type of engineering activity;

[0037] In this embodiment of the invention, the sample size is 512×512. Engineering activity categories are divided based on the map sample, including groundbreaking, construction, mining, and waste disposal / stone crushing. The skip connection structure in the U-Net deep neural network model helps the model inherit small-scale empirical features, improves the preservation of detailed information, and thus improves the accuracy of annotation and recognition. For details, please refer to... Figures 3 to 6 , Figure 3 This is a schematic diagram of an optical remote sensing image of soil breaking through the ground; Figure 4 This is a schematic diagram of optical remote sensing imagery related to construction. Figure 5 This is a schematic diagram of optical remote sensing images related to mining. Figure 6These are schematic diagrams of optical remote sensing images of waste rock and gravel. Since excavation projects only remove the topsoil, the image patches are mostly reddish-brown. When a reddish-brown image patch is detected with little textural variation, it is identified as excavation. Due to construction work such as filling and excavation or surface cement hardening, the image patches are mostly brownish-yellow. When a brownish-yellow image patch is detected with significant textural variation and regular features within it, it is identified as construction. Mining projects, due to long-term mining resulting in exposed rock, often have yellowish-white image patches. When a yellowish-white image patch is detected with extremely significant textural variation, mining steps and their shadows within it, and its size is within a large preset area (i.e., relatively large scale), it is identified as mining. Waste rock and gravel, where stone or waste is piled on the ground, are usually grayish-white or grayish-yellow, and are typically surrounded by connecting roads. When a grayish-white or grayish-yellow image patch is detected with little textural variation, its size is within a small preset area (i.e., relatively small scale), and it is surrounded by connecting roads, it is identified as waste rock and gravel. The detection of connected roads around a patch involves: detecting at least two lines connected to the edge of the patch; calculating the average of the distance differences between different positions of adjacent lines to obtain the target distance difference; selecting the two lines with the smallest target distance difference as a group; and determining whether the width of the two lines in each group at different positions is within a preset spacing. If it is within the preset spacing, it is identified as a road. A highway is a surface of different shapes composed of two lines, such as a long strip or an S-shape. Except in cases with bus stops, the width of a highway is roughly the same at different positions. Therefore, by calculating the average width, if the average width is within the range of standard highway width, the plane composed of the two lines can be identified as a highway.

[0038] Furthermore, the method for acquiring the feature data, based on the U-Net deep neural network model training, includes labeling and identifying engineering activity types. This further involves comparing the labeled image location with construction locations in a preset planning database to determine whether it is illegal construction or illegal mining. If the labeled image location is an approved construction project, it is considered normal construction or mining. This facilitates project construction management for administrators, eliminating the need for real-time on-site monitoring and improving efficiency.

[0039] When illegal construction or mining is identified, on-site video surveillance data from nearby cameras is retrieved. If the marked and identified image location is not an approved construction project, but construction operations have already commenced, it is considered illegal construction or mining. Retrieving nearby camera footage facilitates evidence collection, helps locate illegal construction personnel, and is useful for subsequent rights protection.

[0040] When illegal construction is identified, the location of the illegal construction site is compared with the location of nearby mining construction sites. If the target distance is less than a preset distance, a collapse hazard is indicated. For example, if the preset distance is 50 meters, and the edge of the illegal construction site (A) is less than 45 meters from the edge of the normal mining construction site, the proximity suggests a potential collapse hazard due to weak foundations at the mining site and the resulting weak foundations of nearby construction sites. Management personnel should be alerted, and the person in charge of the illegal construction site (A) should be immediately notified to cease construction to prevent accidents and ensure the safety of life and property.

[0041] This invention divides the open-air engineering activity map samples into two parts: a training set and a validation set. 90% (306 sets) are used for model training, and 10% (34 sets) are used for subsequent accuracy validation. The experiment aims to compare the accuracy of each category in terms of classification precision, and selects KNN and SVM methods for a horizontal comparison with the method of this invention. Method 1 and Method 2 will represent KNN and SVM methods, respectively.

[0042] Table 1 shows the classification accuracy of different types of patches using the three methods. The method of this invention has the highest accuracy in identifying waste rock patches, reaching 91.24%. Moreover, the method of this invention is superior to the other two methods in terms of accuracy across all categories.

[0043] Table 1 Comparison of Classification Accuracy

[0044]

[0045]

[0046] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the specific process of an image annotation and recognition method for mining samples in engineering activities provided by the present invention. It involves acquiring optical remote sensing images, preprocessing them, and extracting color data, texture variation data, image size data, and unique feature data of the image patches. The acquired feature data is then used to train a U-Net deep neural network model for annotation and recognition of engineering activity types. By extracting features from optical remote sensing image data and training the annotation and recognition model, rapid annotation and recognition of engineering activity patches can be achieved without manual on-site visits, resulting in high efficiency.

[0047] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A method for annotating and recognizing mining sample images in engineering activities, characterized in that, Includes the following steps: Acquire optical remote sensing images, perform preprocessing, and extract feature data; wherein, the feature data includes patch color data, texture variation data, image size data, and unique feature data; the specific steps for acquiring and preprocessing optical remote sensing images include: Select the image containing line textures and determine whether it is a complete engineering texture image to be identified; If the line texture is connected to the selected border, it is determined to be an incomplete engineering texture image to be identified; starting from the preset distance of the line texture in the incomplete engineering texture image to be identified that is not connected to the selected border, the image is selected until it is a complete engineering texture image to be identified; If the line texture is not connected to the selected border, it is judged as a complete engineering texture image to be identified; The feature data is obtained by training and labeling the U-Net deep neural network model to identify the types of engineering activities; The location of the labeled and recognized image is compared with the construction location in the preset planning database to determine whether it is illegal construction or illegal mining. When illegal construction is identified, the location of the illegal construction is compared with the location of the nearby mining construction. If the target distance is less than the preset distance, a collapse hazard is indicated.

2. The method for annotating and recognizing engineering activity mining sample images as described in claim 1, characterized in that, The specific steps for obtaining the feature data, based on the training and annotation of the U-Net deep neural network model, include: If the detected patch is reddish-brown and the texture change is not obvious, it is identified as a soil breaking category.

3. The method for annotating and recognizing mining sample images as described in claim 2, characterized in that, The acquisition of the feature data is based on the U-Net deep neural network model, which is used for training and labeling to identify the types of engineering activities. Specific steps also include: If a brownish-yellow patch is detected with obvious texture changes and regular features inside, it is identified as a construction site.

4. The method for annotating and recognizing mining sample images as described in claim 3, characterized in that, The acquisition of the feature data is based on the U-Net deep neural network model, which is used for training and labeling to identify the types of engineering activities. Specific steps also include: If the detected patch is yellowish-white with extremely obvious texture changes, has steps and step shadows inside, and the size of the patch is within a large preset range, then it is labeled and identified as a mining type.

5. The method for annotating and recognizing engineering activity mining sample images as described in claim 4, characterized in that, The acquisition of the feature data is based on the U-Net deep neural network model, which is used for training and labeling to identify the types of engineering activities. Specific steps also include: If the detected patch is grayish-white or grayish-yellow with no obvious texture changes, the size of the patch is within a small preset area, and there are connected roads around the patch, then it is identified as waste and gravel.

6. The method for annotating and recognizing mining sample images as described in claim 5, characterized in that, The detection of connected roads around the image patch includes the following steps: Detect at least two lines connected to the edge of the patch, calculate the average of the distance difference between different positions of adjacent lines to obtain the target distance difference, select the two lines with the smallest target distance difference as a group, and determine whether the width of the two lines in each group at different positions is within the preset spacing. If it is within the preset spacing, it will be marked as a road.

Images