Strip mine area remote sensing image semantic segmentation method, equipment, medium and product

Abstract

The invention discloses a strip mine area remote sensing image semantic segmentation method and device, a medium and a product, and relates to the field of remote sensing image semantic segmentation, and the method comprises the steps: carrying out the data enhancement of remote sensing image data in a strip mine area remote sensing semantic segmentation data set, and obtaining an expansion data set; constructing a remote sensing image semantic segmentation model based on a DeepLabV3 + model; according to the remote sensing image semantic segmentation model, on the basis of a DeepLabV3 + model, a backbone network is constructed based on a MobileNeXt architecture, a DenseASPP module in cross-layer connection is constructed, and two RGA modules are utilized in a decoder to carry out feature extraction on feature map information output by an encoder and shallow semantic features obtained by the backbone network. And training and verifying the remote sensing image semantic segmentation model by using the extended data set. According to the invention, the accuracy of remote sensing image semantic segmentation in a strip mine area scene can be improved.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image semantic segmentation, and in particular to a method, device, medium and product for semantic segmentation of remote sensing images in open-pit mines. Background Art

[0002] Mineral resources are an important energy base to support industrial production and economic development, and coal mines are the most important component of them. As an important source of renewable energy and fossil energy, coal mines play a vital role worldwide. It is widely used in power generation, metallurgy, chemical industry, manufacturing and other fields, providing a stable energy supply for all walks of life. With the rapid growth of the global economy, the demand for mineral resources continues to increase. According to data from the National Bureau of Statistics, China's raw coal production in 2020 was 3.9 billion tons, an increase of 150 million tons compared with the end of 2015; the scale of coal imports in 2020 also increased significantly compared with 2015, and the import volume increased from 204 million tons in 2015 to 304 million tons in 2020. Open-pit coal mining has become a common mining method. Compared with traditional underground mining, open-pit mining has the advantages of low cost, high efficiency and high output, so it has been widely used worldwide.

[0003] However, with the continuous expansion of open-pit mining, mine resource management and environmental monitoring are facing increasingly severe challenges. The impact of coal resource development and utilization on the environment is mainly manifested in land subsidence, water source depletion, vegetation damage, soil erosion, etc. caused by mining, as well as environmental problems caused by waste such as mine water, coal slag and coalbed methane. Based on the above problems, feature segmentation, as one of the key tasks, is crucial to achieving the goals of mine resource management and environmental protection. Traditional feature segmentation methods mainly rely on manually extracted features and classification algorithms. This method is limited by the subjectivity and tediousness of feature selection and cannot adapt well to diverse features and complex mining environments.

[0004] In recent years, with the rapid development of artificial intelligence, computer vision technology has occupied a major position in the field of artificial intelligence, and semantic segmentation based on deep learning is a hot direction in computer vision and digital image processing, which aims to classify each pixel in the image into different semantic categories, so as to achieve fine segmentation and semantic understanding of the image. Compared with traditional image segmentation methods, semantic segmentation based on deep learning can more accurately capture the boundaries and details of different objects in the image, and has higher segmentation accuracy. At present, image semantic segmentation technology is widely used in smart medical care, smart driving, smart security and smart mines. In the field of smart mines, semantic segmentation can be applied to environmental monitoring and resource management of mining areas. By segmenting remote sensing images, it is possible to analyze and identify different landforms in the mining area and the surrounding environment of the mining area, such as pits, soil, water bodies and vegetation, so as to optimize the mining planning, environmental protection and resource utilization of the mine.

[0005] With the advancement of remote sensing technology, semantic segmentation algorithms based on deep learning can better process large-scale remote sensing data and extract more valuable semantic features such as space, spectrum and texture. However, with the continuous improvement of remote sensing image resolution, and the fact that remote sensing images in open-pit mining areas have more complex ground object categories, no obvious distribution of ground objects, and the presence of small targets, the semantic segmentation task of remote sensing images in open-pit mining areas still faces many challenges.

[0006] Based on the above background, there is an urgent need to provide a new remote sensing image semantic segmentation method and system that can improve the accuracy of remote sensing image semantic segmentation in open-pit mining scenarios. Summary of the invention

[0007] The purpose of this application is to provide a method, device, medium and product for semantic segmentation of remote sensing images in open-pit mines, which can improve the accuracy of semantic segmentation of remote sensing images in open-pit mine scenes.

[0008] To achieve the above objectives, this application provides the following solutions:

[0009] In a first aspect, the present application provides a method for semantic segmentation of remote sensing images of an open-pit mine area, the method comprising:

[0010] Based on the acquired open-pit mine area remote sensing image data, a remote sensing semantic segmentation dataset of the open-pit mine area is constructed;

[0011] The remote sensing image data in the open-pit mine area remote sensing semantic segmentation dataset is enhanced to obtain an expanded dataset;

[0012] A remote sensing image semantic segmentation model is constructed based on the DeepLabV3+ model; the remote sensing image semantic segmentation model is a backbone network constructed based on the MobileNeXt architecture on the basis of the DeepLabV3+ model, a DenseASPP module for constructing cross-layer connections, and two RGA modules are used in the decoder to extract feature map information output by the encoder and shallow semantic features obtained by the backbone network;

[0013] Use the extended dataset to train and validate the remote sensing image semantic segmentation model;

[0014] The trained remote sensing image semantic segmentation model is used to perform semantic segmentation of remote sensing images in open-pit mining areas.

[0015] Optionally, constructing a remote sensing semantic segmentation dataset of the open-pit mine area according to the acquired remote sensing image data of the open-pit mine area specifically includes:

[0016] Use drones and satellites to obtain remote sensing image data of open-pit mine areas;

[0017] The sliding window technology is used to intercept image blocks of remote sensing image data to obtain the open-pit mine area remote sensing semantic segmentation dataset.

[0018] Optionally, the step size of the sliding window technique is 100 pixels; the size of each image block is 512 pixels×512 pixels.

[0019] Optionally, data enhancement strategies include: random rotation, mirror flipping, and HSV data enhancement.

[0020] Optionally, the residual module SandGlass in the MobileNeXt architecture fuses the Ghost module and parallel depth convolution.

[0021] Optionally, in the cross-layer connected DenseASPP module, the shallow semantic features obtained by the backbone network are convolved and pooled with four hole convolutions and SP rectangular pooling structures with different expansion rates to obtain five feature maps of different scales.

[0022] In a second aspect, the present application provides a semantic segmentation device for remote sensing images of an open-pit mine area, and the semantic segmentation device for remote sensing images of an open-pit mine area comprises:

[0023] The dataset construction module is used to construct a remote sensing semantic segmentation dataset of the open-pit mine area based on the acquired remote sensing image data of the open-pit mine area;

[0024] An expanded data set determination module is used to perform data enhancement on remote sensing image data in an open-pit mine area remote sensing semantic segmentation data set to obtain an expanded data set;

[0025] A remote sensing image semantic segmentation model construction module is used to construct a remote sensing image semantic segmentation model based on the DeepLabV3+ model; the remote sensing image semantic segmentation model is based on the DeepLabV3+ model, a backbone network constructed based on the MobileNeXt architecture, a DenseASPP module for constructing cross-layer connections, and two RGA modules in the decoder to extract feature map information output by the encoder and shallow semantic features obtained by the backbone network;

[0026] Remote sensing image semantic segmentation model training and verification module, used to train and verify the remote sensing image semantic segmentation model using the expanded data set;

[0027] The remote sensing image semantic segmentation model block is used to perform semantic segmentation of remote sensing images in open-pit mining areas using the trained remote sensing image semantic segmentation model.

[0028] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for semantic segmentation of remote sensing images of open-pit mines.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for semantic segmentation of remote sensing images of open-pit mines.

[0030] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for semantic segmentation of remote sensing images of open-pit mining areas.

[0031] According to the specific embodiments provided in this application, this application has the following technical effects:

[0032] The present application provides a method, device, medium and product for semantic segmentation of remote sensing images in an open-pit mine area. The remote sensing image semantic segmentation model is constructed based on the DeepLabV3+ model. The constructed remote sensing image semantic segmentation model is based on the DeepLabV3+ model, a backbone network constructed based on the MobileNeXt architecture, a DenseASPP module with cross-layer connections, and two RGA modules in the decoder to extract features from the feature map information output by the encoder and the shallow semantic features obtained by the backbone network; an extended data set is used to train the remote sensing image semantic segmentation model, and the trained remote sensing image semantic segmentation model is used to extract features from the feature map information output by the encoder and the shallow semantic features obtained by the backbone network. The lightweight improved encoder can greatly reduce the calculation amount and parameter amount of the model without losing accuracy; the cross-layer connected DenseASPP module is used to fully integrate feature information of different scales; two RGA modules are used to extract feature map information output by the encoder and shallow semantic features obtained by the backbone network, which can pay more attention to the boundary contours of ground objects, thereby reducing the ambiguity of the segmentation results and improving the generalization ability of the remote sensing image semantic segmentation model; then the remote sensing image semantic segmentation is performed after decoding the output to obtain the open-pit mine remote sensing image segmentation result; this application can improve the accuracy of remote sensing image semantic segmentation in open-pit mining scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 This is an application environment diagram of a method for semantic segmentation of remote sensing images of an open-pit mine area in one embodiment of the present application;

[0035] Figure 2 A schematic flow chart of a method for semantic segmentation of remote sensing images of an open-pit mine area provided in one embodiment of the present application;

[0036] Figure 3 This is a schematic diagram of the DeepLabV3+ model structure;

[0037] Figure 4 Schematic diagram of the remote sensing image semantic segmentation model structure. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0040] The method for semantic segmentation of open-pit mine remote sensing images provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the open-pit mine area remote sensing image to be processed to the server 104. After the server 104 receives the open-pit mine area remote sensing image to be processed, for the open-pit mine area remote sensing image to be processed, the server 104 performs semantic segmentation of the open-pit mine area remote sensing image based on the trained remote sensing image semantic segmentation model for the open-pit mine area remote sensing image. The server 104 can feedback the obtained remote sensing image semantic segmentation result to the terminal 102. In addition, in some embodiments, the semantic segmentation method of open-pit mine area remote sensing images can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform semantic segmentation of the open-pit mine area remote sensing images for the open-pit mine area remote sensing images to be processed, or the server 104 can obtain the open-pit mine area remote sensing images to be processed from the data storage system and perform semantic segmentation of the open-pit mine area remote sensing images for the open-pit mine area remote sensing images to be processed.

[0041] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0042] In an exemplary embodiment, Figure 2 As shown, a method for semantic segmentation of remote sensing images of open-pit mines is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following S201 to S205. Among them:

[0043] S201, constructing a remote sensing semantic segmentation dataset of the open-pit mine area according to the acquired remote sensing image data of the open-pit mine area;

[0044] S201 specifically includes:

[0045] S11, using drones and satellites to obtain remote sensing image data of open-pit mine areas;

[0046] In the segmentation of open-pit mines, accurately identifying and extracting the boundaries and features of different objects is of great significance for the management, planning and resource assessment of the mine area. In order to achieve accurate segmentation of objects, it is necessary to obtain remote sensing image data of the open-pit mine area; in order to solve the problem of missing remote sensing data in coal mine scenes, the method of carrying drones is adopted to capture subtle surface features and various targets in coal mine scenes through aerial photography technology. Through route planning and optimization of image acquisition strategies, multi-angle and multi-scale remote sensing images can be obtained, thereby providing high-resolution and accurate remote sensing data.

[0047] S12, the sliding window technology is used to intercept image blocks of remote sensing image data to obtain a remote sensing semantic segmentation dataset of open-pit mine area. The step size of the sliding window technology is 100 pixels; the size of each image block is 512 pixels × 512 pixels, and the sliding window is moved at intervals of 100 pixels in the horizontal and vertical directions, so that there is a certain overlap area between the intercepted image blocks, which can retain edge information and reduce the impact of edge effects on the results. At the same time, a smaller step size can increase the image coverage rate and ensure that the information within the entire image range is taken into account. By analyzing the collected image data and combining the ecological environment of the mining area and its surroundings, the dataset is divided into 8 categories: background, buildings, roads, waters, wastelands, vegetation, agricultural areas, and open-pit coal mining areas.

[0048] S202, performing data enhancement on the remote sensing image data in the open-pit mine area remote sensing semantic segmentation dataset to obtain an expanded dataset;

[0049] When constructing a remote sensing semantic segmentation dataset for an open-pit mine area, the unique characteristics of the scene need to be taken into account. First, the image acquisition process faces the problem of similar texture and color features between different categories. Therefore, color transformation, contrast adjustment and other methods are used to increase the color diversity of the data set, which helps the remote sensing image semantic segmentation model to better learn to distinguish different categories. Secondly, there may be occluded objects in the image, such as large machinery and equipment, buildings and trees. The above objects will make it difficult for the remote sensing image semantic segmentation model to accurately segment the occluded area. Therefore, data expansion methods such as image cropping, flipping and rotation are used to generate more samples and include different occlusion situations, thereby improving the remote sensing image semantic segmentation model's ability to recognize and segment occluded objects. In addition, changes in light intensity and shooting angles will also affect the quality of the picture, which may lead to uneven brightness. In order to make the remote sensing image semantic segmentation model robust, methods such as geometric transformation and light enhancement are used to simulate scene changes under different lighting conditions, so that the remote sensing image semantic segmentation model can adapt to various lighting conditions. In addition, some small target categories that are difficult to identify may appear in the image. In order to improve the remote sensing image semantic segmentation model's ability to recognize these small targets, methods such as data expansion and noise addition can be introduced to increase the number and diversity of small target samples.

[0050] Based on the above problems, this application adopts random rotation, mirror flipping and HSV data enhancement strategies to enhance the diversity of the data set. Among them, the random rotation strategy can improve the remote sensing image semantic segmentation model's ability to recognize and segment target objects at different angles; the mirror flipping strategy can help the remote sensing image semantic segmentation model not be affected by horizontal features, and can also handle the recognition and segmentation of symmetrical objects; and the HSV data enhancement strategy is to adjust the hue, saturation and brightness of the image color, thereby helping the remote sensing image semantic segmentation model to better adapt to scene changes under different color and lighting conditions, and improve the generalization ability and performance of the remote sensing image semantic segmentation model.

[0051] S203, constructing a remote sensing image semantic segmentation model based on the DeepLabV3+ model; the remote sensing image semantic segmentation model is based on the DeepLabV3+ model, a backbone network constructed based on the MobileNeXt architecture, a cross-layer connected DenseASPP module, and two RGA modules in the decoder to extract the feature map information output by the encoder and the shallow semantic features obtained by the backbone network; the residual module SandGlass in the MobileNeXt architecture is fused with the Ghost module and parallel deep convolution. In the cross-layer connected DenseASPP module, the shallow semantic features obtained by the backbone network are convolved and pooled with four hole convolutions with different expansion rates and SP rectangular pooling structures, respectively, to obtain five feature maps of different scales.

[0052] like Figure 3 As shown in the figure, the DeepLabv3+ model is used as the baseline model. In the setting of the experimental parameters, the image size of the input model is controlled at 512×512; the number of samples in each batch of input models is set to 8; the training rounds are set to 100; the random seed is set to 1; the optimizer uses the SGD method; and the cosine annealing algorithm is used as the scheduling strategy for the learning rate to help the model better converge to the optimal solution; finally, the training weights are saved every 10 Epochs.

[0053] Among them, the DeepLabv3+ model is the latest improved version of the DeepLab series of models. It further improves the performance and accuracy of image segmentation by introducing dilated deformable convolution and encoder-decoder structures. When processing complex remote sensing images, smaller targets are often difficult to be effectively segmented. To solve this problem, DeepLabv3+ introduces a dilated deformable convolution operation that can adaptively adjust the sampling position of the convolution kernel to better adapt to the shape changes of objects. The encoder of the DeepLabv3+ model is used to extract high-level semantic features and extract the global semantic information of the image through a series of convolution and pooling layers; while the decoder restores the resolution of the segmentation results layer by layer through upsampling and feature fusion, thereby restoring the details and accuracy of the segmentation results. Depthwise separable convolution is used in the backbone network and ASPP module of the encoder part. Depthwise separable convolution decomposes the convolution operation into two steps: depthwise convolution and pointwise convolution. Deep convolution first applies a convolution kernel to each channel of the input feature map independently to perform a convolution operation, and generates the corresponding output channel, and then uses point-by-point convolution to perform a point-by-point convolution operation on it to achieve feature association between channels.

[0054] One of the key components of the DeepLabv3+ model is the Atrous Convolution Pyramid Pooling (ASPP) module. While the traditional convolution operation keeps the size of the convolution kernel unchanged, its receptive field will be affected as the number of network layers increases. By introducing a dilation rate, the atrous convolution can increase the receptive field, thereby obtaining features in a wider range of remote sensing images. When the dilation rate of the atrous convolution r ≥ 1, for a convolution kernel of size W×H and number of input channels C, after adding a hole, the size of the convolution kernel becomes (W+r-1)×(H+r-1), and the number of channels does not change. In ASPP, the output feature maps of each branch are cascaded together and fused into a feature map through point-by-point convolution operations. The fused feature map not only has rich contextual information, but also maintains multi-scale feature representation. This design enables DeepLabv3+ to better handle objects of different scales in complex remote sensing image segmentation tasks and capture a wider range of contextual semantic information.

[0055] In order to improve the applicability of the scene in the open-pit coal mine area, the DeepLabv3+ model is improved to obtain the following Figure 4 The remote sensing image semantic segmentation model shown;

[0056] The remote sensing image input to the remote sensing image semantic segmentation model is first subjected to feature extraction through the backbone network built based on E-MobileNeXt to obtain deep semantic features and shallow semantic features. The deep semantic features enter the pyramid feature structure of the cross-layer connected DenseASPP module for further feature extraction. In the DenseASPP module, convolution and pooling are performed with four hole convolutions with different expansion rates and the SP rectangular pooling structure to obtain five feature maps of different scales. The five feature maps of different scales can accurately capture contextual information of different scales, thereby obtaining high-level semantic features of the image. Then, the five feature maps are linearly fused and sent to the decoder through two parallel 1×1 convolution kernels. In the decoder, the global attention mechanism RGA module is used to deeply extract the feature map information output by the encoder for one of the high-level features. The shallow semantic features obtained directly through the backbone network are extracted through another RGA module. The deep features and shallow features that have passed through the RGA module, as well as the deep and shallow features that directly enter the decoder are fused. Finally, the size of the feature map is restored to the original image size through 4x upsampling and the segmentation result is output.

[0057] S204, training and verifying the remote sensing image semantic segmentation model using the expanded data set;

[0058] S205, performing semantic segmentation of remote sensing images of open-pit mine areas using the trained remote sensing image semantic segmentation model.

[0059] DeepLabV3+ remote sensing image semantic segmentation takes into account the real-time requirements in field equipment applications, as well as the problems of slow convergence speed and large consumption of computing resources in the training process of the current mainstream model. The model needs to have sufficient lightweight characteristics. The remote sensing image semantic segmentation model provided in this application adopts the improved lightweight backbone network E-MobileNeXt, which is based on the MobileNeXt architecture. The core residual module SandGlass is fused with the Ghost module and parallel deep convolution, which can greatly reduce the number of parameters and computation required for the model while ensuring no loss of accuracy.

[0060] Remote sensing images contain various complex ground backgrounds. Open-pit coal mining areas contain ground objects of different scales, which makes the segmentation of small targets difficult, and large-scale targets are prone to "holes". In addition, the texture features of different categories in the data set are relatively similar, resulting in incomplete edge segmentation of ground objects. These problems have put forward higher requirements on the segmentation algorithm. This application proposes a remote sensing image semantic segmentation model structure. First, the pyramid feature extraction structure ASPP of DeepLabv3+ is improved. By replacing ASPP with a cross-layer connected DenseASPP structure and introducing a rectangular pooling SP module, the remote sensing image semantic segmentation model can obtain a larger receptive field, thereby focusing on the spatial features in the remote sensing image; then in the DeepLabv3+ decoder, the deep and shallow semantic features from the encoding stage are fused with the global attention mechanism RGA, so that the remote sensing image semantic segmentation model pays more attention to the boundary contours of the ground object target, thereby reducing the ambiguity of the segmentation results and improving the generalization ability of the remote sensing image semantic segmentation model.

[0061] This application trains the remote sensing image semantic segmentation model in the same dataset, server and experimental parameters, and tests it in the same CPU server. The ablation experiment and comparative experiment results of the improved structure are compared and analyzed one by one. Finally, the integrated results are analyzed. The experimental results show that the remote sensing image semantic segmentation model is experimentally analyzed on the open-pit mine dataset. The remote sensing image semantic segmentation model provided by this application has good segmentation performance for different land object categories in the mining area, and its average intersection over union ratio mIoU reaches 74.22%, which is 3.15% higher than the baseline model DeepLabv3+. Its parameter amount and computational complexity are reduced by 47.94M and 56.34G respectively compared with the baseline model.

[0062] Based on the same inventive concept, the embodiment of the present application also provides an open-pit mine remote sensing image semantic segmentation device for implementing the open-pit mine remote sensing image semantic segmentation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more open-pit mine remote sensing image semantic segmentation device embodiments provided below can refer to the above limitations on the open-pit mine remote sensing image semantic segmentation method, which will not be repeated here.

[0063] In an exemplary embodiment, a semantic segmentation device for remote sensing images of an open-pit mine is provided, comprising:

[0064] The dataset construction module is used to construct a remote sensing semantic segmentation dataset of the open-pit mine area based on the acquired remote sensing image data of the open-pit mine area;

[0065] An expanded data set determination module is used to perform data enhancement on remote sensing image data in an open-pit mine area remote sensing semantic segmentation data set to obtain an expanded data set;

[0066] A remote sensing image semantic segmentation model construction module is used to construct a remote sensing image semantic segmentation model based on the DeepLabV3+ model; the remote sensing image semantic segmentation model is based on the DeepLabV3+ model, a backbone network constructed based on the MobileNeXt architecture, a DenseASPP module for constructing cross-layer connections, and two RGA modules in the decoder to extract feature map information output by the encoder and shallow semantic features obtained by the backbone network;

[0067] Remote sensing image semantic segmentation model training and verification module, used to train and verify the remote sensing image semantic segmentation model using the expanded data set;

[0068] The remote sensing image semantic segmentation model block is used to perform semantic segmentation of remote sensing images in open-pit mining areas using the trained remote sensing image semantic segmentation model.

[0069] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a semantic segmentation method for remote sensing images of an open-pit mine area is implemented.

[0070] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0071] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0073] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0074] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0075] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0076] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for semantic segmentation of remote sensing images in open-pit mines, characterized in that: The method for semantic segmentation of remote sensing images of open-pit mines comprises: Based on the acquired open-pit mine area remote sensing image data, a remote sensing semantic segmentation dataset of the open-pit mine area is constructed; The remote sensing image data in the open-pit mine area remote sensing semantic segmentation dataset is enhanced to obtain an expanded dataset; A remote sensing image semantic segmentation model is constructed based on the DeepLabV3+ model; the remote sensing image semantic segmentation model is a backbone network constructed based on the MobileNeXt architecture on the basis of the DeepLabV3+ model, a DenseASPP module for constructing cross-layer connections, and two RGA modules are used in the decoder to extract feature map information output by the encoder and shallow semantic features obtained by the backbone network; Use the extended dataset to train and validate the remote sensing image semantic segmentation model; The trained remote sensing image semantic segmentation model is used to perform semantic segmentation of remote sensing images in open-pit mining areas.

2. The method for semantic segmentation of open-pit mine remote sensing images according to claim 1, characterized in that: The method of constructing a remote sensing semantic segmentation dataset of the open-pit mine area based on the acquired open-pit mine area remote sensing image data specifically includes: Use drones and satellites to obtain remote sensing image data of open-pit mine areas; The sliding window technology is used to intercept image blocks of remote sensing image data to obtain the open-pit mine area remote sensing semantic segmentation dataset.

3. The method for semantic segmentation of open-pit mine remote sensing images according to claim 2, characterized in that: The step size of the sliding window technique is 100 pixels; the size of each image block is 512 pixels×512 pixels.

4. The method for semantic segmentation of open-pit mine remote sensing images according to claim 1, characterized in that: Data enhancement strategies include random rotation, mirror flipping, and HSV data enhancement.

5. The method for semantic segmentation of open-pit mine remote sensing images according to claim 1, characterized in that: The residual module SandGlass in the MobileNeXt architecture combines the Ghost module and parallel depth convolution.

6. The method for semantic segmentation of open-pit mine remote sensing images according to claim 1, characterized in that: In the cross-layer connected DenseASPP module, the shallow semantic features obtained by the backbone network are convolved and pooled with four hole convolutions with different expansion rates and SP rectangular pooling structures to obtain five feature maps of different scales.

7. A semantic segmentation device for remote sensing images in open-pit mines, characterized in that: The open-pit mine remote sensing image semantic segmentation device comprises: The dataset construction module is used to construct a remote sensing semantic segmentation dataset of the open-pit mine area based on the acquired remote sensing image data of the open-pit mine area; An expanded data set determination module is used to perform data enhancement on remote sensing image data in an open-pit mine area remote sensing semantic segmentation data set to obtain an expanded data set; A remote sensing image semantic segmentation model construction module is used to construct a remote sensing image semantic segmentation model based on the DeepLabV3+ model; the remote sensing image semantic segmentation model is based on the DeepLabV3+ model, a backbone network constructed based on the MobileNeXt architecture, a DenseASPP module for constructing cross-layer connections, and two RGA modules in the decoder to extract feature map information output by the encoder and shallow semantic features obtained by the backbone network; Remote sensing image semantic segmentation model training and verification module, used to train and verify the remote sensing image semantic segmentation model using the expanded data set; The remote sensing image semantic segmentation model block is used to perform semantic segmentation of remote sensing images in open-pit mining areas using the trained remote sensing image semantic segmentation model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for semantic segmentation of remote sensing images of an open-pit mine area according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for semantic segmentation of remote sensing images of an open-pit mine area according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for semantic segmentation of remote sensing images of an open-pit mine area according to any one of claims 1 to 6 is implemented.

Images