A method, device, equipment and medium for identifying abandoned slag sites in railway engineering

Abstract

The present application relates to a method, device, equipment, and medium for identifying a slag dump for railway engineering, and relates to the technical field of production detection, including: obtaining remote sensing image information and real-time image information, preprocessing the remote sensing image information to obtain spectral image information, inputting the spectral image information into a classification model for training, obtaining engineering image information and annotated vector information, binding the annotated vector information to the engineering image information to obtain an annotated vector file, performing vector-to-raster conversion on the annotated vector file, using the image pixel values ​​and the engineering image information as training samples, performing feature fusion processing on the training samples to obtain edge fusion features, training a preset network model based on the edge fusion features to obtain a trained recognition network model, inputting the real-time image information into the trained recognition network model for training, and obtaining slag dump identification information. The present application has the effect of improving the efficiency of slag dump identification.

Description

Technical Field

[0001] The present application relates to the field of engineering environmental protection, and in particular to a method, device, equipment and medium for identifying a slag dump for railway engineering. Background Art

[0002] Railway engineering refers to various civil engineering facilities on railways. It also refers to the technologies used in various stages of railway construction (survey and design, construction, maintenance, and reconstruction). The construction and development of railways have provided people with convenient transportation and promoted economic and social development.

[0003] Currently, during the implementation of railway construction projects, large amounts of waste slag accumulate, forming waste dumps. These dumps have a serious impact on the surrounding ecological environment, necessitating ecological and environmental impact monitoring during railway construction. This monitoring typically involves using satellite remote sensing imagery to classify and identify surface patches and to measure the impact of construction on land use types. Traditional image classification and recognition methods involve manually labeling ground features using specialized software such as Arcgis. The software then performs calculations based on the labeling, enabling monitoring of the generation of waste dumps and their impact on the ecological environment.

[0004] Regarding the above-mentioned related technologies, the inventors believe that the manual classification and labeling not only has a slow labeling speed, but also cannot guarantee the labeling accuracy due to manual subjective judgment, thus resulting in a low efficiency in identifying waste dumps. Summary of the Invention

[0005] In order to improve the efficiency of identifying waste dumps, the present application provides a waste dump identification method, device, equipment and medium for railway engineering.

[0006] In a first aspect, the present application provides a method for identifying a slag dump for railway engineering, which adopts the following technical solution:

[0007] A method for identifying a slag dump for railway engineering, comprising:

[0008] Acquire remote sensing image information and real-time image information, wherein the remote sensing image information is used to represent satellite remote sensing image information of railway construction in different regional locations, and the real-time image information is used to represent satellite remote sensing image information within a preset range along the current railway line;

[0009] Preprocessing the remote sensing image information to obtain spectral image information;

[0010] Inputting the spectral image information into a trained classification model for training to obtain engineering image information and annotation vector information corresponding to the engineering image information, wherein the engineering image information is used to represent image information of different categories of scenes during the railway construction process, and the annotation vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information;

[0011] Binding the annotation vector information to the engineering image information to obtain an annotation vector file;

[0012] Performing vector-to-raster conversion on the annotated vector file, and using the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​as training samples, wherein the image pixel values ​​are used to represent the pixel values ​​corresponding to each engineering image information in the annotated vector file;

[0013] Performing feature fusion processing on the training samples to obtain edge fusion features;

[0014] Training a preset network model based on the edge fusion features to obtain a trained recognition network model;

[0015] The real-time image information is input into a trained recognition network model for training to obtain waste dump identification information.

[0016] In another possible implementation, preprocessing the remote sensing image information to obtain spectral image information includes:

[0017] Performing geometric correction processing on the remote sensing image information to obtain corrected image information;

[0018] Performing image fusion processing on the corrected image information and the multispectral image to obtain fused image information;

[0019] Perform image mosaic processing on the fused image information to obtain spectral image information.

[0020] In another possible implementation, performing feature fusion processing on the training samples to obtain edge fusion features includes:

[0021] Establishing a first DSM model based on the image pixel values;

[0022] Retrieving feature data information in the first DSM model, and performing DSN-level edge feature extraction on the feature data information and the engineering image information to obtain edge combination results of different scales, wherein the feature data information includes feature category information and spatial coordinate data corresponding to the feature category information;

[0023] The engineering image information, the ground object data information and the edge detection results of different scales are subjected to edge feature fusion to obtain edge fusion features.

[0024] In another possible implementation, the training of a preset network model based on the edge fusion feature to obtain a trained recognition network model includes:

[0025] Creating a first classification network model and a second classification network model, wherein the first classification network model is a network model for recognizing and training the engineering type of the engineering image information, and the second classification network model is a network model for recognizing and training the ground object features in the ground object data information;

[0026] Training the first classification network model based on the engineering image information and the edge fusion features to obtain a trained first classification network model;

[0027] Training the second classification network model based on the DSM model and the edge fusion feature to obtain a trained second classification network model;

[0028] The first classification network model and the second classification network model are subjected to feature fusion to obtain a recognition network model.

[0029] In another possible implementation, the real-time image information is input into a trained recognition network model for training to obtain the waste dump identification information, including:

[0030] Performing overlapping and slicing processing on the real-time image information to obtain cut image information;

[0031] constructing a second DSM model based on the cutting image information, and retrieving DSM data in the second DSM model;

[0032] The cutting image information and the DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information.

[0033] In another possible implementation, the cutting image information and the DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information, and then the method further includes:

[0034] Determining whether there are overlapping slice images in the cut image information;

[0035] If so, determining a pixel prediction value corresponding to each of the overlapping slice images based on the waste dump identification information;

[0036] The pixel prediction values ​​corresponding to each of the overlapping slice images are compared in terms of occurrence rate to obtain a target prediction result.

[0037] In another possible implementation, the method further includes:

[0038] Classify and extract the identification information of the waste dump to obtain optimized identification information;

[0039] Importing corresponding spatial data into the optimized identification information based on the real-time image information to obtain coordinate identification information;

[0040] Performing raster-to-vector processing on the coordinate identification information to obtain vector identification information;

[0041] It is determined whether there is a preset abnormality in the vector identification information. If so, intervention information is generated to inform the staff to intervene and correct the vector identification information.

[0042] In a second aspect, the present application provides a device for identifying a slag dump for railway engineering, which adopts the following technical solution:

[0043] A device for identifying a slag dump for railway engineering, comprising:

[0044] An information acquisition module is used to acquire remote sensing image information and real-time image information, wherein the remote sensing image information is used to represent satellite remote sensing image information of railway construction in different regional locations, and the real-time image information is used to represent satellite remote sensing image information within a preset range along the current railway line;

[0045] An image preprocessing module, used for preprocessing the remote sensing image information to obtain spectral image information;

[0046] An image classification module is configured to input the spectral image information into a trained classification model for training, thereby obtaining engineering image information and annotated vector information corresponding to the engineering image information, wherein the engineering image information is used to represent image information of different categories of scenes during the railway construction process, and the annotated vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information;

[0047] An information binding module, configured to bind the annotation vector information to the engineering image information to obtain an annotation vector file;

[0048] a vector conversion module for performing vector-to-raster processing on the annotated vector file and using the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​as training samples, wherein the image pixel values ​​are used to represent the pixel values ​​corresponding to each engineering image information in the annotated vector file;

[0049] A feature fusion module is used to perform feature fusion processing on the training samples to obtain edge fusion features;

[0050] A network training module is used to train a preset network model based on the edge fusion features to obtain a trained recognition network model;

[0051] The image recognition module is used to input the real-time image information into a trained recognition network model for training to obtain waste dump identification information.

[0052] In a possible implementation, when the image preprocessing module preprocesses the remote sensing image information to obtain spectral image information, it is specifically used to:

[0053] Performing geometric correction processing on the remote sensing image information to obtain corrected image information;

[0054] Performing image fusion processing on the corrected image information and the multispectral image to obtain fused image information;

[0055] Perform image mosaic processing on the fused image information to obtain spectral image information.

[0056] In another possible implementation, when the feature fusion module performs feature fusion processing on the training sample to obtain edge fusion features, it is specifically configured to:

[0057] Establishing a first DSM model based on the image pixel values;

[0058] Retrieving feature data information in the first DSM model, and performing DSN-level edge feature extraction on the feature data information and the engineering image information to obtain edge combination results of different scales, wherein the feature data information includes feature category information and spatial coordinate data corresponding to the feature category information;

[0059] The engineering image information, the ground object data information and the edge detection results of different scales are subjected to edge feature fusion to obtain edge fusion features.

[0060] In another possible implementation, when the network training module trains a preset network model based on the edge fusion feature to obtain a trained recognition network model, the network training module is specifically configured to:

[0061] Creating a first classification network model and a second classification network model, wherein the first classification network model is a network model for recognizing and training the engineering type of the engineering image information, and the second classification network model is a network model for recognizing and training the ground object features in the ground object data information;

[0062] Training the first classification network model based on the engineering image information and the edge fusion features to obtain a trained first classification network model;

[0063] Training the second classification network model based on the DSM model and the edge fusion feature to obtain a trained second classification network model;

[0064] The first classification network model and the second classification network model are subjected to feature fusion to obtain a recognition network model.

[0065] In another possible implementation, when the image recognition module inputs the real-time image information into a trained recognition network model for training to obtain the waste dump identification information, it is specifically used to:

[0066] Performing overlapping and slicing processing on the real-time image information to obtain cut image information;

[0067] constructing a second DSM model based on the cutting image information, and retrieving DSM data in the second DSM model;

[0068] The cutting image information and the DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information.

[0069] In another possible implementation, the device further includes: an overlap determination module, a pixel determination module, and a pixel alignment module, wherein:

[0070] The overlap judgment module is used to judge whether there are overlapping slice images in the cut image information;

[0071] The pixel determination module is configured to determine, when there are overlapping slice images in the cut image information, a pixel prediction value corresponding to each overlapping slice image based on the waste dump identification information;

[0072] The pixel comparison module is used to compare the occurrence rates of the pixel prediction values ​​corresponding to each of the overlapping slice images to obtain a target prediction result.

[0073] In another possible implementation, the device further includes: a detailed classification module, a data import module, a vector conversion module, and a vector judgment module, wherein:

[0074] The fine classification module is used to finely classify and extract the identification information of the waste dump to obtain optimized identification information;

[0075] The data import module is used to import corresponding spatial data to the optimized identification information based on the real-time image information to obtain coordinate identification information;

[0076] The vector conversion module is used to perform raster-to-vector processing on the coordinate identification information to obtain vector identification information;

[0077] The vector judgment module is used to judge whether there is a preset abnormality in the vector identification information. If so, it generates intervention information to inform the staff to intervene and correct the vector identification information.

[0078] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

[0079] An electronic device, comprising:

[0080] one or more processors;

[0081] Memory;

[0082] One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more programs are configured to: execute a method for identifying a slag dump for railway engineering as shown in any possible implementation of the first aspect.

[0083] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0084] A computer-readable storage medium includes: a computer program that can be loaded and executed by a processor to implement a method for identifying a slag dump for railway engineering as shown in any possible implementation of the first aspect.

[0085] In summary, this application includes at least one of the following beneficial technical effects:

[0086] By adopting the above technical solution, when identifying the waste dump, remote sensing image information of railway engineering construction satellites located in different areas and real-time image information of satellites within a preset range along the current railway are obtained, and then the remote sensing image information is preprocessed to obtain spectral image information, and the spectral image information is input into the trained classification model for training to obtain engineering image information and annotation vector information corresponding to the engineering image information, wherein the engineering image information is used to represent image information of different categories of scenes in the process of railway engineering construction, and the annotation vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information, and then the annotation vector information is compared with the engineering image information. The two images should be bound to obtain a labeled vector file, and then the labeled vector file is converted from vector to raster, and the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​are used as training samples, and then the training samples are subjected to feature fusion processing to obtain edge fusion features, and the preset network model is trained based on the edge fusion features to obtain a trained recognition network model, and then the real-time image information is input into the trained recognition network model for training to obtain the waste dump identification information. By adopting the above technical content, not only the overall annotation rate of the engineering image information is improved, but also the image annotation accuracy is guaranteed, thereby achieving the effect of improving the low efficiency of waste dump identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 This is a flow chart of a method for identifying a slag dump in a railway project according to an embodiment of the present application;

[0088] Figure 2 This is a schematic structural diagram of a slag dump identification device for railway engineering according to an embodiment of the present application;

[0089] Figure 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application; DETAILED DESCRIPTION

[0090] The following is combined with Figure 1-3 This application is described in further detail.

[0091] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but as long as they are within the scope of the claims of this application, they are protected by patent law.

[0092] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0093] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0094] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0095] The embodiment of the present application provides a method for identifying waste dumps for railway projects, which is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. Figure 1 As shown, the method includes:

[0096] Step S10: Acquire remote sensing image information and real-time image information.

[0097] Among them, remote sensing image information is used to represent satellite remote sensing image information of railway construction projects in different regional locations, and real-time image information is used to represent satellite remote sensing image information within a preset range along the current railway line.

[0098] In the embodiment of the present application, the collection areas of remote sensing image information include: Menghua Railway, Hangzhou-Changsha Passenger Dedicated Line, Shijiazhuang-Jinan High-speed Railway, Quzhou-Jiujiang Railway and Sichuan-Tibet Railway.

[0099] Specifically, the preset range is a 2-kilometer buffer zone along the current railway line.

[0100] Specifically, satellite remote sensing images are also called satellite images. Remote sensing means distant perception. Satellite remote sensing uses satellites in space to detect the reflection of electromagnetic waves by objects on the earth's surface and the electromagnetic waves they emit to extract ground information, thereby completing the long-distance identification of ground objects. The images obtained by converting and identifying these radio wave information are satellite remote sensing images.

[0101] Specifically, both remote sensing imagery and real-time imagery are composed of pixels. The richer the pixel count, the smaller the details that can be discerned. The density of pixels in an image is often expressed as lines per millimeter, with more lines indicating higher image quality. For example, a satellite image has 250 vertical and horizontal lines per square millimeter, meaning there are 62,500 pixels per square millimeter. The distance between adjacent pixels is only 4 microns, which is related to the camera's focal length and the satellite's flight altitude. If the focal length is 2 meters and the flight altitude is 150 kilometers, then, based on geometric relationships, the ground distance is 0.3 meters. This distance is the image's ground resolution.

[0102] Step S11 , preprocessing the remote sensing image information to obtain spectral image information.

[0103] In the embodiments of this application, preprocessing includes geometric correction, image fusion, and image mosaicking. Due to the influence of various imaging factors, the position, shape, size, and orientation of features in remote sensing image information may deviate from the corresponding real-world features, necessitating geometric correction of the image. The corrected remote sensing image information is then fused using panchromatic and multispectral images, resulting in a fused image with new spatial and spectral resolutions.

[0104] Step S12: inputting the spectral image information into the trained classification model for training to obtain engineering image information and annotation vector information corresponding to the engineering image information.

[0105] Among them, the engineering image information is used to represent the image information of different categories of scenes during the construction of the railway project, and the annotation vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information.

[0106] Step S13: Bind the annotation vector information to the engineering image information to obtain an annotation vector file.

[0107] Step S14, converting the annotated vector file into raster, and using the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​as training samples, the image pixel values ​​are used to represent the pixel values ​​corresponding to each engineering image information in the annotated vector file.

[0108] Specifically, since the training samples will be subsequently input into the ASPP-Aug-HED-DSM convolutional classification network (all input samples are images) for processing, the labeled vector files are converted from vectors to raster. Vector data and raster data are the two most commonly used data types in ArcGIS software. Vector data is internal data stored in a vector structure in a computer and is a direct product of trace digitization. In ArcGIS, vector data usually refers to Shapefile, that is, layer data with the data format suffix .shp in ArcCatalog; raster data is array data arranged in rows and columns of grid cells, with different grayscale or color. The position of each cell is defined by its row and column number, and the entity position represented is implicit in the row and column positions. In ArcGIS, raster data formats are relatively diverse, with common data format suffixes such as .tif, .gif, .img, and .jpg.

[0109] Step S15: Perform feature fusion processing on the training samples to obtain edge fusion features.

[0110] Step S16: training the preset network model based on the edge fusion feature to obtain a trained recognition network model.

[0111] Step S17: input the real-time image information into the trained recognition network model for training to obtain the waste dump identification information.

[0112] In an embodiment of the present application, when identifying a waste dump, remote sensing image information of railway engineering construction satellites located in different areas and real-time image information of satellites within a preset range along the current railway are obtained, and then the remote sensing image information is preprocessed to obtain spectral image information, and the spectral image information is input into the trained classification model for training to obtain engineering image information and annotation vector information corresponding to the engineering image information, wherein the engineering image information is used to represent image information of different categories of scenes in the process of railway engineering construction, and the annotation vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information, and then the annotation vector information is matched with the engineering image information. Binding to obtain the labeled vector file, then performing vector-to-raster processing on the labeled vector file, and using the image pixel values ​​obtained after processing and the engineering image information corresponding to the image pixel values ​​as training samples, and then performing feature fusion processing on the training samples to obtain edge fusion features, and training the preset network model based on the edge fusion features to obtain a trained recognition network model, and then inputting the real-time image information into the trained recognition network model for training to obtain the waste dump identification information. By adopting the above technical content, not only the overall annotation rate of the engineering image information is improved, but also the image annotation accuracy is guaranteed, thereby achieving the effect of improving the low efficiency of waste dump identification.

[0113] In a possible implementation of the embodiment of the present application, step S11 specifically includes step S111 (not shown in the figure) and step S112 (not shown in the figure), wherein:

[0114] Step S111 , performing geometric correction processing on the remote sensing image information to obtain corrected image information.

[0115] Specifically, it is the process of geometrically correcting the geometric distortion of remote sensing images. There are two types of geometric distortion: (1) distortion caused by the performance of the remote sensing instrument itself, including scale distortion, skew distortion, center movement distortion, scanning nonlinear distortion, radial distortion and orthogonal distortion. (2) distortion caused by the flight attitude of the carrier (aircraft or satellite) and the target object. The former includes projection distortion caused by the tilt of the carrier's flight attitude and scale error caused by altitude change; the latter includes distortion caused by terrain undulation and earth curvature. Usually, electronic computers and optical instruments are used for geometric correction. The principle is to transform the elements of a distorted image from their original positions to another correct image through a certain coordinate transformation. Image geometric correction also includes adding coordinate grids, aligning multi-spectral images and transforming remote sensing images obtained from a certain projection into map projections.

[0116] Step S112: performing image fusion processing on the corrected image information and the multispectral image to obtain fused image information.

[0117] Specifically, a multispectral image is one that contains many bands, sometimes as few as three (color images are an example) but sometimes many more, even hundreds. Each band is a grayscale image that represents the brightness of the scene according to the sensitivity of the sensor used to generate that band. In such an image, each pixel is associated with a string of values ​​representing the pixel's position in the different bands, or a vector. This string of values ​​is called the pixel's spectral signature.

[0118] Step S113: performing image mosaic processing on the fused image information to obtain spectral image information.

[0119] In an embodiment of the present application, the method of performing image mosaicking on the fused image information includes: selecting an image with relatively uniform brightness and color from a plurality of image information to be fused as a reference image for mosaicking, and mosaicking other images from near to far based on it.

[0120] In a possible implementation of the embodiment of the present application, step S15 (not shown in the figure) specifically includes step S51 (not shown in the figure), step S52 (not shown in the figure), and step S53 (not shown in the figure), wherein:

[0121] Step S51: establishing a first DSM model based on image pixel values.

[0122] Step S52: retrieve the feature data information in the first DSM model, and perform DSN-level edge feature extraction on the feature data information and engineering image information to obtain edge combination results of different scales. The feature data information includes feature category information and spatial coordinate data corresponding to the feature category information.

[0123] In this embodiment of the present application, DNS-level edge feature extraction includes DNS1-DNS5 stage-level extraction. Each DSN (Deeply-Supervised Net) outputs an edge detection result at a different scale. The edge detection results at different scales of the image and DSM are simply combined to generate DSN-fuse1, DSN-fuse2, DSN-fuse3, DSN-fuse4, and DSN-fuse5. Finally, the five generated DSN-fuse edge detection simple combination results are combined with the original image and original DSM to obtain the edge combination results.

[0124] Step S53 , edge feature fusion is performed on the engineering image information, the ground object data information, and the edge detection results at different scales to obtain edge fusion features.

[0125] In an embodiment of the present application, the first DSM model is the ASPP-Aug-HED-DSM model, which introduces the holistically-nested edge detection network (HED) as a ground feature boundary detection subnetwork into the ASPP-Aug multi-scale dilated convolutional classification network to classify images. While fully leveraging the high accuracy advantage of the HED holistically edge feature detection subnetwork in ground feature boundary detection, the DSM (Digital Surface Model) elevation data is introduced as auxiliary data for network training to obtain the model.

[0126] Specifically, because high-resolution remote sensing data contains rich object information and has large image sizes, even overlapping image slices can result in objects of the same classification being distributed across different slices, hindering the convolutional network's ability to learn the overall characteristics of these objects. Furthermore, because CNNs require a large amount of training data to achieve high-precision classification results, insufficient training data will result in network parameters being highly biased toward the training data. Image enhancement methods can typically be performed, including random cropping, flipping, and random perturbations of brightness, saturation, hue, and contrast. However, these enhancement methods cannot specifically enhance certain objects. Object proposal sampling methods, such as Selective Search and EdgeBoxes, can be used to locate regions within an image that contain potential objects.

[0127] In an embodiment of the present application, a graph-theoretic segmentation method is used to segment a high-resolution remote sensing image into several small regions. Based on the segmentation results, a Selective Search method is then used to generate bounding boxes of potential targets as an enhancement of the sample data, thereby using unsupervised image segmentation methods to obtain more valuable training data than using simple image enhancement. Based on the above method, potential objects and their labels are extracted from the image data as a supplement to the training data to improve the classification accuracy and the generalization ability of the model, thereby forming the ASPP-Aug multi-scale dilated convolutional classification network.

[0128] Specifically, the HED network utilizes a multi-output network structure for edge detection. Based on the VGG-16 network architecture, the convolutional layer preceding each pooling layer in the VGG-16 outputs a side-output feature map. The receptive fields of the convolution operations corresponding to these five side-output feature maps are 5, 14, 40, 92, and 196, respectively. During training, the loss of each of the five side-output feature maps is calculated using the edge images generated by the classified samples as label data, and then backpropagated independently. Unlike traditional CNNs, which have only a single forward-backward propagation flow, the HED network has multiple forward-backward propagation flows. During backpropagation, the gradients of these layers are equal to the weighted fusion of the gradients returned by subsequent layers. Due to the differences in receptive fields, the side-output feature maps closest to the input image have a smaller receptive field and can extract more local image features; the later side-output feature maps have a larger receptive field and can extract higher-level semantic features. Finally, these five side-output feature maps are weightedly fused into the output layer, where the loss is calculated with the label data and backpropagated.

[0129] Specifically, the HED overall edge detection network has the following characteristics compared with traditional edge detection methods:

[0130] 1. For overall image training and prediction, image-by-image edge detection is achieved based on FCNs (Fully Convolutional Networks). The algorithm inputs a multi-channel, high-resolution remote sensing image and outputs five edge detection intensity maps. Based on the FCN's multi-layered structure, multi-level feature learning is embedded within the network. All five feature layers serve as internal edge layers to generate edge detection results at different scales. These five edge detection feature maps are then subjected to a deconvolution layer to restore them to their original size.

[0131] 2. Due to the presence of shadows in high-resolution remote sensing images, the features of objects located in shadowed areas are significantly reduced, resulting in feature loss during the feature extraction process, which directly leads to a decrease in classification accuracy. However, the elevation information of ground objects in the image is not affected by image shadows. Adding data representing the height characteristics of ground objects as auxiliary classification information to the original image during the feature extraction process can reduce the adverse effects of factors such as shadows on the classification results.

[0132] Specifically, a digital elevation model (DEM) is a dataset that represents the plane coordinates (X, Y) and elevations (Z) of regularly gridded points within a certain range. It primarily describes the spatial distribution of landforms within a target study area. It is generated by collecting elevation data using contour lines or similar stereo models and then interpolating the data. A DEM is a subset of the digital terrain model (DTM). A DTM represents the spatial distribution of various landform factors, including elevation, such as aspect and slope, as linear or nonlinear combinations. A digital surface model (DSM) is a ground elevation model that includes height information for surface features such as trees and buildings. Building on the DEM, a DSM further includes height information for features other than the ground. For example, in forested areas, a DSM can be used to monitor forest growth, while in urban areas, a DSM can be used to monitor building construction.

[0133] In a possible implementation of the embodiment of the present application, step S16 specifically includes step S61 (not shown in the figure), step S62 (not shown in the figure), and step S63 (not shown in the figure), wherein:

[0134] Step S61: Create a first classification network model and a second classification network model.

[0135] Among them, the first classification network model is a network model used for identifying the engineering type of engineering image information, and the second classification network model is a network model used for identifying the ground feature in the ground data information.

[0136] Specifically, the first classification network model is the ASPP-Aug-Image model, and the second classification network model is the ASPP-Aug-DSM.

[0137] Step S62: training the first classification network model based on the engineering image information and the edge fusion features to obtain a trained first classification network model.

[0138] Step S63: training the second classification network model based on the DSM model and the edge fusion feature to obtain a trained second classification network model.

[0139] Step S64: Fusing features of the first classification network model and the second classification network model to obtain a recognition network model.

[0140] In a possible implementation of the embodiment of the present application, step S17 specifically includes step S71 (not shown in the figure), step S72 (not shown in the figure), and step S73 (not shown in the figure), wherein:

[0141] Step S71 , performing overlapping slicing processing on the real-time image information to obtain cut image information.

[0142] In the embodiment of the present application, overlapping slicing is performed using multiple different intervals during the slicing process to increase the number of samples and improve the generalization ability of the model.

[0143] Step S72: construct a second DSM model based on the cut image information, and retrieve DSM data in the second DSM model.

[0144] In the embodiment of the present application, the second DSM model is the ASPP-Aug-HED-DSM model.

[0145] In step S73, the cutting image information and DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information.

[0146] In a possible implementation of the embodiment of the present application, step S73 (not shown in the figure) further includes step S731 (not shown in the figure), step S732 (not shown in the figure), and step S733 (not shown in the figure), wherein:

[0147] Step S731: determine whether there are overlapping slice images in the cut image information.

[0148] Step S732: If it exists, determine the pixel prediction value corresponding to each overlapping slice image based on the waste dump identification information.

[0149] Step S733: compare the occurrence rates of the pixel prediction values ​​corresponding to each overlapping slice image to obtain a target prediction result.

[0150] In an embodiment of the present application, the method of comparing pixel prediction values ​​includes: comparing them from large to small, and selecting the pixel prediction value with the largest prediction value as the target prediction result.

[0151] In a possible implementation of the embodiment of the present application, step S17 further includes step S18 (not shown in the figure), step S19 (not shown in the figure), step S20 (not shown in the figure), and step S20a (not shown in the figure), wherein:

[0152] Step S18: Classify and extract the identification information of the waste dump to obtain optimized identification information.

[0153] Step S19: importing corresponding spatial data into the optimized identification information based on the real-time image information to obtain coordinate identification information.

[0154] Step S20 , performing raster-to-vector processing on the coordinate identification information to obtain vector identification information.

[0155] Step S21a: determine whether there is a preset abnormality in the vector identification information. If so, generate intervention information to inform the staff to intervene and correct the vector identification information.

[0156] Specifically, the preset anomalies include: vector recognition and classification errors caused by inaccurate algorithms.

[0157] The above embodiment introduces a method for identifying a slag dump for railway engineering from the perspective of a method flow. The following embodiment introduces a device for identifying a slag dump for railway engineering from the perspective of a virtual module or a virtual unit. For details, please refer to the following embodiment.

[0158] The embodiment of the present application provides a device for identifying a slag dump for railway engineering. As shown in the figure, the device 20 for identifying a slag dump for railway engineering may specifically include: an information acquisition module 21, an image preprocessing module 22, an image classification module 23, an information binding module 24, a vector conversion module 25, a feature fusion module 26, a network training module 27, and an image recognition module 28, wherein:

[0159] An information acquisition module 21 is configured to acquire remote sensing image information and real-time image information. The remote sensing image information is used to represent satellite remote sensing image information of railway construction projects in different locations. The real-time image information is used to represent satellite remote sensing image information within a preset range along the current railway line.

[0160] An image preprocessing module 22 is used to preprocess the remote sensing image information to obtain spectral image information;

[0161] An image classification module 23 is configured to input the spectral image information into a trained classification model for training, thereby obtaining engineering image information and annotated vector information corresponding to the engineering image information. The engineering image information is used to represent image information of different categories of scenes during the railway construction process, and the annotated vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information.

[0162] An information binding module 24 is used to bind the annotation vector information to the engineering image information to obtain an annotation vector file;

[0163] The vector conversion module 25 is used to perform vector-to-raster processing on the annotated vector file and use the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​as training samples. The image pixel values ​​are used to represent the pixel values ​​corresponding to each engineering image information in the annotated vector file;

[0164] The feature fusion module 26 is used to perform feature fusion processing on the training samples to obtain edge fusion features;

[0165] The network training module 27 is used to train the preset network model based on the edge fusion feature to obtain a trained recognition network model;

[0166] The image recognition module 28 is used to input real-time image information into a trained recognition network model for training to obtain waste dump identification information.

[0167] In one possible implementation of the embodiment of the present application, when the image preprocessing module 22 preprocesses the remote sensing image information to obtain spectral image information, it is specifically configured to:

[0168] Performing geometric correction processing on remote sensing image information to obtain corrected image information;

[0169] Performing image fusion processing on the corrected image information and the multispectral image to obtain fused image information;

[0170] The fused image information is subjected to image mosaicking processing to obtain spectral image information.

[0171] In another possible implementation of the embodiment of the present application, when the feature fusion module 26 performs feature fusion processing on the training sample to obtain the edge fusion feature, it is specifically used to:

[0172] Establishing a first DSM model based on the image pixel values;

[0173] Retrieving the feature data information in the first DSM model, and performing DSN-level edge feature extraction on the feature data information and engineering image information to obtain edge combination results of different scales. The feature data information includes feature category information and spatial coordinate data corresponding to the feature category information;

[0174] The engineering image information, ground object data information and edge detection results of different scales are fused to obtain edge fusion features.

[0175] In another possible implementation of the embodiment of the present application, when the network training module 27 trains the preset network model based on the edge fusion feature to obtain the trained recognition network model, it is specifically used to:

[0176] Creating a first classification network model and a second classification network model, wherein the first classification network model is a network model for recognizing and training engineering types of engineering image information, and the second classification network model is a network model for recognizing and training ground object features in ground object data information;

[0177] The first classification network model is trained based on the engineering image information and the edge fusion features to obtain a trained first classification network model;

[0178] The second classification network model is trained based on the DSM model and the edge fusion features to obtain a trained second classification network model;

[0179] The first classification network model and the second classification network model are feature-fused to obtain a recognition network model.

[0180] In another possible implementation of the embodiment of the present application, the image recognition module 28 inputs real-time image information into a trained recognition network model for training to obtain the waste dump identification information, specifically for:

[0181] Perform overlapping slicing processing on the real-time image information to obtain cut image information;

[0182] Constructing a second DSM model based on the cutting image information, and retrieving DSM data in the second DSM model;

[0183] The cutting image information and DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information.

[0184] In another possible implementation, the apparatus 20 further includes: an overlap determination module, a pixel determination module, and a pixel alignment module, wherein:

[0185] An overlap judgment module is used to judge whether there are overlapping slice images in the cut image information;

[0186] A pixel determination module, configured to determine a pixel prediction value corresponding to each overlapping slice image based on the waste dump identification information when overlapping slice images exist in the cut image information;

[0187] The pixel comparison module is used to compare the occurrence rates of the pixel prediction values ​​corresponding to each overlapping slice image to obtain the target prediction result.

[0188] In another possible implementation, the device 20 further includes: a detailed classification module, a data import module, a vector conversion module, and a vector judgment module, wherein:

[0189] The fine classification module is used to finely classify and extract the identification information of the waste dump to obtain optimized identification information;

[0190] A data import module is used to import corresponding spatial data of the optimized recognition information based on real-time image information to obtain coordinate recognition information;

[0191] A vector conversion module is used to perform raster-to-vector processing on the coordinate identification information to obtain vector identification information;

[0192] The vector judgment module is used to judge whether there is a preset abnormality in the vector identification information. If so, intervention information is generated to inform the staff to intervene and correct the vector identification information.

[0193] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0194] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 The electronic device 300 shown includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may further include a transceiver 304. It should be noted that in actual applications, the number of transceivers 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present application.

[0195] Processor 301 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0196] Bus 302 may include a path for transmitting information between the above components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0197] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0198] The memory 303 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the above method embodiment.

[0199] Electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. They may also include servers, etc. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0200] The embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed on a computer, enables the computer to execute the corresponding contents of the aforementioned method embodiment. Compared with the related art, in the embodiment of the present application, in the process of an industrial robot producing a product, by acquiring a production visual frame, the production visual frame is analyzed and screened to obtain a visual key frame, and then a first map point and a co-visual map are determined based on the visual key frame, and a depth value correlation analysis is performed on the first map point and the co-visual map to obtain a visual map point, and then a feature analysis is performed on the feature points in the visual map point to obtain a production analysis result, and then it is determined whether there is a preset production defect in the production analysis result. If so, a position detection is performed on the visual map point with the preset production defect to obtain abnormal position information, so that when an abnormal fault occurs for the first time in the production process of the product, the position of the defective node is promptly and accurately informed to the staff, so that the staff can perform maintenance in time to avoid the subsequent secondary defects, thereby achieving the effect of reducing the maintenance cost of the production product.

[0201] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0202] The above are only some of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying a slag dump in a railway project, characterized in that: include: Acquire remote sensing image information and real-time image information, wherein the remote sensing image information is used to represent satellite remote sensing image information of railway construction in different regional locations, and the real-time image information is used to represent satellite remote sensing image information within a preset range along the current railway line; Preprocessing the remote sensing image information to obtain spectral image information; Inputting the spectral image information into a trained classification model for training to obtain engineering image information and annotation vector information corresponding to the engineering image information, wherein the engineering image information is used to represent image information of different categories of scenes during the railway construction process, and the annotation vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information; Binding the annotation vector information to the engineering image information to obtain an annotation vector file; Performing vector-to-raster conversion on the annotated vector file, and using the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​as training samples, wherein the image pixel values ​​are used to represent the pixel values ​​corresponding to each engineering image information in the annotated vector file; Performing feature fusion processing on the training samples to obtain edge fusion features; Training a preset network model based on the edge fusion features to obtain a trained recognition network model; The real-time image information is input into a trained recognition network model for training to obtain waste dump identification information.

2. The method for identifying a slag dump for railway engineering according to claim 1, characterized in that: The preprocessing of the remote sensing image information to obtain spectral image information includes: Performing geometric correction processing on the remote sensing image information to obtain corrected image information; Performing image fusion processing on the corrected image information and the multispectral image to obtain fused image information; Perform image mosaic processing on the fused image information to obtain spectral image information.

3. The method for identifying a slag dump for railway engineering according to claim 1, characterized in that: The performing feature fusion processing on the training samples to obtain edge fusion features includes: Establishing a first DSM model based on the image pixel values; Retrieving feature data information in the first DSM model, and performing DSN-level edge feature extraction on the feature data information and the engineering image information to obtain edge combination results of different scales, wherein the feature data information includes feature category information and spatial coordinate data corresponding to the feature category information; The engineering image information, the ground object data information and edge detection results of different scales are subjected to edge feature fusion to obtain edge fusion features.

4. The method for identifying a slag dump for railway engineering according to claim 3, characterized in that: The step of training a preset network model based on the edge fusion feature to obtain a trained recognition network model includes: Creating a first classification network model and a second classification network model, wherein the first classification network model is a network model for recognizing and training the engineering type of the engineering image information, and the second classification network model is a network model for recognizing and training the ground object features in the ground object data information; Training the first classification network model based on the engineering image information and the edge fusion features to obtain a trained first classification network model; Training the second classification network model based on the DSM model and the edge fusion feature to obtain a trained second classification network model; The first classification network model and the second classification network model are subjected to feature fusion to obtain a recognition network model.

5. The method for identifying a slag dump for railway engineering according to claim 1, characterized in that: The real-time image information is input into a trained recognition network model for training to obtain waste dump identification information, including: Performing overlapping and slicing processing on the real-time image information to obtain cut image information; constructing a second DSM model based on the cutting image information, and retrieving DSM data in the second DSM model; The cutting image information and the DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information.

6. A method for identifying a slag dump for railway engineering according to claim 5, characterized in that: The cutting image information and the DSM data are input into the recognition network model for prediction training to obtain the waste dump identification information, and then the following steps are further included: Determining whether there are overlapping slice images in the cut image information; If so, determining a pixel prediction value corresponding to each of the overlapping slice images based on the waste dump identification information; The pixel prediction values ​​corresponding to each of the overlapping slice images are compared in terms of occurrence rate to obtain a target prediction result.

7. The method for identifying a slag dump for railway engineering according to claim 1, characterized in that: The method further comprises: Classify and extract the identification information of the waste dump to obtain optimized identification information; Importing corresponding spatial data into the optimized identification information based on the real-time image information to obtain coordinate identification information; Performing raster-to-vector processing on the coordinate identification information to obtain vector identification information; It is determined whether there is a preset abnormality in the vector identification information. If so, intervention information is generated to inform the staff to intervene and correct the vector identification information.

8. A device for identifying a slag dump in a railway project, characterized in that: include: An information acquisition module is used to acquire remote sensing image information and real-time image information, wherein the remote sensing image information is used to represent satellite remote sensing image information of railway construction in different regional locations, and the real-time image information is used to represent satellite remote sensing image information within a preset range along the current railway line; An image preprocessing module, used for preprocessing the remote sensing image information to obtain spectral image information; An image classification module is configured to input the spectral image information into a trained classification model for training, thereby obtaining engineering image information and annotated vector information corresponding to the engineering image information, wherein the engineering image information is used to represent image information of different categories of scenes during the railway construction process, and the annotated vector information is used to represent the three-dimensional geographic coordinate information corresponding to the engineering image information; An information binding module, configured to bind the annotation vector information to the engineering image information to obtain an annotation vector file; a vector conversion module for performing vector-to-raster processing on the annotated vector file and using the image pixel values ​​obtained after the processing and the engineering image information corresponding to the image pixel values ​​as training samples, wherein the image pixel values ​​are used to represent the pixel values ​​corresponding to each engineering image information in the annotated vector file; A feature fusion module is used to perform feature fusion processing on the training samples to obtain edge fusion features; A network training module is used to train a preset network model based on the edge fusion features to obtain a trained recognition network model; The image recognition module is used to input the real-time image information into a trained recognition network model for training to obtain waste dump identification information.

9. An electronic device, characterized in that: It includes: One or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the method for identifying a slag dump for railway engineering according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for identifying a slag dump for railway engineering as claimed in any one of claims 1 to 7 is implemented.

Images