Road ground-object recognition method and related apparatus

Abstract

Embodiments of the present application disclose a road feature recognition method and related device, which are used in the recognition scene of road features in multispectral remote sensing images, facilitate the application landing of the remote sensing image industry, reduce the application cost, and greatly improve the classification accuracy in the recognition process. The method can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, and auxiliary driving. The method comprises: acquiring a multispectral remote sensing image to be recognized; preprocessing the multispectral remote sensing image to be recognized to obtain a first image; taking the first image as the input of N target base classification models to obtain N first classification results predicted by the N target base classification models for the first image; taking the N first classification results as the input of a target integrated classification model to obtain a target classification result, the target classification result being used to indicate the classification of road features in the multispectral remote sensing image to be recognized.

Description

Technical Field

[0001] This application relates to the field of mapping, specifically to a method and related apparatus for road feature identification. Background Technology

[0002] Land feature identification (such as road surface and lane lines) in remote sensing images is one of the key technologies in automatic target recognition theory. As an important component of geographic information databases, accurately identifying urban road features from remote sensing images is of great significance for surface detection, urban structure description, and road updates.

[0003] Remote sensing images have complex backgrounds and are easily affected by the shadows of surrounding objects and the similarity of the spectra of roads and buildings, making it difficult to guarantee recognition accuracy. Figure 1 A schematic diagram illustrating the identification of road features in relevant schemes is shown. For example... Figure 1 As shown, for road feature identification in remote sensing images, the relevant schemes mainly process the original hyperspectral remote sensing image through principal component analysis. That is, the multiple bands (often dozens) of the hyperspectral remote sensing image are reduced to a few bands. Then, the first principal component of the obtained hyperspectral remote sensing image is used as a training sample. The training sample is used to construct a network model through a residual network and a feature fusion module for training. The trained network model is then used to identify the classification result of the hyperspectral remote sensing image.

[0004] However, the existing solutions for identifying ground features in hyperspectral remote sensing imagery rely solely on principal component analysis (PCA) for data preprocessing. For complex applications such as partially cloud-covered images or pixel-level imbalances in positive and negative samples, PCA alone is insufficient for effective data understanding and preprocessing, significantly impacting classification accuracy in subsequent model recognition. Furthermore, PCA itself is computationally expensive, and acquiring hyperspectral remote sensing imagery is costly in practice. Moreover, the solutions are only applicable to hyperspectral remote sensing imagery scenarios, making them unsuitable for identifying road features in other types of remote sensing imagery, thus severely limiting their practical application in the remote sensing imagery industry. Summary of the Invention

[0005] This application provides a method and related apparatus for road feature identification, which can be used in the identification of road features in multispectral remote sensing images, facilitating the application of remote sensing imagery and reducing application costs. Furthermore, by combining the prediction of a target base classification model with a target ensemble classification model to achieve road feature identification in multispectral remote sensing images, rather than relying solely on principal component analysis, the classification accuracy during the identification process is greatly improved.

[0006] In a first aspect, embodiments of this application provide a method for identifying road features. The method includes: acquiring a multispectral remote sensing image to be identified; preprocessing the multispectral remote sensing image to obtain a first image; using the first image as input to N target base classification models to obtain N first classification results obtained by the N target base classification models predicting the first image, wherein each of the N target base classification models is a machine learning model obtained by iteratively training image samples with image samples of a preset image size as training data, where N ≥ 2 and N is an integer; using the N first classification results as input to a target ensemble classification model to obtain target classification results, wherein the target classification results indicate the classification status of road features in the multispectral remote sensing image to be identified, and the target ensemble classification model is a machine learning model obtained by iteratively training the N classification results obtained by the N target base classification models predicting the image samples as training data.

[0007] Secondly, embodiments of this application provide a road feature identification device. This road feature identification device includes, but is not limited to, terminal equipment, servers, etc. The road feature identification device includes an acquisition unit and a processing unit. The system comprises: an acquisition unit for acquiring a multispectral remote sensing image to be identified; a processing unit for preprocessing the multispectral remote sensing image to be identified to obtain a first image; a processing unit for using the first image as input to N target base classification models to obtain N first classification results predicted by the N target base classification models for the first image, wherein each of the N target base classification models uses the predicted classification results of the acquired image samples as training targets and uses image samples of a preset image size as training data for iterative training to obtain a machine learning model, where N ≥ 2 and N is an integer; and a processing unit for using the N first classification results as input to a target ensemble classification model to obtain target classification results, wherein the target classification results are used to indicate the classification status of road features in the multispectral remote sensing image to be identified, and the target ensemble classification model is a machine learning model obtained by iterative training using the N classification results predicted by the N target base classification models for the image samples and the labeled tags of the image samples as training data.

[0008] In some optional examples, the processing unit is used to: perform image segmentation processing on the multispectral remote sensing image to be identified according to a preset image size to obtain N segmented image sets, wherein the segmented images in any two segmented image sets do not overlap; perform sampling processing on the corresponding segmented image set according to the ratio of positive and negative segmented image samples in each segmented image set to obtain a sampled segmented image set; and perform principal component analysis (PCA) processing on the sampled segmented image set to obtain a first image.

[0009] In some alternative examples, the acquisition unit is used to acquire the image matrix corresponding to the sampled cut image set, the image matrix being used to indicate the arrangement of the cut images in the sampled cut image set; the processing unit is used to: calculate the mean vector and covariance matrix corresponding to the image matrix, and the orthogonal matrix corresponding to the covariance matrix; perform principal component transformation processing on the sampled cut image set based on the mean vector and the orthogonal matrix to obtain the principal component transformed image; and perform inverse principal component transformation processing on the principal component transformed image to obtain the target principal components of the cut images in the sampled cut image set, so as to obtain the first image.

[0010] In some alternative examples, the processing unit is used to: before performing principal component analysis (PCA) on the sampled cut image set to obtain a first image, perform denoising on each cut image in the sampled cut image set based on a preset denoising model to obtain a denoised cut image; and perform principal component analysis (PCA) on the denoised cut image to obtain the first image.

[0011] In some alternative examples, the acquisition unit is further configured to: acquire N classification results and the labeled labels of image samples, wherein the N classification results are the classification results obtained after the N target base classification models predict the image samples respectively; the processing unit is further configured to: process the N classification results and the labeled labels of image samples based on a preset logistic regression model to obtain N target weights, wherein each target weight is used to indicate the degree of contribution of the corresponding classification result; and iteratively train the initial ensemble classification model based on the N target weights to obtain the target ensemble classification model.

[0012] In some other optional examples, the acquisition unit is also used to: acquire image samples of a preset image size; the processing unit is also used to: process the image samples of the preset image size based on a preset processing model to obtain the predicted classification result of the image samples; acquire the difference between the predicted classification result of the image samples and the preset label to obtain the target loss value; and iteratively train the initial base classification model based on the target loss value to obtain the target base classification model.

[0013] A third aspect of this application provides a road feature recognition device, including: a memory, an input / output (I / O) interface, and a processor. The memory stores program instructions. The processor executes the program instructions in the memory to perform the road feature recognition method corresponding to the embodiment of the first aspect described above.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method corresponding to the embodiments of the first aspect described above.

[0015] The fifth aspect of this application provides a computer program product containing instructions that, when run on a computer or processor, causes the computer or processor to execute the method described above for performing the implementation method of the first aspect.

[0016] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0017] In this embodiment, each of the N target base classification models uses the predicted classification results of image samples as training targets and image samples of a preset image size as training data for iterative training to obtain a machine learning model. After preprocessing the multispectral remote sensing image to be identified to obtain a first image, this first image is used as input to each target base classification model to obtain the first classification result predicted by each model for the first image. Furthermore, since the target ensemble classification model is a machine learning model obtained by iteratively training the N classification results obtained from the predictions of the image samples by the N target base classification models, the obtained N first classification results can be used as input to the target ensemble classification model to predict the target classification result of the multispectral remote sensing image to be identified. This target classification result can be used to indicate the classification status of road features in the multispectral remote sensing image to be identified. In other words, since the acquisition cost of multispectral remote sensing images is much lower than that of hyperspectral remote sensing images, the road feature recognition method provided in this application can be extended to road feature recognition scenarios in multispectral remote sensing images, facilitating the application of remote sensing imagery and reducing application costs. Furthermore, by combining the prediction of the target base classification model with a target ensemble classification model to achieve road feature recognition in multispectral remote sensing images, rather than relying solely on principal component analysis, the classification accuracy during the recognition process is greatly improved. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram of the identification of road features in the relevant scheme is shown;

[0020] Figure 2 A flowchart of a road feature identification method provided in an embodiment of this application is shown;

[0021] Figure 3 Another flowchart of the road feature identification method provided in this application embodiment is shown;

[0022] Figure 4 This illustration shows a schematic diagram of an interface for acquiring multispectral remote sensing images according to an embodiment of this application;

[0023] Figure 5 This illustration shows a schematic diagram of an interface for road feature recognition provided in an embodiment of this application;

[0024] Figure 6 A schematic diagram illustrating the target classification results provided in an embodiment of this application is shown;

[0025] Figure 7 A schematic diagram of the road feature identification device provided in the embodiments of this application is shown;

[0026] Figure 8 A schematic diagram of the hardware structure of the road feature identification device provided in the embodiments of this application is shown. Detailed Implementation

[0027] This application provides a method and related apparatus for road feature identification, which can be used in the identification of road features in multispectral remote sensing images, facilitating the application of remote sensing imagery and reducing application costs. Furthermore, by combining the prediction of a target base classification model with a target ensemble classification model to achieve road feature identification in multispectral remote sensing images, rather than relying solely on principal component analysis, the classification accuracy during the identification process is greatly improved.

[0028] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that implementations of the application described herein can be implemented, for example, in sequences other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] With the research and advancement of artificial intelligence (AI) technology, AI technology is being researched and applied in many fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, self-driving cars, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI technology will be applied in more fields and play an increasingly important role.

[0032] This application provides a method for road feature identification. The method provided in this application is based on artificial intelligence (AI). AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0033] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech recognition, natural language processing, and machine learning / deep learning.

[0034] In the embodiments of this application, the main artificial intelligence technologies involved include the aforementioned computer vision (CV) technologies. For example, it may involve image compression in image processing, image classification in image semantic understanding, etc.; it may also involve deep learning in machine learning (ML), etc.

[0035] The road feature identification method provided in this application can be applied to road feature identification devices with data processing capabilities, such as terminal devices, servers, and question-answering robots. Terminal devices may include, but are not limited to, smartphones, desktop computers, laptops, tablets, smart speakers, in-vehicle devices, smartwatches, wearable smart devices, smart voice interaction devices, smart home appliances, and aircraft. Servers may be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services; this application does not impose specific limitations. Furthermore, the mentioned terminal devices and servers can be directly or indirectly connected via wired or wireless communication; this application does not impose specific limitations.

[0036] The aforementioned road feature recognition device is capable of implementing the aforementioned computer vision technology. Computer vision technology is a science that studies how to enable machines to "see," specifically, it uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or instrument detection. As a scientific discipline, computer vision researches related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technology typically includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition. In the embodiments of this application, the road feature recognition device can use this computer vision technology to preprocess multispectral remote sensing images.

[0037] Furthermore, this road feature recognition device also possesses machine learning capabilities. Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as neural networks.

[0038] The method for road feature identification provided in this application uses an artificial intelligence model, mainly involving the application of neural networks, which are used to identify and process the classification results of multispectral remote sensing images.

[0039] The following describes a method for road feature identification provided by an embodiment of this application, with reference to the accompanying drawings. Figure 2 A flowchart illustrating a method for road feature identification provided in an embodiment of this application is shown. Figure 2 As shown, the method for road feature identification may include the following steps:

[0040] 201. Acquire the multispectral remote sensing image to be identified.

[0041] In this example, the multispectral remote sensing image refers to an image obtained by extracting multiple single bands from the radiation of road features. In other words, the multispectral remote sensing image contains spectral information from multiple bands. The described road features may include, but are not limited to, road surfaces, lane lines, and other road-related objects; this embodiment does not impose specific limitations. Furthermore, the format of the described multispectral remote sensing image to be identified may include, but is not limited to, TIFF format.

[0042] 202. Preprocess the multispectral remote sensing image to be identified to obtain the first image.

[0043] In this example, after obtaining the multispectral remote sensing image to be identified, the image can be preprocessed to obtain the first image. The preprocessing described may include, but is not limited to, image segmentation, sampling, and principal component analysis; it may also include denoising operations before performing principal component analysis, but the specific implementation in this application is not limited.

[0044] In some examples, the preprocessing of the multispectral remote sensing image to be identified to obtain the first image can be achieved as follows: the multispectral remote sensing image to be identified is segmented according to a preset image size to obtain N segmented image sets, where the segmented images in any two of the N segmented image sets do not overlap; then, the corresponding segmented image set is sampled according to the ratio of positive and negative segmented images in each segmented image set to obtain a sampled segmented image set, and principal component analysis (PCA) is performed on the sampled segmented image set to obtain the first image.

[0045] In this example, after acquiring the multispectral remote sensing image to be identified, the image can be segmented according to different preset image sizes, such as n1×n1, n2×n2, n3×n3, n... N ×n N The multispectral remote sensing image to be identified is segmented into N segments of different sizes. It should be noted that there is no overlap between any two segments in these N segments. Furthermore, the described preset image size can also be understood as the sliding window size; this application does not specify a particular limitation.

[0046] After obtaining these N segmented image sets, for each segmented image in each segmented image set with a preset image size, such as the segmented image set corresponding to segmented images with an image size of n1×n1, the number of positive segmented images P and the number of negative segmented images Q in that segmented image set can be counted. Then, based on the number of positive segmented images P and the number of negative segmented images Q, the ratio of positive to negative segmented image samples in that segmented image set can be determined. When r∈[r1,r2], the number of positive and negative samples in the cut image set is considered to be balanced, and no sampling processing is required. When r∈(0,r1), the number of negative samples in the cut image set is considered to be much greater than the number of positive samples, resulting in an imbalance. In this case, the negative samples in the cut image set can be downsampled, and the resulting negative samples can be added to the cut image set to obtain the sampled cut image set. Similarly, when r∈(r2,∞), the number of negative samples in the cut image set is considered to be much less than the number of positive samples, resulting in an imbalance. In this case, the negative samples in the cut image set can be upsampled, and the resulting negative samples can be added to the cut image set to obtain the sampled cut image set. It should be noted that r described above can belong to the interval [0.8,1.2], etc., and this application embodiment does not specifically limit it. Furthermore, the described positive samples of the cut image can be understood as pixels marked as roads, and the negative samples of the cut image can be understood as pixels marked as non-roads, such as pixels in the image background, etc. This application does not impose specific limitations.

[0047] Thus, after obtaining the sampled and segmented image set, principal component analysis (PCA) can be performed on the segmented images in the sampled and segmented image set to obtain the first image. It should be noted that the described principal component analysis (PCA), also known as principal component transformation, can construct a mapping from high-dimensional to low-dimensional data. For example, it can filter out most of the redundant information in an n-dimensional multispectral remote sensing image to obtain an m-dimensional multispectral remote sensing image that preserves spectral information as much as possible. PCA processing refers to the statistical analysis process of transforming a set of potentially correlated variables into a set of linearly uncorrelated variables (called principal components) through orthogonal transformation. Different bands in multispectral remote sensing images usually exhibit high correlation. The application of principal component transformation in remote sensing images aims to remove redundant information between bands, greatly reduce the total amount of data, and enhance image information.

[0048] Schematic, the aforementioned process of performing principal component analysis (PCA) on the sampled segmented image set to obtain the first image can be implemented as follows: Obtain the image matrix corresponding to the sampled segmented image set, where the image matrix indicates the arrangement of the segmented images in the sampled segmented image set; calculate the mean vector and covariance matrix corresponding to the image matrix, as well as the orthogonal matrix corresponding to the covariance matrix; perform principal component transformation (PCA) on the sampled segmented image set based on the mean vector and the orthogonal matrix to obtain the PCA-transformed image; and perform inverse principal component transformation (IPM) on the PCA-transformed image to obtain the target principal components of the segmented images in the sampled segmented image set, thus obtaining the first image.

[0049] In this example, assume there are T images in the sampled cut image set, each with a size of k = M × H, where M is the number of rows and H is the number of columns. By superimposing these T images, the arrangement of the cut images in the sampled cut image set can be obtained. For example, a two-dimensional image matrix X can be used to represent the sampled cut image set, specifically, X = [X1, ..., X...]. K ], where X i =[X 1i ... X ni ], X i It is represented as the set of all pixel values ​​at position i.

[0050] At this point, the mean vector m of the image matrix X is calculated. x The covariance matrix C x Furthermore, after calculating the covariance matrix C... x Then, calculate the covariance matrix C. x The corresponding orthogonal matrix A is the eigenvector matrix. Furthermore, it can be based on the mean vector m. x And the orthogonal matrix A performs principal component transformation on the sampled segmented sample set, that is, it performs principal component transformation on the image matrix X to obtain the principal component transformed image V = A(Xm). x Then, the principal component transform image V is subjected to inverse principal component transform to obtain the target principal components of the sampled segmented image set, such as... This allows for the compression of information from these T images, resulting in the first image.

[0051] In some examples, before performing principal component analysis (PCA) on the sampled segmented image set to obtain the first image, each segmented image in the sampled image set can be denoised based on a preset denoising model to obtain a denoised segmented image. Then, the denoised segmented image is processed again using the aforementioned principal component analysis (PCA) to obtain the first image.

[0052] It should be noted that the described preset denoising model may include, but is not limited to, graphic wavelet denoising algorithms, etc., and this application does not limit the description. Taking the preset denoising model as a graphic wavelet denoising algorithm as an example, this graphic wavelet denoising algorithm is a new method of wavelet denoising. Since the actual noise of an image cannot be predicted in advance, and wavelet denoising algorithms have a good denoising effect on unknown noise, wavelet denoising algorithms can be selected to denoise the image. In wavelet denoising algorithms, in order to correctly distinguish between image information and noise, threshold denoising is often used to distinguish between image information and noise. The graphic wavelet denoising algorithm is an adaptive threshold wavelet denoising method. Although the actual noise of an image cannot be predicted in advance, most image noise is Gaussian noise. Therefore, for each segmented image X in the sampled segmented image set X, Gaussian white noise with a mean of 0 and a standard deviation of 20 is first added. In this way, performing L-level GWT decomposition on the image after adding Gaussian white noise can obtain the noisy GWT transformed image Y. In this way, the threshold t of the transformed image Y can be calculated based on the noise variance and the image size of the segmented image, i.e. Where σ represents the noise variance, M×H is the image size, and l is the layer where the wavelet decomposition transform coefficients are located (l=1, 2, …, L). Then, the threshold t0 used to distinguish noise and image information can be calculated using this threshold t, i.e., t0=s×t. Thus, based on the determined t and t0, the wavelet coefficients of each sub-band of the transformed image Y are thresholded, and the thresholded coefficients are reconstructed using GWT-based image reconstruction to obtain the denoised segmented image.

[0053] It should be noted that the determination of s described above is within the interval [0,1], with a step size of n0, and the optimal size of s is determined based on the denoising effect. Furthermore, the described noise variance can be obtained based on the independent normal variable decision theory; for example, the commonly used noise variance σ can be estimated as σ=(|w j,k The values ​​such as |) / 0.6745 are not specifically limited in the embodiments of this application. For example, after obtaining the denoised segmented image, objective indicators such as root mean square error (RMSE) and peak signal-to-noise ratio (PSNR), which can be used to evaluate the similarity between the denoised segmented image and the original segmented image, can also be used to evaluate the quality of the denoised segmented image. For instance, the larger the PSNR value, the better the similarity between the denoised segmented image and the original segmented image, and the better the denoising effect. The smaller the RMSE value, the more similar the denoised segmented image is to the original segmented image, and the better the denoising effect.

[0054] 203. Take the first image as input to N target base classification models to obtain N first classification results obtained by the N target base classification models for the first image. Each of the N target base classification models is a machine learning model obtained by iteratively training image samples with image samples of a preset image size as training data, with N≥2 and N being an integer.

[0055] In this example, each of the N target base classification models described is a machine learning model obtained by iteratively training image samples with the predicted classification results of the image samples as the training target and image samples of a preset image size as the training data.

[0056] In other words, after obtaining the multispectral remote sensing data samples and their corresponding labels, the preprocessing methods described in step 202 above can be used to segment, sample, perform wavelet denoising (optional), and principal component analysis (PCA) on the multispectral remote sensing data samples and their corresponding labels to obtain an image sample set. These image sample sets can be understood as segmented sample sets of different image sizes, such as: n1×n1, n2×n2, n3×n3, n... N ×n N A sample set of images of the same size is segmented. Then, these segmented sample sets are further divided according to the ratio of training set:validation set:test set = z1:z2:z3. This yields training data for model training, including image samples of the preset image size. It should be noted that manual annotation of the multispectral remote sensing data samples yields road surface annotation vector data; and data format conversion of this road surface annotation vector data provides the corresponding annotation labels for the multispectral remote sensing data samples. Furthermore, the described annotation label format may include, but is not limited to, PNG format.

[0057] After obtaining image samples of a preset image size, a preset processing model can be used to process these samples to obtain a predicted classification result. The difference between the predicted classification result and the preset label is then calculated to obtain the target loss value. This target loss value is then used to iteratively train the initial base classification model, resulting in the target base classification model. It should be noted that the described preset processing model includes, but is not limited to, the HRNet+OCR model framework. The described semantic segmentation model (HRNet) can accept image segments of different sizes as input. It can obtain feature maps of different resolutions through parallel convolutional streams. Then, it uses a context attention (OCR) mechanism to train on training sets of different image sizes. When the accuracy of the trained model or the number of training iterations reaches a preset threshold, the trained model can be tested using a test set. If the test accuracy meets the requirements, N target base classification models are trained, such as model1, model2, ..., modelN. Similarly, if the accuracy of the trained model or the number of training iterations does not reach the preset threshold, iterative training can be performed based on the target loss value until the accuracy of the trained model or the number of training iterations reaches the preset threshold.

[0058] In this way, after training N target base classification models, the first image obtained above can be used as the input of these N target base classification models. These N target base classification models will then make predictions on the first image, thereby obtaining N first classification results obtained by these N target base classification models for the first image.

[0059] 204. Take the N first classification results as input to the target ensemble classification model to obtain the target classification results. The target classification results are used to indicate the classification of road features in the multispectral remote sensing image to be identified. The target ensemble classification model is a machine learning model obtained by iteratively training the N classification results obtained after the N target base classification models predict the image samples, as well as the labeled labels of the image samples.

[0060] In this example, the target ensemble classification model described is a machine learning model obtained by iteratively training N classification results obtained after predicting image samples using N target base classification models, and the labeled images of the samples as training data. It should be noted that the N target base classification models, image samples, and labeled images mentioned can be understood by referring to the content in step 203 above, and will not be repeated here.

[0061] In other words, the classification results obtained from predicting the classification of image samples by N target base classification models, along with the labeled tags of those image samples, can be used as the sample set for training the target ensemble classification model. After obtaining this sample set, these N classification results can be processed based on a pre-defined logistic regression model to obtain N target weights, which indicate the contribution of each classification result. This allows for iterative training of the initial ensemble classification model based on these N target weights. When the accuracy of the trained model or the number of training iterations reaches a preset threshold, the target ensemble classification model is successfully trained. In other words, the prediction results of N different target base classification models (e.g., p1, p2, ... p...) can be used to train the target ensemble classification model. N This can be achieved by using logistic regression to model and analyze the relationship between the predicted results and the corresponding labels, obtaining the weights between multiple prediction results and the final target classification result. These weights can then be used to train an ensemble classification model, resulting in the final target ensemble classification model. For example, the target ensemble classification model can be expressed as y = w1 × p1 + w2 × p2 + w N ×p N , where w i The prediction result p of the classification model for the i-th target base. i The corresponding weights.

[0062] Thus, after training the aforementioned target ensemble classification model, the N first classification results predicted by the N target base classification models obtained in step 203 can be used as input to the target ensemble classification model to identify the target classification result of the multispectral remote sensing image to be identified. This target classification result can indicate the classification of road features in the multispectral remote sensing image to be identified, such as road networks, lane lines, green belts, etc., which are not specifically limited in this application.

[0063] Figure 3 Another schematic flowchart of the method for road feature identification provided in an embodiment of this application is shown.

[0064] like Figure 3 As shown, this example uses three image sizes and three base classification models. In this example, firstly, multispectral remote sensing data samples are acquired, and corresponding road surface annotation vector data is obtained by labeling these samples. Then, the data format of this road surface annotation vector data is converted to obtain the corresponding annotation labels. For details, please refer to the aforementioned... Figure 2 The content described in step 203 is for your understanding and will not be repeated here.

[0065] Then, the multispectral remote sensing data samples and their corresponding labels are segmented to obtain segmented datasets of different image sizes, such as n1×n1, n2×n2, and n3×n3 segmented datasets. These n1×n1, n2×n2, and n3×n3 segmented datasets are then sampled, and the resulting segmented datasets undergo optional wavelet denoising and principal component analysis (PCA) processing to obtain image sample sets. These image sample sets can be understood as segmented sample sets of different image sizes, such as n1×n1, n2×n2, and n3×n3 segmented sample sets. It should be noted that the described wavelet denoising and PCA processing can also be referenced in the preceding sections. Figure 2 The content described in step 202 is for your understanding and will not be repeated here.

[0066] Then, these segmented sample sets are further divided into training, validation, and test sets. Taking an n1×n1 segmented sample set as an example, the HRNet+OCR model framework can be used to train the model on the corresponding training and validation sets. When the accuracy of the trained model or the number of training iterations reaches a preset threshold, the trained model can be tested using the test set. If the test accuracy meets the requirements, the corresponding target base classification model, such as model1, is trained. Similarly, if the accuracy of the trained model or the number of training iterations does not reach the preset threshold, the model can be iteratively trained based on the loss value until the accuracy of the trained model or the number of training iterations reaches the preset threshold. The training process for target base classification models model2 and model3 can be understood by referring to the training process of model1 described above, and will not be elaborated here.

[0067] After training the three target base classification models mentioned above, namely model1, model2, and model3, the obtained multispectral remote sensing data samples can be used as input to these three target base classification models. Each of these models then predicts the multispectral remote sensing data sample, obtaining the corresponding classification results: p1 for model1, p2 for model2, and p3 for model3. By training these three classification results and their corresponding labels using a pre-set logistic regression model, and when the accuracy of the trained model or the number of training iterations reaches a preset threshold, the ensemble classification model for the target is obtained.

[0068] Figure 4 This illustration shows a schematic diagram of an interface for acquiring multispectral remote sensing images according to an embodiment of this application. Figure 4 As shown, on the data upload interface, the multispectral remote sensing image to be identified, such as road test data, is acquired. After preprocessing the acquired multispectral remote sensing image (including cropping, sampling, wavelet denoising (optional), and principal component analysis), the preprocessed image can be input into the three target base classification models trained above, resulting in three classification results. Then, these three classification results are input into the target ensemble classification model to obtain the final target classification result. For example, Figure 5 This diagram illustrates an interface for road feature recognition provided in an embodiment of this application. Figure 5 As shown, after obtaining the aforementioned multispectral remote sensing image to be identified, the target classification result corresponding to the multispectral remote sensing image can be obtained by calling the "Ground Feature Detection - Highway" task model. The described task model can include the target base classification model and the target ensemble classification model trained above. For example, Figure 6 This diagram illustrates the target classification results provided in an embodiment of this application. Figure 6 As can be seen, taking the road network as an example of the target classification result, in the predicted road network ( Figure 6 After viewing the line segments shown in the image, the road network can be visualized on the interface.

[0069] In this embodiment, each of the N target base classification models uses the predicted classification results of image samples as training targets and image samples of a preset image size as training data for iterative training to obtain a machine learning model. After preprocessing the multispectral remote sensing image to be identified to obtain a first image, this first image is used as input to each target base classification model to obtain the first classification result predicted by each model for the first image. Furthermore, since the target ensemble classification model is a machine learning model obtained by iteratively training the N classification results obtained from the predictions of the N target base classification models on the image samples, the obtained N first classification results can be used as input to the target ensemble classification model to predict the target classification result of the multispectral remote sensing image to be identified. In other words, since the acquisition cost of multispectral remote sensing images is much lower than that of hyperspectral remote sensing images, the road feature recognition method provided in this embodiment can be extended to road feature recognition scenarios in multispectral remote sensing images, facilitating the application of remote sensing imagery and reducing application costs. Furthermore, based on the predictions of the target base classification model, the target ensemble classification model is combined to identify road features in multispectral remote sensing images, rather than relying solely on principal component analysis, which greatly improves the classification accuracy during the identification process.

[0070] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. It is understood that to achieve the above functions, corresponding hardware structures and / or software modules are included to execute each function. Those skilled in the art should readily recognize that, based on the modules and algorithm steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0071] This application embodiment can divide the device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0072] The road feature identification device in the embodiments of this application will be described in detail below. Figure 7 This is a schematic diagram of one embodiment of the road feature identification device provided in this application. Figure 7 As shown, the road feature identification device may include an acquisition unit 701 and a processing unit 702.

[0073] The acquisition unit 701 is used to acquire the multispectral remote sensing image to be identified; the processing unit 702 is used to preprocess the multispectral remote sensing image to be identified to obtain a first image; the processing unit 702 is used to use the first image as input to N target base classification models to obtain N first classification results obtained by the N target base classification models for the first image, wherein each of the N target base classification models is a machine learning model obtained by iterative training with the predicted classification results of the acquired image samples as training targets and image samples of a preset image size as training data, where N≥2 and N is an integer; the processing unit 702 is used to use the N first classification results as input to a target ensemble classification model to obtain the target classification result of the multispectral remote sensing image to be identified, wherein the target ensemble classification model is a machine learning model obtained by iterative training with the N classification results obtained by the N target base classification models for predicting the image samples and the labeled labels of the image samples as training data.

[0074] In some optional examples, the processing unit 702 is used to: perform image segmentation processing on the multispectral remote sensing image to be identified according to a preset image size to obtain N segmented image sets, wherein the segmented images in any two segmented image sets do not overlap; perform sampling processing on the corresponding segmented image set according to the ratio of positive and negative segmented image samples in each segmented image set to obtain a sampled segmented image set; and perform principal component analysis (PCA) processing on the sampled segmented image set to obtain a first image.

[0075] In some alternative examples, the acquisition unit 701 is used to acquire the image matrix corresponding to the sampled cut image set, the image matrix being used to indicate the arrangement of the cut images in the sampled cut image set; the processing unit 702 is used to: calculate the mean vector and covariance matrix corresponding to the image matrix, and the orthogonal matrix corresponding to the covariance matrix; perform principal component transformation processing on the sampled cut image set based on the mean vector and the orthogonal matrix to obtain the principal component transformed image; perform inverse principal component transformation processing on the principal component transformed image to obtain the target principal components of the cut images in the sampled cut image set, so as to obtain the first image.

[0076] In some alternative examples, the processing unit 702 is configured to: before performing principal component analysis (PCA) on the sampled cut image set to obtain a first image, perform denoising processing on each cut image in the sampled cut image set based on a preset denoising model to obtain a denoised cut image; and perform principal component analysis (PCA) on the denoised cut image to obtain the first image.

[0077] In some alternative examples, the acquisition unit 701 is further configured to: acquire N classification results and the labeled labels of image samples, wherein the N classification results are the classification results obtained after the N target base classification models predict the image samples respectively; the processing unit 702 is further configured to: process the N classification results and the labeled labels of image samples based on a preset logistic regression model to obtain N target weights, wherein each target weight is used to indicate the degree of contribution of the corresponding classification result; and iteratively train the initial ensemble classification model based on the N target weights to obtain the target ensemble classification model.

[0078] In some alternative examples, the acquisition unit 701 is further configured to: acquire image samples of a preset image size; the processing unit 702 is further configured to: process the image samples of the preset image size based on a preset processing model to obtain the predicted classification result of the image samples; acquire the difference between the predicted classification result of the image samples and the preset label to obtain the target loss value; and iteratively train the initial base classification model based on the target loss value to obtain the target base classification model.

[0079] The road feature recognition device in this application embodiment has been described above from the perspective of modular functional entities. The road feature recognition device in this application embodiment will now be described below from the perspective of hardware processing. Figure 8 This is a schematic diagram of the road feature identification device provided in an embodiment of this application. The road feature identification device can vary considerably due to differences in configuration or performance. The road feature identification device may include at least one processor 801, a communication line 807, a memory 803, and at least one communication interface 804.

[0080] The processor 801 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (server IC), or one or more integrated circuits used to control the execution of the program of the present application.

[0081] Communication line 807 may include a path for transmitting information between the aforementioned components.

[0082] The communication interface 804 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0083] The memory 803 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions. The memory can exist independently and be connected to the processor via communication line 807. The memory can also be integrated with the processor.

[0084] The memory 803 stores computer execution instructions for implementing the scheme of this application, and its execution is controlled by the processor 801. The processor 801 executes the computer execution instructions stored in the memory 803, thereby implementing the road feature identification method provided in the above embodiments of this application.

[0085] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0086] In a specific implementation, as one example, the road feature recognition device may include multiple processors, such as... Figure 8Processors 801 and 802 are described herein. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0087] In a specific implementation, as one embodiment, the road feature recognition device may further include an output device 805 and an input device 806. The output device 805 communicates with the processor 801 and can display information in various ways. The input device 806 communicates with the processor 801 and can receive input from the target object in various ways. For example, the input device 806 may be a mouse, a touch screen device, or a sensor device, etc.

[0088] The aforementioned road feature identification device can be a general-purpose device or a dedicated device. In specific implementations, the road feature identification device can be a server, a terminal, or other similar device. Figure 8 A similar structure to the one described in this application. The embodiments of this application do not limit the type of road feature identification device.

[0089] It should be noted that Figure 8 The processor 801 can invoke computer execution instructions stored in the memory 803 to enable the road feature recognition device to perform actions such as... Figures 2 to 3 The method in the corresponding method embodiment.

[0090] Specifically, Figure 7 The function / implementation process of the processing unit 702 can be achieved through... Figure 8 The processor 801 in the memory calls computer execution instructions stored in the memory 803 to implement the function. Figure 7 The function / implementation process of the acquisition unit 701 can be achieved through... Figure 8 It is implemented using the 804 communication interface.

[0091] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, they can be implemented in whole or in part in the form of a computer program product.

[0098] A computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, they generate, in whole or in part, the processes or functions according to embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., SSDs), etc.

[0099] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for road feature identification, characterized in that, include: Acquire the multispectral remote sensing image to be identified; The multispectral remote sensing image to be identified is segmented according to a preset image size to obtain N segmented image sets, wherein the segmented images in any two of the N segmented image sets do not overlap. For each of the N cut image sets, count the number of positive cut image samples and the number of negative cut image samples in the cut image set; where positive cut image samples are pixels labeled as roads and negative cut image samples are pixels labeled as non-roads. When the number of negative cut image samples is greater than the number of positive cut image samples, the negative cut image samples in the target cut image set are downsampled, and the downsampled negative cut image samples are added to the target cut image set to obtain the sampled cut image set. When the number of negative cut image samples is less than the number of positive cut image samples, the negative cut image samples in the target cut image set are upsampled, and the upsampled negative cut image samples are added to the target cut image set to obtain the sampled cut image set. Principal component analysis (PCA) is performed on the sampled segmented image set to obtain the first image; The first image is used as input to N target base classification models to obtain N first classification results obtained by the N target base classification models for the first image. Each of the N target base classification models is a machine learning model obtained by iteratively training image samples with image samples of a preset image size as training data, with N≥2 and N being an integer. The N first classification results are used as input to the target ensemble classification model to obtain the target classification results, which are then displayed visually on the interface. The target classification results are used to indicate the classification status of road features in the multispectral remote sensing image to be identified. The target ensemble classification model is a machine learning model obtained by iteratively training the N classification results obtained after the N target base classification models predict the image samples and the labeled data of the image samples.

2. The method according to claim 1, characterized in that, The process of performing principal component analysis (PCA) on the sampled segmented image set to obtain a first image includes: Obtain the image matrix corresponding to the sampled cut image set, the image matrix being used to indicate the arrangement of the cut images in the sampled cut image set; Calculate the mean vector and covariance matrix corresponding to the image matrix, as well as the orthogonal matrix corresponding to the covariance matrix; Based on the mean vector and the orthogonal matrix, the sampled segmented image set is subjected to principal component transformation to obtain a principal component transformed image. The principal component transform image is subjected to inverse principal component transform processing to obtain the target principal components of the sampled segmented image set, so as to obtain the first image.

3. The method according to claim 1, characterized in that, Before performing principal component analysis (PCA) on the sampled segmented image set to obtain the first image, the method further includes: Based on a preset denoising model, each segmented image in the sampled segmented image set is denoised to obtain a denoised segmented image. Principal component analysis (PCA) is performed on the sampled segmented image set to obtain the first image, including: The denoised and segmented image is subjected to principal component analysis (PCA) to obtain the first image.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain N classification results and the labeled tags of the image samples. The N classification results are the classification results obtained after the N target base classification models predict the image samples respectively. The N classification results and the labeled images are processed based on a preset logistic regression model to obtain N target weights, where each target weight is used to indicate the degree of contribution of the corresponding classification result. The initial ensemble classification model is iteratively trained based on the N target weights to obtain the target ensemble classification model.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain an image sample of the preset image size; Based on a preset processing model, image samples of the preset image size are processed to obtain the predicted classification results of the image samples; The difference between the predicted classification result and the preset label of the image sample is obtained to obtain the target loss value; The initial base classification model is iteratively trained based on the target loss value to obtain the target base classification model.

6. A road feature identification device, characterized in that, include: The acquisition unit is used to acquire the multispectral remote sensing image to be identified; The processing unit is configured to perform image segmentation processing on the multispectral remote sensing image to be identified according to a preset image size, obtaining N segmented image sets, wherein the segmented images in any two of the N segmented image sets do not overlap; for each of the N segmented image sets, the number of positive segmented image samples and the number of negative segmented image samples in the targeted segmented image set are counted, wherein positive segmented image samples are pixels labeled as roads, and negative segmented image samples are pixels labeled as non-roads; when the number of negative segmented image samples is greater than the number of positive segmented image samples, the negative segmented image samples in the targeted segmented image set are downsampled, and the downsampled negative segmented image samples are added to the targeted segmented image set to obtain a sampled segmented image set; when the number of negative segmented image samples is less than the number of positive segmented image samples, the negative segmented image samples in the targeted segmented image set are upsampled, and the upsampled negative segmented image samples are added to the targeted segmented image set to obtain a sampled segmented image set; principal component analysis (PCA) is performed on the sampled segmented image set to obtain a first image; The processing unit is used to take the first image as input to N target base classification models to obtain N first classification results obtained by the N target base classification models for the first image. Each of the N target base classification models is a machine learning model obtained by iteratively training image samples with image samples of a preset image size as training data, with N≥2 and N being an integer. The processing unit is used to take the N first classification results as input to the target ensemble classification model to obtain the target classification result, and to display the target classification result visually on the interface. The target classification result is used to indicate the classification status of road features in the multispectral remote sensing image to be identified. The target ensemble classification model is a machine learning model obtained by iteratively training the N classification results obtained after the N target base classification models predict the image samples and the labeled labels of the image samples as training data.

7. The apparatus according to claim 6, characterized in that, The processing unit is used to perform denoising processing on each segmented image in the sampled segmented image set based on a preset denoising model before performing principal component analysis (PCA) processing on the sampled segmented image set to obtain the first image, so as to obtain the denoised segmented image. The denoised and segmented image is subjected to principal component analysis (PCA) to obtain the first image.

8. The apparatus according to any one of claims 6 to 7, characterized in that, The acquisition unit is also used to acquire an image sample of the preset image size; The processing unit is further configured to process the image sample of the preset image size based on the preset processing model to obtain the predicted classification result of the image sample; obtain the difference between the predicted classification result of the image sample and the preset label to obtain the target loss value; and iteratively train the initial base classification model based on the target loss value to obtain the target base classification model.

9. A road feature identification device, characterized in that, include: Input / output (I / O) interface, processor, and memory, wherein program instructions are stored in the memory; The processor is used to execute program instructions stored in the memory to perform the method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer device, cause the computer device to perform the method as described in any one of claims 1 to 5.

11. A computer program product, characterized in that, The computer program product includes instructions that, when executed on a computer device, cause the computer device to perform the method as described in any one of claims 1 to 5.

Images