Image classification method, computer terminal and storage medium

Abstract

This application discloses an image classification method, a computer terminal, and a storage medium. The method comprises: obtaining a target image; processing the target image using an image classification model to obtain a classification result corresponding to the target image. The image classification model is obtained by adjusting a pre-trained model using image samples and target samples, where the target samples are preset samples corresponding to image samples in a preset sample set, and the pre-trained model is trained using the preset sample set. This application addresses the technical problem in related technologies of low image classification accuracy due to a small number of data samples and low accuracy of the image classification model.

Description

Technical Field

[0001] The present application relates to the field of model training, and specifically to an image classification method, a computer terminal, and a storage medium. Background Art

[0002] Predicting the progress of construction projects is a crucial task in urban planning. Construction projects are typically categorized into three categories: unstarted, started, and completed. Compared to on-site surveys, predicting the progress of construction projects based on aerial imagery offers the advantages of ease of implementation, low labor costs, and high real-time performance. Specifically, image classification models are used to extract features and classify specific areas in aerial images, thereby determining the construction progress of the designated areas. However, training image classification models requires a large amount of labeled data. Small datasets can cause severe overfitting in deep neural networks, leading to poor generalization and difficulty in training highly accurate image classification models.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide an image classification method, a computer terminal, and a storage medium to at least solve the technical problem in related technologies of low image classification accuracy due to a small amount of data samples and low accuracy of the image classification model.

[0005] According to one aspect of an embodiment of the present application, an image classification method is provided, including: acquiring a target image; processing the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target samples are preset samples corresponding to image samples in a preset sample set, and the pre-trained model is obtained by training through the preset sample set.

[0006] According to another aspect of an embodiment of the present application, an image classification method is also provided, including: obtaining a building image; processing the building image using an image classification model to obtain a classification result of the building contained in the building image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a spot classification sample corresponding to the image sample in a spot classification sample set, and the pre-trained model spot classification sample set is trained.

[0007] According to another aspect of an embodiment of the present application, an image classification method is also provided, including: a cloud server receives a target image uploaded by a client; the cloud server uses an image classification model to process the target image to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in a preset sample set, and the pre-trained model is trained through the preset sample set; the cloud server feeds back the classification result to the client.

[0008] According to another aspect of an embodiment of the present application, an image classification device is also provided, including: a first acquisition module for acquiring a target image; a first processing module for processing the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in a preset sample set, and the pre-trained model is obtained by training through the preset sample set.

[0009] According to another aspect of an embodiment of the present application, an image classification device is also provided, including: a third acquisition module for acquiring a building image; a third processing module for processing the building image using an image classification model to obtain a classification result of the building contained in the building image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a spot classification sample corresponding to the image sample in the spot classification sample set, and the pre-trained model spot classification sample set is trained.

[0010] According to another aspect of an embodiment of the present application, an image classification device is also provided, including: a first uploading module for receiving a target image uploaded by a client; a fourth processing module for processing the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in a preset sample set, and the pre-trained model is trained through the preset sample set; a first feedback module for feeding back the image classification model to the client.

[0011] According to another aspect of the embodiments of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the image classification method in any of the above embodiments.

[0012] According to another aspect of the embodiments of the present application, a computer terminal is further provided, including: a processor and a memory, wherein the processor is configured to run a program stored in the memory, wherein the image classification method in any one of the above embodiments is executed when the program is run.

[0013] In an embodiment of the present application, after acquiring a target image, the target image is processed using an image classification model to obtain a classification result corresponding to the target image, thereby achieving the purpose of improving the accuracy of the image classification model. It is easy to notice that since the target sample is obtained from a preset sample set based on the target image, the similarity between the target sample and the target image is relatively high. Therefore, the target sample can be used as expansion data for training the image classification model, so that the amount of data used to train the image classification model is sufficiently large. Therefore, the accuracy of the image classification model obtained by training is relatively high, which can achieve the purpose of improving the classification accuracy of the target image, thereby solving the technical problem in the related art that the image classification accuracy is relatively low due to the small amount of data samples and the low accuracy of the image classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for an image classification method according to an embodiment of the present application.

[0016] Figure 2 is a flowchart of an image classification method according to Example 1 of the present application;

[0017] Figure 3 is a schematic diagram of an interactive interface according to an embodiment of the present application;

[0018] Figure 4 is a schematic diagram of an optional interactive interface according to an embodiment of the present application;

[0019] Figure 5 This is a flow chart for improving a residual network according to an embodiment of the present application;

[0020] Figure 6 is a flowchart of another image classification method according to Example 1 of the present application;

[0021] Figure 7 is a flowchart of an image classification method according to Example 2 of the present application;

[0022] Figure 8is a flowchart of an image classification method according to Example 3 of the present application;

[0023] Figure 9 is a flowchart of an image classification method according to Example 4 of the present application;

[0024] Figure 10 is a schematic diagram of an image classification device according to Example 5 of the present application;

[0025] Figure 11 is a schematic diagram of an image classification device according to Example 6 of the present application;

[0026] Figure 12 is a schematic diagram of an image classification device according to Example 7 of the present application;

[0027] Figure 13 is a schematic diagram of an image classification device according to Example 8 of the present application;

[0028] Figure 14 is a flowchart of an image classification method according to Example 9 of the present application;

[0029] Figure 15 This is a flowchart of an image classification method according to Example 10 of the present application.

[0030] Figure 16 This is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0034] Map block: It can refer to a single land type plot, or a single land type plot divided by land ownership boundaries or linear features.

[0035] Mask: It can convert different grayscale values ​​into different transparencies and apply it to the layer where it is located, so that the transparency of different parts of the layer will change accordingly. The white color of the mask can represent the selected area, and the black color of the mask can represent the non-selected area.

[0036] With the development of satellite and airborne sensors, remote sensing, aerial imagery, and video data are becoming increasingly accessible. These data are widely used in urban planning and other areas. Therefore, analyzing the information contained in remote sensing and aerial imagery is becoming increasingly important. However, labeling this data requires significant human resources, significantly increasing the cost of utilizing remote sensing data. Furthermore, when applying remote sensing data to different scenarios, the labeling rules vary depending on the task requirements, making it extremely difficult to reuse existing labeled data.

[0037] Predicting the progress of construction projects is a crucial task in urban planning. Construction projects are typically categorized according to their progress: unstarted, started, and completed. Compared to on-site surveys, predicting construction project progress based on aerial imagery offers the advantages of ease of implementation, low labor costs, and high real-time performance. Specifically, an image classification model is used to classify designated areas in aerial imagery, thereby determining the construction progress status of those areas.

[0038] Convolutional Neural Networks (CNN), as a typical deep learning model, are a type of feedforward neural network with a deep structure that includes convolutional calculations. With the continuous advancement of deep learning technology, CNN has made great breakthroughs in the field of image classification. Using CNN to classify aerial images has important academic research value. However, training deep neural networks requires a large amount of labeled data. Small datasets can cause deep neural networks to suffer from serious overfitting problems, which can easily lead to poor generalization ability of the model. To address this problem, the problem of few-shot learning (FSL) was proposed. Few-shot learning aims to achieve good generalization performance using only a small amount of data. Therefore, few-shot learning can help alleviate the difficulties of collecting large-scale supervised data or manual labeling, making deep learning more convenient to use in industrial environments.

[0039] Convolutional neural networks extract roughly the same feature information in the same scene, so the parameters of layers that focus on shallow features such as contours, colors, and shapes can be reused. However, only gradients are passed back to the deep layers and the difference layers to update the parameters. This parameter update method is called fine-tuning. Fine-tuning is one of the most commonly used methods for small-sample learning. On the one hand, a pre-trained model with good feature extraction capabilities is obtained by training the model with large amounts of data. Fine-tuning the pre-trained model on a scene with a small amount of data and similar distribution can greatly improve the generalization ability of the model. On the other hand, the time and resource overhead of training large-scale datasets is large, so fine-tuning becomes one of the important links in the application of algorithms. Common fine-tuning methods usually include the following steps: (1) construct a large-scale dataset; (2) train the model on the large-scale dataset to obtain a pre-trained model; (3) construct a small-scale dataset with a distribution similar to the large-scale dataset; (4) use the parameters of the pre-trained model as initialization parameters, freeze some low-level parameters, and use a smaller learning rate to train on the small-sample dataset, which is fine-tuning. However, the effect of fine-tuning between two datasets with large distribution differences will be significantly reduced.

[0040] The distribution of construction project progress classification differs significantly from that of conventional ground object scene classification. Therefore, it is worthwhile to study how to use large-scale conventional ground object scene classification data to assist in the classification of small-sample construction project progress. However, there is currently little work in this area.

[0041] In order to solve the above problems, the present application provides an image classification method, which expands the training data used to train the model by supplementing auxiliary data, thereby achieving the purpose of training the image classification model with a large amount of training data.

[0042] Example 1

[0043] According to an embodiment of the present application, an image classification method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image classification method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0045] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image classification method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned image classification method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0047] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0048] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0049] Figure 1 The hardware structure block diagram shown can serve not only as an exemplary block diagram of the computer terminal 10 (or mobile device), but also as an exemplary block diagram of the server. In an optional embodiment, the server can be a locally deployed server or a cloud server, connected to one or more clients via a data network connection or electronic connection. The data network connection can be a local area network connection, a wide area network connection, an Internet connection, or other types of data network connections.

[0050] Under the above operating environment, this application provides Figure 2 The image classification method shown. Figure 2 This is a flow chart of an image classification method according to Example 1 of the present application. Figure 2 As shown, the method may include the following steps:

[0051] Step S202: Acquire a target image.

[0052] The target images mentioned above can be remote sensing images, aerial images taken by drones, or radar images, but are not limited to these. In construction project applications, the target images can include images of buildings, such as those before construction begins, images of construction that has already begun, and images of completed buildings. In agriculture and forestry, the target images can include images of uncultivated farmland, images of cultivated land, and images of completed farmland. In disaster scenarios, the target images can include images of cities before, during, or after a disaster.

[0053] In an optional embodiment, the target image can be taken by a satellite or a drone and transmitted to a server via a network, processed by the server, and the target image can be displayed to the user at the same time. Figure 3 As shown, the target image can be displayed in the image acquisition area; in another optional embodiment, the target image can be taken by a satellite or a drone and actively uploaded to the server by the user and processed by the server, such as Figure 4 As shown, users can upload their target image to the server by clicking the "Upload Image" button in the interactive interface or by dragging the target image directly into the dotted box. The uploaded image can then be displayed in the image acquisition area. The server here can be a local server or a cloud server.

[0054] Step S204: Process the target image using the image classification model to obtain a classification result corresponding to the target image.

[0055] The image classification model is obtained by adjusting the pre-trained model through image samples and target samples. The target samples are preset samples corresponding to the image samples in the preset sample set. The image classification model is obtained by training through the preset sample set.

[0056] The above-mentioned image classification model can be a convolutional neural network (CNN). Among them, the convolutional neural network, as a typical deep learning model, is a type of feedforward neural network that includes convolution calculations and has a deep structure.

[0057] In an optional embodiment, the preset sample set can be extracted from various types of tasks previously processed. Optionally, the field of remote sensing image processing encompasses many types of tasks, such as change detection, object classification, and building extraction. Samples suitable for these types of tasks are more versatile and have a larger number of labeled samples. Thus, images from the datasets for these tasks, their corresponding mask information, and their corresponding label information can be extracted to construct a large-scale dataset, which can then be used as the preset sample set.

[0058] The aforementioned preset sample set may be a large number of fine-grained spot classification samples.

[0059] In another optional embodiment, an image classification model can be used to extract features of a target image and features of samples in a preset sample set. By comparing the similarity between the features of the target image and the features of the samples, the sample corresponding to the feature with the higher similarity can be determined from the preset sample set as the target sample. Furthermore, the target image and target sample can be mixed, and the feature extraction module can be modified using a smaller learning rate, thereby obtaining an image classification model with stronger practicality and robustness. Since the image classification model is modified using the target image and target samples with a high similarity to the target image, the resulting image classification model is more targeted for feature extraction of the target image, and the results obtained are more accurate.

[0060] In another optional embodiment, the server processes the target image and the preset sample set through the image classification model, and after determining the target sample corresponding to the target image in the preset sample set, it can be directly displayed to the user for viewing, such as Figure 3 As shown, the target sample can be displayed in the result feedback area; in another optional embodiment, the server processes the target image and the preset sample set through the image classification model, and after determining the target sample corresponding to the target image in the preset sample set, the target sample can be fed back to the user's client through the network, and the client displays it to the user for viewing, such as Figure 4 As shown, the target sample can be displayed in the result feedback area. Further, after the target sample is displayed to the user, the user feedback can be used to verify whether the target sample is correct. If the user believes that the target sample is incorrect, the user can select a target sample from the preset sample set for feedback, such as Figure 3 and Figure 4 As shown, the user can feedback the correct target sample in the result feedback area and upload it to the server, so that the server can correct the image classification model based on the user feedback and the target image.

[0061] For example, taking the construction process of a building construction project as an example, after satellites, drones, or radars capture images of unfinished, already-finished, or completed construction, these images can be sent directly to a server to determine target samples. Alternatively, they can be transmitted to a client, where the user can select and upload the construction images for which target samples are to be determined. After acquiring the construction images, the server can use a pre-trained model to process the construction images and a preset sample set to determine target samples corresponding to the construction images, where the similarity between the target samples and the construction images is greater than a certain value. Furthermore, the pre-trained model can be used to adjust the construction images and target samples to generate an image classification model, thereby improving the accuracy of the image classification model for extracting features from construction images. After obtaining the target samples, the server can directly display the target samples to the user for viewing, or the server can send the target samples to the client, which then displays them to the user. This allows the user to see whether the target samples are highly similar to the construction images and can optimize the image classification model based on user feedback, thereby improving server performance.

[0062] For example, using the plowing process in an agricultural and forestry scenario as an example, after satellites, drones, or radar capture images of uncultivated, cultivated, or completed farmland, these images can be sent directly to a server to identify target samples. Alternatively, they can be transmitted to a client, where the user can select and upload the farm images for which target samples are to be identified. After acquiring the farm images, the server can use a pre-trained model to process the farm images and a preset sample set to identify target samples corresponding to the farm images. The target samples have a similarity greater than a certain value with the farm images. Furthermore, the pre-trained model can be used to adjust the model using the farm images and target samples to generate an image classification model, thereby improving the accuracy of the image classification model for extracting features from farm images. After obtaining the target samples, the server can directly display them to the user, or the server can send them to the client, which then displays them to the user. This allows the user to see whether the target samples are highly similar to the farm images. User feedback can also be used to optimize the image classification model and improve server performance.

[0063] For example, in a disaster application scenario, after satellites, drones, or radar capture images of a city before, during, or after a disaster, these images can be sent directly to a server to identify target samples. Alternatively, they can be transmitted to a client, where the user can select and upload the city images for target sample identification to the server. After acquiring the city images, the server can use a pre-trained model to process the city images and a preset sample set to identify target samples corresponding to the city images. The target samples have a similarity greater than a certain value to the city images. Furthermore, the pre-trained model can be used to adjust the city images and target samples to generate an image classification model, thereby improving the accuracy of the image classification model for extracting features from city images. After obtaining the target samples, the server can directly display them to the user, or send them to the client, which then displays them to the user. This allows the user to determine whether the target samples are highly similar to the city images. User feedback can also be used to optimize the image classification model, thereby improving server performance.

[0064] Through the solution provided by the above embodiment of the present application, after acquiring the target image, the target image is processed using the image classification model to obtain the classification result corresponding to the target image, thereby achieving the purpose of improving the accuracy of the image classification model. It is easy to notice that since the target sample is obtained from a preset sample set based on the target image, the similarity between it and the target image is relatively high. Therefore, the target sample can be used as expanded data for training the image classification model, so that the amount of data used to train the image classification model is large enough. Therefore, the accuracy of the image classification model obtained by training is relatively high, which can achieve the purpose of improving the classification accuracy of the target image, thereby solving the technical problem in the related art that the image classification accuracy is relatively low due to the small amount of data samples and the low accuracy of the image classification model.

[0065] In the above embodiment of the present application, the method also includes: using an image classification model to process the target image and multiple preset samples contained in the preset sample set to obtain a first feature of the target image and a second feature of the multiple preset samples; and determining the target sample based on the similarity between the first feature and the multiple second features.

[0066] Optionally, determining the target sample based on the similarity between the first feature and multiple second features includes: obtaining the similarity between the first feature and each second feature to obtain multiple similarities; and determining a preset sample corresponding to the second feature whose similarity is greater than a preset similarity as the target sample.

[0067] In an optional embodiment, in order to obtain the target sample that is most similar to the target image, cosine similarity can be used as a metric. The target sample can be determined by calculating the cosine similarity between the first feature of the target image and the second features of multiple preset samples. The calculation formula for the cosine similarity can be as follows:

[0068]

[0069] Among them, d AB It can represent the cosine similarity between the first feature and the second feature, A can be the first feature, and B can be the second feature.

[0070] For example, an empty set of data to be supplemented can be set, S = {}, and according to the cosine similarity obtained above, for the target image x i , you can select 200 and preset sample S from the preset sample set i , given the state label of its target image is I pi , added to the supplementary data set, that is, S=S+S i , the pre-training model is corrected by S to improve the accuracy of the image classification model.

[0071] In the above embodiment of the present application, the method also includes: using the target image and target sample to correct the network parameters of the target layer of the pre-trained model, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0072] The target layer may be a fully connected layer for feature recognition or a residual block.

[0073] In an optional embodiment, the target image and the target sample can be mixed to obtain mixed training data, and the target layer in the feature extraction network can be trained using the training data to update the network parameters of the target layer, thereby achieving the purpose of correcting the network parameters of the target layer and improving the accuracy of the image classification model.

[0074] In the above embodiment of the present application, the method also includes: obtaining a preset sample set, wherein the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; using the initial classification model to extract features of the multiple preset samples and the masks corresponding to the multiple preset samples, and performing prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; based on the category recognition results and category labels corresponding to the multiple preset samples, obtaining the loss value of the initial classification model; adjusting the initial classification model based on the comparison result of the loss value and the preset loss value to obtain a pre-trained model.

[0075] The mask described above can convert different grayscale values ​​into different levels of transparency and apply them to the layer it resides on, causing the transparency of different parts of the layer to change accordingly. The white color of the mask can represent the selected area, and the black color can represent the unselected area. The mask described above can be a single-channel mask.

[0076] In an optional embodiment, an image corresponding to a target task in a preset sample can be selected through a mask. For example, if the target task may be to obtain construction progress, an image corresponding to a construction area in a preset sample can be selected through a mask, that is, information about the construction area can be represented by a mask.

[0077] In another optional embodiment, multiple preset samples can be identified through the initial classification model in combination with the mask, and category recognition results corresponding to the multiple preset samples can be obtained. By comparing the category recognition results and the category labels, the loss value of the initial classification model is calculated. The loss value can be used to determine whether the initial classification model has a high accuracy. If the loss value is less than a certain value (that is, the above-mentioned preset loss value), it means that the initial classification model has a high accuracy. At this time, there is no need to continue training the initial classification model to obtain a pre-trained model; if the loss value is greater than a certain value, it means that the initial classification model has a low accuracy. At this time, it is necessary to continue training the initial classification model and adjust the network parameters of the initial classification model.

[0078] In the above embodiment of the present application, the method also includes: using the first two residual blocks in the image classification model to process each preset sample to obtain the sample features of each preset sample; using the convolution layer in the image classification model to process the mask corresponding to each preset sample to obtain the mask features of each preset sample; superimposing the sample features and the mask features to obtain the first image features of each preset sample; using the last two residual blocks in the image classification model to process the first image features to obtain the second image features of each preset sample; using the output layer in the image classification model to process the second image features to obtain the category recognition results of each preset sample.

[0079] In an optional embodiment, in order to integrate the information corresponding to the target task into the image classification model, the first two residual blocks in the image classification model can be used to process each preset sample to obtain the features of each preset sample, so as to be later fused with the mask features corresponding to the target task; then the convolution layer in the image classification model is used to process the mask corresponding to each preset sample. Specifically, the corresponding mask can be adjusted according to the feature size of the preset sample to obtain the mask features of each preset sample; finally, the sample features and the mask features can be superimposed to obtain the first image features of each preset sample, wherein the first image features are features containing target task information. The last two residual blocks in the image classification model can be used to perform a global pooling operation on the first image features, so that a layer of full connection is mapped to the category space, and a classifier (softmax) is used to obtain the probability corresponding to each category feature, that is, the above-mentioned category recognition result.

[0080] The above-mentioned target task may be to obtain construction area information, and the first image feature may include the construction area information.

[0081] For example, in order to integrate the construction area information into the feature extraction network, a mask information fusion branch can be added on the basis of the residual convolutional neural network (ResNeSt50). First, the mask size is adjusted to align with the output features of the second residual block using the nearest neighbor method, and then a single convolution operation is used to expand the single-channel mask to 512 dimensions to obtain the mask feature. Finally, the mask feature is added to the sample feature and input into the subsequent network.

[0082] like Figure 5As shown, improvements can be made based on ResNeSt50. In order to integrate construction area information into the network, a single-channel mask can be used to represent the construction area information. The mask is adjusted to the same size as the output feature of the second residual block using the nearest neighbor difference. Then, a convolution operation with a kernel size of 3*3 is used to map the single-channel mask to 512 channels. The obtained mask feature is added to the sample feature to obtain the second image feature containing the area information. The obtained second image feature is sent to the subsequent operation. The image feature obtained by the fourth residual block is then subjected to the global pooling operation and mapped to the category space using a fully connected layer. The probability corresponding to each category is obtained using softmax, which is the category recognition result mentioned above.

[0083] In the above embodiment of the present application, before using the image classification model to process multiple preset samples and the masks corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, the method also includes: using a bilinear interpolation algorithm to adjust the size of each preset sample to a first preset size; using a nearest neighbor difference algorithm to adjust the size of the mask corresponding to each preset sample to a second preset size, wherein the second preset size is the same as the size of the sample feature of each preset sample.

[0084] The first preset size and the second preset size can be set arbitrarily. For example, the first preset size can be set to 600*600, and the second preset size can be set to 600*600.

[0085] In an optional embodiment, bilinear interpolation can be used to adjust the size of the preset sample to 600*600, and correspondingly, the mask size can be adjusted to 600*600 using the nearest neighbor difference, and the feature extraction network can be trained using multiple preset samples after resizing and the resized mask.

[0086] In another optional embodiment, the preset samples can be divided into a training set and a test set in a ratio of 4:1, wherein the training set is used to train the feature extraction network, and the test set is used to test the feature extraction network. Specifically, during the process of training the feature extraction network using the training set, the cross-entropy loss function can be used to adjust the parameters of the network so that the parameters of the image classification model can achieve a high classification accuracy during verification, thereby obtaining a better image classification model.

[0087] For example, the pre-set samples can be divided into a training set and a test set in a 4:1 ratio. Model parameters can be adjusted based on the image classification model's performance on the test set to obtain a more optimal image classification model. Because both the image classification data and the construction project data are remote sensing data, the image classification model's ability to extract shallow features based on the image classification data can be well generalized to the construction project data.

[0088] In the above embodiment of the present application, obtaining the preset sample set includes: determining the target size of the target image; and filtering the preset sample set from the data set based on the target size, wherein the sizes of the multiple preset samples included in the preset sample set are the same as the target size.

[0089] In an optional embodiment, by determining the target size of the target image and screening a preset sample set from the data set, the obtained preset sample set can maintain similarity with the target image, and the amount of data for subsequent calculation of feature similarity can be reduced. The image classification model can then be corrected through the preset sample set, so that the obtained image classification model has higher accuracy.

[0090] Because the dataset contains a sufficient number of categories, the features extracted by the image classification model in the dataset contain information that distinguishes most land object categories. The convolution and global pooling layers of the image classification model can be used to extract multiple preset samples corresponding to features in the dataset that have the same feature size as the target image, i.e., a preset sample set. The mapping between the preset samples in the preset sample set and the target image is a 2048-dimensional vector.

[0091] For example, the target image can be represented as P = {p1, p2, ..., p n}, where p i ={I pi ,M pi ,l pi}Contains image data I pi , mask data M pi and category label l pi . Use the image classification model F(*) to extract features from the target image P and the preset sample set, that is, x i =F(p i ), y j =F(a j ), where x i and y j Both are 2048-dimensional vectors.

[0092] In the above embodiment of the present application, the method also includes: outputting the target sample; receiving feedback results of the target sample, wherein the feedback results are obtained by modifying the target sample; and correcting the image classification model using the target image and the feedback results.

[0093] In an optional embodiment, since the selection result of the target sample will affect the accuracy of the final image classification model, in order to ensure the accuracy of the image classification model, in an optional embodiment, the server can directly display the target sample to the user for viewing, that is, display the target sample on the interactive interface. In another optional embodiment, the server can send the target sample to the client through the network, and the client displays it to the user for viewing, that is, display the target sample on the interactive interface. Furthermore, the target sample can be confirmed by the user. If the user confirms that it is correct, the image classification model can be directly corrected based on the target sample and the target image; if the user confirms that it is incorrect, the user can adjust the target sample on the interactive interface, obtain the corresponding adjustment result, and feed the adjustment result back to the server. The image classification model can then be corrected based on the adjustment result for the target sample and the target image. Moreover, the image classification model can be optimized based on the adjustment result to improve the performance of the server.

[0094] In the above embodiment of the present application, the method also includes: displaying multiple task types; receiving a target task type selected from the multiple task types; displaying a pre-trained model corresponding to the target task type; receiving image samples, and using the image samples and the target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0095] The multiple task types in the above steps are respectively for different application scenarios, for example, object classification tasks, building detection tasks, building extraction tasks, building change detection tasks, etc., but are not limited thereto.

[0096] As an optional embodiment, task types for multiple application scenarios can be displayed on the interactive interface. The user can operate on the interactive interface and select a task type from multiple task types as the target task type. The server can build a corresponding model based on the user's selection, or search for a corresponding model based on the user's selection and display it to the user for viewing, and then receive the image sample provided by the user. Finally, the image sample and the target sample are fused to obtain the final training sample, and the network parameters of the pre-trained model are adjusted through the training sample to obtain the image classification model corresponding to the target task type.

[0097] Through the above solution, it is possible to provide external applications for model construction in specific scenarios. Users can choose the model training method to use, provide data in the corresponding scenario to train the model, and output the trained model as feedback to the user.

[0098] The following combination Figure 6 A preferred embodiment of the present application is described in detail. The method can be executed by a computer terminal or a server.

[0099] Step S601: introduce sufficient fine-grained pattern classification samples as auxiliary data.

[0100] The above-mentioned fine-grained pattern classification samples may be a preset sample set.

[0101] Optionally, the size distribution of construction project patches can be statistically analyzed, and patch samples with similar sizes can be selected from large-scale datasets for object classification, change detection, and building extraction. Synonymous categories between different datasets can be merged to obtain 34 categories, forming the auxiliary data A = {a1, a2, ..., a n}, where a i ={I ai ,M ai ,l ai}Contains image data I ai , mask data M ai and category label l ai .

[0102] Step S602: Build a deep neural network that can integrate construction area information and image information.

[0103] like Figure 5 As shown, improvements can be made based on ResNeSt. To integrate construction area information into the image classification network, a single-channel mask can be used to represent the construction area information. The mask is adjusted to the same size as the output feature of the second residual block using nearest neighbor interpolation. Then, a convolution operation with a kernel size of 3*3 is used to map the single-channel mask to 512 channels. The resulting mask features are added to the image features to obtain image features containing regional information, which are then fed into subsequent operations. The image features obtained from the fourth residual block are subjected to global pooling and mapped to the category space using a fully connected layer. The probability corresponding to each category is obtained using softmax.

[0104] Step S603: Use the auxiliary data to pre-train the network to obtain a pre-trained model.

[0105] The above-mentioned pre-trained model may be an image classification model.

[0106] Optionally, the deep neural network constructed in step S602 can be pre-trained using the fine-grained image classification samples in step S601. First, use bilinear interpolation to resize the image to 600*600. Correspondingly, use the nearest neighbor difference to resize the mask to 600*600. Divide the auxiliary data A into a training set and a test set in a ratio of 4:1. Use the training set to train the network, using the cross entropy loss function. Adjust the hyperparameters of the model so that the classification accuracy of the model on the validation set is high, and then obtain the pre-trained model F(*).

[0107] Step S604: extract features of construction project samples and fine-grained patch classification samples using the pre-trained model.

[0108] Optionally, the construction project data is represented as P = {p1, p2, ..., p n}, where p i ={I pi ,M pi ,l pi}Contains image data I pi , mask data M pi and category label l pi . Use model F(*) to extract features from construction project data P and pattern classification data A, that is, x i =F(p i ), y j =F(a j ), where x i and y j Both are 2048-dimensional vectors.

[0109] Step S605 , calculating the cosine similarity between the features of the construction project sample and the fine-grained patch classification sample.

[0110] Optional, for any pair {x i ,y j}, the calculation method of cosine similarity is:

[0111]

[0112] Step S606: Select the fine-grained patch classification sample that is most similar to the construction project sample as supplementary data.

[0113] Optionally, an empty set of supplementary data may be set, S = {}, and the cosine similarity D calculated in step S505 is used for each construction project sample feature x i , we select the 200 most similar samples S in the spot classification data i , given its construction status label is I pi , added to the supplementary data set, that is, S=S+Si .

[0114] Step S607: Modify the pre-trained model using construction project samples and supplementary data.

[0115] Optionally, the supplementary data can maintain an approximate data distribution while expanding the construction project data. Therefore, the supplementary data and the original construction project data can be mixed, and the pre-trained model obtained in step S503 is used to fix the first three residual blocks of the model, and a smaller learning rate is used to fine-tune the fourth residual block and the fully connected layer, ultimately obtaining a construction project progress prediction model with strong practicality and robustness.

[0116] Through the above steps, we can use the fine-tuning method in small sample learning to select supplementary data based on the sample feature distance to overcome the problem of differences in data distribution. In addition, we use the regional scope information of the construction project to propose a convolutional neural network structure that integrates regional information and image information, thereby improving the classification accuracy in the construction project progress prediction task.

[0117] The effect of the present invention can be further illustrated by the following experiments:

[0118] Experimental conditions: This experiment uses a scene dataset, pytorch (deep learning framework), and the GPU (Graphics Processing Unit, image processing chip) is configured as NVIDIA Tesla P100.

[0119] Experimental Details: This experiment utilizes 300 construction project patch data and 31,436 fine-grained patch classification data. To accurately evaluate the model's classification accuracy in small sample sizes, we used a five-fold cross-validation method. Specifically, the 300 construction project patch data were randomly divided into five equal parts. One of these parts was selected as the validation set, and the remaining parts were used as the training set. The average classification accuracy of the five folds was used as the evaluation criterion.

[0120] Comparison method: In order to verify the superiority of the proposed method, the following four comparison methods are provided:

[0121] Method 1: directly use construction project patch data to train the classification model;

[0122] Method 2: Use fine-grained patch classification data to pre-train the model and fine-tune the model using only construction project patch data;

[0123] Method 3: Use the fine-grained patch classification data to pre-train the model, and directly use the data matching the construction project categories in the fine-grained patch data as supplementary data to fine-tune the model;

[0124] Method 4 removes the masked convolution and the training process is consistent with the method proposed in this paper.

[0125] Experimental results:

[0126] The average classification accuracy (meanOA) of the five-fold cross validation can be used as the evaluation indicator. The specific results are shown in Table 1 below:

[0127] Table 1

[0128]

[0129] As can be seen from the table, this application can be well applied to the construction project status prediction task, and has higher classification accuracy than the comparison method.

[0130] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

[0131] Example 2

[0132] According to an embodiment of the present application, an image classification method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0133] Figure 7 This is a flow chart of an image classification method according to Example 2 of the present application. Figure 7 As shown, the method may include the following steps:

[0134] Step S702: Acquire a building image.

[0135] Step S704: Process the building image using the image classification model to obtain a classification result of the buildings contained in the building image.

[0136] Among them, the image classification model is obtained by adjusting the pre-training model through image samples and target samples. The target samples are the spot classification samples corresponding to the image samples in the spot classification sample set, and the pre-training model is trained by the spot classification sample set.

[0137] In the above embodiment of the present application, the method further includes: determining a marking method for the building based on the classification result; marking the building according to the marking method to obtain a first marked image; and displaying the first marked image.

[0138] The marking method in the above steps can be the color, line, etc. of different areas in the image, but is not limited thereto.

[0139] In an optional embodiment, in order to facilitate users to view the classification results of buildings, different categories of buildings can be marked using different area division methods and displayed to users for viewing. For example, the area where the buildings of the unconstructed category are located can be green; the area where the buildings of the completed construction category are located can be blue, but it is not limited to this.

[0140] In the above embodiment of the present application, the method further includes: marking the building based on the classification result to obtain a second marked image; and displaying the second marked image.

[0141] In an optional embodiment, in order to facilitate users in determining the categories of different buildings, the classification results may be marked next to the buildings in the building image.

[0142] In the above embodiment of the present application, the method further includes: displaying the classification result; receiving feedback information corresponding to the classification result, wherein the feedback information is obtained by modifying the classification result; and adjusting the image classification model based on the feedback information.

[0143] In an optional embodiment, the classification results can be displayed in an interactive interface so that users can view the classification results of each building. Furthermore, users can confirm the classification results. If the user finds that the classification results are wrong, the classification results can be modified on the interactive interface and the feedback information can be uploaded to the server. The server will optimize the model based on the user's feedback to further improve the accuracy of the image classification model.

[0144] In the above embodiment of the present application, the method also includes: using an image classification model to process a building image and multiple pattern classification samples contained in a pattern classification sample set to obtain a first feature of the building image and a second feature of the multiple pattern classification samples; and determining a target sample based on the similarity between the first feature and the multiple second features.

[0145] Optionally, based on the similarity between the first feature and multiple second features, determining the target sample includes: obtaining the similarity between the first feature and each second feature to obtain multiple similarities; and determining the patch classification sample corresponding to the second feature whose similarity is greater than a preset similarity as the target sample.

[0146] Optionally, the method further includes: using the building image and the target sample to correct the network parameters of the target layer of the pre-trained model, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0147] Optionally, the method also includes: obtaining a set of pattern classification samples, wherein the pattern classification sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; using the initial classification model to extract features of the multiple preset samples and the masks corresponding to the multiple preset samples, and performing prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; based on the category recognition results and category labels corresponding to the multiple preset samples, the loss value of the initial classification model is obtained; based on the comparison result of the loss value and the preset loss value, the initial classification model is adjusted to obtain a pre-trained model.

[0148] Optionally, the method further includes: outputting a target sample; receiving feedback results of the target sample, wherein the feedback results are obtained by modifying the target sample; and correcting the image classification model using the building image and the feedback results.

[0149] In the above embodiment of the present application, the method also includes: displaying multiple task types; receiving a target task type selected from the multiple task types; displaying a pre-trained model corresponding to the target task type; receiving image samples, and using the image samples and the target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0150] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0151] Example 3

[0152] According to an embodiment of the present application, an image classification method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0153] Figure 8 This is a flow chart of an image classification method according to Example 3 of the present application. Figure 8 As shown, the method may include the following steps:

[0154] Step S802: The cloud server receives the target image uploaded by the client.

[0155] In step S804, the cloud server processes the target image using the image classification model to obtain a classification result corresponding to the target image.

[0156] Among them, the image classification model is obtained by adjusting the pre-training model through image samples and target samples. The target samples are preset samples corresponding to the image samples in the preset sample set, and the image classification model is obtained by training the preset sample set.

[0157] In step S806, the cloud server feeds back the classification results to the client.

[0158] Optionally, the method also includes: the cloud server uses an image classification model to process the target image and multiple preset samples contained in the preset sample set to obtain a first feature of the target image and a second feature of the multiple preset samples; the cloud server determines the target sample based on the similarity between the first feature and the multiple second features.

[0159] Optionally, the cloud server determines the target sample based on the similarity between the first feature and multiple second features, including: the cloud server obtains the similarity between the first feature and each second feature to obtain multiple similarities; the cloud server determines that the preset sample corresponding to the second feature whose similarity is greater than the preset similarity is the target sample.

[0160] Optionally, the method also includes: the cloud server uses the target image and target sample to correct the network parameters of the target layer of the pre-trained model, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0161] Optionally, the method also includes: the cloud server obtains a preset sample set, wherein the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; the cloud server uses the initial classification model to extract features of the multiple preset samples and the masks corresponding to the multiple preset samples, and performs prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; the cloud server obtains the loss value of the initial classification model based on the category recognition results and category labels corresponding to the multiple preset samples; the cloud server adjusts the initial classification model based on the comparison result of the loss value and the preset loss value to obtain a pre-trained model.

[0162] Optionally, the method also includes: the cloud server outputs multiple task types; the cloud server receives a target task type selected from the multiple task types; the cloud server outputs a pre-trained model corresponding to the target task type; the cloud server receives image samples, and uses the image samples and target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0163] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0164] Example 4

[0165] According to an embodiment of the present application, an image classification method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0166] Figure 9 This is a flow chart of an image classification method according to Example 4 of the present application. Figure 9 As shown, the method may include the following steps:

[0167] Step S902: The cloud server receives the model training request uploaded by the client.

[0168] Step S904: The cloud server obtains the target image corresponding to the model training request.

[0169] In step S906 , the cloud server processes the target image and the preset sample set using the pre-trained model to determine the target sample corresponding to the target image in the preset sample set.

[0170] Among them, the pre-trained model is obtained by training a preset sample set.

[0171] In step S908, the cloud server adjusts the pre-trained model using the target image and target sample to obtain an image classification model.

[0172] In step S910, the cloud server feeds back the image classification model to the client.

[0173] Optionally, the cloud server uses an image classification model to process the target image and a preset sample set to determine the target sample corresponding to the target image in the preset sample set, including: the cloud server uses an image classification model to process the target image and multiple preset samples contained in the preset sample set to obtain a first feature of the target image and second features of multiple preset samples; the cloud server determines the target sample based on the similarity between the first feature and the multiple second features.

[0174] Optionally, the cloud server determines the target sample based on the similarity between the first feature and multiple second features, including: the cloud server obtains the similarity between the first feature and each second feature to obtain multiple similarities; the cloud server determines that the preset sample corresponding to the second feature whose similarity is greater than the preset similarity is the target sample.

[0175] Optionally, the method also includes: the cloud server uses the target image and target sample to correct the network parameters of the target layer of the pre-trained model, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0176] Optionally, the method also includes: the cloud server obtains a preset sample set, wherein the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; the cloud server uses the initial classification model to extract features of the multiple preset samples and the masks corresponding to the multiple preset samples, and performs prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; the cloud server obtains the loss value of the initial classification model based on the category recognition results and category labels corresponding to the multiple preset samples; the cloud server adjusts the initial classification model based on the comparison result of the loss value and the preset loss value to obtain a pre-trained model.

[0177] Optionally, the method also includes: the cloud server outputs multiple task types; the cloud server receives a target task type selected from the multiple task types; the cloud server outputs a pre-trained model corresponding to the target task type; the cloud server receives image samples, and uses the image samples and target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0178] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0179] Example 5

[0180] According to an embodiment of the present application, an image classification device for implementing the above-mentioned image classification method is also provided. Figure 10 As shown, the device 1000 includes: a first acquisition module 1002 and a first processing module 1004.

[0181] Among them, the first acquisition module is used to acquire the target image; the first processing module is used to process the target image using the image classification model to obtain the classification result corresponding to the target image, wherein the image classification model is obtained by adjusting the pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in the preset sample set, and the pre-trained model is trained through the preset sample set.

[0182] It should be noted that the first acquisition module 1002 and the first processing module 1004 correspond to steps S202 to S204 in Example 1. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0183] In the above embodiment of the present application, the device further includes: a first processing unit and a first determining unit.

[0184] Among them, the first processing unit is used to use the image classification model to process the target image and multiple preset samples contained in the preset sample set to obtain the first feature of the target image and the second features of the multiple preset samples; the first determination unit is used to determine the target sample based on the similarity between the first feature and the multiple second features.

[0185] In the above embodiment of the present application, the first determination unit is further configured to obtain the similarity between the first feature and each second feature, obtain multiple similarities, and determine the preset sample corresponding to the second feature whose similarity is greater than the preset similarity as the target sample.

[0186] In the above embodiment of the present application, the device further includes: a first correction unit.

[0187] The first correction unit is used to correct the network parameters of the target layer of the pre-trained model using the target image and the target sample, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0188] In the above embodiment of the present application, the device further includes: a second acquisition module, a second processing module, a loss module, and a judgment module.

[0189] Among them, the second acquisition module is used to obtain a preset sample set, wherein the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; the second processing module is used to use the initial classification model to extract features of the multiple preset samples and the masks corresponding to the multiple preset samples, and perform prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; the loss module is used to obtain the loss value of the initial classification model based on the category recognition results and category labels corresponding to the multiple preset samples; the judgment module is used to adjust the initial classification model based on the comparison result of the loss value and the preset loss value to obtain a pre-trained model.

[0190] In the above embodiment of the present application, the second processing module includes: a second processing unit, a third processing unit, a superposition unit, a fourth processing unit, and a fifth processing unit.

[0191] Among them, the second processing unit is used to use the first two residual blocks in the image classification model to process each preset sample to obtain the sample features of each preset sample; the third processing unit is used to use the convolution layer in the image classification model to process the mask corresponding to each preset sample to obtain the mask features of each preset sample; the superposition unit is used to superimpose the sample features and the mask features to obtain the first image features of each preset sample; the fourth processing unit is used to use the last two residual blocks in the image classification model to process the first image features to obtain the second image features of each preset sample; the fifth processing unit is used to use the output layer in the image classification model to process the second image features to obtain the category recognition results of each preset sample.

[0192] In the above embodiments of the present application, the device further includes: a first adjustment unit and a second adjustment unit.

[0193] Among them, the first adjustment unit is used to adjust the size of each preset sample to a first preset size using a bilinear interpolation algorithm; the second adjustment unit is used to adjust the size of the mask corresponding to each preset sample to a second preset size using a nearest neighbor difference algorithm, wherein the second preset size is the same as the size of the sample feature of each preset sample.

[0194] In the above embodiment of the present application, the second acquisition module includes: a second determination unit and a first screening unit.

[0195] The second determining unit is used to determine the target size of the target image; the first screening unit is used to screen the data set based on the target size to obtain a preset sample set, wherein the sizes of multiple preset samples included in the preset sample set are the same as the target size.

[0196] In the above embodiment of the present application, the device further includes: a first output module, a first receiving module, and a first correction module.

[0197] Among them, the first output module is used to output the target sample; the first receiving module is used to receive the feedback result of the target sample, wherein the feedback result is obtained by modifying the target sample; the first correction module is used to correct the image classification model using the target image and the feedback result.

[0198] In the above embodiment of the present application, the device further includes: a first display module, a second receiving module, a second display module, and a second correction module.

[0199] Among them, the first display module is used to display multiple task types; the second receiving module is used to receive the target task type selected from the multiple task types; the second display module is used to display the pre-trained model corresponding to the target task type; the second correction module is used to receive image samples, and use image samples and target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0200] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0201] Example 6

[0202] According to an embodiment of the present application, an image classification device for implementing the above-mentioned image classification method is also provided. Figure 11 As shown, the device 1100 includes: a third acquisition module 1102 and a third processing module 1104.

[0203] The third acquisition module is used to acquire a building image; the third processing module is used to process the building image using an image classification model to obtain a classification result of the building contained in the building image.

[0204] It should be noted that the third acquisition module 1102 and the third processing module 1104 correspond to steps S702 to S704 in Example 2. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 2. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0205] In the above embodiment of the present application, the device further includes: a first determination module, a first marking module, and a third display module.

[0206] The first determining module is used to determine a marking method for the building based on the classification result; the first marking module is used to mark the building according to the marking method to obtain a first marked image; and the third display module is used to display the first marked image.

[0207] In the above embodiment of the present application, the device further includes: a second marking module and a fourth display module.

[0208] The second marking module is used to mark the building based on the classification result to obtain a second marked image; and the fourth display module is used to display the second marked image.

[0209] In the above embodiment of the present application, the device further includes: a fifth display module, a third receiving module, and a third correction module.

[0210] The fifth display module is used to display the classification results; the third receiving module is used to receive feedback information corresponding to the classification results, wherein the feedback information is obtained by modifying the classification results; and the third correction module is used to adjust the image classification model based on the feedback information.

[0211] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0212] Example 7

[0213] According to an embodiment of the present application, an image classification device for implementing the above-mentioned image classification method is also provided. Figure 12 As shown, the device 1200 includes: a first uploading module 1202 , a fourth processing module 1204 , and a first feedback module 1206 .

[0214] Among them, the first upload module is used to receive the target image uploaded by the client; the fourth processing module is used to process the target image using the image classification model to obtain the classification result corresponding to the target image, wherein the image classification model is obtained by adjusting the pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in the preset sample set, and the image classification model is trained by the preset sample set; the first feedback module is used to feed back the image classification model to the client.

[0215] It should be noted that the first uploading module 1202, the fourth processing module 1204, and the first feedback module 1206 correspond to steps S802 to S806 in Example 3. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 3. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0216] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0217] Example 8

[0218] According to an embodiment of the present application, an image classification device for implementing the above-mentioned image classification method is also provided. Figure 13 As shown, the apparatus 1300 includes: a second uploading module 1302 , a fourth obtaining module 1304 , a fifth processing module 1306 , a fourth correcting module 1308 , and a second feedback module 1310 .

[0219] Among them, the second uploading module is used for the cloud server to receive the model training request uploaded by the client; the fourth acquisition module is used to obtain the target image corresponding to the model training request; the fifth processing module is used to use the pre-training model to process the target image and the preset sample set, and determine the target sample corresponding to the target image in the preset sample set, wherein the pre-training model is obtained by training the preset sample set; the fourth correction module is used to adjust the pre-training model using the target image and target sample to obtain an image classification model; the second feedback module is used to feed back the image classification model to the client.

[0220] It should be noted that the second uploading module 1302, the fourth obtaining module 1304, the fifth processing module 1306, the fourth correction module 1308, and the second feedback module 1310 correspond to steps S902 to S910 in Example 4. The examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 4. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0221] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0222] Example 9

[0223] According to an embodiment of the present application, an image classification method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0224] Figure 14 This is a flow chart of an image classification method according to Example 9 of the present application. Figure 14 As shown, the method may include the following steps:

[0225] Step S1402: Acquire natural resource images.

[0226] Step S1404: Process the natural resource image using the feature classification model to obtain the feature type corresponding to the natural resource image.

[0227] Among them, the ground feature classification model is obtained by adjusting the pre-training model through image samples and target samples. The target samples are preset samples corresponding to the image samples in the preset sample set, and the pre-training model is trained through the preset sample set.

[0228] It should be noted that the embodiments of the present invention can be applied to, but are not limited to, actual application scenarios of feature classification, where a feature classification model is used to analyze the natural resource image to obtain the feature type corresponding to the natural resource image. For example, feature types such as cultivated land, urban areas, water areas, sea areas, minerals, forest vegetation, deserts, and tropical rainforests can be obtained.

[0229] For example, it can also be applied to the following technical fields: meteorology (for example, cloud extraction, weather forecast, weather warning, etc.); natural resources and ecological environment (for example, weather forecast, change detection, ecological red line change detection, multi-classification change detection, feature classification, greenhouse extraction, road network extraction, building extraction, building change detection (satellite, drone), etc.); water conservancy (for example, water area change detection, greenhouse extraction, water body extraction (optical, radar), forest extraction, cage aquaculture extraction, sand mining site extraction, riverside house extraction, dam extraction, photovoltaic power plant extraction, etc.); agriculture and forestry Industry fields (for example, crop extraction (wheat, rice, potatoes, etc.), drone crop identification (corn, flue-cured tobacco, Job's tears rice, etc.), plot identification, growth monitoring (index calculation), agricultural yield estimation, pest and disease monitoring, planting recommendation push, etc.); secondary disaster fields (for example, disaster monitoring, disaster warning, etc.); life services (travel, food delivery, logistics) fields (for example, travel route planning, travel recommendation push, personnel transfer, price adjustment, etc.); urban planning fields (for example, road network extraction (satellite, drone), building extraction, building change detection (satellite, drone), fire protection, etc.).

[0230] In the above embodiment of the present invention, the method further includes: displaying the feature type; receiving feedback information corresponding to the feature type, wherein the feedback information is obtained by modifying the feature type; and adjusting the feature classification model based on the feedback information.

[0231] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0232] Example 10

[0233] According to an embodiment of the present application, an image classification method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0234] Figure 15 This is a flow chart of an image classification method according to embodiment 10 of the present application. Figure 15 As shown, the method may include the following steps:

[0235] Step S1502: Acquire a building image.

[0236] Step S1504: Process the building image using the change detection model to obtain a detection result of whether the building image has changed.

[0237] Among them, the change detection model is obtained by adjusting the pre-trained model through image samples and target samples. The target sample is a preset sample corresponding to the image sample in the preset sample set, and the pre-trained model is trained through the preset sample set.

[0238] It should be noted that embodiments of the present invention can be applied to, but are not limited to, actual application scenarios for feature classification. For example, in a building feature change detection scenario, a change detection model can be used to detect whether the building feature has changed, thereby obtaining a detection result indicating whether the building image has changed. For example, if a building has changed from existing to wasteland, it indicates that the building in the building image has been demolished, or if a building has changed from wasteland to existing, it indicates that the building in the building image has been newly constructed.

[0239] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0240] Example 11

[0241] The embodiment of the present application can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0242] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0243] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image classification method: obtaining a target image; processing the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting the pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in the preset sample set, and the pre-trained model is trained through the preset sample set.

[0244] Optionally, Figure 16 A structural block diagram of a computer terminal according to an embodiment of the present application. Figure 16 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1602 and a memory 1604.

[0245] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image classification method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image classification method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0246] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the target image; use the image classification model to process the target image to obtain the classification result corresponding to the target image, wherein the image classification model is obtained by adjusting the pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in the preset sample set, and the pre-trained model is trained through the preset sample set.

[0247] Optionally, the above-mentioned processor can also execute the program code of the following steps: using the image classification model to process the target image and multiple preset samples contained in the preset sample set to obtain the first feature of the target image and the second features of the multiple preset samples; based on the similarity between the first feature and the multiple second features, determine the target sample.

[0248] Optionally, the processor may further execute program code of the following steps: obtaining the similarity between the first feature and each second feature to obtain multiple similarities; and determining a preset sample corresponding to the second feature having a similarity greater than a preset similarity as a target sample.

[0249] Optionally, the processor may also execute the program code of the following steps: modifying the network parameters of the target layer of the pre-trained model using the target image and target sample, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0250] Optionally, the processor may also execute the program code of the following steps: obtaining a preset sample set, wherein the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; using the initial classification model to perform feature extraction on the multiple preset samples and the masks corresponding to the multiple preset samples, and performing prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; based on the category recognition results and category labels corresponding to the multiple preset samples, obtaining the loss value of the initial classification model; adjusting the initial classification model based on the comparison result of the loss value and the preset loss value to obtain a pre-trained model.

[0251] Optionally, the processor may also execute the program code of the following steps: using the first two residual blocks in the image classification model to process each preset sample to obtain the sample features of each preset sample; using the convolutional layer in the image classification model to process the mask corresponding to each preset sample to obtain the mask features of each preset sample; superimposing the sample features and the mask features to obtain the first image features of each preset sample; using the last two residual blocks in the image classification model to process the first image features to obtain the second image features of each preset sample; and using the output layer in the image classification model to process the second image features to obtain the category recognition results of each preset sample.

[0252] Optionally, the processor may also execute the program code of the following steps: using a bilinear interpolation algorithm to adjust the size of each preset sample to a first preset size; using a nearest neighbor difference algorithm to adjust the size of the mask corresponding to each preset sample to a second preset size, wherein the second preset size is the same as the size of the sample feature of each preset sample.

[0253] Optionally, the processor may further execute program code of the following steps: determining a target size of a target image; and obtaining a preset sample set from a data set based on the target size, wherein the sizes of multiple preset samples included in the preset sample set are the same as the target size.

[0254] Optionally, the processor may also execute program code for the following steps: outputting a target sample; receiving feedback results of the target sample, wherein the feedback results are obtained by modifying the target sample; and correcting the image classification model using the target image and the feedback results.

[0255] Optionally, the above-mentioned processor can also execute the program code of the following steps: displaying multiple task types; receiving a target task type selected from multiple task types; displaying a pre-trained model corresponding to the target task type; receiving image samples, and using image samples and target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0256] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a building image; use the image classification model to process the building image to obtain the classification result of the building contained in the building image, wherein the image classification model is obtained by adjusting the pre-trained model through image samples and target samples, and the target sample is the pattern classification sample corresponding to the image sample in the pattern classification sample set, and the pre-trained model pattern classification sample set is trained.

[0257] Optionally, the processor may further execute program codes of the following steps: determining a marking method for the building based on the classification result; marking the building according to the marking method to obtain a first marked image; and displaying the first marked image.

[0258] Optionally, the processor may further execute program codes of the following steps: marking the building based on the classification result to obtain a second marked image; and displaying the second marked image.

[0259] Optionally, the processor may further execute program code of the following steps: displaying the classification result; receiving feedback information corresponding to the classification result, wherein the feedback information is obtained by modifying the classification result; and adjusting the image classification model based on the feedback information.

[0260] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the cloud server receives the target image uploaded by the client; the cloud server uses the image classification model to process the target image to obtain the classification result corresponding to the target image, wherein the image classification model is obtained by adjusting the pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in the preset sample set, and the image classification model is trained through the preset sample set; the cloud server feeds back the classification result to the client.

[0261] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the cloud server receives the model training request uploaded by the client; the cloud server obtains the target image corresponding to the model training request; the cloud server uses the pre-trained model to process the target image and the preset sample set, and determines the target sample corresponding to the target image in the preset sample set, wherein the pre-trained model is obtained by training the preset sample set; the cloud server uses the target image and target sample to adjust the pre-trained model to obtain an image classification model; the cloud server feeds back the image classification model to the client.

[0262] It can be understood by those skilled in the art that Figure 16 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 16 It does not limit the structure of the above electronic device. For example, the computer terminal A may also include Figure 16 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 16 Different configurations shown.

[0263] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0264] Example 12

[0265] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image classification method provided in the above embodiment.

[0266] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0267] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: obtaining a target image; processing the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in a preset sample set, and the pre-trained model is trained through the preset sample set.

[0268] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: using an image classification model to process a target image and multiple preset samples contained in a preset sample set to obtain a first feature of the target image and a second feature of the multiple preset samples; and determining the target sample based on the similarity between the first feature and the multiple second features.

[0269] Optionally, the storage medium is further configured to store program code for executing the following steps: obtaining a plurality of similarities by obtaining the similarity between the first feature and each second feature; and determining a preset sample corresponding to the second feature whose similarity is greater than a preset similarity as a target sample.

[0270] Optionally, the above-mentioned storage medium is also configured to store program code for performing the following steps: using the target image and target sample to correct the network parameters of the target layer of the pre-trained model, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

[0271] Optionally, the storage medium is also configured to store program code for executing the following steps: obtaining a preset sample set, wherein the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area; using the initial classification model to extract features of the multiple preset samples and the masks corresponding to the multiple preset samples, and performing prediction processing on the fusion features corresponding to the multiple preset samples to obtain category recognition results corresponding to the multiple preset samples, wherein the fusion feature is obtained by fusing the third feature of the corresponding preset sample with the fourth feature of the corresponding mask, and the category recognition result is the category recognition result of the target area; based on the category recognition results and category labels corresponding to the multiple preset samples, obtaining the loss value of the initial classification model; adjusting the initial classification model based on the comparison result of the loss value and the preset loss value to obtain a pre-trained model.

[0272] Optionally, the storage medium is also configured to store program code for executing the following steps: using the first two residual blocks in the image classification model to process each preset sample to obtain the sample features of each preset sample; using the convolutional layer in the image classification model to process the mask corresponding to each preset sample to obtain the mask features of each preset sample; superimposing the sample features and the mask features to obtain the first image features of each preset sample; using the last two residual blocks in the image classification model to process the first image features to obtain the second image features of each preset sample; and using the output layer in the image classification model to process the second image features to obtain the category recognition results of each preset sample.

[0273] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: adjusting the size of each preset sample to a first preset size using a bilinear interpolation algorithm; adjusting the size of the mask corresponding to each preset sample to a second preset size using a nearest neighbor difference algorithm, wherein the second preset size is the same as the size of the sample feature of each preset sample.

[0274] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: determining the target size of the target image; filtering out a preset sample set from the data set based on the target size, wherein the sizes of multiple preset samples contained in the preset sample set are the same as the target size.

[0275] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: outputting a target sample; receiving feedback results of the target sample, wherein the feedback results are obtained by modifying the target sample; and correcting the image classification model using the target image and the feedback results.

[0276] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: displaying multiple task types; receiving a target task type selected from multiple task types; displaying a pre-trained model corresponding to the target task type; receiving image samples, and using image samples and target samples to adjust the pre-trained model to obtain an image classification model corresponding to the target task type.

[0277] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: acquiring a building image; processing the building image using an image classification model to obtain a classification result of the building contained in the building image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a spot classification sample corresponding to the image sample in the spot classification sample set, and the pre-trained model spot classification sample set is trained.

[0278] Optionally, the storage medium is further configured to store program codes for executing the following steps: determining a marking method for a building based on a classification result; marking the building according to the marking method to obtain a first marked image; and displaying the first marked image.

[0279] Optionally, the storage medium is further configured to store program codes for executing the following steps: marking the building based on the classification result to obtain a second marked image; and displaying the second marked image.

[0280] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: displaying the classification result; receiving feedback information corresponding to the classification result, wherein the feedback information is obtained by modifying the classification result; and adjusting the image classification model based on the feedback information.

[0281] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: the cloud server receives a target image uploaded by the client; the cloud server processes the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target sample is a preset sample corresponding to the image sample in a preset sample set, and the image classification model is trained through the preset sample set; the cloud server feeds back the classification result to the client.

[0282] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: the cloud server receives a model training request uploaded by the client; the cloud server obtains a target image corresponding to the model training request; the cloud server uses a pre-trained model to process the target image and a preset sample set, and determines a target sample corresponding to the target image in the preset sample set, wherein the pre-trained model is obtained by training the preset sample set; the cloud server uses the target image and target sample to adjust the pre-trained model to obtain an image classification model; the cloud server feeds back the image classification model to the client.

[0283] It should be noted that the collection, storage, use, processing, transmission, provision and disclosure of images involved in the above embodiments of this application are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0284] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0285] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0286] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0287] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0288] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0289] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0290] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image classification method, characterized in that: include: Acquire the target image; The target image is processed by an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, and the target sample is a preset sample corresponding to the image sample in a preset sample set, and the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area. The pre-trained model is obtained by training an initial classification model through the category recognition result and category label corresponding to the preset sample, and the category recognition result corresponding to the preset sample is obtained by using the initial classification model to predict the fusion feature corresponding to the preset sample, and the fusion feature is obtained by fusing the feature of the preset sample with the feature of the corresponding mask, and the feature of the preset sample and the feature of the corresponding mask are obtained by using the initial classification model to extract features of the preset sample and the mask corresponding to the preset sample.

2. The method according to claim 1, characterized in that The method further comprises: Using the image classification model, the image sample and a plurality of preset samples included in the preset sample set are processed to obtain a first feature of the image sample and a second feature of the plurality of preset samples; The target sample is determined based on the similarity between the first feature and multiple second features.

3. The method according to claim 2, characterized in that Determining the target sample based on similarities between the first feature and the plurality of second features includes: Obtaining a similarity between the first feature and each second feature to obtain multiple similarities; The preset sample corresponding to the second feature whose similarity is greater than the preset similarity is determined as the target sample.

4. The method according to claim 1, wherein The method further comprises: The network parameters of the target layer of the pre-trained model are adjusted using the image samples and the target samples, wherein the network parameters of other layers in the pre-trained model except the target layer remain unchanged.

5. The method according to claim 1, wherein The method further comprises: Obtaining a loss value of the initial classification model based on the category recognition results and category labels corresponding to the plurality of preset samples; The initial classification model is adjusted based on the comparison result of the loss value and the preset loss value to obtain the pre-trained model.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Display multiple task types; receiving a target task type selected from the plurality of task types; Display the pre-trained model corresponding to the target task type; The image sample is received, and the pre-trained model is adjusted using the image sample and the target sample to obtain the image classification model corresponding to the target task type.

7. An image classification method, characterized in that: include: Acquire building images; The building image is processed using an image classification model to obtain a classification result of the building contained in the building image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, the target samples are spot classification samples corresponding to the image samples in a spot classification sample set, the spot classification sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, the category label is the category label of the target area, the pre-trained model is obtained by training an initial classification model through the category recognition results and category labels corresponding to the preset samples, the category recognition results corresponding to the preset samples are obtained by using the initial classification model to predict the fusion features corresponding to the preset samples, the fusion features are obtained by fusing the features of the preset samples with the features of the corresponding masks, and the features of the preset samples and the features of the corresponding masks are obtained by using the initial classification model to extract features from the preset samples and the masks corresponding to the preset samples.

8. The method according to claim 7, characterized in that The method further comprises: Determining a marking method for the building based on the classification result; Marking the building according to the marking method to obtain a first marked image; The first marker image is displayed.

9. The method according to claim 8, characterized in that The method further comprises: Marking the building based on the classification result to obtain a second marked image; The second marker image is displayed.

10. The method according to any one of claims 7 to 9, characterized in that The method further comprises: displaying the classification results; receiving feedback information corresponding to the classification result, wherein the feedback information is obtained by modifying the classification result; The image classification model is adjusted based on the feedback information.

11. An image classification method, characterized in that: include: The cloud server receives the target image uploaded by the client; The cloud server processes the target image using an image classification model to obtain a classification result corresponding to the target image, wherein the image classification model is obtained by adjusting a pre-trained model through image samples and target samples, and the target sample is a preset sample corresponding to the image sample in a preset sample set, and the preset sample set includes: multiple preset samples, a mask and a category label corresponding to each preset sample, the mask is used to characterize the target area in the corresponding preset sample, and the category label is the category label of the target area. The pre-trained model is obtained by training an initial classification model through the category recognition result and category label corresponding to the preset sample, and the category recognition result corresponding to the preset sample is obtained by using the initial classification model to predict the fusion feature corresponding to the preset sample, and the fusion feature is obtained by fusing the feature of the preset sample with the feature of the corresponding mask, and the feature of the preset sample and the feature of the corresponding mask are obtained by using the initial classification model to extract features from the preset sample and the mask corresponding to the preset sample; The cloud server feeds back the classification result to the client.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the image classification method according to any one of claims 1 to 11.

13. A computer terminal, characterized in that: include: A processor and a memory, wherein the processor is used to run a program stored in the memory, wherein the image classification method according to any one of claims 1 to 9 is executed when the program is run.

Images