Classification model training and image feature extraction method and apparatus

Abstract

The application discloses a kind of classification model training and image feature extraction method and device.The application includes: the latest training sample set is sequentially acquired, and the corresponding first feature extraction model is trained by the sample image in each acquired training sample set;According to the description value of the corresponding sample image output by each trained first feature extraction model, a target sample set is selected from the current training sample set and the training sample set is updated, to determine the target feature extraction model corresponding to each selected target sample set, the target feature extraction model is the first feature extraction model or the second feature extraction model, and the second feature extraction model is trained from the target sample set;Based on the target feature extraction model and the corresponding target sample set obtained by multiple training, a data classification model is trained.The application solves the problem that the prior art is not suitable for extracting the features of complex scene images, and improves the applicability of the feature extraction method.

Description

Technical Field

[0001] This application relates to the field of image technology, and in particular to a method and apparatus for classification model training and image feature extraction. Background Technology

[0002] With the rapid development of image processing technology, image feature extraction technology has become increasingly mature. Image feature data is widely used in various image processing tasks, such as 3D map reconstruction, object detection, and type recognition. Image feature extraction is one of the important steps in image processing tasks, and effective feature data is key to the success of image processing tasks.

[0003] Currently, when extracting effective feature data from images, the images are not classified; instead, feature data is directly extracted from the images using deep learning models. When the image dataset covers a wide range of data types, this deep learning model can only extract effective feature data from specific image types. The feature data from other images cannot accurately represent the information in the images, affecting the feature extraction results and reducing the success rate of image processing tasks. Therefore, the approach of directly extracting features using deep learning models is not suitable for feature extraction from complex scene images. Summary of the Invention

[0004] This application provides a classification model training and image feature extraction method and apparatus, which solves the problem that the existing technology is not suitable for extracting features from complex scene images, improves the applicability of feature extraction methods, efficiently extracts effective feature data from various complex scene images, and improves the success rate of image processing tasks.

[0005] Firstly, this application provides a method for training a classification model, including:

[0006] The latest training sample set is obtained sequentially, and the corresponding first feature extraction model is trained using the sample images in each obtained training sample set;

[0007] Based on the description value of the corresponding sample image output by the first feature extraction model trained each time, a target sample set is selected from the current training sample set and the training sample set is updated. The target feature extraction model corresponding to the target sample set selected each time is determined. The target feature extraction model is either the first feature extraction model or the second feature extraction model. The second feature extraction model is trained from the target sample set.

[0008] Based on the target feature extraction model obtained through multiple training sessions and the corresponding target sample set, a data classification model is trained.

[0009] Secondly, this application provides an image feature extraction method, including:

[0010] Image data containing features to be extracted is acquired, and the image data is input into a preset data classification model to obtain the classification result output by the data classification model; wherein, the data classification model is trained by the classification model training method described in the first aspect;

[0011] Based on the classification result, a target sample set corresponding to the image data is determined, and the target feature extraction model corresponding to the target sample set is determined as the target feature extraction model corresponding to the image data; wherein, the target feature extraction model is trained by the classification model training method described in the first aspect;

[0012] The image data is input into the corresponding target feature extraction model to obtain the feature information of the image data output by the target feature extraction model.

[0013] Thirdly, this application provides a classification model training device, comprising:

[0014] The first training module is configured to sequentially acquire the latest training sample set and train the corresponding first feature extraction model using the sample images in each acquired training sample set.

[0015] The second training module is configured to select a target sample set from the current training sample set and update the training sample set based on the description value of the corresponding sample image output by the first feature extraction model trained each time, and determine the target feature extraction model corresponding to the target sample set selected each time. The target feature extraction model is either the first feature extraction model or the second feature extraction model, and the second feature extraction model is trained from the target sample set.

[0016] The third training module is configured to train a data classification model based on the target feature extraction model obtained from multiple training sessions and the corresponding target sample set.

[0017] Fourthly, this application provides an image feature extraction apparatus, comprising:

[0018] The data classification module is configured to acquire image data containing features to be extracted, input the image data into a preset data classification model, and obtain the classification result output by the data classification model; wherein the data classification model is trained by the classification model training method described in the first aspect.

[0019] The first model determination module is configured to determine the target sample set corresponding to the image data based on the classification result, and determine the target feature extraction model corresponding to the target sample set as the target feature extraction model corresponding to the image data; wherein, the target feature extraction model is trained by the classification model training method described in the first aspect;

[0020] The first feature extraction module is configured to input the image data into the corresponding target feature extraction model to obtain the feature information of the image data output by the target feature extraction model.

[0021] Fifthly, this application provides an electronic device, comprising:

[0022] One or more processors; a storage device storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the classification model training method as described in the first aspect or the image feature extraction method as described in the second aspect.

[0023] In a sixth aspect, this application provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a classification model training method as described in the first aspect or an image feature extraction method as described in the second aspect.

[0024] This application uses the descriptive values ​​output by the first feature extraction model to divide the sample images from which the first feature extraction model can extract effective feature information into a target sample set. The effective feature information in the sample images in the target sample set is distributed in the distribution area that the first feature extraction model focuses on. Therefore, the first feature extraction model can serve as the target feature extraction model corresponding to the target sample set, used to extract feature information from the distribution area where the effective feature information of the sample images in the target sample set is located. The second feature extraction model is trained based on the target sample set, and it focuses on training to extract features from the distribution area where the effective feature information of the sample images in the target sample set is located, resulting in a more convergent model. For the sample images remaining after the target sample set is selected from the training sample set, the effective feature information of these sample images is distributed in other distribution areas. During the subsequent training of the target feature extraction model, a target feature extraction model that focuses on other distribution areas can be trained, realizing the training of multiple target feature extraction models so that effective feature information can be extracted from various types of image data through multiple target feature extraction models. Since the sample images in the target sample set corresponding to the target feature extraction model belong to the same data type, the target sample set to which a sample image belongs can be used as the label information of that sample image. A data classification model is trained using multiple target sample sets. This allows for the subsequent determination of target sample sets with similar feature information types to the image data based on the data classification model. The effective feature information in the image data is then extracted using the target feature extraction model corresponding to the target sample set, ensuring the reliability of the feature information and improving the success rate of image processing tasks. By using multiple target feature extraction models, effective feature data can be efficiently extracted from images of various complex scenes, improving the applicability of feature extraction methods. Attached Figure Description

[0025] Figure 1 This is a flowchart of a classification model training method provided in an embodiment of this application;

[0026] Figure 2 This is a flowchart illustrating the training of a target feature extraction model using a training sample set, as provided in an embodiment of this application.

[0027] Figure 3 This is a flowchart illustrating the process of dividing the training sample set into batches, as provided in an embodiment of this application.

[0028] Figure 4 This is a flowchart of training a target feature extraction model using a target sample set, provided in an embodiment of this application.

[0029] Figure 5 This is a schematic diagram of the loss values ​​of the first feature extraction model and the second feature extraction model provided in the embodiments of this application;

[0030] Figure 6 This is a flowchart of the data classification model training phase provided in the embodiments of this application;

[0031] Figure 7 This is a flowchart of an image feature extraction method provided in an embodiment of this application;

[0032] Figure 8 This is a flowchart of extracting feature information from an image sequence according to an embodiment of this application;

[0033] Figure 9 This is a schematic diagram of the structure of a classification model training device provided in an embodiment of this application;

[0034] Figure 10 This is a schematic diagram of the structure of an image feature extraction device provided in an embodiment of this application;

[0035] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0037] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0038] The classification model training method provided in this embodiment can be executed by a classification model training device, and the image feature extraction method can be executed by an image feature extraction device. The classification model training device and the image feature extraction device can be the same electronic device or different electronic devices. This electronic device can be implemented through software and / or hardware, and can consist of two or more physical entities, or a single physical entity. For example, the electronic device can be a server or other device with strong computing power, or it can be an intelligent device that collects image data.

[0039] The electronic device is equipped with at least one type of operating system, including but not limited to Android, Linux, and Windows. The electronic device can install at least one application based on the operating system. This application can be a built-in application of the operating system or an application downloaded from a third-party device or server. In this embodiment, the electronic device at least has an application capable of performing classification model training methods and / or image feature extraction methods; therefore, the electronic device itself can also be the application.

[0040] For ease of understanding, this embodiment uses a server as the main entity executing the classification model training method and the image feature extraction method as an example for description.

[0041] In one embodiment, the image processing task is described using the reconstruction of a 3D map of a surveyed area as an example. For instance, an unmanned device collects remote sensing image data of the surveyed area and constructs a 3D map of the area based on the feature information of the remote sensing image data. The accuracy of the feature information in the remote sensing image data directly affects the accuracy of the 3D map. Traditional image feature extraction methods extract feature data from remote sensing image data using deep learning models. Since different types of image data have different effective feature regions, feature extraction methods like deep learning models, which can only extract feature data of a single type, focus primarily on the effective feature regions of that type. This results in only some images in the remote sensing image data having feature data that effectively expresses the image information of the corresponding image. The feature data of the remaining images cannot accurately express the image information because the feature regions focused on by the deep learning model include irrelevant or unimportant information. Therefore, when the surveyed area has a large geographical span and the image dataset covers a wide range of data types, traditional image feature extraction methods suffer from poor feature data extraction results, reducing the success rate of image processing tasks.

[0042] To address the aforementioned issues, this embodiment provides a classification model training method and an image feature extraction method to optimize the feature extraction results and improve the success rate of image processing tasks.

[0043] Figure 1 A flowchart of a classification model training method provided in an embodiment of this application is given. Figure 1 As shown, the specific training method for this classification model includes:

[0044] S110. Sequentially obtain the latest training sample set, train the corresponding first feature extraction model using the sample images in each obtained training sample set, and select the target sample set from the current training sample set based on the description value of the corresponding sample image output by the first feature extraction model each time, and update the training sample set. Determine the target feature extraction model corresponding to each selected target sample set. The target feature extraction model is either the first feature extraction model or the second feature extraction model. The second feature extraction model is trained from the target sample set.

[0045] The training sample set includes multiple sample images, which are image data used to train the target feature extraction model during the training phase. For example, the unmanned equipment can collect remote sensing image data in advance through multiple flight sorties and use this remote sensing image data as sample images. In this embodiment, multiple target feature extraction models are trained in batches, with each batch corresponding to a training sample set. A target feature extraction model is trained based on each batch of training sample sets. The next batch of training sample sets is obtained from the previous batch, so a new training sample set is obtained during each batch of training.

[0046] In one embodiment, a total training sample set is obtained beforehand. After the first batch of training using this total training sample set, a first target feature extraction model is obtained. A target sample set is then selected from the total training sample set. The remaining sample images in the total training sample set are used to update the second batch of training sample sets. A second target feature extraction model is then trained using the updated training sample set, and so on, until the number of target feature extraction models equals a preset number. Here, the target sample set can be considered as a set of sample images of the same data type. The target feature extraction model is a neural network model used to focus on the distribution area of ​​effective feature information of the sample images in the corresponding target sample set. It is understood that the sample images used to train the target feature extraction model in each batch are different, and each batch corresponds to training a target feature extraction model for one image data type. Multiple batches can train target feature extraction models for multiple image data types. Accordingly, during the image feature extraction process, if the data type of the image data to be extracted is the same as the data type of the sample images in a certain target sample set, that is, the distribution area of ​​effective feature information in the image data is the same as the distribution area of ​​effective feature information in the sample images of the same type. When a target feature extraction model extracts feature information from image data using a corresponding target sample set, the target feature extraction model will focus on the region where the effective feature information is located, and thus the target feature extraction model will extract the effective feature information of the image data.

[0047] This embodiment provides two implementation methods for generating target feature extraction models. The first method uses a first feature extraction model trained on a training sample set as the target feature extraction model, and the second method uses a second feature extraction model trained on a target sample set as the target feature extraction model. The specific steps of the two implementation methods are described in the following examples.

[0048] In an embodiment where the first feature extraction model trained on the training sample set is used as the target feature extraction model, Figure 2 This is a flowchart illustrating the training of a target feature extraction model using a training sample set, as provided in an embodiment of this application. Figure 2 As shown, the specific steps for training the target feature extraction model using the training sample set include S1101-S1105:

[0049] S1101. Train the preset first neural network model using sample images from the current training sample set to obtain the current first feature extraction model.

[0050] The first neural network model consists of a convolutional network, which can be used to extract depth features from image data. Each batch of the same first neural network model is trained using a corresponding set of training samples. With the network model structure remaining unchanged, the differences in the network model parameters reflect the differences in the first feature extraction models across each batch. The model parameters of the first feature extraction model specify the model's focus on different regions of the image, thus demonstrating the differences in feature information extracted by different batches of the first feature extraction model.

[0051] For example, in the first batch of training, the total set of training samples is used as the first batch of training sample set D1. Each sample image in the training sample set D1 is initialized, such as by assigning each sample image the same initial weight value, and the first neural network model is trained using the initialized sample images. Specifically, the initialized sample images are input into the first neural network model to obtain the feature information and descriptive values ​​output by the first neural network model. These descriptive values ​​can be loss values ​​or confidence values, etc. The model parameters are adjusted based on the feature information and descriptive values ​​output by the first neural network model to train the first neural network model to focus on the distribution regions of feature information in sample images with higher descriptive values, resulting in the first batch of first feature extraction model M1.

[0052] S1102. Input each sample image in the current training sample set into the current first feature extraction model to obtain the description value of the corresponding sample image output by the current first feature extraction model.

[0053] For example, a sample image is input into a first feature extraction model M1, yielding feature information and descriptive values ​​output by M1. Since M1 focuses on feature information within a specific distribution region of the image, it outputs a higher descriptive value when it extracts valid feature information from this region. Conversely, it outputs a lower descriptive value when it extracts invalid feature information from the same region. Therefore, the descriptive values ​​of the sample image determine whether valid feature information is distributed within the region focused on by the first feature extraction model, and consequently, whether the data type of the sample image corresponds to the data type specified by the first feature extraction model.

[0054] S1103. Select the current target sample set from the current training sample set based on the description value, and update the remaining sample images in the current training sample set to the training sample set for the next training session. The description value of the sample images in the target sample set is greater than the description value of the remaining sample images.

[0055] For example, when the data type of the sample image is the same as that of the first feature extraction model M1, the sample image can be assigned to the first batch of target sample set D1' so that the data classification model can be trained subsequently using the target sample set and the corresponding target feature extraction model. When the data type of the sample image is not the same as that of the first feature extraction model M1, the sample image can be assigned to the second batch of training sample set D2 so that target feature extraction models corresponding to different data types can be trained subsequently.

[0056] In one embodiment, a descriptive value threshold is set, which can be understood as the lowest descriptive value of the sample images in each batch of the target sample set. When the descriptive value of a sample image is greater than the descriptive value threshold, the sample image is determined to belong to the corresponding batch of the target sample set; when the descriptive value of a sample image is less than or equal to the descriptive value threshold, the sample image is determined not to belong to the corresponding batch of the target sample set. For example, the training sample set D1 is divided into a first batch of target sample sets D1' and a second batch of training sample sets D2 based on the descriptive value threshold. In this embodiment, the average descriptive value is used as the descriptive value threshold to divide each batch of the training sample set. Figure 3 This is a flowchart illustrating the process of dividing the training sample set into batches, as provided in an embodiment of this application. For example... Figure 3 As shown, the steps for dividing the training sample set into each batch specifically include S11031-S11032:

[0057] S11031. Calculate the descriptive average value of the current training sample set based on the descriptive value of each sample image in the current training sample set.

[0058] Let's take the confidence value as an example. The confidence value of each sample image in the first batch of training sample set D1 is summed and then divided by the number of sample images to obtain the average confidence value Vavg.

[0059] S11032. Assign sample images with descriptive values ​​greater than the descriptive mean to the current target sample set, and assign sample images with descriptive values ​​less than or equal to the descriptive mean to the training sample set for the next training iteration.

[0060] For example, sample images with confidence values ​​greater than Vavg are assigned to the first batch of target sample set D1', and sample images with confidence values ​​less than or equal to Vavg are assigned to the second batch of training sample set D2.

[0061] In another embodiment, a Gaussian distribution is used to fit the mean mu and variance sigma of the confidence distribution of each sample image in the first batch of training sample set D1. The confidence of the sample image is then calculated to determine whether it falls within the range of the overall distribution (mu-0.5*sigma, mu+0.5*sigma). If it does, the sample image is assigned to the first batch of target sample set D1'; otherwise, it is assigned to the second batch of training sample set D2.

[0062] S1104. Set the current first feature extraction model to the target feature extraction model corresponding to the currently selected target sample set.

[0063] For example, since the sample images in the target sample set are divided based on the descriptive values ​​output by the first feature extraction model, the distribution area where the effective feature information of the sample images in the target sample set is located is exactly the distribution area that the first feature extraction model focuses on. Therefore, the first feature extraction model can be used as the target feature extraction model corresponding to the target sample set, so as to focus on extracting the feature information of the distribution area where the effective feature information of this type of sample image in the target sample set is located, thereby optimizing the feature extraction effect of various data types.

[0064] S1105. When the number of first feature extraction models is less than the preset number, train the next batch of first feature extraction models using the next batch of training sample sets until the number of first feature extraction models equals the preset number.

[0065] Assume the preset quantity is N, where N>1. For example, after obtaining the first batch of the first feature extraction model M1 and the target sample set D1', a second batch of the first feature extraction model M2 is trained using the second batch of training sample set D2, and this second batch of trained first feature extraction model M2 is used as the target feature extraction model. For example, the sample images in the second batch of training sample set D2 are assigned the same weight value, and the first neural network model is trained using the weighted sample images to obtain the second batch of the first feature extraction model M2. The sample images from the second batch of training sample set D2 are input into the second batch of the first feature extraction model M2 to obtain the descriptive values ​​output by the second batch of the first feature extraction model M2. Based on the descriptive values, the second batch of training sample set D2 is divided into the second batch of the target sample set D2' and the third batch of training sample set D3. Similarly, during the training phase of the i-th batch, the first feature extraction model Mi for the i-th batch is trained using the training sample set Di of the i-th batch. The training sample set Di of the i-th batch is then input into the first feature extraction model Mi for the i-th batch. Based on the descriptive values ​​output by the first feature extraction model Mi for the i-th batch, the target sample set Di' for the i-th batch and the training sample set Di+1 for the (i+1)-th batch are determined. When i=N, training stops after obtaining the first feature extraction model MN for the N-th batch and the target sample set DN'.

[0066] In an embodiment where the second feature extraction model trained on the target sample set is used as the target feature extraction model, Figure 4 This is a flowchart illustrating the training of a target feature extraction model using a set of target samples, as provided in an embodiment of this application. Figure 4 As shown, the specific steps for training the target feature extraction model using the target sample set include S1106-S1110:

[0067] S1106. Train the preset first neural network model using sample images from the current training sample set to obtain the current first feature extraction model.

[0068] S1107. Input each sample image in the current training sample set into the current first feature extraction model to obtain the description value of the corresponding sample image output by the current first feature extraction model.

[0069] S1108. Based on the description value, select the current target sample set from the current training sample set, and update the remaining sample images in the current training sample set to the training sample set for the next training session. The description value of the sample images in the target sample set is greater than the description value of the remaining sample images.

[0070] For example, steps S1106-S1108 can refer to steps S1101-S1103. It should be noted that the first feature extraction model in this embodiment is a model used to screen the target sample set, rather than a target feature extraction model.

[0071] S1109. Train the preset first neural network model using the currently selected target sample set to obtain the current second feature extraction model, and set the current second feature extraction model as the target feature extraction model corresponding to the currently selected target sample set.

[0072] For example, a first neural network model is trained using the target sample set Di' of the i-th batch to obtain the second feature extraction model Mi' of the i-th batch, and the second feature extraction model Mi' of the i-th batch is used as the target feature extraction model corresponding to the target sample set of the i-th batch.

[0073] S1110. When the number of second feature extraction models is less than the preset number, obtain the next batch of target sample sets from the latest generated training sample set, and train the next batch of second feature extraction models based on the next batch of target sample sets, until the number of second feature extraction models is equal to the preset number.

[0074] For example, a first feature extraction model Mi for the i-th batch is trained using the training sample set Di of the i-th batch. The sample images from the training sample set Di are then input into the first feature extraction model Mi to obtain the descriptive value output by Mi. Based on this descriptive value, the training sample set Di is divided into a training sample set Di+1 for the (i+1)-th batch and a target sample set Di' for the i-th batch. A second feature extraction model Mi' for the i-th batch is trained using the target sample set Di' for the i-th batch, and this second feature extraction model Mi' is used as the target feature extraction model for the i-th batch. When i=N, training stops after obtaining the second feature extraction model MN' for the N-th batch and the target sample set DN'.

[0075] Figure 5 This is a schematic diagram illustrating the loss values ​​of the first and second feature extraction models provided in the embodiments of this application. For example... Figure 5 As shown, in the target feature extraction models of the i-th batch, the loss value Li of the first feature extraction model Xi is greater than the loss value Li' of the second feature extraction model Xi'. The loss value characterizes the convergence of the model; a larger loss value indicates a weaker convergence, and a smaller loss value indicates a stronger convergence. Therefore, the second feature extraction model Xi' is more convergent than the first feature extraction model Xi, and the second feature extraction model Xi' better describes the feature information of the corresponding target sample set Di'.

[0076] S120. Based on the target feature extraction model obtained through multiple training sessions and the corresponding target sample set, a data classification model is trained.

[0077] The data classification model is a neural network model used to determine the data type of image data. For example, the data classification model includes a convolutional network and a fully connected layer. The convolutional network extracts feature information from the image data and inputs this feature information into the fully connected layer. The fully connected layer then outputs the classification result of the image data based on the feature information.

[0078] In this embodiment, the sample images in the target sample set corresponding to the target feature extraction model belong to the same data type. Therefore, the target sample set to which the sample image belongs can be used as the label information of the sample image. The second neural network model is trained through the sample image and the corresponding label information to obtain the data classification model.

[0079] In one embodiment, Figure 6 This is a flowchart of the data classification model training phase provided in an embodiment of this application. For example... Figure 6 As shown, the specific steps in the data classification model training phase include S1201-S1203:

[0080] S1201. Use the target sample set to which the sample image belongs as the label information of the corresponding sample image.

[0081] S1202. Input the sample image into the preset second neural network model to obtain the classification result of the corresponding sample image output by the second neural network model.

[0082] S1203. Based on the classification results and label information corresponding to the sample images, adjust the parameters of the second neural network model to obtain the data classification model.

[0083] This embodiment uses the target feature extraction model as the second feature extraction model as an example. After training multiple target feature extraction models in batches, the target sample set {D1', D2', ..., DN'} corresponding to each target feature extraction model (M1', M2', ..., MN') is obtained. The data classification model Mc is trained using the sample images in the target sample set.

[0084] In another embodiment, during the training phase of the Nth batch, sample images other than the target sample set DN' are also acquired, and these sample images are assigned to the hard sample set DN+1'. A data classification model Mc is trained using the sample images from the target sample set {D1', D2', ..., DN'} and the hard sample set DN+1'. During the training phase of the data classification model, the sample images are input into the second neural network model to obtain the classification results output by the second neural network model. The classification results and corresponding label information of the sample images are substituted into a preset loss function, and the model parameters of the second neural network model are adjusted according to the loss result output by the loss function. When the second neural network model converges or the number of iterations reaches its upper limit, the training is completed and the data classification model Mc is obtained.

[0085] In this embodiment, the image data is input into the data classification model Mc. The data classification model Mc determines that the image data belongs to the hard sample set DN+1', meaning that the image data does not belong to any target sample set. Therefore, it is determined that there is no valid feature information in the image data, and thus the image data can be considered invalid data, and its feature information is not extracted.

[0086] Based on the above embodiments, the data classification model and target feature extraction model trained by the classification model training method are applied to the image feature extraction method to extract effective feature information from the image data. Figure 7 This is a flowchart of an image feature extraction method provided in an embodiment of this application. (Reference) Figure 7 The image feature extraction method specifically includes:

[0087] S210. Obtain image data of the features to be extracted, input the image data into a preset data classification model, and obtain the classification result output by the data classification model.

[0088] In this embodiment, the data classification model is trained using the classification training method described in steps S110-S120. The image data from which features are to be extracted is remote sensing image data containing the surveyed area, collected by the unmanned equipment during the surveying task. After completing the surveying task, the unmanned equipment transmits the image data to the server via cellular network or wireless communication. Upon receiving the image data, the server executes the image feature extraction method of this embodiment to obtain feature information about the surveyed area from the image data.

[0089] In one embodiment, the classification result output by the data classification model can be a classification value. For example, the data classification model may pre-set five data types, each corresponding to a classification value range, such as [0, 1], (1, 2], (2, 3], (3, 4], and (4, 5). The data type of the image data is determined based on the classification value range in which the data classification model outputs the classification value.

[0090] S220. Determine the target sample set corresponding to the image data based on the classification results, and determine the target feature extraction model corresponding to the target sample set as the target feature extraction model corresponding to the image data.

[0091] In this embodiment, the target feature extraction model is trained using the classification training method described in steps S110-S120.

[0092] In one embodiment, each target sample set corresponds to a classification value range. Based on the classification value range of the fully connected layer output, the target sample set corresponding to the image data is determined, and then the target feature extraction model corresponding to the image data is determined.

[0093] S230. Input the image data into the corresponding target feature extraction model to obtain the feature information of the image data output by the target feature extraction model.

[0094] For example, image data is input into the corresponding target feature extraction model, and the convolutional network in the target feature extraction model extracts depth features from the distribution area of ​​effective feature information of the image data to obtain the effective feature information of the image data.

[0095] Based on the above embodiments, Figure 8 This is a flowchart illustrating the extraction of feature information from an image sequence, as provided in an embodiment of this application. For example... Figure 8 As shown, the steps for extracting feature information from an image sequence specifically include S240-S260:

[0096] S240. Obtain multiple image data from the set of flight images from which features are to be extracted, input the multiple image data into the data classification model, and obtain the classification result output by the data classification model.

[0097] Here, the image set for a sortie refers to the remote sensing image data collected by the unmanned aerial vehicle (UAV) in a single sortie. For example, if the image data collected by the UAV in a sortie are largely similar in content, multiple image data can be selected from this sortie image set. The data type of the sortie image set is determined based on these multiple image data, and the target feature extraction model corresponding to this data type extracts the feature information of all image data from that sortie. These multiple image data are then input into the data classification model Mc to obtain the classification values ​​output by Mc. If the score output by the data classification model corresponds to a set of hard samples, the image data is discarded.

[0098] S250. The classification results of multiple image data are weighted to obtain the classification results of the flight image set. Based on the classification results of the flight image set, the target feature extraction model corresponding to the flight image set is determined.

[0099] For example, the scores of multiple image data are weighted and fused, and the resulting score is used as the classification value of the flight image set. Based on the score range corresponding to this classification value, it is determined that the sample images in the target sample set corresponding to this score range belong to the same data type as the image data in the flight image set, and the target feature extraction model corresponding to this target sample set is determined as the target feature extraction model for the corresponding flight.

[0100] S260. Input each image data in the flight image set into the corresponding target feature extraction model to obtain the feature information output by the target feature extraction model.

[0101] For example, all image data from the flight image set is input into the corresponding target feature extraction model to obtain the feature information of each image data output by the target feature extraction model. This embodiment eliminates the need to determine the data type of each image data, thus improving the efficiency of feature information extraction.

[0102] In summary, the classification model training method and image feature extraction method provided in this application, through the descriptive values ​​output by the first feature extraction model, divide the sample images from which the first feature extraction model can extract effective feature information into a target sample set. The effective feature information in the sample images in the target sample set is distributed in the distribution area that the first feature extraction model focuses on. Therefore, the first feature extraction model can serve as the target feature extraction model corresponding to the target sample set, used to extract feature information from the distribution area where the effective feature information of the sample images in the target sample set is located. The second feature extraction model is trained based on the target sample set, and it focuses on training to extract features from the distribution area where the effective feature information of the sample images in the target sample set is located, making the model more convergent. For the sample images remaining after the target sample set is selected from the training sample set, the effective feature information of these sample images is distributed in other distribution areas. In the subsequent training process of the target feature extraction model, a target feature extraction model that focuses on other distribution areas can be trained, realizing the training of multiple target feature extraction models, so that effective feature information can be extracted from various types of image data through multiple target feature extraction models. Since the sample images in the target sample set corresponding to the target feature extraction model belong to the same data type, the target sample set to which a sample image belongs can be used as the label information of that sample image. A data classification model is trained using multiple target sample sets. This allows for the subsequent determination of target sample sets with similar feature information types to the image data based on the data classification model. The effective feature information in the image data is then extracted using the target feature extraction model corresponding to the target sample set, ensuring the reliability of the feature information and improving the success rate of image processing tasks. By using multiple target feature extraction models, effective feature data can be efficiently extracted from images of various complex scenes, improving the applicability of feature extraction methods.

[0103] Based on the above embodiments, Figure 9 This is a schematic diagram of a classification model training device provided in an embodiment of this application. (Reference) Figure 9 The classification model training device provided in this embodiment specifically includes: a first training module 31, a second training module 32, and a third training module 33.

[0104] The first training module is configured to sequentially acquire the latest training sample set and train the corresponding first feature extraction model using the sample images in each acquired training sample set.

[0105] The second training module is configured to select a target sample set from the current training sample set and update the training sample set based on the description value of the corresponding sample image output by the first feature extraction model trained each time, and determine the target feature extraction model corresponding to the target sample set selected each time. The target feature extraction model is either the first feature extraction model or the second feature extraction model, and the second feature extraction model is trained from the target sample set.

[0106] The third training module is configured to train a data classification model based on the target feature extraction model obtained from multiple training sessions and the corresponding target sample set.

[0107] Based on the above embodiments, the first training module includes: a first model training unit, configured to train a preset first neural network model using sample images in the current training sample set to obtain a current first feature extraction model.

[0108] Based on the above embodiments, the second training module includes: a description value determination unit, configured to input each sample image in the current training sample set into the current first feature extraction model to obtain the description value of the corresponding sample image output by the current first feature extraction model; and a target sample set determination unit, configured to select the current target sample set from the current training sample set according to the description value, and update the remaining sample images in the current training sample set to the training sample set for the next training, wherein the description value of the sample images in the target sample set is greater than the description value of the remaining sample images.

[0109] Based on the above embodiments, the target sample set determination unit includes: a descriptive average determination subunit, configured to calculate the descriptive average of the current training sample set based on the descriptive value of each sample image in the current training sample set; and a sample image partitioning subunit, configured to partition sample images with descriptive values ​​greater than the descriptive average to the current target sample set, and partition sample images with descriptive values ​​less than or equal to the descriptive average to the training sample set for the next training iteration.

[0110] Based on the above embodiments, the second training module includes: a first model setting unit, configured to set the current first feature extraction model as the target feature extraction model corresponding to the currently selected target sample set.

[0111] Based on the above embodiments, the second training module includes: a second model setting unit, configured to train a preset first neural network model using the currently selected target sample set to obtain a current second feature extraction model, and set the current second feature extraction model as the target feature extraction model corresponding to the currently selected target sample set.

[0112] Based on the above embodiments, the third training module includes: a label information determination unit, configured to use the target sample set to which the sample image belongs as the label information of the corresponding sample image; a model classification unit, configured to input the sample image into a preset second neural network model to obtain the classification result of the corresponding sample image output by the second neural network model; and a parameter adjustment unit, configured to adjust the parameters of the second neural network model according to the classification result and label information corresponding to the sample image to obtain a data classification model.

[0113] Based on the above embodiments, Figure 10 This is a schematic diagram of an image feature extraction device provided in an embodiment of this application. (Reference) Figure 10 The image feature extraction device provided in this embodiment specifically includes: a data classification module 41, a first model determination module 42, and a first feature extraction module 43.

[0114] The data classification module is configured to acquire image data containing features to be extracted, input the image data into a preset data classification model, and obtain the classification result output by the data classification model; wherein, the data classification model is trained using the classification model training method described above;

[0115] The first model determination module is configured to determine the target sample set corresponding to the image data based on the classification results, and to determine the target feature extraction model corresponding to the target sample set as the target feature extraction model corresponding to the image data; wherein, the target feature extraction model is trained by the classification model training method described above;

[0116] The first feature extraction module is configured to input image data into the corresponding target feature extraction model to obtain the feature information of the image data output by the target feature extraction model.

[0117] Based on the above embodiments, the image feature extraction device further includes: a multi-image acquisition module, configured to acquire multiple image data from a set of flight images for which features are to be extracted, input the multiple image data into a data classification model, and obtain the classification result output by the data classification model; a second model determination module, configured to perform weighted processing on the classification results of the multiple image data to obtain the classification result of the flight image set, and determine the target feature extraction model corresponding to the flight image set based on the classification result of the flight image set; and a second feature extraction module, configured to input each image data in the flight image set into the corresponding target feature extraction model to obtain the feature information output by the target feature extraction model.

[0118] The classification model training device and image feature extraction device provided in this application, through the descriptive values ​​output by the first feature extraction model, divide the sample images from which the first feature extraction model can extract effective feature information into a target sample set. The effective feature information in the sample images in the target sample set is distributed in the distribution area that the first feature extraction model focuses on. Therefore, the first feature extraction model can serve as the target feature extraction model corresponding to the target sample set, used to extract the feature information of the distribution area where the effective feature information of the sample images in the target sample set is located. The second feature extraction model is trained based on the target sample set, and it focuses on training the feature extraction of the distribution area where the effective feature information of the sample images in the target sample set is located, making the model more convergent. For the sample images remaining after the target sample set is selected from the training sample set, the effective feature information of these sample images is distributed in other distribution areas. In the subsequent training process of the target feature extraction model, a target feature extraction model that focuses on other distribution areas can be trained, realizing the training of multiple target feature extraction models, so that effective feature information can be extracted from various types of image data through multiple target feature extraction models. Since the sample images in the target sample set corresponding to the target feature extraction model belong to the same data type, the target sample set to which a sample image belongs can be used as the label information of that sample image. A data classification model is trained using multiple target sample sets. This allows for the subsequent determination of target sample sets with similar feature information types to the image data based on the data classification model. The effective feature information in the image data is then extracted using the target feature extraction model corresponding to the target sample set, ensuring the reliability of the feature information and improving the success rate of image processing tasks. By using multiple target feature extraction models, effective feature data can be efficiently extracted from images of various complex scenes, improving the applicability of feature extraction methods.

[0119] The classification model training apparatus provided in this application can be used to execute the classification model training method provided in the above embodiments, and has corresponding functions and beneficial effects. Similarly, the image feature extraction apparatus provided in this application can be used to execute the image feature extraction method provided in the above embodiments, and has corresponding functions and beneficial effects. Technical details not described in detail in the above embodiments can be found in the classification model training method or image feature extraction method provided in any embodiment of this application.

[0120] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be a classification model training device or an image feature extraction device. (Reference) Figure 11The electronic device includes a processor 51, a memory 52, a communication device 53, an input device 54, and an output device 55. The electronic device may have one or more processors 51 and one or more memories 52. The processor 51, memory 52, communication device 53, input device 54, and output device 55 can be connected via a bus or other means.

[0121] The memory 52, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the classification model training method or image feature extraction method in any embodiment of this application (e.g., the first training module 31, the second training module 32, and the third training module 33 in the classification model training device, or the data classification module 41, the first model determination module 42, and the first feature extraction module 43 in the image feature extraction device). The memory 52 may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created according to the use of the device, etc. Furthermore, the memory 52 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0122] The communication device 53 is used for data transmission.

[0123] The processor 51 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 52, that is, implementing the above-mentioned classification model training method or image feature extraction method.

[0124] Input device 54 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 55 may include display devices such as a display screen.

[0125] The electronic device provided above can be used to execute the classification model training method or image feature extraction method provided in the above embodiments, and has corresponding functions and beneficial effects.

[0126] This application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the above-described classification model training method or image feature extraction method.

[0127] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0128] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the classification model training method or image feature extraction method described above, but can also execute related operations in the classification model training method or image feature extraction method provided in any embodiment of this application.

[0129] The storage medium and electronic device provided in the above embodiments can execute the classification model training method or image feature extraction method provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the classification model training method or image feature extraction method provided in any embodiment of this application.

[0130] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application. The scope of this application is determined by the scope of the claims.

Claims

1. A classification model training method, characterized in that, Comprise: Obtain the latest training sample set in turn, and train the corresponding first feature extraction model through the sample image in each obtained training sample set; Input each sample image in the current training sample set into the current first feature extraction model to obtain the description value of the corresponding sample image output by the current first feature extraction model; according to the description value, filter out the current target sample set from the current training sample set, and update the remaining sample images in the current training sample set as the training sample set for the next training, the description value of the sample image in the target sample set is greater than that of the remaining sample image, and the description value is a confidence value; determine the first feature extraction model as the target feature extraction model corresponding to the target sample set, or train the corresponding second feature extraction model through the target sample set, and determine the second feature extraction model as the target feature extraction model corresponding to the target sample set; Based on the target feature extraction model obtained through multiple trainings and the corresponding target sample set, a data classification model is trained, and the label information of the sample image in the target sample set is the target sample set to which it belongs.

2. The classification model training method of claim 1, wherein, The method comprises: Train the preset first neural network model through the sample image in the current training sample set to obtain the current first feature extraction model. 3.The method of claim 1, wherein, The method comprises: According to the description value of each sample image in the current training sample set, calculate the description average value of the current training sample set; Divide the sample image with a description value greater than the description average value into the current target sample set, and divide the sample image with a description value less than or equal to the description average value into the training sample set for the next training. 4.The method of any one of claims 1-3, wherein, The method comprises: Train the preset first neural network model through the current filtered target sample set to obtain the current second feature extraction model. 5.The method of any one of claims 1-3, wherein, The method comprises: Input the sample image into the preset second neural network model to obtain the classification result of the corresponding sample image output by the second neural network model; According to the classification result and the label information corresponding to the sample image, adjust the parameters of the second neural network model to obtain the data classification model.

6. An image feature extraction method characterized by, The method comprises: Obtain image data to be extracted features, input the image data into the preset data classification model to obtain the classification result output by the data classification model; wherein the data classification model is trained by the classification model training method of any one of claims 1-5; According to the classification result, a target sample set corresponding to the image data is determined, and a target feature extraction model corresponding to the target sample set is determined as a target feature extraction model corresponding to the image data; wherein the target feature extraction model is obtained by training the classification model training method in any one of claims 1-5; The image data is input into the corresponding target feature extraction model to obtain feature information of the image data output by the target feature extraction model.

7. The image feature extraction method of claim 6, wherein, The image feature extraction method further comprises: A plurality of image data are obtained from the flight image set to be extracted, and the plurality of image data are input into the data classification model to obtain a classification result output by the data classification model; The classification results of the plurality of image data are weighted to obtain a classification result of the flight image set, and a target feature extraction model corresponding to the flight image set is determined according to the classification result of the flight image set; Each image data in the flight image set is input into the corresponding target feature extraction model to obtain feature information output by the target feature extraction model. 8.A device for training a classification model, characterized in that, Comprise: The first training module is configured to obtain a latest training sample set in sequence, and train a corresponding first feature extraction model by sample images in each obtained training sample set; The second training module is configured to input each sample image in the current training sample set into the current first feature extraction model to obtain a description value of the corresponding sample image output by the current first feature extraction model; according to the description value, a current target sample set is screened out from the current training sample set, and the remaining sample images of the current training sample set are updated as training sample sets for next time training, the description value of the sample images in the target sample set is greater than the description value of the remaining sample images, and the description value is a confidence value; the first feature extraction model is determined as a target feature extraction model corresponding to the target sample set, or a corresponding second feature extraction model is trained by the target sample set, and the second feature extraction model is determined as the target feature extraction model corresponding to the target sample set; The third training module is configured to train a data classification model based on the target feature extraction model obtained by multiple times of training and the corresponding target sample set, and the label information of the sample images in the target sample set is the target sample set to which the sample images belong.

9. An image feature extraction apparatus characterized by comprising: Comprise: The data classification module is configured to obtain image data to be extracted, input the image data into a preset data classification model, and obtain a classification result output by the data classification model; wherein the data classification model is obtained by training the classification model training method in any one of claims 1-5; The first model determination module is configured to determine a target sample set corresponding to the image data according to the classification result, and determine a target feature extraction model corresponding to the target sample set as a target feature extraction model corresponding to the image data; wherein the target feature extraction model is obtained by training the classification model training method in any one of claims 1-5; The first feature extraction module is configured to input the image data into a corresponding target feature extraction model to obtain feature information of the image data output by the target feature extraction model.

10. An electronic device, comprising: The method comprises: one or more processors; a storage device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the classification model training method as claimed in any one of claims 1-5 or the image feature extraction method as claimed in any one of claims 6-7.

11. A storage medium containing computer-executable instructions, wherein: The computer executable instructions, when executed by a computer processor, are used to perform the classification model training method as claimed in any one of claims 1-5 or the image feature extraction method as claimed in any one of claims 6-7.

Images