Classification method, device, electronic device and storage medium

Abstract

The present invention relates to the field of data classification technology, and provides a classification method, device, electronic device, and storage medium. By inputting an image set collected by an operating device into a preset classification model, the category of the objects in the image set is obtained; then, based on the corresponding images of objects of the same category in the training set of the preset classification model and the image set, it is determined whether the preset classification model is degraded; if the preset classification model is degraded, the preset classification model is optimized, and the optimized preset classification model is used to classify objects in the target image collected by the operating device. Based on the training data of the classification model and the actual data classified by the classification model, it is possible to automatically determine whether the classification model is degraded, that is, whether it is suitable for the actual scene. If the classification model is degraded, it is optimized so that the optimized classification model matches the actual scene to improve the accuracy of classification.

Description

Technical Field

[0001] The present invention relates to the technical field of data classification, and in particular to a classification method, device, electronic device and storage medium. Background Art

[0002] With technological innovations in automated agricultural operations, equipment such as drones and unmanned vehicles have become indispensable tools in agricultural production. Classification models can categorize information collected by unmanned equipment during operations, such as crop classification. However, due to the diverse growing environments and complex variety of crops in real-world scenarios, classification models may not be suitable for actual applications, resulting in low classification accuracy. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a classification method, device, electronic device and storage medium.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, the present invention provides a classification method, comprising:

[0006] Inputting the image set collected by the operating equipment into a preset classification model to obtain the category of the object in the image set;

[0007] Determining whether the preset classification model is degraded based on images corresponding to objects of the same category in the training set of the preset classification model and the image set;

[0008] If the preset classification model is degraded, the preset classification model is optimized, and the optimized preset classification model is used to classify objects in the target image captured by the operating equipment.

[0009] In an optional embodiment, the step of determining whether the preset classification model is degraded based on images corresponding to objects of the same category in the training set of the preset classification model and the image set includes:

[0010] Acquire the training set, where the training set includes a plurality of training images and a category of an object in each training image;

[0011] Obtaining a comparison result for each category based on the training images corresponding to the objects of the same category in the training set and the images corresponding to the objects in the image set;

[0012] According to the comparison result of each category, it is determined whether the preset classification model is degraded.

[0013] In an optional embodiment, the step of obtaining a comparison result for each category based on images corresponding to objects of the same category in the training set of the preset classification model and the image set includes:

[0014] For each category, obtaining a first probability distribution and a second distribution probability of the category based on training images corresponding to objects of the category in the training set and images corresponding to the objects in the image set;

[0015] For each category, the similarity between the first probability distribution and the second probability distribution of the category is calculated, and the similarity is used as the comparison result of the category.

[0016] In an optional embodiment, the step of calculating the similarity between the first probability distribution and the second probability distribution of the category includes:

[0017] Calculating a joint distribution of a first probability distribution and a second probability distribution of the category; the joint distribution includes a plurality of probability intervals and a first frequency corresponding to each of the probability intervals in the first probability distribution and a second frequency corresponding to the second probability distribution;

[0018] For each of the probability intervals, taking the absolute value of the difference between the first frequency and the second frequency corresponding to the probability interval as the distance value of the probability interval, to obtain the distance value of each of the probability intervals;

[0019] Calculate the expected value of the distance value of all probability intervals to obtain the similarity between the first probability distribution and the second probability distribution of the category.

[0020] In an optional embodiment, the step of determining whether the preset classification model is degraded according to the comparison result of each category includes:

[0021] If the similarity of each category is less than the preset threshold, the preset classification model is not degraded;

[0022] If there is a similarity equal to or greater than the preset threshold, the preset classification model is degraded.

[0023] In an optional embodiment, the step of optimizing the preset classification model includes:

[0024] Obtaining a confidence score of an object in each image in the image set;

[0025] The preset classification model is optimized based on an image to be processed obtained from the image set; the confidence level of the object in the image to be processed is less than a preset confidence level.

[0026] In an optional embodiment, the object includes one of the following: crops, obstacles, ground film, and ground.

[0027] In an optional embodiment, the objects are crops and obstacles; and the method further comprises:

[0028] Obtaining the categories of crops and obstacles in the target image;

[0029] Performing semantic segmentation on the target image according to the category of the crops and the category of the obstacles.

[0030] In a second aspect, the present invention provides a classification device, comprising:

[0031] An acquisition module, configured to input the image set collected by the operating equipment into a preset classification model to obtain the category of the object in the image set;

[0032] a determination module, configured to determine whether the preset classification model is degraded based on images corresponding to objects of the same category in the training set of the preset classification model and the image set;

[0033] The optimization module is used to optimize the preset classification model if the preset classification model is degraded, and the optimized preset classification model is used to classify objects in the target image collected by the operating equipment.

[0034] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any one of the aforementioned embodiments is implemented.

[0035] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the aforementioned embodiments.

[0036] The classification method, apparatus, electronic device, and storage medium provided by the embodiments of the present invention obtain the categories of objects in the image set by inputting the image set collected by the operating equipment into a preset classification model. The classification model is then determined to be degraded based on the corresponding images of objects of the same category in the training set and image set of the preset classification model. If the preset classification model is degraded, the preset classification model is optimized, and the optimized preset classification model is used to classify objects in the target image collected by the operating equipment. Based on the training data of the classification model and the actual data classified by the classification model, it is automatically determined whether the classification model is degraded, that is, whether it is suitable for the actual scene. If the classification model is degraded, it is optimized so that the optimized classification model matches the actual scene to improve classification accuracy.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown;

[0040] Figure 2 A schematic diagram of a process flow of a classification method provided by an embodiment of the present invention is shown;

[0041] Figure 3 Another schematic diagram of a process of the classification method provided by an embodiment of the present invention is shown;

[0042] Figure 4 Another schematic diagram of a process of the classification method provided by an embodiment of the present invention is shown;

[0043] Figure 5 An example diagram of a classification method provided by an embodiment of the present invention is shown;

[0044] Figure 6 Another example diagram of the classification method provided by an embodiment of the present invention is shown;

[0045] Figure 7 A functional module diagram of a classification device provided by an embodiment of the present invention is shown.

[0046] Icon: 110 - bus; 120 - processor; 130 - memory; 170 - communication interface; 300 - classification device; 310 - acquisition module; 330 - determination module; 350 - optimization module. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0049] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0050] With the technological innovation of agricultural automation operations, equipment such as drones and unmanned vehicles have become one of the indispensable tools in agricultural production. Classification models can classify information collected by unmanned equipment during operation, such as crop classification and obstacle classification, and the classification results can also be used for semantic segmentation. Since the data set for training the classification model is limited, and the characteristics of the classification objects in actual application scenarios may be more complex, for example, the growth environment of crops is different and their types are more complex. As a result, the classification model may not be applicable to the actual application scenario, resulting in a phenomenon of mismatch with the actual scenario, that is, model degradation, resulting in low classification accuracy. Furthermore, an embodiment of the present invention provides a classification method to solve the above problems.

[0051] Please refer to Figure 1 , is a block diagram of an electronic device provided by an embodiment of the present invention. The electronic device includes a bus 110, a processor 120, a memory 130, and a communication interface 170.

[0052] The bus 110 may be a circuit that connects the above elements to each other and transfers communications (eg, control messages) between the above elements.

[0053] The processor 120 may receive commands from the other components (eg, the memory 130 , the communication interface 170 , etc.) through the bus 110 , may interpret the received commands, and may perform calculations or data processing according to the interpreted commands.

[0054] The processor 120 may be an integrated circuit chip with signal processing capabilities. The processor 120 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0055] The memory 130 may store commands or data received from the processor 120 or other elements (eg, the communication interface 170 , etc.) or commands or data generated by the processor 120 or other elements.

[0056] The memory 130 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0057] The communication interface 170 may be used to communicate signaling or data with other node devices.

[0058] It is understandable that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0059] It is understood that the electronic device provided in the embodiments of the present invention may be an operating device such as an unmanned vehicle; it may also be a smartphone, personal computer, tablet computer, wearable device, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. that is communicatively connected to an operating device such as an unmanned vehicle or drone. The embodiments of the present invention do not impose any limitations on this.

[0060] The above-mentioned electronic device will be used as the execution subject to execute each step of each method provided in the embodiment of the present invention and achieve corresponding technical effects.

[0061] See also Figure 2 , Figure 2 It is a flowchart of a classification method provided by an embodiment of the present invention.

[0062] Step S202: Input the image set collected by the operating equipment into a preset classification model to obtain the category of the object in the image set;

[0063] Among them, the preset classification model is a classification model obtained based on the deep learning model and a preset training set, which is used to classify objects in the image.

[0064] In this embodiment, while operating along a planned route, the drone can capture multiple images of the operating area. The electronic device, in communication with the drone, can obtain the multiple images actually captured by the drone, i.e., an image set. By inputting the image set into a preset classification model, the classification of the objects in the image set can be determined.

[0065] For example, a preset classification model can be used to determine the categories of crops in a field based on images of a field captured by a drone. A preset classification model can also be used to determine the categories of obstacles in a field based on images of a field captured by a drone.

[0066] Step S204, determining whether the preset classification model is degraded based on the images corresponding to the objects of the same category in the training set and image set of the preset classification model;

[0067] It can be understood that the training set of the preset classification model includes categories of objects.

[0068] In this embodiment, for each category, images corresponding to objects of the category can be obtained in the training set, and images corresponding to objects of the category can be obtained in the image set, to obtain two sets of the category, namely the set obtained from the training set and the set obtained from the image set, and to obtain two sets for each category.

[0069] The two sets of each category may be compared to determine whether the classification ability of the preset classification model for each category is reduced, thereby determining whether the preset classification model is degraded.

[0070] Step S206: If the preset classification model is degraded, the preset classification model is optimized, and the optimized preset classification model is used to classify objects in the target image captured by the operating equipment;

[0071] In this embodiment, if the preset classification model is determined to be degraded, it indicates that the current preset classification model does not match the actual application scenario and has low classification accuracy. The preset classification model can be optimized to match the application scenario. The optimized model can then be used to classify objects in target images subsequently captured by the operating equipment, thereby improving classification accuracy.

[0072] Optionally, the images to be processed can be obtained from the image set, and the preset classification model can be trained based on the images to be processed to optimize the preset classification model; the parameters of the preset classification model can also be adjusted according to the preset algorithm to optimize the preset classification model.

[0073] As can be seen from the above design, by inputting the image set collected by the operating equipment into the preset classification model, the categories of the objects in the image set are obtained. Then, based on the corresponding images of objects of the same category in the preset classification model's training set and the image set, it is determined whether the preset classification model is degraded. If the preset classification model is degraded, the preset classification model is optimized and used to classify objects in the target images collected by the operating equipment. Based on the classification model's training data and the actual data classified by the classification model, it is determined whether the classification model is degraded, that is, whether it is suitable for the actual scene. If the classification model is degraded, it is optimized to match the optimized classification model with the actual scene to improve classification accuracy.

[0074] Optionally, the optimized model can be used to classify objects in target images subsequently captured by the operating equipment. If the object is a crop, the optimized preset classification model can accurately determine the crop category in the target image, allowing the operating equipment to perform plant protection operations based on that crop category. If the object is an obstacle, the optimized preset classification model can accurately determine the obstacle category in the target image to plan the flight path of the operating equipment.

[0075] If the object is ground film, the optimized preset classification model can accurately determine the area of ​​the ground film in the target image. Ground film is used to cover vegetation, so the area where the ground film is located is also the area where the vegetation is located. This allows operating equipment such as unmanned vehicles to perform plant protection operations based on the area where the ground film is located. If the object is the ground, the optimized preset classification model can accurately determine the area where the ground is located in the target image. Based on the area where the ground is located, the route of operating equipment such as unmanned vehicles can be planned.

[0076] If the objects are crops and obstacles, the optimized preset classification model can accurately obtain the categories of crops and obstacles in the target image, and perform semantic segmentation on the target image based on the categories of crops and obstacles to obtain the areas of crops and obstacles in the target image, which can be used to set the plant protection operations and flight routes of the operating equipment.

[0077] Optionally, after obtaining the category of crops and the category of obstacles in the target image through a preset classification model, the outline of the crops, that is, the area of ​​the crops in the target image, can be obtained based on the category of the crops, and the outline of the obstacles, that is, the area of ​​the obstacles in the target image, can be obtained based on the category of the obstacles; the areas where the crops and obstacles are located are marked, thereby completing the semantic segmentation of the target image.

[0078] Optionally, the training set may reflect the preset classification capability of the classification model, and the actual classification capability of the classification model may be compared with the preset classification capability to determine whether the classification model is degraded. Furthermore, for the above step S206, the embodiment of the present invention provides a possible implementation method. Figure 3 , wherein step S206 includes the following steps:

[0079] Step S204-1, obtaining a training set, where the training set includes a plurality of training images and a category of an object in each training image;

[0080] In this embodiment, the training set is the data used to train the preset classification model, which includes multiple training images and the category of objects in each training image. The category of objects in the training set can represent the preset classification ability of the preset classification model for each category.

[0081] It should be noted that the training set for the preset classification model will differ depending on the object being classified. For example, if the object is crops, the training images in the training set will include crops. If the object is obstacles, the training images in the training set will also include obstacles. This can be understood as the training set being associated with the object classified by the preset classification model.

[0082] Step S204-3, obtaining a comparison result for each category based on the training images corresponding to the objects of the same category in the training set and the images corresponding to the objects in the image set;

[0083] It is understandable that the categories of objects in the training set can represent the preset classification capabilities of the preset classification model for each category, and the categories of objects in the image set can represent the actual classification capabilities of the preset classification model for each category.

[0084] In this embodiment, for each category, training images corresponding to objects of that category, i.e., a first set, can be obtained from the training set, and images corresponding to objects of that category, i.e., a second set, can be obtained from the image set, to obtain the first set and the second set for each category. By comparing the first set and the second set for each category, a comparison result for each category can be obtained.

[0085] The first set and the second set of a category can respectively represent the preset classification capability and the actual classification capability of the preset classification model for the category. The comparison result of a category can be understood as the comparison result of the preset classification capability and the actual classification capability of the preset classification model for the category.

[0086] Step S204-5, determining whether the preset classification model is degraded based on the comparison results of each category;

[0087] In this embodiment, the comparison result of each category may indicate whether the classification capability of the preset classification model in each category dimension is reduced, and whether the preset classification model is degraded may be determined based on the comparison result of each category.

[0088] By comparing the training images corresponding to objects of the same category in the training set with the corresponding images in the image set, we can obtain comparison results for each category. This allows us to compare the classification model's preset and actual capabilities based on data from real-world scenarios, enabling automated evaluation of the classification model and facilitating its optimization to improve classification accuracy.

[0089] Optionally, for the above step S204-3, the embodiment of the present invention provides a possible implementation method, please refer to Figure 4 , wherein step S204-3 includes the following steps:

[0090] It is understood that by inputting an image set into a preset classification model, the categories and probabilities of the objects in the images can be obtained. The training set can include the categories and probabilities of the objects in each training image. Comparison results for each category can be obtained based on the distribution of probability values ​​for objects of the same category in the training set and the image set.

[0091] For ease of calculation, a normalization function such as softmax or sigmoid can be used to normalize the probability values ​​of objects in the image set and training images respectively so that the probability values ​​belong to [0,1].

[0092] It should be noted that the method for obtaining the comparison results of each category is similar, and the obtaining method is described as follows.

[0093] Step S204-3-1, for each category, obtain a first probability distribution and a second probability distribution of the category based on the training images corresponding to the objects of the category in the training set and the images corresponding to the objects in the image set;

[0094] In this embodiment, for each category, the probability value of the object in each training image in the training set is obtained, resulting in multiple probability values. Based on the multiple probability values ​​and the total number of training images, a first probability distribution for the category in the training set is obtained. Furthermore, in the image set, the probability value of the object in each image in the training set is obtained, resulting in multiple probability values. Based on the multiple probability values ​​and the total number of images, a second probability distribution for the category in the image set is obtained. Thus, a first probability distribution and a second probability distribution are obtained for each category.

[0095] For ease of understanding, an embodiment of the present invention provides an example. For example, if the object is crops, the preset classification model can classify crops into two categories, class 1 and class 2.

[0096] Obtain the first probability distribution of class1 and class2 in the training set respectively. For class1, in the training set, obtain the probability value of the crop in each training image being class1; obtain the first probability distribution of class1 based on multiple probability values ​​and the total number of training images. Figure 5 (a1) in the figure is the probability histogram of class1, which can reflect the distribution of class1 in the training set. The horizontal axis represents the probability interval and the vertical axis represents the frequency.

[0097] For class 2, in the training set, the probability value of each training image being a crop of class 2 is obtained; based on the multiple probability values ​​and the total number of training images, the first probability distribution of class 2 is obtained. Figure 5 (b1) in the figure is the probability histogram of class2, which can reflect the distribution of class2 in the training set. The horizontal axis represents the probability interval and the vertical axis represents the frequency.

[0098] The second probability distribution of class1 and class2 in the image set is obtained respectively. For class1 in the above example, in the image set, the probability value of the crop in each image being class1 is obtained; based on multiple probability values ​​and the total number of images, the second probability distribution of class1 in the image set is obtained. Figure 6 (a2) in the figure is the probability histogram of class 1, which can reflect the distribution of class 1 in the image set. The horizontal axis represents the probability interval and the vertical axis represents the frequency.

[0099] For class 2, in the image set, obtain the probability value of the crop in each image being class 2; based on the multiple probability values ​​and the total number of images, obtain the second probability distribution of class 2. Figure 6 (b2) in the figure is the probability histogram of class2, which can reflect the distribution of class2 in the image set. The horizontal axis represents the probability interval and the vertical axis represents the frequency.

[0100] Step S204-3-3: for each category, calculate the similarity between the first probability distribution and the second probability distribution of the category, and use the similarity as the category comparison result;

[0101] In this embodiment, for each category, the similarity of the two probability distributions can be calculated based on the first probability distribution and the second probability distribution of the category, and a comparison result for the category can be obtained, thereby obtaining a comparison result for each category. The distance between the two probability distributions can be calculated to measure the similarity between the two, and the distance between the two probability distributions can be calculated using KL divergence (Kullback-Leible divergence) or Wasserstein distance.

[0102] The following will introduce how to use Wasserstein distance to calculate the distance between two probability distributions.

[0103] The joint distribution of the first probability distribution and the second probability distribution of the category may be calculated first, wherein the joint distribution includes multiple probability intervals and a first frequency corresponding to each probability interval in the first probability distribution and a second frequency corresponding to each probability interval in the second probability distribution.

[0104] Then, for each probability interval, the absolute value of the difference between the first frequency and the second frequency corresponding to the probability interval is used as the distance value of the probability interval to obtain the distance value of each probability interval.

[0105] Then calculate the expected value of the distance values ​​of all probability intervals, which is the similarity between the first probability distribution and the second probability distribution of the category.

[0106] In a possible implementation, the above calculation of the distance between the two probability distributions can be expressed by a preset formula, where the preset formula is:

[0107] W(P1, P2)=E (x，y)～γ [||xy||]

[0108] Among them, P1 represents the first probability distribution; P2 represents the second probability distribution; W(P1, P2) represents the distance between P1 and P2; γ represents the joint distribution of P1 and P2; x represents the first frequency; y represents the second frequency; E represents the expected value.

[0109] The similarity between the first probability distribution and the second probability distribution of the category is used as the comparison result of the category. It can be understood that the greater the distance between the two probability distributions, the greater the difference; if the distance between the two probability distributions is 0, it means that the two probability distributions are exactly the same.

[0110] For example, for class1 in the above example, the similarity W1 of the two probability distributions of class1 can be obtained according to a preset formula, the first probability distribution and the second probability distribution of class1, and the similarity W1 is used as the comparison result of class1.

[0111] For class2 in the above example, the similarity W2 of the two probability distributions of class2 can be obtained according to the preset formula, the first probability distribution and the second probability distribution of class2, and the similarity W2 is used as the comparison result of class2.

[0112] By calculating the similarity between the first probability distribution of each category in the training set and the second probability distribution in the image set, we can obtain a comparison result for each category. The similarity between the probability distributions of a category in the two datasets represents the difference between the classification model's preset and actual classification capabilities for that category. This allows us to more accurately determine whether the classification model's classification capabilities have degraded, facilitating optimization of the classification model to improve classification accuracy.

[0113] Optionally, based on the above steps, the similarities of the two probabilities of the categories are calculated. Then, for the above step S204-5, the embodiment of the present invention provides a possible implementation method, wherein step S204-5 includes the following steps:

[0114] Step S204-5-1: If the similarity of each category is less than the preset threshold, the preset classification model is not degraded;

[0115] In this embodiment, the similarity of each category is compared with a preset threshold. If the similarity of each category is less than the preset threshold, it means that the difference between the preset classification ability and the actual classification ability of the preset classification model for each category is small, and the preset classification model has not degenerated.

[0116] Step S204-5-3: If there is a similarity equal to or greater than a preset threshold, the preset classification model is degraded.

[0117] In this embodiment, if after comparing the similarity of each category with the preset threshold, there is a category whose similarity is equal to or greater than the preset threshold, it means that the preset classification ability and actual classification ability of the preset classification model for this category are significantly different, the classification ability for this category is reduced, and the preset classification model is degraded.

[0118] In order to understand the degradation of the classification model more intuitively, we will combine the above example Figure 5 and Figure 6 Provide explanation.

[0119] For class1 in the above example, Figure 5 In (a1), the probability corresponding to the maximum frequency of class1 in the training set is about 0.9. Figure 6 In (a2), the probability corresponding to the maximum frequency of class 1 in the image set is about 0.7, and the number of images with a probability of the object being class 1 higher than 0.8 is 0.

[0120] It can be seen that for class1, the probability that the preset classification model in the training set classifies the object in the image as class1 can reach 0.9, while the maximum probability that the preset classification model in the image set classifies the object in the image as class1 is less than 0.8. It can be considered that the preset classification model has a large deviation in the classification judgment of class1, which means that the classification ability of the preset classification model for the class1 category is reduced in actual application, and the preset classification model is degraded.

[0121] For class2 in the above example, Figure 5 In (b2), the probability corresponding to the maximum frequency of class2 in the training set is about 0.8. Figure 6 In (b2), the probability corresponding to the maximum frequency of class2 in the image set is about 0.6, and the number of images with a probability of being class2 higher than 0.7 is 0.

[0122] It can be seen that for class2, the probability that the preset classification model in the training set classifies the object in the image as class2 can reach 0.8, while the maximum probability that the preset classification model in the image set classifies the object in the image as class1 is less than 0.7. It can be considered that the preset classification model has a large deviation in the classification judgment of class2, which means that the classification ability of the preset classification model for the class2 category is reduced in actual application, and the preset classification model is degraded.

[0123] Optionally, for the above step S206, an embodiment of the present invention provides a possible implementation, wherein step S206 includes the following steps:

[0124] Step S206-1, obtaining the confidence of the object in each image in the image set;

[0125] Among them, for the category and probability value of the object in each image output by the preset classification model, the category corresponding to the maximum probability value can be taken as the predicted category of the object in the image, and the maximum probability value is the confidence level of the object in the image.

[0126] In this embodiment, in the image set, the maximum probability value of the object in each image is obtained, and the maximum probability value of each object is used as the confidence of each object.

[0127] Step S206-3: Optimizing a preset classification model based on the image to be processed obtained from the image set; the confidence level of the object in the image to be processed is less than a preset confidence level.

[0128] In this embodiment, an image to be processed in which the confidence of the object in the image is less than a preset confidence can be obtained from the image set, which can be understood as an image with low confidence. After the category of the object in the image to be processed is marked, it is input into the preset classification model and the preset classification model is iteratively trained to achieve optimization.

[0129] Optionally, for pending categories with a similarity equal to or greater than a preset threshold, indicating that the preset classification model's ability to classify the pending category has decreased, an image predicted to be the pending category, i.e., the pending image, can be obtained. The pending image can be annotated with the category of the object and then input into the preset classification model for iterative training to achieve optimization and improve the preset model's ability to classify the pending category. Alternatively, images with object confidence levels lower than a preset confidence level can be selected from all pending images, annotated, and then iteratively optimized using the preset classification model.

[0130] By comparing the similarity of each category with a preset threshold, it can be seen that if the similarity of each category is less than the preset threshold, the preset classification model has not degraded. If there is a category whose similarity is not less than the preset threshold, the preset classification model has degraded. Unprocessed images with object confidence less than the preset confidence can be obtained from the image set to optimize the preset classification model. This achieves automated evaluation of the classification model and optimizes the classification model based on the evaluation results, making the optimized classification model suitable for actual application scenarios and thus improving classification accuracy.

[0131] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a classification device is given below. Figure 7 , Figure 7This is a functional block diagram of a classification device 300 provided in an embodiment of the present invention. It should be noted that the basic principles and technical effects of the classification device 300 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding contents of the above-mentioned embodiments. The classification device 300 includes:

[0132] An acquisition module 310 is configured to input the image set collected by the operating equipment into a preset classification model to obtain the category of the object in the image set;

[0133] A determination module 330 is configured to determine whether the preset classification model is degraded based on images corresponding to objects of the same category in the training set and image set of the preset classification model;

[0134] The optimization module 350 is used to optimize the preset classification model if the preset classification model is degraded, and the optimized preset classification model is used to classify objects in the target image collected by the operating equipment.

[0135] Optionally, the determination module 330 is specifically used to: obtain a training set, the training set including multiple training images and the category of objects in each training image; obtain a comparison result for each category based on the training images corresponding to objects of the same category in the training set and the corresponding images in the image set; and determine whether the preset classification model is degraded based on the comparison result of each category.

[0136] Optionally, the determination module 330 is specifically used to: for each category, obtain a first probability distribution and a second distribution probability of the category based on the training images corresponding to the objects of the category in the training set and the images corresponding to the objects in the image set; for each category, calculate the similarity between the first probability distribution and the second probability distribution of the category, and use the similarity as the comparison result of the category to obtain a comparison result for each category.

[0137] Optionally, the determination module 330 is specifically configured to: calculate a joint distribution of a first probability distribution and a second probability distribution of the category; the joint distribution includes a plurality of probability intervals and a first frequency corresponding to each of the probability intervals in the first probability distribution and a second frequency corresponding to the second probability distribution;

[0138] For each of the probability intervals, taking the absolute value of the difference between the first frequency and the second frequency corresponding to the probability interval as the distance value of the probability interval, to obtain the distance value of each of the probability intervals;

[0139] Calculate the expected value of the distance value of all probability intervals to obtain the similarity between the first probability distribution and the second probability distribution of the category.

[0140] Optionally, the determination module 330 is specifically configured to: if the similarity of each category is less than a preset threshold, then the preset classification model is not degraded; if there is a similarity equal to or greater than the preset threshold, then the preset classification model is degraded.

[0141] Optionally, the optimization module 350 is specifically used to: obtain the confidence of the object in each image in the image set; optimize the preset classification model based on the image to be processed obtained from the image set; the confidence of the object in the image to be processed is less than the preset confidence.

[0142] An embodiment of the present invention further provides an electronic device, including a processor 120 and a memory 130. The memory 130 stores a computer program. When the processor executes the computer program, the classification method disclosed in the above embodiment is implemented.

[0143] The embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by the processor 120, the classification method disclosed in the embodiment of the present invention is implemented.

[0144] In summary, the classification method, apparatus, electronic device, and storage medium provided by the embodiments of the present invention obtain the categories of objects in the image set by inputting the image set collected by the operating equipment into a preset classification model. The classification model is then determined to be degraded based on the corresponding images of objects of the same category in the training set and image set of the preset classification model. If the preset classification model is degraded, the preset classification model is optimized, and the optimized preset classification model is used to classify objects in the target image collected by the operating equipment. Based on the training data of the classification model and the actual data classified by the classification model, it is determined whether the classification model is degraded, that is, whether it is suitable for the actual scene. If the classification model is degraded, it is optimized so that the optimized classification model matches the actual scene to improve the accuracy of the classification.

[0145] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0146] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0147] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0148] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A classification method, characterized in that The method comprises: Inputting the image set collected by the operating equipment into a preset classification model to obtain the category of the object in the image set; Obtaining a training set of the preset classification model, wherein the training set includes a plurality of training images and a category of an object in each training image; Obtaining a comparison result for each category based on the training images corresponding to the objects of the same category in the training set and the images corresponding to the objects in the image set; wherein the comparison result for each category is the similarity between the probability distribution of the objects of the category in the training set and the probability distribution in the image set; Determining whether the preset classification model is degraded according to the comparison result of each category; If the preset classification model is degraded, the preset classification model is optimized, and the optimized preset classification model is used to classify objects in the target image captured by the operating equipment.

2. The method according to claim 1, characterized in that The step of obtaining a comparison result for each category based on the images corresponding to objects of the same category in the training set of the preset classification model and the image set includes: For each category, obtaining a first probability distribution and a second probability distribution of the category according to the training images corresponding to the objects of the category in the training set and the images corresponding to the objects in the image set; For each category, the similarity between the first probability distribution and the second probability distribution of the category is calculated, and the similarity is used as the comparison result of the category.

3. The method according to claim 2, characterized in that The step of calculating the similarity between the first probability distribution and the second probability distribution of the category comprises: Calculating a joint distribution of a first probability distribution and a second probability distribution of the category; the joint distribution includes a plurality of probability intervals and a first frequency corresponding to each of the probability intervals in the first probability distribution and a second frequency corresponding to the second probability distribution; For each of the probability intervals, taking the absolute value of the difference between the first frequency and the second frequency corresponding to the probability interval as the distance value of the probability interval, to obtain the distance value of each of the probability intervals; Calculate the expected value of the distance value of all probability intervals to obtain the similarity between the first probability distribution and the second probability distribution of the category.

4. The method according to claim 2, characterized in that The step of determining whether the preset classification model is degraded according to the comparison result of each category includes: If the similarity of each category is less than the preset threshold, the preset classification model is not degraded; If there is a similarity equal to or greater than the preset threshold, the preset classification model is degraded.

5. The method according to claim 1, wherein The step of optimizing the preset classification model includes: Obtaining a confidence score of an object in each image in the image set; The preset classification model is optimized based on an image to be processed obtained from the image set; the confidence level of the object in the image to be processed is less than a preset confidence level.

6. The method according to claim 1, characterized in that The object includes one of the following: crops, obstacles, ground film, and ground.

7. The method according to claim 1, characterized in that The objects are crops and obstacles; the method further comprises: Obtaining the categories of crops and obstacles in the target image; Performing semantic segmentation on the target image according to the category of the crops and the category of the obstacles.

8. A classification device, characterized in that: The device comprises: An acquisition module, configured to input the image set collected by the operating equipment into a preset classification model to obtain the category of the object in the image set; a determination module configured to obtain a training set for the preset classification model, the training set comprising a plurality of training images and the category of objects in each training image; obtain a comparison result for each category based on the training images corresponding to objects of the same category in the training set and the images corresponding to the image set; wherein the category comparison result is the similarity between the probability distribution of objects of the category in the training set and the probability distribution in the image set; and determine whether the preset classification model is degraded based on the comparison result for each category; The optimization module is used to optimize the preset classification model if the preset classification model is degraded, and the optimized preset classification model is used to classify objects in the target image collected by the operating equipment.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images