Method and device for determining data category attributes, storage medium, and electronic device
By using the target network model in data processing to extract data characteristics and determine their category attributes, the problem of inaccurate external data classification in the prior art is solved, and more accurate data classification is achieved.
Patent Information
- Application Number
- CN202110796429.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-14
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2041-07-14
AI Technical Summary
The prior art classification is inaccurate when processing external data, and it is impossible to effectively distinguish internal data from external data, resulting in external data being misclassified into known categories.
By entering the target data into the target network model, the characteristic mean and characteristic variance of the target data are extracted, these characteristics are used to determine the category to which the data belongs and its probability, and the category attributes of the data are determined based on the probability.
On the basis of fixed number of classifications, external data are effectively identified to avoid misclassifying external data into known categories, and to improve the accuracy of classification.
Smart Images

Figure CN113469265B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of data processing, and in particular, to a method and device for determining data category attributes, a storage medium, and an electronic device. Background Art
[0002] In the real world, almost all classification models face a problem when they are actually used. They need to process external data that does not belong to any known category. General classification methods will force the data to be classified into one of the known categories.
[0003] The open-set classification problem not only includes the digital categories of 0 to 9, but also other unknown categories such as A to Z, etc. However, these unknown categories do not have labels, and the classifier cannot know the specific categories of the images in these unknown categories, such as: whether it is A. These many different categories of images together constitute a category: the unknown category, which we call the background category in detection. The open-set classification problem is to distinguish these 10 categories and reject other unknown categories.
[0004] The open set classification method disclosed in the prior art adds many categories during training, but still faces the problem of external data during prediction.
[0005] With respect to the above technical problems, no effective solutions have been proposed in the relevant technologies. Summary of the invention
[0006] The embodiments of the present invention provide a method and device for determining data category attributes, a storage medium, and an electronic device, so as to at least solve the problem of inaccurate classification of external data in the related art.
[0007] According to one embodiment of the present invention, a method for determining data category attributes is provided, comprising: inputting target data into a target network model to obtain a first feature and a second feature of the target data output by the target network model, wherein the first feature is used to represent a feature mean of the target data, and the second feature is used to represent a feature variance of the target data; using the first feature and the second feature to determine a target data category to which the target data belongs, and a probability that the target data belongs to the target data category; determining the category attribute of the target data based on the target data category to which the target data belongs, and the probability that the target data belongs to the target data category.
[0008] According to another embodiment of the present invention, there is provided a device for determining data category attributes, comprising: a first input module, used to input target data into a target network model, and obtain a first feature and a second feature of the target data output by the target network model, wherein the first feature is used to represent a feature mean of the target data, and the second feature is used to represent a feature variance of the target data; a first determination module, used to determine a target data category to which the target data belongs, and a probability that the target data belongs to the target data category, using the first feature and the second feature; and a second determination module, used to determine the category attribute of the target data based on the target data category to which the target data belongs, and the probability that the target data belongs to the target data category.
[0009] In an exemplary embodiment, the above-mentioned first determination module includes: a first input unit, used to input the above-mentioned first feature into a classifier to obtain the above-mentioned target data category to which the above-mentioned target data belongs calculated by the above-mentioned classifier, wherein the above-mentioned second feature is also used to represent the output feature of the above-mentioned classifier; a first determination unit, used to determine the probability that the target data belongs to the target data category based on the feature mean of the first feature and the feature variance of the second feature, the first parameter and the preset data category.
[0010] In an exemplary embodiment, the above-mentioned second determination module includes one of the following: a second determination unit, used to determine that the above-mentioned target data belongs to the above-mentioned target data category when the above-mentioned probability is greater than or equal to the above-mentioned preset threshold; a third determination unit, used to determine that the above-mentioned target data belongs to the abnormal data category when the above-mentioned probability is less than the above-mentioned preset threshold.
[0011] In an exemplary embodiment, the above-mentioned device also includes: a second input module, used to input the target data into the target network model, and before obtaining the first feature and the second feature of the above-mentioned target data output by the above-mentioned target network model, the internal data and the external data in the determined sample data are input into the original network model according to a first preset ratio to obtain the third feature and the fourth feature output by the above-mentioned original network model, wherein the above-mentioned third feature is used to represent the feature mean of the above-mentioned internal data and the above-mentioned external data, the above-mentioned fourth feature is used to represent the feature variance of the above-mentioned internal data and the above-mentioned external data, the above-mentioned external data is used to represent data without an attribution category, and the above-mentioned internal data is used to represent data with an attribution category; a first separation module, used to separate the above-mentioned third feature and the above-mentioned fourth feature according to a second preset ratio; a first training module, used to train the above-mentioned original network model according to the above-mentioned third feature and the above-mentioned fourth feature to obtain the above-mentioned target network model.
[0012] In an exemplary embodiment, the first training module includes: a fourth determination unit, used to determine a feature memory library, wherein the feature memory library is used to store the feature distribution of the sample data; a fifth determination unit, used to determine a first distance between the third feature and the feature memory library to determine a first probability of the data category to which the internal data belongs; a sixth determination unit, used to determine a second distance between the fourth feature and the feature memory library to determine a second probability of the data category to which the external data belongs; and a first training unit, used to train the original network model based on the first probability and the second probability to obtain the target network model.
[0013] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.
[0014] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0015] Through the present invention, by inputting the target data into the target network model, the first feature and the second feature of the target data output by the target network model are obtained, wherein the first feature is used to represent the feature mean of the target data, and the second feature is used to represent the feature variance of the target data; the target data category to which the target data belongs and the probability that the target data belongs to the target data category are determined by using the first feature and the second feature; the category attribute of the target data is determined based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category. On the basis of a fixed number of classifications, it can be effectively determined whether the target data belongs to internal data or external data, thereby ensuring the accuracy of classification and avoiding misclassification of external data into known categories. Therefore, the problem of inaccurate classification of external data in the related art can be solved, and the effect of accurately classifying data can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for determining a data category attribute according to an embodiment of the present invention;
[0017] Figure 2 is a flow chart of a method for determining a data category attribute according to an embodiment of the present invention;
[0018] Figure 3 is a training flow chart according to an embodiment of the present invention;
[0019] Figure 4is a flow chart of internal data processing in an uncertainty estimation module according to an embodiment of the present invention;
[0020] Figure 5 is a flowchart of processing external data in an uncertainty estimation module according to an embodiment of the present invention;
[0021] Figure 6 is a test flow chart according to an embodiment of the present invention;
[0022] Figure 7 4 is a structural block diagram of a device for determining data category attributes according to an embodiment of the present invention. DETAILED DESCRIPTION
[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.
[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0025] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for determining a data category attribute according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining data category attributes in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0028] In this embodiment, a method for determining a data category attribute is provided. Figure 2 is a flow chart of a method for determining a data category attribute according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:
[0029] Step S202, inputting the target data into the target network model, obtaining a first feature and a second feature of the target data output by the target network model, wherein the first feature is used to represent the feature mean of the target data, and the second feature is used to represent the feature variance of the target data;
[0030] Step S204, using the first feature and the second feature to determine the target data category to which the target data belongs, and the probability that the target data belongs to the target data category;
[0031] Step S206 , determining the category attribute of the target data based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category.
[0032] The execution subject of the above steps may be a terminal, etc., but is not limited thereto.
[0033] This embodiment includes, but is not limited to, application in scenarios of classifying data, for example, classifying data of known categories and classifying data of unknown categories.
[0034] In this embodiment, the target data includes but is not limited to internal data (known categories) and external data (unknown categories). The target network model includes but is not limited to a cellular neural network (Cellular Neural Network, referred to as CNN).
[0035] Through the above steps, by inputting the target data into the target network model, the first feature and the second feature of the target data output by the target network model are obtained, wherein the first feature is used to represent the feature mean of the target data, and the second feature is used to represent the feature variance of the target data; the target data category to which the target data belongs and the probability that the target data belongs to the target data category are determined by using the first feature and the second feature; the category attribute of the target data is determined based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category. On the basis of a fixed number of categories, it can be effectively determined whether the target data belongs to internal data or external data, thereby ensuring the accuracy of the classification and avoiding misclassification of external data into known categories. Therefore, the problem of inaccurate classification of external data in the related art can be solved, and the effect of accurately classifying data can be achieved.
[0036] In an exemplary embodiment, determining the probability that the target data belongs to the target data category using the first feature and the second feature includes:
[0037] S1, inputting the first feature into a classifier to obtain a target data category to which the target data calculated by the classifier belongs, wherein the second feature is also used to represent an output feature of the classifier;
[0038] S2, determining the probability that the target data belongs to the target data category based on the feature mean of the first feature, the feature variance of the second feature, the first parameter, and the preset data category.
[0039] In this embodiment, the probability that the target data belongs to the target data category is determined by the following formula:
[0040]
[0041] Among them, P is used to represent the probability of the target data category, μ is used to represent the feature mean of the first feature, σ is used to represent the feature variance of the second feature, m is used to represent the first parameter, c is used to represent the preset data category, and i is used to represent the order of the feature mean of the first feature.
[0042] In an exemplary embodiment, determining the category attribute of the target data based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category includes one of the following:
[0043] S1, when the probability is greater than or equal to a preset threshold, determining that the target data belongs to the target data category;
[0044] S2: When the probability is less than a preset threshold, it is determined that the target data belongs to the abnormal data category.
[0045] In this embodiment, for example, if the probability is greater than a threshold value T, the classification is considered correct, and if it is less than T, it is considered to be uncertain data.
[0046] In an exemplary embodiment, before inputting the target data into the target network model and obtaining the first feature and the second feature of the target data output by the target network model, the method further includes:
[0047] S1, inputting the internal data and the external data in the determined sample data into the original network model according to a first preset ratio, and obtaining the third feature and the fourth feature output by the original network model, wherein the third feature is used to represent the feature mean of the internal data and the external data, the fourth feature is used to represent the feature variance of the internal data and the external data, the external data is used to represent the data without an attribution category, and the internal data is used to represent the data with an attribution category;
[0048] S2, separating the third feature and the fourth feature according to a second preset ratio;
[0049] S3, training the original network model according to the third feature and the fourth feature to obtain the target network model.
[0050] In this embodiment, the original network model includes but is not limited to a CNN network model. Separating the third feature and the fourth feature according to the second preset ratio is to tear apart the third feature and the fourth feature.
[0051] In an exemplary embodiment, training the original network model according to the third feature and the fourth feature to obtain the target network model includes:
[0052] S1, determining a feature memory library, wherein the feature memory library is used to store feature distribution of sample data;
[0053] S2, determining a first distance between the third feature and the feature memory library to determine a first probability of the data category to which the internal data belongs;
[0054] S3, determining a second distance between the fourth feature and the feature memory to determine a second probability of the data category to which the external data belongs;
[0055] S4, training the original network model based on the first probability and the second probability to obtain a target network model.
[0056] In this embodiment, the feature memory is a library for storing the feature distribution of samples within a class, and is updated slowly. Each class maintains a feature memory library, which is updated using a momentum update method.
[0057] The present invention will be described below in conjunction with specific embodiments:
[0058] The problem to be solved in this embodiment is to effectively identify external data and avoid misclassifying external data into known categories while ensuring classification accuracy on the basis of a fixed number of categories.
[0059] like Figure 3 As shown, it is a training flow chart in this embodiment, which includes the following steps:
[0060] S301, input the intra-class data and external data in a certain ratio. The intra-class data is used to represent the data within the category of supervised classification, and the external data is used to represent the diverse external data that does not belong to any of the categories. In terms of data input, this embodiment is different from the general supervised classification task. External data is added. The introduction of external data will have little effect on the classification probability, but the uncertainty estimation of external data will be significantly improved, that is, the external data will not be classified into a known category with a high probability. The internal data and external data of the input data are pre-processed in a certain ratio (recommended 1:1) and sent to the network.
[0061] S302, after the data passes through the CNN network forward process, a certain N-dimensional feature is obtained. The feature in this embodiment includes two N-dimensional features, one representing the feature mean of the sample, denoted by μ, and the other representing the feature variance of the sample, denoted by σ. A normal distribution σ(μ, σ 2 ) to represent this feature, then the representation range of this feature will be expanded.
[0062] Among them, f(x) is used to represent the representation range of the feature;
[0063] S303, after obtaining the features, record them as fea(μ, σ) and split them according to the ratio. There are two reasons for splitting. First, the cross entropy loss does not need to be calculated for the external data part, because this part does not have the correct classification label and cannot contribute to the correct loss. Second, the operations of internal data and external data in the uncertainty estimation part are also different, and they need to be processed separately.
[0064] S304-S306, classification branch, when the internal data is used for classification, it needs to be concretized, using formula 2, where ε∈N(0,1), the introduction of ε makes the model more robust. After the features are solidified, cross entropy loss is used for classification. Cross entropy loss is denoted as L softmax .
[0065] fea=u+ε×σ (2);
[0066] S307, uncertainty estimation branch. The uncertainty estimation part is introduced in detail in two parts, namely the processing of internal data and the processing of external data.
[0067] First, let me introduce the feature memory library. The feature memory library is a library that memorizes the feature distribution of samples within a class and is updated slowly. Each class maintains a distribution , and the momentum update method is used to update, as shown in formulas (3) and (4). m , σ m0 It represents the parameters before the update, which is the distribution parameter obtained from all historical data. New information can be added slowly after the update. c , σ c is the mean of fea(μ, σ) of all samples in this batch.
[0068] μ m = k × μ m0 +(1-k)×μ c (3);
[0069] σ m = k × σ m0 +(1-k)×σ c (4);
[0070] Here, k is a preset parameter less than 1.
[0071] For internal data, such as Figure 4 As shown, through an L in To estimate the distance between a single sample distribution and the feature memory. The specific implementation method is shown in formula (5-7). In formula 5, DM is used to represent the normalized distance. Formula 6 represents the measure of the distance between the same category in the memory library. The determination of the threshold refers to the 3σ principle of the normal distribution. Formula 7 represents the measure of the distance from other categories in the memory library.
[0072]
[0073]
[0074]
[0075] For external data, such as Figure 5 As shown, through an L out To estimate the distance between a single sample distribution and the feature memory. The specific implementation method is shown in formula (8-9). In formula 8, DM is used to represent the normalized distance. Formula 9 specifically measures the distance between the external data and the intra-class center of the feature memory to ensure that the external data is outside the 3σ range.
[0076]
[0077]
[0078] The overall LOSS consists of three parts
[0079] L = a × L softmax +b×L in +c×L out (10).
[0080] like Figure 6 FIG. 1 is a test flow chart of this embodiment, which includes the following steps:
[0081] S601, input the test data into the CNN network model;
[0082] S602-S603, the test data is subjected to the CNN network to extract features and obtain classification features fea (μ, σ);
[0083] S604-S607, when passing through the classifier, directly take μ as the output feature and calculate the category to which it belongs. Assuming it belongs to category c, use formula 11 to calculate the probability of belonging to the current category. If the probability is greater than the threshold T, the classification is considered correct, and if it is less than T, it is considered to be uncertain data.
[0084]
[0085] In summary, this embodiment solves the problem of predicting out-of-distribution data. It can correctly predict the category of in-class data and correctly predict out-of-class data as external data, with a low probability of belonging to the in-class, significantly reducing misclassification. The above is achieved by predicting the distribution of features rather than the specifics to increase the model tolerance, and the model's ability to distinguish external data is enhanced by introducing uncertainty estimation modules and external data.
[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0087] In this embodiment, a device for determining data category attributes is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0088] Figure 7 is a structural block diagram of a device for determining data category attributes according to an embodiment of the present invention, such as Figure 7 As shown, the device comprises:
[0089] A first input module 72 is used to input the target data into the target network model to obtain a first feature and a second feature of the target data output by the target network model, wherein the first feature is used to represent a feature mean of the target data, and the second feature is used to represent a feature variance of the target data;
[0090] A first determination module 74, configured to determine the target data category to which the target data belongs, and the probability that the target data belongs to the target data category, using the first feature and the second feature;
[0091] The second determination module 76 is used to determine the category attribute of the target data based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category.
[0092] In an exemplary embodiment, the first determining module includes:
[0093] A first input unit is used to input the first feature into a classifier to obtain the target data category to which the target data calculated by the classifier belongs, wherein the second feature is also used to represent an output feature of the classifier;
[0094] The first determining unit is used to determine the probability that the target data belongs to the target data category based on the feature mean of the first feature, the feature variance of the second feature, the first parameter, and the preset data category.
[0095] In an exemplary embodiment, the second determining module includes one of the following:
[0096] A second determining unit, configured to determine that the target data belongs to the target data category when the probability is greater than or equal to a preset threshold;
[0097] The third determining unit is used to determine that the target data belongs to the abnormal data category when the probability is less than the preset threshold.
[0098] In an exemplary embodiment, the above device further comprises:
[0099] A second input module is used to input the target data into the target network model, and before obtaining the first feature and the second feature of the target data output by the target network model, input the internal data and the external data in the determined sample data into the original network model according to a first preset ratio, and obtain the third feature and the fourth feature output by the original network model, wherein the third feature is used to represent the feature mean of the internal data and the external data, the fourth feature is used to represent the feature variance of the internal data and the external data, the external data is used to represent data without an attribution category, and the internal data is used to represent data with an attribution category;
[0100] A first separation module, used for separating the third feature and the fourth feature according to a second preset ratio;
[0101] The first training module is used to train the original network model according to the third feature and the fourth feature to obtain the target network model.
[0102] In an exemplary embodiment, the first training module includes:
[0103] A fourth determining unit, used to determine a feature memory library, wherein the feature memory library is used to store the feature distribution of the sample data;
[0104] a fifth determining unit, configured to determine a first distance between the third feature and the feature memory library, so as to determine a first probability of a data category to which the internal data belongs;
[0105] a sixth determining unit, configured to determine a second distance between the fourth feature and the feature memory library, so as to determine a second probability of the data category to which the external data belongs;
[0106] The first training unit is used to train the original network model based on the first probability and the second probability to obtain the target network model.
[0107] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0108] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0109] In this embodiment, the computer-readable storage medium may be configured to store a computer program for executing the above steps.
[0110] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0111] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0112] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0113] In an exemplary embodiment, the processor may be configured to execute the above steps through a computer program.
[0114] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0115] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for determining a data category attribute, characterized in that: include: Input the target data into the target network model, and obtain the first feature and the second feature of the target data output by the target network model, wherein the first feature is used to represent the feature mean of the target data, and the second feature is used to represent the feature variance of the target data, the target data is image data that does not belong to the data category that has been trained in the target network model, and the target network model is a CNN network; Determine the target data category to which the target data belongs and the probability that the target data belongs to the target data category by using the first feature and the second feature; Detecting the resolution capability of the CNN network for the target data based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category; Before inputting the target data into the target network model to obtain the first feature and the second feature of the target data output by the target network model, the method also includes: inputting the internal data and the external data in the determined sample data into the original network model according to a first preset ratio to obtain the third feature and the fourth feature output by the original network model, wherein the third feature is used to represent the feature mean of the internal data and the external data, the fourth feature is used to represent the feature variance of the internal data and the external data, the external data is used to represent data without an attribution category, and the internal data is used to represent data with an attribution category; separating the third feature and the fourth feature according to a second preset ratio; training the original network model according to the third feature and the fourth feature to obtain the target network model.
2. The method according to claim 1, characterized in that Determining the target data category to which the target data belongs and the probability that the target data belongs to the target data category by using the first feature and the second feature includes: Inputting the first feature into a classifier to obtain the target data category to which the target data belongs calculated by the classifier, wherein the second feature is also used to represent an output feature of the classifier; The probability that the target data belongs to the target data category is determined based on the feature mean of the first feature, the feature variance of the second feature, a first parameter, and a preset data category.
3. The method according to claim 1, characterized in that Determining the category attribute of the target data based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category includes one of the following: When the probability is greater than or equal to a preset threshold, determining that the target data belongs to the target data category; When the probability is less than the preset threshold, it is determined that the target data belongs to the abnormal data category.
4. The method according to claim 1, characterized in that: Training the original network model according to the third feature and the fourth feature to obtain the target network model includes: Determine a feature memory library, wherein the feature memory library is used to store feature distribution of the sample data; determining a first distance between the third feature and the feature memory to determine a first probability of a data category to which the internal data belongs; determining a second distance between the fourth feature and the feature memory to determine a second probability of the data category to which the external data belongs; The original network model is trained based on the first probability and the second probability to obtain the target network model.
5. A device for determining data category attributes, characterized in that: include: A first input module is used to input target data into a target network model to obtain a first feature and a second feature of the target data output by the target network model, wherein the first feature is used to represent a feature mean of the target data, and the second feature is used to represent a feature variance of the target data, the target data is image data that does not belong to a data category that has been trained in the target network model, and the target network model is a CNN network; A first determination module, configured to determine the target data category to which the target data belongs, and a probability that the target data belongs to the target data category, by using the first feature and the second feature; A second determination module is used to detect the resolution capability of the CNN network for the target data based on the target data category to which the target data belongs and the probability that the target data belongs to the target data category; The device is also used to input the target data into the target network model, and before obtaining the first feature and the second feature of the target data output by the target network model, input the internal data and the external data in the determined sample data into the original network model according to a first preset ratio, to obtain the third feature and the fourth feature output by the original network model, wherein the third feature is used to represent the feature mean of the internal data and the external data, the fourth feature is used to represent the feature variance of the internal data and the external data, the external data is used to represent data without an attribution category, and the internal data is used to represent data with an attribution category; separate the third feature and the fourth feature according to a second preset ratio; train the original network model according to the third feature and the fourth feature to obtain the target network model.
6. The device according to claim 5, characterized in that The first determining module includes: A first input unit, used for inputting the first feature into a classifier to obtain the target data category to which the target data belongs calculated by the classifier, wherein the second feature is also used for representing an output feature of the classifier; The first determining unit is used to determine the probability that the target data belongs to the target data category based on the feature mean of the first feature, the feature variance of the second feature, a first parameter, and a preset data category.
7. The device according to claim 5, characterized in that The second determining module includes one of the following: A second determining unit, configured to determine that the target data belongs to the target data category when the probability is greater than or equal to a preset threshold; A third determining unit is configured to determine, when the probability is less than the preset threshold, that the target data belongs to an abnormal data category.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the method described in any one of claims 1 to 4 when executed by a processor.
9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Open set domain adaptation method and system based on entropy minimization
CN110750665A
Neural network training method and device, storage medium and electronic device
CN110837889A
Image classification method and device, storage medium and electronic equipment
CN111046933A
Digital hearing aid-oriented scene matching method and apparatus and computer device
CN111144482A
Image recognition method and device, electronic equipment and storage medium
CN111368893A