System and method for predicting set of possible categories of test data

Abstract

A system and method for predicting a set of possible categories of test data is described. The method includes obtaining, from a memory, a training data set, a plurality of categories, and a plurality of corresponding training feature vectors for each of the plurality of categories. The method includes receiving an input indicating a target probability required to test data. The method includes determining a set of membership probabilities for the test data, the set of membership probabilities including a corresponding membership probability associated with each of the plurality of categories, the corresponding membership probability indicating a probability that the test data belongs to a corresponding category of the plurality of categories. The method includes determining a set of possible categories of the test data from a plurality of categories based on the input and the set of membership probabilities.

Description

Technical Field

[0001] This disclosure relates to the field of data classification. For example, this disclosure relates to methods and systems for predicting a set of possible categories for test data based on the target probability of a classification desired by a user. Background Technology

[0002] Classification is a fundamental concept in machine learning (ML) applications. Classification involves grouping data into predefined categories or classes. Classification is a supervised ML method where an ML model attempts to predict the correct category for given input data. To classify correctly, an ML model can be trained using pre-defined training data. Furthermore, the ML model can be evaluated based on the training and / or test data before being deployed to perform predictions on real-time data. Therefore, the ML model can learn patterns and relationships from the training data to perform successful predictions and / or classifications on real-time data. Some classification algorithms utilize techniques such as decision trees, support vector machines, logistic regression, and neural networks. Summary of the Invention

[0003] Solution to the problem

[0004] According to an example embodiment of this disclosure, a method for predicting a set of possible categories for test data is disclosed. The method includes: retrieving from a memory including a training dataset a plurality of categories and a plurality of corresponding training feature vectors for each of the plurality of categories. The method includes: receiving an input indicating a target probability required for the test data. The method includes: determining a set of membership probabilities for the test data. The set of membership probabilities includes corresponding membership probabilities associated with each of the plurality of categories. Corresponding membership probabilities indicate the probability that the test data belongs to a corresponding category among the plurality of categories. The method includes: determining a set of possible categories for the test data from the plurality of categories based on the input and the set of membership probabilities.

[0005] According to an example embodiment of this disclosure, a system for predicting a set of possible categories of test data is disclosed. The system includes a memory and at least one processor, the at least one processor including processing circuitry communicatively coupled to the memory. The at least one processor is individually and / or collectively configured to: retrieve from the memory a plurality of categories and a plurality of corresponding training feature vectors for each of the plurality of categories. The at least one processor is individually and / or collectively configured to: receive an input indicating a target probability desired for the test data. The at least one processor is individually and / or collectively configured to: determine a set of membership probabilities for the test data. The set of membership probabilities includes corresponding membership probabilities associated with each of the plurality of categories. Corresponding membership probabilities indicate the probability that the test data belongs to a corresponding category among the plurality of categories. The at least one processor is individually and / or collectively configured to determine a set of possible categories of the test data from the plurality of categories based on the input and the set of membership probabilities.

[0006] To further illustrate the advantages and features of this disclosure, a more detailed description of the disclosure will be presented with reference to various exemplary embodiments of the disclosure illustrated in the accompanying drawings. It should be understood that these drawings depict only exemplary embodiments and should not be considered as limiting the scope of the disclosure. The disclosure will be described and explained with reference to the accompanying drawings, incorporating additional features and details. Attached Figure Description

[0007] These and other features, aspects, characteristics, and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, wherein similar reference numerals denote similar parts throughout the drawings, and in the drawings:

[0008] Figure 1A This is a diagram illustrating the feature space and training data corresponding to the ML model according to the prior art;

[0009] Figure 1B , Figure 1C , Figure 1D , Figure 1E , Figure 1F and Figure 1G This is a diagram illustrating various example use case scenarios where a classifier based on one or more existing technologies might misclassify input data;

[0010] Figure 2 This is a block diagram illustrating an example environment for predicting a set of possible categories for test data according to various embodiments;

[0011] Figure 3 This is a block diagram illustrating example configurations of user devices and systems for predicting a set of possible categories of test data according to various embodiments;

[0012] Figure 4 This is a block diagram illustrating an example configuration of one or more modules of a system for predicting a set of possible categories of test data according to various embodiments;

[0013] Figure 5 , Figure 6A , Figure 6B , Figure 7 , Figure 8 , Figure 9 and Figure 10 It is a diagram illustrating example use cases for predicting a set of possible categories by a system, according to various embodiments; and

[0014] Figure 11A , Figure 11B and Figure 11C This is a flowchart illustrating example methods for predicting a set of possible categories for test data according to various embodiments.

[0015] Furthermore, those skilled in the art will understand that the elements in the accompanying drawings are shown for simplicity and may not necessarily be drawn to scale. For example, flowcharts illustrate methods from the perspective of steps that help improve understanding of aspects of this disclosure. Additionally, regarding the construction of the device, one or more components of the device may already be represented by conventional symbols in the drawings, and the drawings may illustrate those specific details relevant to understanding embodiments of this disclosure, without obscuring the drawings with details that would be obvious to those of ordinary skill in the art who would benefit from the description herein. Detailed Implementation

[0016] Various exemplary embodiments will now be referenced, and these exemplary embodiments will be described using specific language. However, it will be understood that this is not intended to limit the scope of the present disclosure, and such changes and further modifications in the illustrated systems, as well as such further applications of the principles of the present disclosure as shown therein, are contemplated as would normally occur to those skilled in the art to which this disclosure pertains.

[0017] Those skilled in the art will understand that the foregoing general description and the following detailed description are illustrative and not intended to be limiting.

[0018] References to “aspect,” “on the other hand,” or similar language throughout this specification may refer to a specific feature, structure, or characteristic described, for example, in connection with an embodiment included in at least one embodiment of this disclosure. Therefore, the phrases “in an embodiment,” “in another embodiment,” and similar language throughout this disclosure may, but not necessarily, refer to the same embodiment.

[0019] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that includes a list of steps may include not only those steps but may also include other steps not expressly listed or inherent to such a process or method. Similarly, without further constraints, a list of one or more devices, subsystems, elements, structures, or components beginning with “comprising…” does not exclude the presence of other devices or subsystems or elements or structures or components, or additional devices or subsystems or elements or structures or components.

[0020] This disclosure relates to a method and system for predicting a set of possible classes for test data given a target probability desired by a user. The method includes using user input indicating the target probability required for the test data to predict the set of possible classes. Therefore, the disclosed method provides a set of outputs that allows the user to view possible options and further select the most relevant result based on the user's requirements, rather than providing incorrect output due to misclassification.

[0021] Misclassification is a major problem associated with ML models that use classification algorithms. Despite significant advancements in classification algorithms, misclassification remains a key challenge in ML applications. For ML models, the misclassification rate is a measure of the percentage of observations incorrectly predicted by the ML model. The misclassification rate can be determined based on the number of incorrect predictions out of the total number of predictions. Misclassification occurs when an ML model incorrectly assigns data to categories, leading to incorrect predictions and potentially undesirable consequences.

[0022] Among various classification techniques, a set of hyperplanes can be identified to classify different types of data. However, it may be impossible to identify hyperplanes that completely isolate related data points, thus leading to errors in classification, as some points may be misclassified while others may be missed.

[0023] In most real-world problems, the training data associated with the ML model is not cleanly separated. Figure 1A This is a graph showing the feature space 102 corresponding to the ML model and the training data 104. The training data 104 can be associated with different categories. For example... Figure 1A As shown, most of the feature space 102 consists of misclassified regions, where the training data 104 overlap with each other. Regions within bounding boxes 106 can be considered correctly classified regions, while regions outside bounding boxes 106 can be considered misclassified regions.

[0024] Figure 1BThis diagram illustrates an example use case scenario where a classifier (ML model) might misclassify input data. The input data can be test images. Training data 104 can be provided to the classifier, including dog training data, hyena training data, and fox training data. Initially, at box 110, the classifier can receive the test image 108 of a hyena. Within feature space 102, the test image 108 can overlap with different training data, such as... Figure 1B As shown. In box 112, the classifier can predict that the test data matches the dog more strongly than the hyena, and in box 114, the classifier can incorrectly provide the output classification as a dog.

[0025] Figure 1C This is a diagram illustrating another example use case scenario where a classifier might misclassify the input data. For example... Figure 1C As shown, in box 116, disease symptoms can be provided to the classifier as input data. In box 118, the classifier can predict that the symptoms strongly match Disease 1 among multiple diseases (Disease 2, Disease 3, etc.). In box 120, the classifier may incorrectly provide an output classification as Disease 1. Incorrect classification in the field of medical diagnostics can lead to the omission of critical diseases during diagnosis and serious consequences for personal safety. Specifically, incorrect classification can lead to misdiagnosis, which results in inappropriate treatment plans or delayed interventions, potentially harming the patient's health.

[0026] Figure 1D This is a diagram illustrating another example use case scenario where a classifier might misclassify the input data. Figure 1D In this system, the classifier may be provided with an authentication system 122 that grants access rights to administrators / authorized personnel. When user 124 attempts to pass through authentication system 122, authentication system 122 examines the input data from user 124, such as biometric information, and incorrectly classifies user 124 as not being an administrator. For example, authentication system 122 may have an authentication threshold of 90% and determine that the probability of user 124 being an administrator is less than 90% (e.g., 55%). Therefore, authentication system 122 may not classify user 124 as an administrator and may restrict further access; however, user 124 may actually be an authorized user / administrator.

[0027] Figure 1E This is a diagram illustrating another example use case scenario where a classifier might misclassify the input data. Figure 1E In this context, a classifier can be provided by a facial recognition engine 126. However, the classifier may incorrectly classify the user 128's face, which could lead to facial recognition failure.

[0028] Figure 1FThis diagram illustrates another example use case scenario where a classifier might misclassify input data. The input data could be historical data, such as data associated with finance, the economy, natural disasters, etc. At box 130, the input data can be provided to the classifier. At box 132, the classifier can predict various scenarios (Scenario 1, Scenario 2, etc.) and predicts Scenario 1 as the strongest match. However, the classifier may fail to predict a strong match for the worst-case scenario. At box 134, the classifier can provide Scenario 1 as the classification output, omitting the worst-case scenario (worst-case minimum, possible worst-case disaster, etc.).

[0029] Figure 1G This diagram illustrates another example use case scenario where a classifier might misclassify input data. The input data could be voice input from user 136. A voice recognition engine could be provided in user device 138. The voice input could include the words "search Korean pledge". The classifier could process the voice input, and regarding the voice input containing the word "pledge", the classifier might classify the voice input as more strongly corresponding to the word "placed" than the word "pledge". As a result, an incorrect output could be provided to the user.

[0030] As is evident from various example use cases, misclassification can have serious consequences in real-world applications, such as misdiagnosis in the medical field, failure of biometric detection, failure of facial recognition, false positives or false negatives in fraud detection systems, incorrect identification in autonomous vehicles, and incorrect speech recognition.

[0031] Therefore, the above-mentioned problems need to be addressed. For example, systems and methods that provide fault-tolerant predictions during classification and enhance reliability in practical applications are needed.

[0032] Figure 2 This is a block diagram illustrating an example environment 200 for predicting a set of possible categories of test data according to various embodiments. Environment 200 may include a plurality of user devices 210a, 210b, 210c and a system 220 communicatively coupled to the plurality of user devices 210a, 210b, 210c. It should be understood that details may be provided with respect to the user device (hereinafter referred to as 'user device 210') among the plurality of user devices 210a, 210b, 210c, and the details are equally applicable to each of the plurality of user devices 210a, 210b, 210c.

[0033] In various embodiments, user equipment 210 may be associated with a user. In various embodiments, user equipment 210 may include any device, such as, but not limited to, a user's smartphone, laptop computer, desktop computer, smartwatch, tablet computer, or personal digital assistant (PDA). In various embodiments, user equipment 210 may be configured to generate test data. Various non-limiting examples of test data include voice, photos, videos, text, biometric information, etc. In other words, test data may refer to data to be classified into one or more categories from multiple categories. In various embodiments, test data may be associated with either an identification type or a detection type, as will be further described below.

[0034] System 220 can be configured to conformally predict a set of possible categories of test data. System 220 can be communicatively coupled to user equipment 210 via communication device 230. In various embodiments, system 220 can be an on-device system, i.e., system 220 can be integrated with user equipment 210 and can be configured to combine user equipment 210 to predict a set of possible categories. In various embodiments, system 220 can be a cloud-based system. In various embodiments, system 220 can be provided in a distributed manner, i.e., one or more components and / or functions of system 220 can be provided through user equipment 210, and one or more components and / or functions of system 220 can be provided through cloud-based units (e.g., cloud storage or cloud-based servers).

[0035] Communication device 230 may include, for example, a communication network, such as, but not limited to, direct interconnection, a local area network (LAN), a wide area network (WAN), a wireless network (e.g., using the Wireless Application Protocol (WAP)), the Internet, etc. In various embodiments, communication device 230 may include the internal communication bus and interface of user equipment 210.

[0036] Figure 3 This is a block diagram illustrating example configurations of user equipment 210 and system 220 for predicting a set of possible categories of test data according to various embodiments.

[0037] User equipment 210 may include a transceiver 302 configured to receive signals from and / or transmit signals to system 220 and any other devices / units connected thereto. User equipment 210 may include an input / output (I / O) unit (e.g., including various input / output circuitry) 304. In various embodiments, I / O unit 304 enables user equipment 210 to receive and / or generate test data for predicting a set of possible categories. I / O unit 304 may allow input to and output from user equipment 210 using suitable devices such as, but not limited to, a camera, keyboard, mouse, pointer, sensor, printer, microphone, speaker, etc. In various embodiments, I / O unit 304 may provide display functionality, such as through a display and / or graphical user interface (GUI) and one or more physical buttons on user equipment 210. In various embodiments, I / O unit 304 may be configured to receive user input from the user and / or any external components / devices and facilitate prediction of a set of possible categories based on the user input. It should be understood that although I / O unit 304 is depicted as a single entity, I / O unit 304 is intended to include multiple units associated with user equipment 210.

[0038] In various embodiments, the I / O unit 304, which communicates with the transceiver, can facilitate communication with the system 220 and can employ communication protocols / standards such as, but not limited to, Code Division Multiple Access (CDMA), High-Speed ​​Packet Access (HSPA+), Global System for Mobile Communications (GSM), 3rd Generation Cellular, Long Term Evolution (LTE), 5th Generation Cellular, WiMax, WiFi, Bluetooth, Bluetooth Low Energy (BLE), etc.

[0039] The embodiments are non-limiting examples, and user equipment 210 may include any additional components necessary to achieve the desired functionality of user equipment 210, such as, but not limited to, processor(s), memory(s), etc., for example, to provide test data. Due to the general nature of the components, descriptions of the components may be omitted for brevity.

[0040] System 220 may include memory 306, one or more modules (e.g., including various circuitry and / or executable program instructions) 308, and processor / controller (e.g., including processing circuitry) 310 (hereinafter referred to as 'processor 310'). In various embodiments, one or more modules 308 may be included within memory 306. In various embodiments, memory 306 may be communicatively coupled to processor 310. Memory 306 may be configured to store data and instructions executable by processor 310. Memory 306 may include a database 306A configured to store data.

[0041] In various embodiments, one or more modules 308 may include a set of instructions executable to cause system 220 to perform any or more of the methods disclosed herein. As discussed throughout this disclosure, one or more modules 308 may be configured to perform the steps of this disclosure using data stored in database 306A to facilitate the prediction of a set of possible categories. In embodiments, each of the one or more modules 308 may be a hardware unit that may be external to memory 306. Furthermore, memory 306 may include operating system 306B for performing one or more tasks of system 220, such as those performed by a general-purpose operating system in a communications domain.

[0042] Memory 306 may include a training dataset unit 306C, which includes a training dataset and is configured to store training data. Processor 310, in conjunction with module 308, can determine a set of possible categories for test data based on this training data. Memory 306 is operable to store instructions executable by processor 310. The functions, actions, or tasks shown or described in the figures can be executed by a programmed processor 310, which executes the instructions stored in memory 306. Functions, actions, or tasks are independent of a specific type of instruction set, storage medium, processor, or processing strategy, and can be executed by software, hardware, integrated circuits, firmware, microcode, etc., operating individually or in combination. Similarly, processing strategies may include multiprocessing, multitasking, parallel processing, etc.

[0043] For the sake of brevity, the architecture and standard operation of the operating system 306B, memory 306, database 306A, and processor 310 are not discussed in detail. In an embodiment, database 306A may be configured to store information required by one or more modules 308 and processor 310 to perform one or more functions to predict a set of possible categories of test data.

[0044] In various embodiments, memory 306 may communicate via a bus within system 220. Memory 306 may include, but is not limited to, non-transitory computer-readable storage media, such as various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media, etc. In examples, memory 306 may include a cache or random access memory for the processor. In alternative examples, memory 306 is decoupled from the processor, such as the processor's cache memory, system memory, or other memory.

[0045] Furthermore, this disclosure envisions a computer-readable medium that includes instructions or receives and executes instructions in response to propagated signals, enabling devices connected to a network to transmit voice, video, audio, images, or any other data over the network. Additionally, instructions can be sent or received over the network via a communication port or interface or using a bus (not shown). The communication port or interface may be part of processor 310 or may be a separate component. The communication port may be created in software or may be a physical connection in hardware. The communication port may be configured to connect to a network, external media, a display, or any other component or combination thereof in the system. The connection to the network may be a physical connection such as a wired Ethernet connection, or it may be established wirelessly. Similarly, additional connections to other components of system 220 may be physical or can be established wirelessly. The network may alternatively be directly connected to the bus.

[0046] In embodiments, processor 310 may include dedicated processing units, such as an integrated system (bus) controller, a memory management control unit, a floating-point unit, a graphics processing unit, a digital signal processing unit, etc. In embodiments, processor 310 may include a central processing unit (CPU), a graphics processing unit (GPU), or both. Processor 310 may be one or more general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), servers, networks, digital circuits, analog circuits, combinations thereof, or other devices now known or later developed for analyzing and processing data. In various embodiments, processor 310 may include one or more processors. One or more processors may be general-purpose processors (such as central processing units (CPUs), application processors (APs), etc.), graphics-only units (such as graphics processing units (GPUs), vision processing units (VPUs)), and / or AI-specific processors (such as neural processing units (NPUs)). Processor 310 may implement software programs, such as manually generated (e.g., programmed) code. In other words, processor 310 may include various processing circuits and / or multiple processors. For example, as used herein (including the claims), the term "processor" can include various processing circuitry, including at least one processor, wherein one or more of the at least one processor can be configured individually and / or collectively in a distributed manner to perform the various functions described herein. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform a number of functions, these terms cover, for example, but not limited to, a situation where one processor performs some of the functions and another processor performs other functions, and a situation where a single processor can perform all of the functions. Additionally, at least one processor can include, for example, a combination of processors performing the various described / disclosed functions in a distributed manner. At least one processor can execute program instructions to implement or perform the various functions.

[0047] In various embodiments, processor 310 may be configured to communicate with user equipment 210 via a network interface (not shown). In various embodiments, where system 220 is integrated within user equipment 210, the network interface may function as an I / O unit, such as I / O unit 304. The network interface may be connected to a communication network, such as communication device 230. The network interface may employ connectivity protocols, including but not limited to direct connection, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), Transmission Control Protocol / Internet Protocol (TCP / IP), Token Ring, IEEE 802.11a / b / g / n / x, etc.

[0048] In various embodiments, as described above, system 220 can be provided in a distributed manner. For example, processor 310 and associated functions can be provided through user equipment 210, i.e., processor 310 can be integrated within user equipment 210. Furthermore, memory 306 and associated functions can be provided through a cloud-based system.

[0049] In various embodiments, the processor can control the processing of input data based on predefined operating rules or artificial intelligence (AI) models stored in non-volatile memory and volatile memory. The predefined operating rules or AI models are provided through training or learning.

[0050] Here, learning is used to provide, for example, by applying learning techniques to multiple learning datasets to formulate predefined operating rules or AI models with desired characteristics. Learning can be performed within the device itself, according to the embodiment of the AI, and / or can be implemented via a separate server / system.

[0051] AI models can include multiple neural network layers. Each layer can have multiple weight values, and layer operations can be performed by computing the operations of previous layers and multiple weights. Examples of neural networks include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), and deep Q-networks.

[0052] Learning techniques can refer to methods, such as using multiple learning data to train a predetermined target device (e.g., a robot) to induce, permit, or control the target device to make a determination or prediction. Examples of learning techniques include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0053] According to this disclosure, a method for predicting a set of possible categories can use an artificial intelligence (AI) model to process test data. The processor can perform preprocessing operations on the data to transform it into a form suitable for use as input to the AI ​​model. The AI ​​model can be obtained through training. Here, "obtained through training" can refer to, for example, training a basic AI model using multiple training data sets using training techniques to obtain a predefined operating rule or AI model configured to perform a desired feature (or purpose). The AI ​​model can include multiple neural network layers. Each of the multiple neural network layers can include multiple weight values, and neural network computation can be performed by computation between the results of previous layers and the multiple weight values.

[0054] Reasoning and prediction can refer to techniques for logical reasoning and prediction based on known information, and includes, for example, knowledge-based reasoning, optimization prediction, preference-based planning, or recommendation.

[0055] Figure 4 This is a block diagram illustrating an example configuration of one or more modules 308 of a system 220 for predicting a set of possible categories for test data according to various embodiments. In embodiments, one or more modules 308 may include a feature extractor 402, an average distance calculator 404, a first membership probability calculator 406, a normalization factor calculator 408, a second membership probability calculator 410, a category membership predictor 412, and a category selector 414, each of which may include various circuitry and / or executable program instructions. Furthermore, one or more modules 308, in conjunction with a memory 306 and a processor 310, perform their designated functions. (The following is a continuation of the previous paragraph...) Figure 3 Detailed descriptions of each module in discussion module 308.

[0056] In various embodiments, one or more modules 308 may be communicatively coupled to other components of system 220, such as memory 306. One or more modules 308 may also be coupled to user equipment 210. One or more modules 308 may be configured to receive one or more inputs from user equipment 210.

[0057] refer to Figure 3 and Figure 4 The processor 310 can be configured to receive test data from the user equipment 210 in conjunction with the feature extractor 402. The processor 310 can also be configured to extract a test feature vector from the test data in conjunction with the feature extractor 402. In various embodiments, the test feature vector can be associated with multiple features corresponding to the test data. In various embodiments, the test feature vector can be represented by an array of features associated with the test data.

[0058] Processor 310 can also be configured to retrieve multiple categories and multiple corresponding training feature vectors for each category from memory 306 (specifically from training dataset unit 306C). In various embodiments, the multiple categories may be pre-stored in training dataset 306C. Processor 310, in conjunction with feature extractor 402, can be configured to extract multiple corresponding training feature vectors for each category from training data associated with training data stored in training dataset unit 306C.

[0059] As an example, processor 310 can be configured to extract test feature vectors from test data in conjunction with feature extractor 402. Furthermore, the training dataset 306C can have multiple classes C1, C2, ... C N For category C1, multiple corresponding training feature vectors {f11, f12, f13, ...} can be extracted. Similarly, for category C2, multiple corresponding training feature vectors {f21, f22, f23, ...} can be extracted. Accordingly, for each category Cn (where... ), multiple corresponding feature vectors It can be extracted by processor 310 in conjunction with feature extractor 402.

[0060] In various embodiments, considering one or more modules 308, the feature extractor 402 can receive test data as input and retrieve multiple categories from memory 306. Furthermore, the feature extractor 402 can extract test feature vectors from the test data. And extract multiple corresponding training feature vectors from multiple categories. Furthermore, the feature extractor 402 can provide multiple corresponding training feature vectors to the first membership probability calculator 406. Furthermore, feature extractor 402 can extract test feature vectors. and multiple corresponding training feature vectors Both are provided to the average distance calculator 404, the normalization factor calculator 408, and the second membership probability calculator 410.

[0061] Processor 310 can be configured to determine a set of membership probabilities for test data. The set of membership probabilities includes a corresponding membership probability associated with each of a plurality of categories. The corresponding membership probability indicates the probability that the test data belongs to a corresponding category among the plurality of categories.

[0062] Continuing with the example above, for each category The corresponding membership probability can be determined. In other words, for categories The corresponding membership probability Test data can belong to a category. The probability. When considering each category Determine the corresponding membership probability At that time, the set of membership probabilities is obtained { }, which will be referred to interchangeably as { in the following text. }

[0063] Processor 310 can be configured to receive user input indicating the type of test data. The type of test data can include a recognition type or a detection type. In a recognition type, members of a category are considered approximate representations of the characteristics of the category. In a detection type, members of a category are considered true and complete instances of the category. Examples of recognition types can include fingerprint recognition, where each training data point is an approximate representation. Examples of detection problems can include detecting all faces in an image, where each training data point can be considered an independent version of a category.

[0064] When the type associated with the test data is an identification type, the processor 310 can be configured to combine the average distance calculator 404 and the first membership probability calculator 406 to determine the set of membership probabilities.

[0065] The processor 310 can be configured to combine with the average distance calculator 404 to determine the corresponding average vector for each of the multiple categories based on multiple corresponding training feature vectors. Furthermore, the processor 310 can be configured to combine with the average distance calculator 404 to determine the distance vector for each of the multiple categories based on the corresponding average vector and the test feature vector.

[0066] In various embodiments, processor 310 may be configured to select a category from a plurality of categories and, for the selected category, perform a first processing step, the first processing step including accessing a plurality of corresponding training feature vectors of the selected category, determining a corresponding average vector of the selected category, and determining a corresponding distance vector of the selected category. Processor 310 may be configured to repeat the first processing step for each selected category among the plurality of categories.

[0067] Continuing with the example above, for each category The corresponding average vector can be determined. Furthermore, for each category The corresponding distance vector can be determined. In other words, for categories The average vector can be determined. and distance vector For categories The average vector can be determined. and distance vector ,etc.

[0068] In various embodiments, the average vector can be determined based on equation (1). :

[0069] …(1)

[0070] In various embodiments, the distance vector It can be determined based on equation (2):

[0071] …(2)

[0072] In various embodiments, considering one or more modules 308, the average distance calculator 404 can receive test feature vectors from the feature extractor 402. and multiple corresponding training feature vectors As input. Furthermore, the average distance calculator 404 can determine the corresponding average vector. and the corresponding distance vector And output it to the first membership probability calculator 406.

[0073] Processor 310 can be configured to select a category from a plurality of categories and, in conjunction with first membership probability calculator 406, perform a second processing step for the selected category. In the second processing step, processor 310 can be configured to determine a difference parameter for each of a plurality of corresponding feature vectors of the selected category. As a result, a set of difference parameters associated with the selected category is obtained.

[0074] Furthermore, in the second processing step, processor 310 can be configured to determine the standard deviation associated with the selected category based on the set of difference parameters. Additionally, in the second processing step, processor 310 can be configured to determine the distribution of a plurality of corresponding training feature vectors relative to their corresponding mean vectors. In various embodiments, a normal distribution can be determined by processor 310.

[0075] Furthermore, in the second processing step, processor 310 can be configured to select a probability density function associated with the determined distribution. Additionally, in the second processing step, processor 310 can be configured to determine the corresponding membership probability of the test feature vector for the selected category based on the probability density function, the magnitude of the corresponding distance vector, and the determined standard deviation. Processor 310 can be configured to repeat the second processing step for each selected category among multiple categories, in conjunction with the first membership probability calculator 406.

[0076] Once the membership probability of the test feature vector has been determined for each of the multiple categories, the processor 310 can determine a set of membership probabilities for the test feature vector. The set of membership probabilities can indicate the probability that the test data belongs to multiple categories, such as the probability that the test data belongs to the first category, the probability that the test data belongs to the second category, etc.

[0077] Continuing with the example above, we can target each category. Accessing multiple training feature vectors Furthermore, multiple training feature vectors can be calculated. The corresponding difference parameter for each of them Difference parameters It can indicate along the distance vector The corresponding training feature vector and average vector of the projection The difference. That is, for categories. Difference parameters It can indicate along the distance vector The corresponding training feature vector of the projection With average vector The difference. In various embodiments, the corresponding difference parameter can be determined based on equation (3). :

[0078] …(3)

[0079] In addition, when targeting categories Multiple associated corresponding training feature vectors Each of the calculations corresponds to a difference parameter. At that time, it can be targeted by category Obtain difference parameters A set of values. Furthermore, it can be based on the difference parameter. The set of values ​​is used to calculate the standard deviation. For categories Therefore, we can obtain the distance vector along the corresponding path. The standard deviation of the associated training data for the projection.

[0080] In addition, for each category It can be based on the probability density function of the determined distribution, category standard deviation and categories Distance vector The size determines the corresponding membership probability. In various embodiments, if the probability density function is expressed as Then each category Corresponding membership probability It can be given by the following equation (4):

[0081]

[0082] When for each category Determine the corresponding membership probability Therefore, we obtain the set of membership probabilities { }

[0083] In various embodiments, considering one or more modules 308, the first membership probability calculator 406 may receive a distance vector from the average distance calculator 404. and average vector As input, and receiving multiple corresponding training feature vectors from feature extractor 402. As input. Furthermore, the first membership probability calculator 406 can determine the set of membership probabilities { } and output it to the category membership predictor 412.

[0084] When the type associated with the test data is a detection type, the processor 310 can be configured to combine the normalization factor calculator 408 and the second membership probability calculator 410 to determine the set of membership probabilities.

[0085] Processor 310 can be configured to receive multiple corresponding training feature vectors and test feature vectors from feature extractor 402 in conjunction with normalization factor calculator 408. Processor 310 can also be configured to receive user input from user device 210 indicating a system index. The system index can indicate the sensitivity of the learning model to outliers (e.g., data far from the mean). In various embodiments, a larger system index means that outliers are given less weight when building the learning model, while a smaller index means that outliers are given relatively more weight when building the learning model.

[0086] The processor 310 can be configured to combine with the normalization factor calculator 408 to determine the normalization factor based on the received system index and multiple corresponding training feature vectors for each of the multiple categories.

[0087] Continuing with the example above, the normalization factor calculator 408 is used to obtain the values ​​for each category. Multiple corresponding training feature vectors Test feature vectors and system index In addition, the normalization factor The normalization factor is calculated by the normalization factor calculator 408 and provided as output to the second membership probability calculator 410. In various embodiments, the normalization factor can be determined based on equation (5). :

[0088] …(5)

[0089] Processor 310 can be configured to combine with second membership probability calculator 410 to determine a set of membership probabilities. In various embodiments, processor 310 can be configured to select a category from multiple categories, and for each selected category, perform a third processing step. In the third processing step, processor 310 can be configured to determine the corresponding membership probability for the selected category based on a normalization factor, a test feature vector, a system index, and multiple corresponding training feature vectors of the selected category.

[0090] In the third processing step, processor 310 may be configured to determine a set of membership probabilities based on the corresponding membership probabilities determined for each selected category among the plurality of categories. Processor 310 may be configured to repeat the third processing step in conjunction with a second membership probability calculator 410 for each selected category among the plurality of categories.

[0091] Continuing with the example above, the second membership probability calculator 410 can obtain the normalization factor from the normalization factor calculator 408. Obtain the system index from user equipment 210 And obtain each category from feature extractor 402 Multiple corresponding training feature vectors and test feature vectors .

[0092] For each category The membership probability can be determined based on equation (6). :

[0093] …(6)

[0094] Once for each category Determine the membership probability Therefore, we can obtain the set of membership probabilities { }. The set of membership probabilities { This can be provided as output to the category membership predictor 412.

[0095] Processor 310 can be configured to, in conjunction with category membership predictor 412, receive a set of membership probabilities from first membership probability calculator 406 when test data can be associated with a recognition type, and a set of membership probabilities from second membership probability calculator 410 when test data can be associated with a detection type. Processor 310 can also be configured to sort the set of membership probabilities to determine a sorted probability array. In various embodiments, the sorted probability array may include a set of membership probabilities, for example, the corresponding membership probabilities of each of a plurality of categories sorted based on the values ​​of the corresponding membership probabilities. In various embodiments, the sorting may be in descending order, such that the corresponding membership probability with the highest value is followed by corresponding membership probabilities with decreasing values.

[0096] Continuing with the example above, processor 310 can be configured to support a set of membership probabilities { Sort the array to determine the sorted probability array. For multiple categories Each category in the sorted array It can include a set of membership probabilities sorted from high to low. For example, a sorted array. It can include values ​​therein Category { } related { }. Based on the set of membership probabilities { The value of the corresponding membership probability of} }, sorted array Indicator test data has a category Highest probability, category The next highest probability, etc. A sorted array. It can be provided as output to category selector 414.

[0097] In various embodiments, considering one or more modules 308, the category membership predictor 412 may receive a set of membership probabilities from either a first membership probability calculator 406 (in the case of identifying types) or a second membership probability calculator 410 (in the case of detecting types). }, and can also determine the sorted array. And output it to category selector 414.

[0098] Processor 310 can be configured to receive user input from user equipment 210 in conjunction with class selector 414, the user input indicating the target probability required for test data. The target probability can indicate the expected probability or minimum probability guarantee of test data falling into one or more of a plurality of classes.

[0099] Processor 310 can be configured to combine with category selector 414 to determine a set of possible categories from a plurality of categories. In various embodiments, processor 310 can be configured to select the set of possible categories based on a sorted array of probabilities and a target probability received via user input. The set of possible categories can be selected such that the combined probability of the set of possible categories is greater than the target probability.

[0100] Continuing with the example above, processor 310 can be configured based on a sorted array of probabilities. and target probability From multiple categories Select several possible categories Sorted probability array Including the sorted set of membership probabilities { Processor 310, in conjunction with category selector 414, can select from a sorted array. Select the highest There are several probabilities that make the highest chosen probability... The combined probability of these probabilities is greater than the target probability. Once chosen The highest probability will be with The highest probability (e.g., The associated categories are determined as a set of possible categories. For example, suppose a sorted array... Including sorted and categorized A set of associated membership probabilities. Membership probability. The combined probability can be greater than the target probability. The result is related to the corresponding membership probability. Related categories Form a set of possible categories Similarly, in the combination probability of membership probability... greater than the target probability In the case of [the following], then [it is related to] the corresponding membership probability. Related categories Form a set that is identified as possible categories. The probability of a combination.

[0101] In various embodiments, the selection from the sorted array can be based on equation (7). highest One probability:

[0102] …(7)

[0103] In other words, select the minimum number of probabilities that can be combined to give a combined probability greater than the target probability. In various embodiments, this comes from a sorted array. of A probability can be selected as a membership probability of a combination of probabilities that has a minimum possible value greater than the target probability.

[0104] Processor 310 can be configured to handle a set of possible categories. The output, along with the corresponding membership probabilities of the set of possible categories, is sent to user device 210. In various embodiments, the output may be visual or audiovisual. In various embodiments, the output may be provided via an application programming interface (API) for use in one or more additional user devices.

[0105] In various embodiments, the set of possible categories The corresponding membership probabilities of the set of possible categories can be displayed on a user interface associated with user device 210, such as via a display of user device 210. Therefore, the user can view the probabilities that the test data belongs to different categories, such that the combination of probabilities is greater than the user's minimum expected probability. For example, if the user expects a 90% probability, rather than simply choosing the category with the highest probability, a set of categories with a total combined probability greater than 90% is provided. Accordingly, conformal predictions can be provided instead of misclassification, and the output set of categories is always guaranteed to have a high success probability of being selected by the user.

[0106] Figure 5 This is a diagram illustrating example use cases for predicting a set of possible categories by system 220 according to various embodiments. (See diagram for example.) Figure 5 As shown, at box 510, system 220 can receive a test image 502 of a hyena. At box 512, system 220 can also receive user input 504 indicating a target probability (e.g., a user-guaranteed expected probability). For example, the target probability could be 90%. System 220 can access training data 506, which may include dog training data, hyena training data, and fox training data. The test image 502 can be overlapped with different training data. At box 514, system 220 can predict that the test image 502 belongs to various categories with corresponding probabilities. For example, system 220 can determine that the test image belongs to the 'dog' category with a 55% probability, the 'hyena' category with a 36% probability, and the 'fox' category with a 19% probability. At box 516, system 220 can determine the set of possible categories as {dog, hyena} because the combined probability of the set of possible categories (55% and 36%) is greater than the target probability (90%). Accordingly, system 220 can output a set of possible categories {dog, hyena} to the user.

[0107] Figure 6AThis diagram illustrates an example use case for system 220 to predict a set of possible categories according to various embodiments. Voice data from user 602 can be provided as input to system 220 integrated with user device 604. The voice data may include the words “search Korean pledge”. System 220 can process the voice data, and regarding the voice data for the word “pledge”, system 220 can classify the voice data into two words, “placed” and “pledge”, corresponding with corresponding probabilities. System 220 can display a set of possible categories to user device 604, in this case, “placed” and “pledge” as indicated by arrow 606. Therefore, user 602 can select the correct option from the set of possible categories, in this case, “pledge” as indicated by arrow 608. Accordingly, the correct output can be provided to the associated user 602. This can thus make the speech-to-text application more reliable. Therefore, user 602 can quickly select the relevant words from the available options, thus avoiding deletion and re-speaking. Furthermore, this will promote the increasing adoption of voice-based speech-to-text typing in devices.

[0108] Figure 6B This diagram illustrates example use cases for predicting a set of possible categories by system 220 according to various embodiments. Audio or video data can be provided as input to system 220, and system 220 can process the input to generate subtitles. For example, original Korean oath 610 can be provided as input to system 220. System 220 can process the audio and determine a set of possible categories for words that are not clearly identified. System 220 can display the set of possible categories for unclear words on a display, allowing a user to select an appropriate option from the set of possible categories. For example, subtitle 612 can be generated by system 220. Figure 7As shown, the input word "pledge" has been processed by system 220, and the user is provided with a set of possible categories {placed, pledge} to select the relevant option. The target probability of system 220 can be, for example, 90%, and the combined probability of the set of possible categories {placed, pledge} determined by system 220 (e.g., 91%) can be greater than the target probability. Therefore, the user is provided with a set of possible categories {placed, pledge} to select the relevant option. Similarly, for the input word "Taegeuk", a set of possible categories {take book, textbook, Taegeuk} is provided; for the input word "allegiance", a set of possible categories {elite jeans, allegiance} is provided; and for the input word "eternal", a set of possible categories {terminal, eternal} is provided for the user to select the appropriate option. As a result, the user can mentally select the correct word to read based on sound and context. Furthermore, hearing-impaired users can mentally select the correct word to read based on context.

[0109] Figure 7 This is a diagram illustrating example use cases for predicting a set of possible categories by system 220 according to various embodiments. (See diagram for example.) Figure 7 As shown, at box 702, disease symptoms can be provided to system 220 as input data. At box 704, system 220 can process the input data and determine multiple categories with corresponding membership probabilities. For example, system 220 can determine the probability that the input data (symptoms) belongs to each category (disease). For example, system 220 can determine that the probability of symptoms matching disease 1 is 55%, the probability of symptoms matching disease 2 is 36%, and the probability of symptoms matching disease 3 is 8%. At box 706, system 220 can receive user input indicating a target probability of 90%. Furthermore, system 220 can determine that the set of possible categories is {disease 1, disease 2, disease 3} because the combined probability of the set of possible categories is greater than the target probability. The set of possible categories can be displayed on the user interface to allow the user to view diseases that may be associated with the symptoms. As a result, key diseases, such as disease 3, are not missed, and the risk of symptoms being associated with key disease 3 is identified. Accordingly, in medical diagnosis, the detection of missed key diseases is eliminated.

[0110] Figure 8This is a diagram illustrating example use cases for system 220 to predict a set of possible categories, according to various embodiments. System 220 can be integrated with authentication system 802. When user 804 attempts to pass through authentication system 802, authentication system 802 examines input data from user 804, such as biometrics. System 220 may have a threshold of 90% required for successful access. System 220 can process the input data from user 804 to determine the set of possible categories for user 804 as {administrator, employee}. Since the probability of the combination of this set of possible categories is greater than the threshold of 90%, access and privileges are granted. Accordingly, the authentication failure rate is low, and in some cases, minimum privileges can be granted instead of complete denial of access.

[0111] Figure 9 This diagram illustrates example use cases for predicting a set of possible categories by system 220 according to various embodiments. System 220 can be integrated with facial recognition engine 902. System 220 can process facial features as input from user 904 and determine a set of possible categories that match the facial features. For example, system 220 can have a 90% threshold, and system 220 can determine that user 904 can be one of the set of users {A, B} because the probability of combining the set of users can be greater than the threshold. Therefore, facial recognition can be successful, and the chance of facial recognition failure is small.

[0112] Figure 10 This diagram illustrates example use cases for system 220 to predict a set of possible categories according to various embodiments. At box 1002, system 220 may receive input data. The input data may be historical data, such as data associated with finance, economics, natural disasters, etc. At box 1004, system 220 may predict probabilities associated with multiple scenarios, such as scenario 1 having a 55% probability, scenario 2 having a 36% probability, and scenario 3 having an 8% probability. At box 1006, system 220 may determine the set of possible categories based on a prediction threshold (e.g., 90%). System 220 may predict that the set of possible categories includes {scenario 1, scenario 2, scenario 3}. Therefore, no worst-case scenario, such as scenario 3, is missed; instead, the scenario is identified as one of the scenarios in the set of possible categories. Accordingly, there is no risk of missing the identification of worst-case scenarios (such as worst-case minimum, possible worst-case disaster, etc.).

[0113] refer to Figure 11A This is a flowchart illustrating an example method 1100 for predicting a set of possible categories for test data according to various embodiments. In embodiments, the steps or operations of method 1100 may be performed by system 220, as described above.

[0114] At 1102, method 1100 includes: retrieving from a memory containing training dataset units a plurality of classes and a plurality of corresponding training feature vectors for each of the plurality of classes.

[0115] At 1104, method 1100 includes: receiving an input indicating the target probability required for the test data, such as user input.

[0116] At 1106, method 1100 includes: determining a set of membership probabilities for the test data. The set of membership probabilities includes a corresponding membership probability associated with each of the multiple categories. The corresponding membership probability indicates the probability that the test data belongs to the corresponding category among the multiple categories.

[0117] At 1108, method 1100 includes: determining a set of possible categories of test data from multiple categories based on user input and a set of membership probabilities.

[0118] In various embodiments, user input may indicate the type associated with the test data. This type may be either an identification type or a detection type.

[0119] In various embodiments, when the type associated with the test data is an identification type, method 1100 may include sub-steps 1106A-1106J to determine a set of membership probabilities for the test data, as referenced below. Figure 11B More detailed illustrations are provided.

[0120] At 1106A, method 1100 includes: selecting a category from multiple categories. At 1106B, method 1100 includes: for each selected category, accessing multiple corresponding training feature vectors.

[0121] At 1106C, method 1100 includes: determining a corresponding average vector based on multiple corresponding training feature vectors. At 1106D, method 1100 includes: determining a corresponding distance vector associated with the selected class based on the corresponding average vector and a test feature vector associated with the test data.

[0122] At 1106E, method 1100 includes: determining a corresponding difference parameter for each of a plurality of corresponding feature vectors of the selected category based on the corresponding distance vector and the corresponding average vector, thereby determining a set of difference parameters associated with the selected category.

[0123] At 1106F, method 1100 includes: determining the standard deviation associated with the selected category based on a set of difference parameters. At 1106G, method 1100 includes: determining the distribution of a plurality of corresponding training feature vectors relative to corresponding mean vectors.

[0124] At 1106H, method 1100 includes: selecting a probability density function associated with the determined distribution. At 1106I, method 1100 includes: determining the corresponding membership probability of the test feature vector for the selected category based on the probability density function, the magnitude of the corresponding distance vector, and the determined standard deviation.

[0125] At 1106J, method 1100 includes: determining a set of membership probabilities of a test feature vector based on the corresponding membership probabilities determined for each selected category among a plurality of categories.

[0126] In various embodiments, when the type associated with the test data is a detection type, method 1100 may include substeps 1106K-1106N to determine a set of membership probabilities for the test data, as referenced below. Figure 11C More detailed illustrations are provided.

[0127] At 1106K, method 1100 includes: receiving user input indicating a system index. At 1106L, method 1100 includes: determining a normalization factor based on the received system index and multiple corresponding training feature vectors for each of the multiple categories.

[0128] At 1106M, method 1100 includes: selecting a category from multiple categories, and for each selected category, determining a corresponding membership probability for the selected category based on a normalization factor, a test feature vector associated with the test data, a system index, and multiple corresponding training feature vectors of the selected category.

[0129] At 1106N, method 1100 includes: determining a set of membership probabilities based on the corresponding membership probabilities determined for each selected category among a plurality of categories.

[0130] In various embodiments, method 1100 may further include: extracting test feature vectors from test data. The test feature vectors are associated with multiple features corresponding to the test data. In various embodiments, method 1100 may further include: extracting multiple corresponding training feature vectors for multiple categories from training data associated with training data stored in training dataset unit 306C.

[0131] In various embodiments, method 1100 may further include: sorting the set of membership probabilities to form a sorted probability array. In various embodiments, method 1100 may further include: selecting a set of possible classes based on the sorted probability array and a target probability. The combined probability of the set of possible classes is greater than the target probability.

[0132] In various embodiments, method 1100 may further include providing an output indicating a set of possible categories via a user device.

[0133] Although shown and described in a specific order Figure 11A , Figure 11B and Figure 11C The above operations are performed, but the order of steps can vary depending on the specific embodiments. Furthermore, with... Figure 11A , Figure 11B and Figure 11C Detailed descriptions of each step are covered in the relevant section. Figures 2-4 The relevant descriptions are already available, and for the sake of brevity, they will not be repeated here.

[0134] This disclosure provides various technological advancements based on the key features discussed above. This disclosure provides methods and systems that guarantee class prediction probabilities in a fault-tolerant (e.g., fail-safe) manner. There is no risk of misclassification because the systems and methods disclosed herein provide fault-tolerant classification to conformally predict the set of possible classes for test data while taking into account the minimum probability guarantee of user expectations.

[0135] Furthermore, classification failures may not lead to incorrect classifications; instead, the system and method provide a more general set of output categories. The output set of categories is always guaranteed to have a high probability of success, as this probability is based on the target probability provided by the user. Moreover, efficiency is improved as failed classifications are eliminated; for example, categories that are not part of the output set of categories can be excluded with high confidence.

[0136] This disclosure provides methods and systems that are highly useful in a variety of applications, as referenced in [reference]. Figure 5 , Figure 6A , Figure 6B , Figure 7 , Figure 8 , Figure 9 and Figure 10 The use cases described herein. This disclosure provides methods and systems that are highly beneficial in high-risk settings, particularly in areas where the cost of incorrect classification is high. For example, in the fields of medical diagnostics and security, the risk of any prediction errors is significantly reduced. In the field of speech-to-text processing, multiple predictions can be provided instead of incorrect predictions, allowing users to select the relevant options. In the field of authentication, least privilege can be provided and access can be allowed instead of failed authentication.

[0137] While this disclosure has been described in specific language, it is not intended to create any limitation. It will be apparent to those skilled in the art that various modifications can be made to achieve the disclosure as taught herein. The accompanying drawings and the foregoing description provide examples of various embodiments. Those skilled in the art will understand that one or more of the described elements can be well combined into a single functional element. Alternatively, certain elements may be divided into multiple functional elements. It should also be understood that any embodiment described herein can be used in conjunction with any other embodiment described herein.

Claims

1. A method for predicting a set of possible categories for test data, the method comprising: Obtain multiple categories and multiple corresponding training feature vectors for each of the multiple categories from a memory that includes the training dataset; Receive input indicating the target probability required for the test data; Determine a set of membership probabilities for the test data, wherein the set of membership probabilities includes a corresponding membership probability associated with each of the plurality of categories, and wherein the corresponding membership probability indicates the probability that the test data belongs to a corresponding category among the plurality of categories; and Based on the input and the set of membership probabilities, determine the set of possible categories of the test data from the plurality of categories.

2. The method according to claim 1, comprising: Receive input indicating the type associated with the test data, wherein the type is one of an identification type or a detection type.

3. The method according to claim 2, wherein, The set of membership probabilities for the test data, determined based on the type associated with the test data being the identification type, includes: Select a category from the multiple categories, and for each selected category: Access the multiple corresponding training feature vectors; Based on the multiple corresponding training feature vectors, determine the corresponding average vector; and Based on the corresponding average vector and the test feature vector associated with the test data, a corresponding distance vector associated with the selected category is determined.

4. The method according to claim 3, comprising: For each selected category, Based on the corresponding distance vector and the corresponding average vector, determine the corresponding difference parameter for each of the plurality of corresponding feature vectors of the selected category, so as to determine the set of difference parameters associated with the selected category; and Based on the set of difference parameters, the standard deviation associated with the selected category is determined.

5. The method according to claim 4, comprising: Determine the distribution of the plurality of corresponding training feature vectors relative to the corresponding average vector; Choose a probability density function that is associated with the determined distribution; Based on the probability density function, the magnitude of the corresponding distance vector, and the determined standard deviation, the corresponding membership probability of the test feature vector is determined for the selected category; as well as The set of membership probabilities of the test feature vector is determined based on the corresponding membership probability determined for each selected category among the plurality of categories.

6. The method according to claim 3, comprising: Extract the test feature vector from the test data, wherein the test feature vector is associated with multiple features corresponding to the test data; and Multiple corresponding training feature vectors for the multiple categories are extracted from the training data associated with the training dataset.

7. The method according to claim 2, wherein, The set of membership probabilities for the test data, determined based on the type associated with the test data being the detection type, includes: Receive input indicating the system index; and A normalization factor is determined based on the received system index and the corresponding training feature vectors for each of the plurality of categories.

8. The method of claim 7, comprising: Select a category from the multiple categories; For each selected category, the corresponding membership probability is determined based on the normalization factor, the test feature vector associated with the test data, the system index, and the multiple corresponding training feature vectors of the selected category. as well as The set of membership probabilities is determined based on the corresponding membership probability determined for each of the plurality of categories.

9. The method of claim 8, comprising: Extract the test feature vector from the test data, wherein the test feature vector is associated with multiple features corresponding to the test data; and Multiple corresponding training feature vectors for the multiple categories are extracted from the training data associated with the training dataset.

10. The method according to claim 1, wherein, Determining the set of possible categories includes: The set of membership probabilities is sorted to form a sorted probability array; and The set of possible categories is selected based on the sorted probability array and the target probability, wherein the combined probability of the set of possible categories is greater than the target probability.

11. The method according to claim 1, comprising: The user equipment provides an output indicating the set of possible categories. The output is either visual output or audiovisual output.

12. A system configured to predict a set of possible categories of test data, the system comprising: Memory; as well as At least one processor, including processing circuitry, is communicatively coupled to the memory, and the at least one processor is individually and / or collectively configured to: Retrieve from the memory the training dataset, multiple categories, and multiple corresponding training feature vectors for each of the multiple categories; Receive input indicating the target probability required for the test data; Determine a set of membership probabilities for the test data, wherein the set of membership probabilities includes a corresponding membership probability associated with each of the plurality of categories, and wherein the corresponding membership probability indicates the probability that the test data belongs to a corresponding category among the plurality of categories; and Based on the input and the set of membership probabilities, determine the set of possible categories of the test data from the plurality of categories.

13. The system according to claim 12, wherein, The at least one processor is configured individually and / or collectively to: Receive input indicating the type associated with the test data, wherein the type is one of an identification type or a detection type.

14. The system according to claim 13, wherein, Based on the fact that the type associated with the test data is the identification type, in order to determine the set of membership probabilities of the test data, the at least one processor is individually and / or collectively configured to: Select a category from the multiple categories, and for each selected category: Access the multiple corresponding training feature vectors; Based on the multiple corresponding training feature vectors, determine the corresponding average vector; and Based on the corresponding average vector and the test feature vector associated with the test data, a corresponding distance vector associated with the selected category is determined.

15. The system according to claim 14, wherein, The at least one processor is configured individually and / or collectively to: for each selected category, Based on the corresponding distance vector and the corresponding average vector, determine the corresponding difference parameter for each of the plurality of corresponding feature vectors of the selected category, so as to determine the set of difference parameters associated with the selected category; and Based on the set of difference parameters, the standard deviation associated with the selected category is determined.

Images