Out-of-distribution sample detection method, device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an out-of-distribution sample detection method and device, equipment and a medium. The method is applied to a medical scene, and comprises the following steps: obtaining a plurality of visual features of a to-be-detected sample, so as to determine a preceding text relationship of the to-be-detected sample according to the plurality of visual features, increase the detection capability of the to-be-detected sample, and increase the text description of the to-be-detected sample by performing text enhancement processing on a label. Therefore, when the multiple visual features and the enhanced label are combined to detect the to-be-detected sample, the context relation can be considered, and the detection accuracy of the to-be-detected sample is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for detecting out-of-distribution samples. Background Art

[0002] Different from the sample data used to train the model, i.e., in-distribution samples, out-of-distribution samples refer to real samples that are semantically offset from in-distribution samples and do not belong to any of the categories to be classified. Existing deep neural network models tend to give high confidence judgments to out-of-distribution samples. When a deep neural network model receives an out-of-distribution sample, it usually cannot give a reliable result, and may even give a high-confidence but incorrect prediction result. For example, in medical image analysis, when encountering an abnormal structure that has never appeared before, the preset model may make an incorrect detection result for the out-of-distribution sample. Especially when facing new diseases or uncommon conditions, the preset model has low detection accuracy for abnormal structures. Therefore, in the process of detecting out-of-distribution samples, how to improve the detection accuracy has become an urgent problem that needs to be solved. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an out-of-distribution sample detection method, apparatus, device, and medium to solve the problem of low detection accuracy during the detection of out-of-distribution samples.

[0004] In a first aspect, an embodiment of the present invention provides a method for detecting out-of-distribution samples, the method comprising: Obtain N visual features of the sample to be detected, where N is an integer greater than zero; Obtaining labels of the samples to be detected, and performing text enhancement processing on the labels based on the N visual features to obtain N enhanced labels; The sample to be detected is detected according to the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected.

[0005] In a second aspect, an embodiment of the present invention provides an out-of-distribution sample detection device, the out-of-distribution sample detection device comprising: An acquisition module is used to obtain N visual features of the sample to be detected, where N is an integer greater than zero; a processing module, configured to obtain labels of the samples to be detected, and perform text enhancement processing on the labels according to the N visual features to obtain N enhanced labels; The detection module is used to detect the sample to be detected based on the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected.

[0006] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the out-of-distribution sample detection method as described in the first aspect when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the out-of-distribution sample detection method as described in the first aspect is implemented.

[0008] Compared with the prior art, the present invention has the following beneficial effects: In the present invention, multiple visual features of the sample to be detected are obtained so that the contextual relationship of the sample to be detected can be determined based on the multiple visual features, thereby increasing the detection capability of the sample to be detected. By performing text enhancement processing on the label, a text description of the sample to be detected is added, so that when the sample to be detected is detected in combination with the multiple visual features and the enhanced label, the contextual relationship can be considered, thereby improving the detection accuracy of the sample to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0010] Figure 1 This is a schematic diagram of an application environment of an out-of-distribution sample detection method provided by an embodiment of the present invention; Figure 2 1 is a flow chart of a method for detecting out-of-distribution samples provided by an embodiment of the present invention; Figure 3 1 is a schematic structural diagram of an out-of-distribution sample detection device provided by an embodiment of the present invention; Figure 4 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0012] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0013] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0014] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0015] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0016] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0017] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0018] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0019] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0020] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0021] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0022] An out-of-distribution sample detection method provided by an embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, clients communicate with servers. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, personal digital assistants (PDAs), and other computer devices. Servers can be standalone servers or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0023] See also Figure 2 , is a flow chart of a method for detecting out-of-distribution samples provided by an embodiment of the present invention. The above-mentioned method for detecting out-of-distribution samples can be applied to Figure 1 The server in Figure 2 As shown, the out-of-distribution sample detection method may include the following steps.

[0024] S201: Obtain N visual features of a sample to be detected, where N is an integer greater than zero.

[0025] In step S201 , the sample to be detected is the corresponding object to be detected, the sample to be detected is an in-distribution sample, or an out-of-distribution sample, and the visual feature refers to a representative feature extracted from the image.

[0026] In this embodiment, N visual features of the sample to be detected are obtained, where the sample to be detected is an image of a certain disease corresponding to a medical center. The disease can be pneumonia, and the medical center can be a hospital, pharmaceutical company, etc. If a detection model is needed to detect pneumonia in an image, a large number of different pneumonia images need to be used as sample data to train the detection model. However, the pneumonia image data in the medical center may only contain images of common pneumonia symptoms. In actual application, the detection model detects the pneumonia symptoms contained in the pneumonia images. If new symptoms appear, which are different from the pneumonia images in the sample data, the detection model may not produce satisfactory results when processing pneumonia detection for new symptoms, resulting in false detection or missed detection of new symptoms. Therefore, it is necessary to expand the pneumonia images in the medical center and add out-of-distribution samples to increase the number of pneumonia images corresponding to new symptoms that may appear, so as to help the detection model better identify new symptoms. In order to enable the detection model to better identify new symptoms, it is necessary to improve the accuracy of the detection model's detection of out-of-distribution samples, that is, to improve the accuracy of detecting new symptoms that appear.

[0027] In this example, N visual features of the sample to be detected are obtained. The visual features may include the shape, appearance, position and surrounding environment of the sample to be detected, and N is an integer greater than zero.

[0028] It should be noted that when extracting visual features from the sample to be detected, a preset image encoder can be used for feature extraction, such as the image encoder of the Clip (Contrastive Language-Image Pre-Training) model, to obtain N visual features of the sample to be detected. Alternatively, an image encoder can be used for feature extraction, which is not limited in this embodiment.

[0029] S202: Obtain labels of samples to be detected, and perform text enhancement processing on the labels based on N visual features to obtain N enhanced labels.

[0030] In step S202, a label of the sample to be detected is obtained, wherein the label of the sample to be detected is a single word text. Text enhancement processing is performed on the label to obtain an enhanced label, wherein the enhanced label is a sentence text, so that the enhanced label can better describe the sample to be detected.

[0031] In this example, when performing text enhancement on a label, the label is expanded by combining the label with visual features, i.e., the number of words in the label is increased. The combination of visual features with the label makes the enhanced label dependent on the visual features, thereby improving the accuracy of the enhanced label.

[0032] In this example, text enhancement is performed on the labels, that is, the labels are expanded so that the labels are transformed from a single word into a sentence text, thereby fully reflecting the diversity of the space.

[0033] Optionally, text enhancement processing is performed on the label based on N visual features to obtain N enhanced labels, including: Determine N prompt templates based on N visual features; Generate N enhanced labels based on N prompt templates and labels.

[0034] In this example, N hint templates are determined based on N visual features, and N enhanced labels are generated based on these N hint templates and the label. Visual hints are used to indicate the corresponding visual features. For example, for a blurred background, the corresponding hint template is "A photo of {label} with a blurred background," ensuring that the enhanced label reflects the visual context of the sample being tested. For an internally blurred visual hint, the corresponding hint template is "A photo of {label} with blurred details," allowing the enhanced label to capture the detailed attention paid to the sample's environment.

[0035] According to each prompt template and label, a corresponding enhanced label is generated. The formula for generating the corresponding enhanced label is as follows: in, For the enhanced label, is the i-th label, is the hint template corresponding to the p-th visual feature.

[0036] In this embodiment, N prompt templates are determined based on N visual features, and N enhanced labels are generated based on the N prompt templates and labels. This allows the generated enhanced labels to better reflect the corresponding visual features, ensuring a closer fit between the visual features of the sample to be tested and the features of the enhanced labels.

[0037] S203: Detect the sample to be detected based on the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected.

[0038] In step S203, the sample to be detected is detected based on the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected, wherein the detection result includes out-of-distribution samples and in-distribution samples.

[0039] In this embodiment, the sample to be detected is detected based on N visual features and N enhanced labels, that is, the detection is performed in combination with multiple visual features and multiple enhanced labels. During the detection, the detection can be performed based on the similarity between the N visual features and the text features of the N enhanced labels, that is, the similarity between each visual feature and the text feature of the corresponding enhanced label is calculated to obtain N similarity values, and the average of the N similarity values ​​is calculated. When the average of the N similarity values ​​is larger, it is considered that the visual feature and the text feature of the enhanced label are more similar, and the sample to be detected is the corresponding in-distribution sample. When the average of the N similarity values ​​is smaller, it is considered that the visual feature and the text feature of the enhanced label are less similar, and the sample to be detected is the corresponding out-distribution sample. If the average of the N similarity values ​​is greater than the preset average, the detected sample is determined to be the corresponding in-distribution sample. If the average of the N similarity values ​​is not greater than the preset average, the detected sample is determined to be the corresponding out-distribution sample.

[0040] It should be noted that the text features of the N enhanced labels can be extracted using the text encoder of the Clip (Contrastive Language-Image Pre-Training) model to obtain the text features corresponding to the N enhanced labels. Other encoder models may also be used and are not limited in this embodiment.

[0041] It should be noted that when calculating the similarity between each visual feature and the text feature of the corresponding enhanced label, the cosine formula can be used for calculation.

[0042] In this embodiment, the sample to be detected is detected based on N visual features and N enhanced labels, and the contextual relationship of the sample to be detected is taken into consideration, thereby improving the detection accuracy.

[0043] Optionally, the sample to be detected is detected based on the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected, including: According to N visual features, the embedded features of the sample to be detected are calculated; Perform feature extraction on each enhanced label to obtain the text features corresponding to each enhanced label, and calculate the target text features of the sample to be detected based on the text features corresponding to each enhanced label; Calculate the uncertainty of the sample to be tested based on the embedded features and the target text features; Based on the uncertainty, the test results of the sample to be tested are determined.

[0044] In this embodiment, an embedded feature of the sample to be detected is calculated based on N visual features, where the embedded feature is a comprehensive feature that includes the N visual features and represents the contextual characteristics of the sample to be detected. When calculating the embedded feature of the sample to be detected based on the N visual features, the average of the N visual features can be calculated and used as the embedded feature of the sample to be detected.

[0045] In this embodiment, multiple visual features are integrated to obtain embedded features, thereby retaining important environmental information, so that when using the detection model to perform target detection on the sample to be detected, the context features of the corresponding target can be focused on to improve detection accuracy.

[0046] Perform feature extraction on each enhanced label to obtain the text features corresponding to each enhanced label. When performing feature extraction on each enhanced label, a text encoder can be used for text feature extraction, such as using a text encoder of the Clip (Contrastive Language-Image Pre-Training) model to extract the text features of each enhanced label. Other encoder models can also be used, and this embodiment does not limit this. Based on the text features corresponding to each enhanced label, the target text features of the sample to be detected are calculated. The mean of the text features corresponding to each enhanced label can be calculated, and the corresponding mean is used as the target text feature of the sample to be detected.

[0047] In this example, the target text features of the sample to be detected are calculated based on the text features corresponding to each enhanced label, so as to take into account the visual cues corresponding to the visual features, so that the target text features cover the descriptions of different visual features and improve the accuracy of the target text features.

[0048] The uncertainty of the sample to be tested is calculated based on the embedded features and the target text features. The uncertainty represents the likelihood that the sample to be tested is out-of-distribution. Based on the uncertainty, the test result of the sample to be tested is determined. In other words, the greater the uncertainty, the closer the corresponding test result is to being out-of-distribution.

[0049] It should be noted that when calculating the uncertainty of a sample to be tested based on the embedded features and the target text features, the difference between the embedded features and the target text features can be calculated. When the difference is larger, the uncertainty of the sample to be tested is considered to be larger, and when the difference is smaller, the uncertainty of the sample to be tested is considered to be smaller. A larger uncertainty value corresponds to a test result that is closer to an out-of-distribution sample, and a smaller uncertainty value corresponds to a test result that is closer to an in-distribution sample.

[0050] It should be noted that when calculating the difference between the embedded feature and the target text feature, the difference between the embedded feature and the target text feature can be calculated and the corresponding difference is used as the difference between the embedded feature and the target text feature. Other methods can also be used for calculation, which is not limited in this embodiment.

[0051] Optionally, based on N visual features, an embedded feature of the sample to be detected is calculated, including: N visual features are added together to obtain the embedded features of the sample to be detected.

[0052] In this embodiment, N visual features are added to obtain the embedded features of the sample to be detected. During the addition, each visual feature is converted into a feature of equal dimension so that the values ​​of the same dimension in different visual features can be added. After the addition, normalization is performed to obtain the embedded features of the sample to be detected. The calculation formula of the embedded features is as follows: in, is the embedding feature, is the pth visual feature, N is the number of N visual features represents the L2 norm.

[0053] In this example, the embedded features are obtained by adding N visual features. Therefore, the embedded features cover the overall structure of the sample to be detected, allowing the model to more comprehensively understand the relationship between the target and its environment.

[0054] Optionally, the target text features of the sample to be detected are calculated based on the text features corresponding to each enhanced label, including: The text features corresponding to each enhanced label are added together to obtain the target text features of the sample to be detected.

[0055] In this embodiment, the text features corresponding to each enhanced label are added to obtain the target text features of the sample to be detected. When adding, each text feature is converted into a feature with equal dimension so that the values ​​of the same dimension in different text features can be added. After addition, normalization is performed to obtain the target text features of the sample to be detected. The calculation formula of the target text features is as follows: in, is the target text feature, is the p-th enhanced label The text features of N are the number of N enhanced labels. For the enhanced label, represents the L2 norm.

[0056] In this embodiment, the target text feature is obtained by adding the text features corresponding to N enhanced labels. Therefore, the target text feature represents all visual features of the sample to be detected, thereby improving the accuracy of the label corresponding to the sample to be detected.

[0057] Optionally, the uncertainty of the sample to be detected is calculated based on the embedded features and the target text features, including: Calculate the feature similarity between the embedded features and the target text features based on the embedded features and the target text features; According to the feature similarity, the uncertainty of the sample to be tested is calculated.

[0058] In this embodiment, the feature similarity between the embedded feature and the target text feature is calculated based on the embedded feature and the target text feature. When calculating the feature similarity, the cosine formula can be used for calculation. Based on the feature similarity, the uncertainty of the sample to be detected is calculated. The calculation formula for uncertainty is as follows: in, is the target text feature corresponding to the i-th label of the sample to be detected, is the number of labels of the samples to be tested, The target text features corresponding to different labels form a text space, is the uncertainty value, Z is the embedded feature, is the similarity between the embedded feature and the target text feature corresponding to the i-th label, is the temperature factor, and K is the number of labels of the samples to be detected.

[0059] Optionally, determining the test result of the sample to be tested based on the uncertainty includes: If the uncertainty value is greater than the preset threshold, the test result of the sample to be tested is determined to be an out-of-distribution sample; If the uncertainty value is not greater than the preset threshold, the detection result of the sample to be detected is determined to be an in-distribution sample.

[0060] In this embodiment, the test result of the sample to be tested is determined based on the uncertainty, and the calculation formula for determining the test result is as follows: in, For the test results, is a sample within the distribution, is an out-of-distribution sample, is the uncertainty value, is the preset threshold, This ensures that more than 95% of samples within the distribution are correctly detected and identified. This means that when the uncertainty is greater than 95%, the sample is considered out-of-distribution. When the uncertainty is less than 95%, the sample is considered in-distribution. Other values ​​may also be used, which are not limited in this embodiment.

[0061] In the present invention, multiple visual features of the sample to be detected are obtained so that the contextual relationship of the sample to be detected can be determined based on the multiple visual features, thereby increasing the detection capability of the sample to be detected. By performing text enhancement processing on the label, the text description of the sample to be detected is increased, so that when the sample to be detected is detected in combination with the multiple visual features and the enhanced label, the contextual relationship can be considered, thereby improving the detection accuracy of the sample to be detected.

[0062] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an out-of-distribution sample detection device provided by an embodiment of the present invention. This out-of-distribution sample detection device corresponds one-to-one with the out-of-distribution sample detection method in the above embodiment. Figure 2 as well as Figure 2 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 3 The out-of-distribution sample detection device 30 includes: an acquisition module 31 , a processing module 32 , and a detection module 33 .

[0063] An acquisition module 31 is used to acquire N visual features of a sample to be detected, where N is an integer greater than zero; The processing module 32 is used to obtain labels of samples to be detected, and perform text enhancement processing on the labels according to N visual features to obtain N enhanced labels.

[0064] The detection module 33 is configured to detect the sample to be detected based on the N visual features and the N enhanced labels, and obtain a detection result of the sample to be detected.

[0065] Optionally, the processing module 32 includes: The determination unit is used to determine N prompt templates according to N visual features.

[0066] The generating unit is configured to generate N enhanced labels according to the N prompt templates and labels.

[0067] Optionally, the detection module 33 includes: The first calculation unit is used to calculate the embedded features of the sample to be detected based on N visual features.

[0068] The second calculation unit is used to extract features from each enhanced label to obtain text features corresponding to each enhanced label, and calculate target text features of the sample to be detected based on the text features corresponding to each enhanced label.

[0069] The third calculation unit is used to calculate the uncertainty of the sample to be detected based on the embedded features and the target text features.

[0070] The second determining unit is used to determine the detection result of the sample to be detected according to the uncertainty.

[0071] Optionally, the first calculation unit includes: The first adding subunit is used to add N visual features to obtain the embedded features of the sample to be detected.

[0072] Optionally, the second calculation unit includes: The second adding subunit is used to add the text features corresponding to each enhanced label to obtain the target text features of the sample to be detected.

[0073] Optionally, the third calculation unit includes: The first calculation subunit is used to calculate the feature similarity between the embedded feature and the target text feature based on the embedded feature and the target text feature.

[0074] The second calculation subunit is used to calculate the uncertainty of the sample to be detected based on the feature similarity.

[0075] Optionally, the second determining unit includes: The first judgment unit is configured to determine that the detection result of the sample to be detected is an out-of-distribution sample if the uncertainty value is greater than a preset threshold.

[0076] The second judgment unit is configured to determine that the detection result of the sample to be detected is an in-distribution sample if the uncertainty value is not greater than a preset threshold.

[0077] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0078] Figure 4 This is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps of any of the above-mentioned out-of-distribution sample detection method embodiments are implemented.

[0079] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 4 The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0080] The processor may be a CPU, other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0081] Memory includes readable storage media, internal memory, and the like. Internal memory can be the internal memory of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage medium. The readable storage medium can be the computer device's hard drive. In other embodiments, it can also be an external storage device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash memory card. Furthermore, memory can include both the computer device's internal storage unit and external storage devices. Memory is used to store the operating system, application programs, boot loaders, data, and other programs, such as the program code of computer programs. Memory can also be used to temporarily store data that has been output or is about to be output.

[0082] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include at least: any entity or device capable of carrying computer program code, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunications signals.

[0083] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0084] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0085] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0086] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0087] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for detecting out-of-distribution samples, characterized in that: The out-of-distribution sample detection method comprises: Obtain N visual features of the sample to be detected, where N is an integer greater than zero; Obtaining labels of the samples to be detected, and performing text enhancement processing on the labels based on the N visual features to obtain N enhanced labels; The sample to be detected is detected according to the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected.

2. The out-of-distribution sample detection method according to claim 1, wherein: The step of performing text enhancement processing on the label according to the N visual features to obtain N enhanced labels includes: Determining N prompt templates according to the N visual features; N enhanced labels are generated according to the N prompt templates and the labels.

3. The out-of-distribution sample detection method according to claim 1, wherein: The detecting the sample to be detected according to the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected includes: Calculating the embedded features of the sample to be detected based on the N visual features; Performing feature extraction on each enhanced label to obtain text features corresponding to each enhanced label, and calculating target text features of the sample to be detected based on the text features corresponding to each enhanced label; Calculating the uncertainty of the sample to be detected based on the embedded features and the target text features; Determine the test result of the sample to be tested based on the uncertainty.

4. The out-of-distribution sample detection method according to claim 3, wherein: The calculating, based on the N visual features, the embedded features of the sample to be detected includes: adding the N visual features to obtain the embedded features of the sample to be detected.

5. The out-of-distribution sample detection method according to claim 3, wherein: The step of calculating the target text features of the sample to be detected based on the text features corresponding to each enhanced label includes: The text features corresponding to each enhanced label are added together to obtain the target text features of the sample to be detected.

6. The out-of-distribution sample detection method according to any one of claims 4 to 5, characterized in that: The calculating the uncertainty of the sample to be detected according to the embedded feature and the target text feature includes: Calculating a feature similarity between the embedded feature and the target text feature based on the embedded feature and the target text feature; The uncertainty of the sample to be detected is calculated according to the feature similarity.

7. The out-of-distribution sample detection method according to claim 3, wherein: The test results include out-of-distribution samples and in-distribution samples; Determining the test result of the sample to be tested based on the uncertainty includes: If the uncertainty value is greater than a preset threshold, determining that the detection result of the sample to be detected is an out-of-distribution sample; If the uncertainty value is not greater than a preset threshold, the detection result of the sample to be detected is determined to be an in-distribution sample.

8. An out-of-distribution sample detection device, characterized in that: The out-of-distribution sample detection device comprises: An acquisition module is used to obtain N visual features of the sample to be detected, where N is an integer greater than zero; A processing module is used to obtain the label of the sample to be detected, and perform text enhancement processing on the label according to the N visual features to obtain N enhanced labels The detection module is used to detect the sample to be detected based on the N visual features and the N enhanced labels to obtain a detection result of the sample to be detected.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the out-of-distribution sample detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the out-of-distribution sample detection method according to any one of claims 1 to 7 is implemented.

Images