Target detection method and apparatus, electronic device, and readable storage medium

Abstract

The present disclosure relates to a target detection method and apparatus, an electronic device, and a readable storage medium. The method comprises: acquiring data to be detected, wherein said data comprises an original image and text data; acquiring first similarity data between image features of the original image and text features of the text data, wherein the image features include features obtained by using a preset neural network model to perform weighted fusion processing; and acquiring, on the basis of the first similarity data, a first detection result of the data to be detected. According to the present embodiment, by using a slot attention unit to perform weighted fusion processing on some image features, interaction and fusion results of different image slots can be reflected, improving the accuracy of the image features, and helping to improve the accuracy of detection results.

Description

Target detection method, device, electronic device and readable storage medium Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a target detection method, device, electronic device, and readable storage medium. Background Art

[0002] The fundamental idea of ​​zero-shot learning is to transfer the knowledge contained in the training samples to the test sample classification task. Therefore, zero-shot learning can be considered a transfer learning problem. In transfer learning, when building a model for a specific task in the target domain, the knowledge contained in the source domain and the source task is transferred to the target domain.

[0003] In zero-shot learning, although the training samples and test samples are in the same feature space, the category set to which the training samples belong is different from the category set to which the test samples belong. Therefore, zero-shot learning falls into the category of heterogeneous transfer learning.

[0004] Zero-shot learning addresses the problem of unlabeled samples for unknown classes by introducing auxiliary information. Each unknown class requires auxiliary information, and this auxiliary information must be associated with instances in the feature space. A common auxiliary information is the semantic information of the unknown class. Inaccurate semantic information can affect detection results.

[0005] Summary of the Invention

[0006] The present disclosure provides a target detection method, device, electronic device and readable storage medium to address the deficiencies of related technologies.

[0007] According to a first aspect of an embodiment of the present disclosure, a target detection method is provided, the method comprising:

[0008] Acquiring data to be detected, wherein the data to be detected includes original images and text data;

[0009] Obtaining first similarity data between image features of the original image and text features of the text data; the image features include features obtained by weighted fusion processing using a preset neural network model;

[0010] A first detection result of the data to be detected is obtained according to the first similarity data; the first detection result is used to indicate that an abnormal target is detected from the original image.

[0011] Optionally, obtaining first similarity data between the image feature of the original image and the text feature of the text data includes:

[0012] Obtain multi-level image features of the original image in the data to be detected; the first-level image features are features extracted from the original image, the remaining levels of image features are features extracted from the previous level image features, and the last level of image features are features obtained by weighted fusion processing of the extracted features by the slot attention unit;

[0013] Respectively obtaining feature similarities between the image features at each level of the original image and the text features of the text data, and obtaining feature similarities corresponding to the image features at each level;

[0014] A weighted cumulative value of feature similarities corresponding to image features at each level is obtained as the first similarity data.

[0015] Optionally, obtaining first similarity data between the image feature of the original image and the text feature of the text data includes:

[0016] The data to be detected is input into a first preset detection model to obtain first similarity data output by the first preset detection model, wherein the first similarity data represents a weighted cumulative value of feature similarities between image features at different levels of the original image and text features of the text data.

[0017] Optionally, the first preset detection model includes a plurality of cascaded feature extraction units and a preset network model connected to the last-stage feature extraction unit;

[0018] Each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes the original image or the image features output by the previous level of feature extraction unit;

[0019] The preset network model is used to perform weighted fusion processing on the features output by the last level feature extraction unit.

[0020] Optionally, the preset network model includes a slot attention unit, which is used to achieve cross-attention between multiple feature sequences among the output features of the last-level feature extraction unit.

[0021] Optionally, the preset network model further includes a plurality of adapter units; except for the last-level feature extraction unit, each level of feature extraction unit is connected to a matching adapter unit, and the adapter unit corresponding to the last-level feature extraction unit is connected to the slot attention unit;

[0022] The adapter unit is used to perform mapping processing on the received image features to obtain image mapping features, and the image mapping features are used to belong to the same mapping space as the text features corresponding to the text data.

[0023] Optionally, the first preset detection model further includes a similarity superposition unit and a plurality of similarity calculation units, each similarity calculation unit is respectively connected to each adapter unit; the similarity superposition unit is respectively connected to the plurality of similarity calculation units;

[0024] The similarity calculation unit is used to calculate the similarity between the text features and the image mapping features at each level to obtain feature similarity;

[0025] The similarity superposition unit is used to obtain a weighted cumulative value of feature similarities output by multiple similarity calculation units to obtain the first similarity data.

[0026] Optionally, the first preset detection model is trained by the following steps, including:

[0027] Acquire training sample data, wherein the training sample data includes sample images and text data;

[0028] Inputting the sample data into the first preset detection model in sequence to obtain first similarity data output by the first preset detection model, wherein the first similarity data refers to a weighted cumulative value of feature similarities between image features of the sample image and text features of the text data;

[0029] Calculate the loss function value corresponding to this training according to the first similarity data;

[0030] In response to the loss function value being greater than a preset loss threshold or the difference between the loss function value and a previous loss function value being greater than a preset difference threshold, continuing to sequentially input the sample data into the first preset detection model;

[0031] In response to the loss function value being less than or equal to a preset loss threshold or the difference between the loss function value and a previous loss function value being less than or equal to a preset difference threshold, it is determined that the first preset detection model has completed training.

[0032] Optionally, the first detection result includes a first detection image, and the first detection image is used to represent the probability of an abnormality occurring at each position of the original image. Obtaining the first detection result of the data to be detected based on the first similarity data includes:

[0033] Performing visualization processing on the first similarity data to obtain a first similarity image;

[0034] Perform image optimization processing on the first similarity image to obtain the first detection image as the first detection result.

[0035] Optionally, the method further includes:

[0036] Obtaining a classification feature vector of the original image; the classification feature vector includes category information of the original image;

[0037] Obtaining a first similarity value between the classification feature vector and the text feature of the text data;

[0038] The first similarity value is normalized to obtain a first anomaly score corresponding to the original image, where the first anomaly score is used to indicate a probability of an anomaly occurring in the original image when there is no reference image. A larger first anomaly score indicates a greater probability of an anomaly occurring at a position in the original image.

[0039] Optionally, the data to be detected further includes a reference image; and the method further includes:

[0040] Acquiring reference features of the reference image;

[0041] Acquire second similarity data between the reference feature of the reference image and the image feature of the original image;

[0042] A second detection result of the data to be detected is obtained according to the first similarity data and the second similarity data; the second detection result is used to indicate that an abnormal target is detected from the original image when there is a reference image.

[0043] Optionally, obtaining a reference feature of the reference image includes:

[0044] Acquire multi-level reference features of the reference image, where each level of reference features matches each level of image features of the original image;

[0045] Store reference features of all levels in a preset storage area.

[0046] Optionally, obtaining second similarity data between the reference feature of the reference image and the image feature of the original image includes:

[0047] Sequentially obtaining feature similarities between reference features at various levels of the reference image and image features at various levels of the original image;

[0048] A weighted cumulative value of feature similarities at each level is obtained as the second similarity data.

[0049] Optionally, the second detection result includes a second detection image, where the second detection image is used to represent the probability of anomalies occurring at various positions of the original image when the reference image is used as a benchmark. Obtaining the second detection result of the data to be detected based on the first similarity data and the second similarity data includes:

[0050] Obtaining a weighted cumulative value of the first similarity data and the second similarity data to obtain third similarity data;

[0051] Performing visualization processing on the third similarity data to obtain a second similarity image;

[0052] Perform image optimization processing on the second similarity image to obtain the second detection image as the second detection result.

[0053] Optionally, the method further includes:

[0054] Obtaining a maximum value of similarity values ​​in the third similarity data to obtain a second similarity value;

[0055] Obtaining a sum of the second similarity value and the first similarity value to obtain a third similarity value;

[0056] The third similarity value is normalized to obtain a second anomaly score corresponding to the original image, where the second anomaly score is used to indicate a probability of an anomaly occurring in the original image when a reference image is present. A larger second anomaly score indicates that the location is more likely to be abnormal.

[0057] Optionally, obtaining second similarity data between the reference feature of the reference image and the image feature of the original image includes:

[0058] The image features of the reference image and the original image are respectively input into a second preset detection model to obtain second similarity data output by the second preset detection model; the second similarity data represents the weighted cumulative value of the feature similarity between the image features of different levels of the original image and the reference features of the reference image.

[0059] Optionally, the second preset detection model includes a plurality of cascaded feature extraction units; each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes a reference image or image features output by a previous level of feature extraction unit;

[0060] The feature extraction units at the same level in the second preset detection model and the first preset detection model are implemented using the same model parameter weights.

[0061] According to a second aspect of an embodiment of the present disclosure, there is provided an object detection device, the device comprising:

[0062] A detection data acquisition module is used to acquire the data to be detected, wherein the data to be detected includes original images and text data;

[0063] A first similarity acquisition module is configured to acquire first similarity data between image features of the original image and text features of the text data; the image features include features obtained by weighted fusion processing using a preset neural network model;

[0064] The first detection result acquisition module is used to obtain a first detection result of the data to be detected based on the first similarity data; the first detection result is used to indicate that an abnormal target is detected from the original image.

[0065] Optionally, the first similarity acquisition module includes:

[0066] An image feature acquisition submodule is used to acquire multi-level image features of the original image in the data to be detected; the first-level image features are features extracted from the original image, the remaining levels of image features are features extracted from the image features of the previous level, and the last level of image features are features obtained by weighted fusion processing of the extracted features by the slot attention unit;

[0067] A feature similarity acquisition submodule is used to respectively acquire feature similarities between the image features at various levels of the original image and the text features of the text data, and obtain feature similarities corresponding to the image features at various levels;

[0068] The first similarity acquisition submodule is configured to acquire a weighted cumulative value of feature similarities corresponding to image features at each level as the first similarity data.

[0069] Optionally, the first similarity acquisition module includes:

[0070] A first similarity acquisition submodule is used to input the data to be detected into a first preset detection model to obtain first similarity data output by the first preset detection model, wherein the first similarity data represents a weighted cumulative value of feature similarities between image features at different levels of the original image and text features of the text data.

[0071] Optionally, the first preset detection model includes a plurality of cascaded feature extraction units and a preset network model connected to the last-stage feature extraction unit;

[0072] Each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes the original image or the image features output by the previous level of feature extraction unit;

[0073] The preset network model is used to perform weighted fusion processing on the features output by the last level feature extraction unit.

[0074] Optionally, the preset network model includes a slot attention unit, which is used to achieve cross-attention between multiple feature sequences among the output features of the last-level feature extraction unit.

[0075] Optionally, the preset network model further includes a plurality of adapter units; except for the last-level feature extraction unit, each level of feature extraction unit is connected to a matching adapter unit, and the adapter unit corresponding to the last-level feature extraction unit is connected to the slot attention unit;

[0076] The adapter unit is used to perform mapping processing on the received image features to obtain image mapping features, and the image mapping features are used to belong to the same mapping space as the text features corresponding to the text data.

[0077] Optionally, the first preset detection model further includes a similarity superposition unit and a plurality of similarity calculation units, each similarity calculation unit is respectively connected to each adapter unit; the similarity superposition unit is respectively connected to the plurality of similarity calculation units;

[0078] The similarity calculation unit is used to calculate the similarity between the text features and the image mapping features at each level to obtain feature similarity;

[0079] The similarity superposition unit is used to obtain a weighted cumulative value of feature similarities output by multiple similarity calculation units to obtain the first similarity data.

[0080] Optionally, the first preset detection model is trained by the following steps, including:

[0081] Acquire training sample data, wherein the training sample data includes sample images and text data;

[0082] Inputting the sample data into the first preset detection model in sequence to obtain first similarity data output by the first preset detection model, wherein the first similarity data refers to a weighted cumulative value of feature similarities between image features of the sample image and text features of the text data;

[0083] Calculate the loss function value corresponding to this training according to the first similarity data;

[0084] In response to the loss function value being greater than a preset loss threshold or the difference between the loss function value and a previous loss function value being greater than a preset difference threshold, continuing to sequentially input the sample data into the first preset detection model;

[0085] In response to the loss function value being less than or equal to a preset loss threshold or the difference between the loss function value and a previous loss function value being less than or equal to a preset difference threshold, it is determined that the first preset detection model has completed training.

[0086] Optionally, the first detection result includes a first detection image, and the first detection image is used to represent the probability of abnormality occurring at each position of the original image. The first detection result module includes:

[0087] a similarity image acquisition submodule, configured to perform visualization processing on the first similarity data to obtain a first similarity image;

[0088] The detection image acquisition submodule is used to perform image optimization processing on the first similarity image to obtain the first detection image as the first detection result.

[0089] Optionally, the device further comprises:

[0090] A classification vector acquisition module, configured to acquire a classification feature vector of the original image; the classification feature vector includes category information of the original image;

[0091] A first similarity acquisition module, configured to acquire a first similarity value between the classification feature vector and the text feature of the text data;

[0092] A first similarity acquisition module is configured to normalize the first similarity value to obtain a first anomaly score corresponding to the original image, where the first anomaly score indicates a probability of an anomaly occurring in the original image when no reference image is present. A larger first anomaly score indicates a more likely abnormal location.

[0093] Optionally, the data to be detected further includes a reference image; and the device further includes:

[0094] A reference feature acquisition module, configured to acquire reference features of the reference image;

[0095] a similarity data acquisition module, configured to acquire second similarity data between the reference feature of the reference image and the image feature of the original image;

[0096] The second result acquisition module is used to obtain a second detection result of the data to be detected based on the first similarity data and the second similarity data; the second detection result is used to indicate that an abnormal target is detected from the original image when there is a reference image.

[0097] Optionally, the reference feature acquisition module includes:

[0098] A reference feature acquisition submodule, configured to acquire multi-level reference features of the reference image, wherein each level of reference features matches each level of image features of the original image;

[0099] The reference feature storage submodule is used to store reference features of various levels in a preset storage area.

[0100] Optionally, the similarity data acquisition module includes:

[0101] A feature similarity acquisition submodule, configured to sequentially acquire feature similarities between reference features of each level of the reference image and image features of each level of the original image;

[0102] The similarity data acquisition submodule is used to obtain a weighted cumulative value of feature similarities at each level as the second similarity data.

[0103] Optionally, the second detection result includes a second detection image, and the second detection image is used to represent the probability of abnormality occurring at each position of the original image when the reference image is used as a reference; the second result acquisition module includes:

[0104] a similarity data acquisition submodule, configured to acquire a weighted cumulative value of the first similarity data and the second similarity data to obtain third similarity data;

[0105] A second image acquisition submodule is configured to perform visualization processing on the third similarity data to obtain a second similarity image;

[0106] The second detection result acquisition submodule is configured to perform image optimization processing on the second similarity image to obtain the second detection image as the second detection result.

[0107] Optionally, the device further comprises:

[0108] a second similarity value obtaining module, configured to obtain a maximum value of similarity values ​​in the third similarity data to obtain a second similarity value;

[0109] A third similarity value obtaining module is configured to obtain a sum of the second similarity value and the first similarity value to obtain a third similarity value;

[0110] a second anomaly score acquisition module, configured to normalize the third similarity value to obtain a second anomaly score corresponding to the original image, wherein the second anomaly score is used to indicate a probability of an anomaly occurring in the original image when a reference image is present, and a larger second anomaly score indicates a more likely abnormal location.

[0111] Optionally, the similarity data acquisition module includes:

[0112] The similarity data acquisition submodule is used to input the image features of the reference image and the original image into a second preset detection model respectively to obtain second similarity data output by the second preset detection model; the second similarity data represents the weighted cumulative value of the feature similarity between the image features of different levels of the original image and the reference features of the reference image.

[0113] Optionally, the second preset detection model includes a plurality of cascaded feature extraction units; each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes a reference image or image features output by a previous level of feature extraction unit;

[0114] The feature extraction units at the same level in the second preset detection model and the first preset detection model are implemented using the same model parameter weights.

[0115] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising

[0116] processor;

[0117] a memory for storing a computer program executable by the processor;

[0118] The processor is configured to execute the computer program in the memory to implement the method described in the first aspect.

[0119] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, which, when an executable computer program in the storage medium is executed by a processor, can implement the method described in the first aspect.

[0120] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0121] As can be seen from the above embodiments, the scheme provided by the embodiments of the present disclosure can obtain data to be detected, wherein the data to be detected includes original images and text data; then, first similarity data between the image features of the original image and the text features of the text data is obtained; the image features include features obtained by weighted fusion processing using a preset neural network model; thereafter, a first detection result of the data to be detected is obtained based on the first similarity data. In this way, this embodiment can reflect the interaction and fusion results of different image slots by using the slot attention unit to perform weighted fusion processing on some image features, thereby improving the accuracy of image features and helping to improve the accuracy of detection results.

[0122] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0124] Fig. 1 is a flow chart showing a method for detecting an object according to an exemplary embodiment.

[0125] Fig. 2 is a flow chart showing a method of obtaining first similarity data according to an exemplary embodiment.

[0126] Fig. 3 is a schematic structural diagram of a first preset detection model according to an exemplary embodiment.

[0127] Fig. 4 is a schematic structural diagram of an adapter unit according to an exemplary embodiment.

[0128] Fig. 5 is a schematic diagram showing a first detection image according to an exemplary embodiment.

[0129] Fig. 6 is a flowchart showing a method of obtaining a first anomaly score according to an exemplary embodiment.

[0130] Fig. 7 is a flow chart showing a method of obtaining a second detection result according to an exemplary embodiment.

[0131] Fig. 8 is a schematic structural diagram showing a first preset detection model and a second preset detection model according to an exemplary embodiment.

[0132] Fig. 9 is a block diagram showing an object detection device according to an exemplary embodiment. DETAILED DESCRIPTION

[0133] Exemplary embodiments will be described in detail herein, with examples shown in the accompanying drawings. When the following description refers to the drawings, identical numbers in different drawings represent identical or similar elements, unless otherwise indicated. The exemplary embodiments described below do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices consistent with certain aspects of the present disclosure, as detailed in the appended claims. It should be noted that, unless there is a conflict, the features of the following embodiments and implementations may be combined with each other.

[0134] To solve the above technical problems, the embodiments of the present disclosure provide a target detection method, device, electronic device and readable storage medium. The embodiments of the present disclosure provide a target detection method that can be applied to electronic devices, which may include but are not limited to devices with processing capabilities such as smart phones, tablet computers, smart displays or electronic whiteboards. Figure 1 is a flow chart of a target detection method according to an exemplary embodiment. Referring to Figure 1, a target detection method includes steps 11 to 13:

[0135] In step 11, data to be detected is obtained, where the data to be detected includes original images and text data.

[0136] In this step, the processor of the electronic device can obtain the data to be detected, which may include original images and text data. The data to be detected may be data detected in real time or according to a set period. For example, a detection device is provided on the production line, and the detection device may use the original image of the product on the production line, and then use a preset image recognition method to identify abnormalities such as flaws and defects in the original image, and then generate description data, i.e., text data, to obtain the data to be detected including the original image and text data. For another example, the detection device of the production line may use the original image of the product, and the original image may be uploaded to the host computer. The detection personnel may analyze the original image based on experience and input text data, so that the original image and text data can form the data to be detected.

[0137] In one example, the data to be detected may also be data imported from an external device, and the source of the data to be detected may be set according to specific scenarios, and the corresponding solutions fall within the protection scope of this disclosure.

[0138] In step 12, first similarity data between the image features of the original image and the text features of the text data is obtained; the image features include features obtained by weighted fusion processing using a preset neural network model.

[0139] In this step, the processor may obtain first similarity data between the image features of the original image and the text features of the text data, as shown in FIG. 2 , which includes steps 21 to 23 .

[0140] In step 21, the processor can obtain multi-level image features of the original image in the data to be detected; the first-level image features are features obtained by extracting the original image, the remaining levels of image features are features obtained by extracting the image features of the previous level, and the last level of image features are features obtained by weighted fusion processing of the penultimate level features by the slot attention unit.

[0141] In other words, the processor extracts features from the original image to obtain first-level image features; then, the processor can extract features from the first-level image features to obtain second-level image features; after that, the processor can extract features from the second-level image features to obtain third-level image features; and so on, to obtain first-level image features to the penultimate image features; for the penultimate image features, features are first extracted, and then the extracted features are weighted fused using the slot attention unit to obtain the last-level image features.

[0142] It should be noted that the slot attention unit can measure the importance of different image slots (each image slot is used to obtain image features) by calculating the attention weight to obtain the attention weight; the image features extracted by the image slot are weightedly fused according to the above attention weight to obtain the final level of image features.

[0143] In one example, the slot attention unit has two input sequences, sequence A and sequence B. Each sequence has N feature vectors, represented as A = [a1, a2, ..., an] and B = [b1, b2, ..., bn], where a i and b i Denotes the i-th feature vector in sequence A and sequence B respectively. Cross-slot attention calculates the attention weight between each feature vector in sequence A and all feature vectors in sequence B by the following formula: i j=q(a i )·k(bj); where q(a i ) means that the feature vector a i Mapped to the query space, k(bj) represents mapping the feature vector bj to the key space.

[0144] Through the dot product operation, the correlation between each feature vector in sequence A and each feature vector in sequence B is calculated. By normalizing the correlation, the attention weight is obtained: α i j=softmax(e i j).

[0145] Use the attention weights to weight the feature vectors in sequence B and get the cross-slot attention representation of sequence A: c i =∑(α i j·v(bj)). Where v(bj) represents mapping the feature vector bj to the value space.

[0146] Finally, by taking the cross-slot attention representation c i and the original feature vector a in sequence A i Splice and get the final output representation: i =[a i ,c i ].

[0147] In this way, the cross-slot attention mechanism is able to capture the interaction information between sequence A and sequence B.

[0148] In step 22 , the processor may respectively obtain feature similarities between image features at various levels of the original image and text features of the text data, and obtain feature similarities corresponding to image features at various levels.

[0149] In this step, the image feature is a feature tensor, and the text feature is a feature tensor. The feature similarity between the image and text features can be converted into the distance between the two feature tensors in feature space. The processor can then calculate the distance between the two feature architectures. A closer distance indicates a higher similarity between the image and text features.

[0150] In this step, the original image includes multiple levels of image features. The feature similarity of each level of image features is calculated with the text features to obtain multiple feature similarities. It is understood that each feature similarity and the image feature are tensors with the same dimension.

[0151] In step 23, the processor can obtain the weighted cumulative value of the feature similarities corresponding to each level of image features as the first similarity data. In this step, the processor can sequentially obtain the similarity values ​​for the same location for each feature similarity, calculate the product of each similarity value and its weight, and obtain multiple products; then, the processor can accumulate these multiple products to obtain the similarity value for that location. Repeating these steps, after obtaining the similarity value for each location, the first similarity data can be obtained. It is understood that the first similarity data is a tensor with the same dimension as the image feature.

[0152] It should be noted that the above-mentioned positions in this step may include the pixel value of the pixel point at that position or the feature value of the feature point, or the normalized value of the pixel value or feature value, which can be selected according to the specific scenario, and the corresponding scheme falls within the protection scope of this disclosure.

[0153] In one example, a first preset detection model is stored in an electronic device, and the first preset detection model is used to obtain first similarity data. A processor can input the data to be detected into the first preset detection model to obtain first similarity data output by the first preset detection model. The first similarity data represents a weighted cumulative value of feature similarities between image features at different levels of the original image and text features of the text data. It is understood that the meaning of the first similarity representation can be seen in the solution illustrated in Figure 2.

[0154] In one example, the first preset detection model includes a plurality of cascaded feature extraction units and a preset network model connected to the last-level feature extraction unit. The feature extraction units at each level are used to extract image features corresponding to the input data; the input data includes the original image or the image features output by the previous-level feature extraction unit; the preset network model is used to perform weighted fusion processing on the features output by the last-level feature extraction unit. In one example, the preset network model includes a slot attention unit, which is used to achieve cross-attention between multiple feature sequences between the output features of the last-level feature extraction unit. The slot attention unit can be implemented using the structure of the self-attention module in the transformer model.

[0155] 3 , the first preset detection model may include four cascaded feature extraction units, namely, feature extraction unit 41, feature extraction unit 42, feature extraction unit 43, and feature extraction unit 44. In one example, the multiple cascaded feature extraction units of the first preset detection model may be implemented using a transformer model, wherein the first six layers of the transformer model constitute feature extraction unit 41, the 7th to 12th layers constitute feature extraction unit 42, the 13th to 18th layers constitute feature extraction unit 43, and the 19th to 24th layers constitute feature extraction unit 44; in other words, the image features F01, F02, F03, and F04 of the original image F00 are features extracted from the 6th, 12th, 18th, and 24th layers of the transformer model, respectively.

[0156] It should be noted that the first preset detection model can also be implemented using the Resnet model. For specific implementation methods, please refer to the implementation method of the transformer model. The corresponding solutions fall within the protection scope of this disclosure.

[0157] Continuing to refer to Figure 3, the slot attention unit is implemented using the transformer model, where the features corresponding to the 24th layer of the transformer model are used as the last level of image features.

[0158] In another example, the preset network model also includes multiple adapter units; except for the last level feature extraction unit, each level of feature extraction unit is connected to a matching adapter unit, and the adapter unit corresponding to the last level feature extraction unit is connected to the slot attention unit. The adapter unit is used to map the received image features to obtain image mapping features, and the above-mentioned image mapping features are used to belong to the same mapping space as the text features corresponding to the text data. Continuing to refer to Figure 3, the input data of the adapter unit adapter1 is image feature F01, the input data of the adapter unit adapter2 is image feature F02, the input data of the adapter unit adapter3 is image feature F03, and the input data of the adapter unit adapter4 is image feature F04. In this way, in this example, by mapping the image features and text features into the same mapping space, it is convenient to subsequently calculate the similarity between the image features and the text features, which is conducive to improving the accuracy of the calculation results.

[0159] In one example, each adapter unit includes multiple fully connected layers, each containing multiple neurons. Nonlinear activation functions are applied between adjacent layers to capture relationships between image features and high-level features. See Figure 4, which illustrates an adapter unit comprising an input layer X, a hidden layer H, and an output layer Y.

[0160] In one example, the first preset detection model also includes a similarity superposition unit and multiple similarity calculation units. Each similarity calculation unit is connected to each adapter unit respectively; the similarity superposition unit is connected to multiple similarity calculation units respectively; the similarity calculation unit is used to calculate the similarity between the text features and the image mapping features at all levels to obtain feature similarity; the similarity superposition unit is used to obtain the weighted cumulative value of the feature similarities output by multiple similarity calculation units to obtain the first similarity data. Continuing to refer to Figure 3, the input data of the similarity calculation unit (31, 32, 33 or 34) are image features and text features respectively, and the output result is feature similarity (tensor). The similarity superposition unit 35 can calculate the weighted cumulative value of the above four feature similarities to obtain the first similarity data. The above first similarity data can be used to obtain the first detection image.

[0161] Based on the structure of the first preset detection model shown in FIG3 , the first preset detection model may be trained, including the following steps:

[0162] (1) Obtain training sample data, wherein the training sample data includes sample images and text data. In one example, the training sample data may be from an MVTec-AD test dataset or a VisA test dataset.

[0163] (2) Inputting the sample data into the first preset detection model in sequence to obtain first similarity data output by the first preset detection model, wherein the first similarity data refers to a weighted cumulative value of feature similarities between image features of the sample image and text features of the text data.

[0164] (3) Calculate the loss function value corresponding to this training based on the first similarity data. The loss function in this step can be implemented using a focal loss function or a binary classification loss function.

[0165] Among them, the focal loss function is shown in formula (1). Focalloss = -(1-p t ) γ log(p t ); (1)

[0166] In formula (1), pt represents the predicted probability of the true type of the training sample, and γ represents the hyperparameter for adjusting the weight factor.

[0167] The binary classification loss function is shown in formula (2).

[0168] In formula (2), y i Represents the predicted value of each pixel, Represents the true value of each pixel, and N represents the total number of pixels.

[0169] (4) In response to the loss function value being greater than a preset loss threshold (such as 90% to 99%, adjustable) or the difference between the loss function value and the previous loss function value being greater than a preset difference threshold (such as 0.1 to 5, adjustable), continue to execute the step of inputting the sample data into the first preset detection model in sequence, that is, jump to step (2).

[0170] (5) In response to the loss function value being less than or equal to a preset loss threshold or the difference between the loss function value and the previous loss function value being less than or equal to a preset difference threshold, it is determined that the first preset detection model has completed training.

[0171] This embodiment uses a cross-slot attention mechanism to add style to the source domain image during the training phase, so that the first preset detection model can be exposed to images with diverse styles, which can increase the robustness of the model to images of different styles, improve the migration ability from the source domain to the target domain, and thus improve the recognition ability in the target domain during the testing phase.

[0172] In one example, after obtaining the data to be detected, the processor can input the data to be detected into a first preset detection model. The first preset detection model obtains the weighted cumulative value of the feature similarity of image features at different levels of the original image and text features of the text data to obtain first similarity data.

[0173] In step 13, a first detection result of the data to be detected is obtained according to the first similarity data; the first detection result is used to indicate that an abnormal target is detected from the original image.

[0174] In one example, the first detection result includes a first detection image, and the first detection image is used to represent the probability of abnormality occurring at each position of the original image.

[0175] In this step, the processor can obtain a first detection result for the data to be detected based on the first similarity data. For example, the processor can perform visualization processing on the first similarity data to obtain a first similarity image. The processor can then perform image optimization processing on the first similarity image, such as sharpening and color adjustment, to obtain a first detection image, which serves as the first detection result. See Figure 5, which illustrates a schematic effect of an abnormality in the central area 51.

[0176] At this point, this embodiment uses the slot attention unit to perform weighted fusion processing on some image features, which can reflect the interaction and fusion results of different image slots, improve the accuracy of image features, and help improve the accuracy of detection results.

[0177] Based on the target detection method shown in FIG1 , the embodiment of the present disclosure further provides another testing method, as shown in FIG6 , which also includes steps 61 to 63 .

[0178] In step 61, a classification feature vector of the original image is obtained; each classification value in the classification feature vector is used to represent the probability of the original image belonging to each classification after classification processing.

[0179] In this step, the processor can obtain a classification feature vector of the original image. Continuing with FIG3 , the first preset detection model includes a classification layer, and the processor can extract classification features from the classification layer to obtain a classification feature vector. The classification feature vector includes the category information of the original image.

[0180] In step 62, a first similarity value between the classification feature vector and the text feature of the text data is obtained.

[0181] In this step, the processor can obtain a first similarity value between the classification feature vector and the text feature of the text data. It is understandable that the classification feature vector is a vector, and the text feature is also a vector. The similarity of the two vectors can be calculated by calculating the cosine value to obtain a numerical value.

[0182] In step 63, the first similarity value is normalized to obtain a first anomaly score corresponding to the original image. The first anomaly score is used to indicate the probability of an anomaly occurring in the original image when there is no reference image. A larger first anomaly score indicates a greater probability of an anomaly occurring at a position in the original image.

[0183] In this step, the processor may perform normalization processing on the first similarity value, such as a logistic function, to convert the first similarity value into a value between 0 and 1, i.e., a first anomaly score. The first anomaly score is negatively correlated with the first similarity value, i.e., the larger the first similarity value, the smaller the first anomaly score, and vice versa.

[0184] It is understandable that in this step, only the original image is used without the reference image when calculating the first anomaly score. It can be understood that in the case of zero samples, the first preset detection model can still classify the original image and calculate the probability that the original image belongs to the anomaly type, which can improve the accuracy of the detection results.

[0185] Based on the target detection method shown in FIG6 , the embodiment of the present disclosure further provides another testing method, see FIG7 , which also includes steps 71 to 73 .

[0186] In step 71, reference features of the reference image are obtained.

[0187] In this step, the data to be tested also includes a reference image. It is understood that the reference image can be an image without abnormalities (i.e., a normal image) and / or an image with problems (i.e., an abnormal image), and its test classification results meet preset conditions, such as a classification success rate exceeding a preset threshold (e.g., 95% to 100%). The reference image can be selected according to the specific scenario, and the corresponding solution falls within the scope of protection of this disclosure.

[0188] In this step, the processor can obtain reference features of the reference image. For example, the processor can obtain multiple levels of reference features for the reference image, with each level of reference features matching each level of image features of the original image. The processor can then store the reference features in a memory within a pre-set storage area. It is understood that pre-storing the reference features in memory in this step can reduce computing resource utilization. Furthermore, providing separate memory for each level of reference image in this step improves access efficiency, thereby improving the efficiency of similarity calculation.

[0189] In step 72, second similarity data between the reference feature of the reference image and the image feature of the original image is obtained.

[0190] In this step, the processor can sequentially obtain the feature similarities of each level of reference features of the reference image and each level of image features of the original image. The method for calculating the feature similarity between the reference features and the image features is the same as the method for calculating the feature similarity between the image features and the text features, and will not be repeated here. The processor can then obtain a weighted cumulative value of the feature similarities at each level as the second similarity data. The method for obtaining the second similarity data by calculating the weighted cumulative value is the same as the method for obtaining the first similarity data by calculating the weighted cumulative value, except that the weights used in the two weighted accumulations can be different, which will not be repeated here.

[0191] In step 73, a second detection result of the data to be detected is obtained based on the first similarity data and the second similarity data; the second detection result is used to indicate that an abnormal target is detected from the original image when there is a reference image.

[0192] In this step, the processor can obtain third similarity data based on the weighted cumulative value of the first similarity data and the second similarity data. It is understood that the third similarity data contains correlated features of text features, image features, and reference features, which can increase the scope of classification and recognition. The processor can then perform visualization processing on the third similarity data to obtain a second similarity image. Finally, the processor can perform image optimization processing on the second similarity image to obtain a second detection image as the second detection result.

[0193] In one example, a second preset detection model is stored in an electronic device for implementing the scheme of steps 71 to 73. Referring to Figure 8, the second preset detection model includes a plurality of cascaded feature extraction units (801, 802, 803, and 804); the feature extraction units at each level are used to extract reference features corresponding to the input data; the input data includes a reference image or a reference feature output by the feature extraction unit of the previous level. It is understandable that since the feature extraction unit in the second preset detection model extracts reference features (F11, F12, F13, and F14) of the reference image, and the reference features correspond one-to-one to the image features of the original image, the feature extraction unit of the second preset detection model uses the same model parameter weights as the feature extraction unit at the same level in the first preset detection model, which can reduce the amount of training. Each feature extraction unit can store the reference features in a memory in a preset storage area, wherein the memory in the preset storage area is such as Memory1, Memory2, Memory3, and Memory4.

[0194] In one example, the second preset detection model also includes a similarity superposition unit (such as 85) and multiple similarity calculation units (such as 81, 82, 83 and 84); the input data of the similarity unit is reference features and image features, which is used to calculate the similarity of text features and image mapping features at all levels to obtain feature similarity; the similarity superposition unit is used to obtain the weighted cumulative value of the feature similarities output by multiple similarity calculation units to obtain second similarity data.

[0195] In one example, the second preset detection model further includes an integration unit, a maximum value selection unit, and a summing unit. Continuing with FIG8 , the integration unit 86 is used to obtain the sum vector of the first similarity data and the second similarity data to obtain third similarity data; the maximum value selection unit 87 is used to obtain the maximum value of the similarity value in the third similarity data to obtain a second similarity value. The summing unit 88 is used to obtain the sum of the second similarity value and the first similarity value to obtain a third similarity value. The processor can then normalize the third similarity value to obtain a second anomaly score corresponding to the original image. The second anomaly score is used to indicate the probability of an anomaly in the original image when there is a reference image. The larger the second anomaly score, the more likely the location is to be abnormal.

[0196] Based on the second preset detection model shown in FIG8 , the processor can input the image features of the reference image and the original image into the second preset detection model, respectively, to obtain second similarity data output by the second preset detection model. In other words, the first preset detection model and the second preset detection model can be used in conjunction to implement classification detection of the original image in the presence of a reference image.

[0197] In this way, this embodiment can use text data guidance to adapt visual features, thereby increasing the accuracy of zero-shot and small-sample anomaly classification and detection. The solution of this embodiment demonstrates good results on the MVTec-AD and VisA datasets, verifying its accuracy in real-world zero-shot and small-sample tests.

[0198] Based on the target detection method provided in the embodiment of the present disclosure, this embodiment further provides a target detection device. Referring to FIG9 , the device includes:

[0199] A detection data acquisition module 91 is used to acquire data to be detected, wherein the data to be detected includes original images and text data;

[0200] A first similarity acquisition module 92 is configured to acquire first similarity data between image features of the original image and text features of the text data; the image features include features obtained by weighted fusion processing using a preset neural network model;

[0201] The first detection result acquisition module 93 is used to obtain a first detection result of the data to be detected based on the first similarity data; the first detection result is used to indicate that an abnormal target is detected from the original image.

[0202] In one embodiment, the first similarity acquisition module includes:

[0203] An image feature acquisition submodule is used to acquire multi-level image features of the original image in the data to be detected; the first-level image features are features extracted from the original image, the remaining levels of image features are features extracted from the image features of the previous level, and the last level of image features are features obtained by weighted fusion processing of the extracted features by the slot attention unit;

[0204] A feature similarity acquisition submodule is used to respectively acquire feature similarities between the image features at various levels of the original image and the text features of the text data, and obtain feature similarities corresponding to the image features at various levels;

[0205] The first similarity acquisition submodule is configured to acquire a weighted cumulative value of feature similarities corresponding to image features at each level as the first similarity data.

[0206] In one embodiment, the first similarity acquisition module includes:

[0207] A first similarity acquisition submodule is used to input the data to be detected into a first preset detection model to obtain first similarity data output by the first preset detection model, wherein the first similarity data represents a weighted cumulative value of feature similarities between image features at different levels of the original image and text features of the text data.

[0208] In one embodiment, the first preset detection model includes a plurality of cascaded feature extraction units and a preset network model connected to the last stage feature extraction unit;

[0209] Each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes the original image or the image features output by the previous level of feature extraction unit;

[0210] The preset network model is used to perform weighted fusion processing on the features output by the last level feature extraction unit.

[0211] In one embodiment, the preset network model includes a slot attention unit, which is used to achieve cross-attention between multiple feature sequences among the output features of the last-level feature extraction unit.

[0212] In one embodiment, the preset network model further includes a plurality of adapter units; except for the last-level feature extraction unit, each level of feature extraction unit is connected to a matching adapter unit, and the adapter unit corresponding to the last-level feature extraction unit is connected to the slot attention unit;

[0213] The adapter unit is used to perform mapping processing on the received image features to obtain image mapping features, and the image mapping features are used to belong to the same mapping space as the text features corresponding to the text data.

[0214] In one embodiment, the first preset detection model further includes a similarity superposition unit and a plurality of similarity calculation units, each similarity calculation unit is respectively connected to each adapter unit; the similarity superposition unit is respectively connected to the plurality of similarity calculation units;

[0215] The similarity calculation unit is used to calculate the similarity between the text features and the image mapping features at each level to obtain feature similarity;

[0216] The similarity superposition unit is used to obtain a weighted cumulative value of feature similarities output by multiple similarity calculation units to obtain the first similarity data.

[0217] In one embodiment, the first preset detection model is trained by the following steps, including:

[0218] Acquire training sample data, wherein the training sample data includes sample images and text data;

[0219] Inputting the sample data into the first preset detection model in sequence to obtain first similarity data output by the first preset detection model, wherein the first similarity data refers to a weighted cumulative value of feature similarities between image features of the sample image and text features of the text data;

[0220] Calculate the loss function value corresponding to this training according to the first similarity data;

[0221] In response to the loss function value being greater than a preset loss threshold or the difference between the loss function value and a previous loss function value being greater than a preset difference threshold, continuing to sequentially input the sample data into the first preset detection model;

[0222] In response to the loss function value being less than or equal to a preset loss threshold or the difference between the loss function value and a previous loss function value being less than or equal to a preset difference threshold, it is determined that the first preset detection model has completed training.

[0223] In one embodiment, the first detection result includes a first detection image, and the first detection image is used to represent the probability of an abnormality occurring at each position of the original image. The first detection result module includes:

[0224] a similarity image acquisition submodule, configured to perform visualization processing on the first similarity data to obtain a first similarity image;

[0225] The detection image acquisition submodule is used to perform image optimization processing on the first similarity image to obtain the first detection image as the first detection result.

[0226] In one embodiment, the apparatus further comprises:

[0227] A classification vector acquisition module, configured to acquire a classification feature vector of the original image; the classification feature vector includes category information of the original image;

[0228] A first similarity acquisition module, configured to acquire a first similarity value between the classification feature vector and the text feature of the text data;

[0229] A first similarity acquisition module is configured to normalize the first similarity value to obtain a first anomaly score corresponding to the original image, where the first anomaly score indicates a probability of an anomaly occurring in the original image when no reference image is present. A larger first anomaly score indicates a more likely abnormal location.

[0230] In one embodiment, the data to be detected further includes a reference image; and the apparatus further includes:

[0231] A reference feature acquisition module, configured to acquire reference features of the reference image;

[0232] a similarity data acquisition module, configured to acquire second similarity data between the reference feature of the reference image and the image feature of the original image;

[0233] The second result acquisition module is used to obtain a second detection result of the data to be detected based on the first similarity data and the second similarity data; the second detection result is used to indicate that an abnormal target is detected from the original image when there is a reference image.

[0234] In one embodiment, the reference feature acquisition module includes:

[0235] A reference feature acquisition submodule, configured to acquire multi-level reference features of the reference image, wherein each level of reference features matches each level of image features of the original image;

[0236] The reference feature storage submodule is used to store reference features of various levels in a preset storage area.

[0237] In one embodiment, the similarity data acquisition module includes:

[0238] A feature similarity acquisition submodule, configured to sequentially acquire feature similarities between reference features of each level of the reference image and image features of each level of the original image;

[0239] The similarity data acquisition submodule is used to obtain a weighted cumulative value of feature similarities at each level as the second similarity data.

[0240] In one embodiment, the second detection result includes a second detection image, and the second detection image is used to represent the probability of anomalies occurring at various locations in the original image when the reference image is used as a reference; and the second result acquisition module includes:

[0241] a similarity data acquisition submodule, configured to acquire a weighted cumulative value of the first similarity data and the second similarity data to obtain third similarity data;

[0242] A second image acquisition submodule is configured to perform visualization processing on the third similarity data to obtain a second similarity image;

[0243] The second detection result acquisition submodule is configured to perform image optimization processing on the second similarity image to obtain the second detection image as the second detection result.

[0244] In one embodiment, the apparatus further comprises:

[0245] a second similarity value obtaining module, configured to obtain a maximum value of similarity values ​​in the third similarity data to obtain a second similarity value;

[0246] A third similarity value obtaining module is configured to obtain a sum of the second similarity value and the first similarity value to obtain a third similarity value;

[0247] a second anomaly score acquisition module, configured to normalize the third similarity value to obtain a second anomaly score corresponding to the original image, wherein the second anomaly score is used to indicate a probability of an anomaly occurring in the original image when a reference image is present, and a larger second anomaly score indicates a more likely abnormal location.

[0248] In one embodiment, the similarity data acquisition module includes:

[0249] The similarity data acquisition submodule is used to input the image features of the reference image and the original image into a second preset detection model respectively to obtain second similarity data output by the second preset detection model; the second similarity data represents the weighted cumulative value of the feature similarity between the image features of different levels of the original image and the reference features of the reference image.

[0250] In one embodiment, the second preset detection model includes a plurality of cascaded feature extraction units; each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes a reference image or image features output by a previous level of feature extraction unit;

[0251] The feature extraction units at the same level in the second preset detection model and the first preset detection model are implemented using the same model parameter weights.

[0252] It should be noted that the device embodiment shown in this embodiment matches the content of the above-mentioned method embodiment. You can refer to the content of the above-mentioned method embodiment and will not repeat it here.

[0253] In an exemplary embodiment, an electronic device is also provided, including:

[0254] processor;

[0255] a memory for storing a computer program executable by the processor;

[0256] The processor is configured to execute the computer program in the memory to implement the above method.

[0257] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including an executable computer program. The executable computer program can be executed by a processor to implement the method of the above embodiment. The computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0258] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0259] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquiring data to be detected, wherein the data to be detected includes original images and text data; Obtaining first similarity data between image features of the original image and text features of the text data; the image features include features obtained by weighted fusion processing using a preset network model; A first detection result of the data to be detected is obtained according to the first similarity data, where the first detection result is used to indicate that an abnormal target is detected from the original image.

2. The method according to claim 1, characterized in that Acquiring first similarity data between the image feature of the original image and the text feature of the text data, including: Obtain multi-level image features of the original image in the data to be detected; the first-level image features are features extracted from the original image, the remaining levels of image features are features extracted from the image features of the previous level, and the last level of image features are features obtained by weighted fusion processing of the extracted features by a preset network model; Respectively obtaining feature similarities between the image features at each level of the original image and the text features of the text data, and obtaining feature similarities corresponding to the image features at each level; A weighted cumulative value of feature similarities corresponding to image features at each level is obtained as the first similarity data.

3. The method according to claim 1, characterized in that Acquiring first similarity data between the image feature of the original image and the text feature of the text data, including: The data to be detected is input into a first preset detection model to obtain first similarity data output by the first preset detection model, wherein the first similarity data represents a weighted cumulative value of feature similarities between image features at different levels of the original image and text features of the text data.

4. The method according to claim 3, characterized in that The first preset detection model includes a plurality of cascaded feature extraction units and a preset network model connected to the last level feature extraction unit; Each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes the original image or the image features output by the previous level of feature extraction unit; The preset network model is used to perform weighted fusion processing on the features output by the last level feature extraction unit.

5. The method according to claim 4, characterized in that The preset network model includes a slot attention unit, which is used to achieve cross-attention between multiple feature sequences among the output features of the last-level feature extraction unit.

6. The method according to claim 5, characterized in that The preset network model also includes multiple adapter units; except for the last level feature extraction unit, each level of feature extraction unit is connected to a matching adapter unit, and the adapter unit corresponding to the last level feature extraction unit is connected to the slot attention unit; The adapter unit is used to perform mapping processing on the received image features to obtain image mapping features, and the image mapping features are used to belong to the same mapping space as the text features corresponding to the text data.

7. The method according to claim 6, characterized in that The first preset detection model further includes a similarity superposition unit and a plurality of similarity calculation units, each similarity calculation unit is respectively connected to each adapter unit; the similarity superposition unit is respectively connected to the plurality of similarity calculation units; The similarity calculation unit is used to calculate the similarity between the text features and the image mapping features at each level to obtain feature similarity; The similarity superposition unit is used to obtain a weighted cumulative value of feature similarities output by multiple similarity calculation units to obtain the first similarity data.

8. The method according to any one of claims 3 to 7, characterized in that The first preset detection model is trained through the following steps, including: Acquire training sample data, wherein the training sample data includes sample images and text data; Inputting the sample data into the first preset detection model in sequence to obtain first similarity data output by the first preset detection model, wherein the first similarity data refers to a weighted cumulative value of feature similarities between image features of the sample image and text features of the text data; Calculate the loss function value corresponding to this training according to the first similarity data; In response to the loss function value being greater than a preset loss threshold or the difference between the loss function value and a previous loss function value being greater than a preset difference threshold, continuing to sequentially input the sample data into the first preset detection model; In response to the loss function value being less than or equal to a preset loss threshold or the difference between the loss function value and a previous loss function value being less than or equal to a preset difference threshold, it is determined that the first preset detection model has completed training.

9. The method according to claim 1, characterized in that The first detection result includes a first detection image, and the first detection image is used to represent the probability of an abnormality occurring at each position of the original image. Obtaining the first detection result of the data to be detected based on the first similarity data includes: Performing visualization processing on the first similarity data to obtain a first similarity image; Perform image optimization processing on the first similarity image to obtain the first detection image as the first detection result.

10. The method according to claim 1, characterized in that The method further comprises: Obtaining a classification feature vector of the original image; the classification feature vector includes category information of the original image; Obtaining a first similarity value between the classification feature vector and the text feature of the text data; The first similarity value is normalized to obtain a first anomaly score corresponding to the original image, where the first anomaly score is used to indicate a probability of an anomaly occurring in the original image when there is no reference image. A larger first anomaly score indicates a greater probability of an anomaly occurring at a position in the original image.

11. The method according to claim 1, wherein The data to be detected further includes a reference image; and the method further includes: Acquiring reference features of the reference image; Acquire second similarity data between the reference feature of the reference image and the image feature of the original image; A second detection result of the data to be detected is obtained according to the first similarity data and the second similarity data; the second detection result is used to indicate that an abnormal target is detected from the original image when there is a reference image.

12. The method according to claim 11, characterized in that Acquiring a reference feature of the reference image includes: Acquire multi-level reference features of the reference image, where each level of reference features matches each level of image features of the original image; Store reference features of all levels in a preset storage area.

13. The method according to claim 12, characterized in that Acquiring second similarity data between the reference feature of the reference image and the image feature of the original image, comprising: Sequentially obtaining feature similarities between reference features at various levels of the reference image and image features at various levels of the original image; A weighted cumulative value of feature similarities at each level is obtained as the second similarity data.

14. The method according to claim 11, characterized in that The second detection result includes a second detection image, and the second detection image is used to represent the probability of abnormality occurring at each position of the original image when the reference image is used as a reference; Obtaining a second detection result of the data to be detected according to the first similarity data and the second similarity data includes: Obtaining a weighted cumulative value of the first similarity data and the second similarity data to obtain third similarity data; Performing visualization processing on the third similarity data to obtain a second similarity image; Perform image optimization processing on the second similarity image to obtain the second detection image as the second detection result.

15. The method according to claim 14, characterized in that The method further comprises: Obtaining a maximum value of similarity values in the third similarity data to obtain a second similarity value; Obtaining a sum of the second similarity value and the first similarity value to obtain a third similarity value; The third similarity value is normalized to obtain a second anomaly score corresponding to the original image, where the second anomaly score is used to indicate a probability of an anomaly occurring in the original image when a reference image is present. A larger second anomaly score indicates that the location is more likely to be abnormal.

16. The method according to claim 13, characterized in that Acquiring second similarity data between the reference feature of the reference image and the image feature of the original image, comprising: The image features of the reference image and the original image are respectively input into a second preset detection model to obtain second similarity data output by the second preset detection model; the second similarity data represents the weighted cumulative value of the feature similarity between the image features of different levels of the original image and the reference features of the reference image.

17. The method according to claim 16, characterized in that The second preset detection model includes a plurality of cascaded feature extraction units; each level of feature extraction unit is used to extract image features corresponding to input data; the input data includes a reference image or image features output by a previous level of feature extraction unit; The feature extraction units at the same level in the second preset detection model and the first preset detection model are implemented using the same model parameter weights.

18. A target detection device, characterized in that: The device comprises: A detection data acquisition module is used to acquire the data to be detected, wherein the data to be detected includes original images and text data; A first similarity acquisition module is configured to acquire first similarity data between image features of the original image and text features of the text data; the image features include features obtained by weighted fusion processing using a preset neural network model; The first detection result acquisition module is used to obtain a first detection result of the data to be detected based on the first similarity data; the first detection result is used to indicate that an abnormal target is detected from the original image.

19. An electronic device, characterized in that: include a processor; a memory for storing a computer program executable by the processor; The processor is configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that When the executable computer program in the storage medium is executed by a processor, the method according to any one of claims 1 to 17 can be implemented.

Images