Target recognition method, electronic device and computer-readable storage medium

Abstract

The present application discloses a target recognition method, an electronic device and a computer-readable storage medium, the method comprising: obtaining an image to be recognized and its corresponding recognition instruction text; wherein the recognition instruction text includes the category to be recognized of the target; obtaining the visual features of the image to be recognized and the text features of the recognition instruction text, and obtaining the average memory features corresponding to the image to be recognized based on the visual features and the text features; obtaining the fused similarity based on the visual features and the text features, and obtaining the comprehensive similarity based on the visual features and the average memory features; determining the target recognition result that matches the image to be recognized with the category to be recognized based on the fused similarity and the comprehensive similarity. The above scheme can improve the accuracy of target recognition.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target recognition method, an electronic device, and a computer-readable storage medium. Background Art

[0002] As one of the most important tasks in the field of computer vision, the development of cross-modal target recognition technology has laid the foundation for downstream computer tasks. The cross-modal target recognition task is mainly to combine multiple different modal information to perform target recognition tasks and give the category of image targets. At present, the most important method is to combine pictures and texts, integrate natural language processing technology and computer vision technology, and form a text- and picture-based target recognition algorithm. However, when the text features and picture visual features are very different, it is difficult to obtain the correct matching results by directly matching the two, resulting in low accuracy of target recognition. Therefore, how to improve the accuracy of target recognition has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a target recognition method, an electronic device and a computer-readable storage medium, which can improve the accuracy of target recognition.

[0004] To solve the above technical problems, the first aspect of the present application provides a target recognition method, comprising: obtaining an image to be recognized and its corresponding recognition indication text; wherein the recognition indication text includes the category of the target to be recognized; obtaining the visual features of the image to be recognized and the text features of the recognition indication text, and based on the visual features and the text features, obtaining the average memory features corresponding to the image to be recognized; obtaining a fused similarity based on the visual features and the text features, and obtaining a comprehensive similarity based on the visual features and the average memory features; and determining a target recognition result that matches the image to be recognized with the category to be recognized based on the fused similarity and the comprehensive similarity.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method described in the first aspect above.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the method described in the first aspect above.

[0007] The above scheme obtains an image to be identified and an identification instruction text corresponding to the image to be identified, wherein the identification instruction text includes the category of the target to be identified, obtains the visual features of the image to be identified and the text features of the identification instruction text, and obtains the average memory features corresponding to the image to be identified based on the visual features and the text features, obtains the fused similarity based on the acquired visual features and the text features, and obtains the comprehensive similarity based on the acquired visual features and the average memory features, and obtains the target recognition result that matches the category of the target to be identified in the image to be identified based on the fused similarity and the comprehensive similarity. On the basis of using the similarity of visual features and text features for target recognition, by combining the similarity of visual features and the average memory features, it is possible to better alleviate the problem that the gap between the two modalities is large, making the fused features non-discriminative and resulting in matching failure, thereby improving the accuracy of target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flowchart of an implementation method of the target recognition method of the present application;

[0010] Figure 2 It is a flowchart of another implementation method of the target recognition method of the present application;

[0011] Figure 3 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0012] Figure 4 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] See also Figure 1 , Figure 1 : is a flow chart of an implementation method of a target recognition method of the present application, the method comprising:

[0015] S101: Acquire an image to be identified and its corresponding identification indication text; wherein the identification indication text includes the category of the target to be identified.

[0016] Specifically, an image to be recognized is obtained, and recognition indication text corresponding to the image to be recognized is obtained, wherein the recognition indication text includes the category of the target to be recognized.

[0017] In one application mode, an image to be recognized and a recognition instruction text input together with the image to be recognized are obtained, and the recognition instruction text includes the category of the target to be recognized.

[0018] In another application, an image to be identified is obtained and a reference image of the image to be identified is determined, and matching identification indication text is generated based on the reference image. The identification indication text includes the category of the target to be identified, and the identification indication text corresponds to the image to be identified.

[0019] In an application scenario, an image to be recognized and a corresponding recognition instruction text manually input by a user are obtained, and the recognition instruction text includes a category of a target to be recognized, which is used to indicate a target that the user wants to recognize.

[0020] In another application scenario, an image to be identified is obtained and a reference image corresponding to the image to be identified is determined. The reference image is processed using a deep neural network model to generate recognition indication text that matches the reference image. The recognition indication text includes the level of the target to be identified, and the recognition indication text corresponds to the image to be identified.

[0021] In some implementation scenarios, each image to be identified includes multiple categories, and the identification indication text is used to indicate the categories to be identified corresponding to multiple different targets that the user wants to identify.

[0022] In some implementation scenarios, each image to be identified includes multiple categories, and the identification indication text is used to indicate the category to be identified corresponding to a single target that the user wants to identify.

[0023] In some implementation scenarios, each image to be identified includes only a single category, and the identification indication text is used to indicate the category to be identified corresponding to the single target that the user wants to identify.

[0024] S102: Obtain visual features of the image to be recognized and text features of the recognition instruction text, and obtain average memory features corresponding to the image to be recognized based on the visual features and the text features.

[0025] Specifically, the visual features of the image to be recognized and the text features of the recognition instruction text are obtained, and based on the visual features and the text features, the average memory features corresponding to the image to be recognized are obtained.

[0026] In one application method, a target region of interest in an image to be identified is determined, a feature vector of the image to be identified is extracted from the target region of interest as a visual feature of the image to be identified, a target text label in an identification indication text is determined, a feature vector of the identification indication text is extracted from the target text label as a text feature of the identification indication text, and based on the visual feature and the text feature, an average memory feature corresponding to the image to be identified is obtained.

[0027] It should be noted that in machine vision and image processing, the region of interest is the area to be processed in the image being processed, which is outlined in the form of a box, circle, ellipse, irregular polygon, etc. Various operators and functions are often used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest and perform the next step of image processing.

[0028] In another application method, image feature point extraction is performed on the image to be identified to obtain multiple feature points in the image to be identified, visual features of the image to be identified are obtained based on the multiple feature points, some sentences in the identification instruction text are filtered, the target text label in the filtered identification instruction text is determined, the feature vector of the identification instruction text is extracted from the target text label as the text feature of the identification instruction text, and based on the visual feature and the text feature, the average memory feature corresponding to the image to be identified is obtained.

[0029] S103: Obtaining fusion similarity based on the visual features and the text features, and obtaining comprehensive similarity based on the visual features and the average memory features.

[0030] Specifically, based on the acquired visual features and text features, a fusion similarity is obtained, and based on the acquired visual features and average memory features, a comprehensive similarity is obtained.

[0031] In one application, an attention mechanism is used to calculate the fusion similarity between visual features and text features, and cosine similarity is used to calculate the sum similarity between visual features and average memory features.

[0032] In another application, a pre-trained cross-modal model is used to calculate the fusion similarity between visual features and text features, and cosine similarity is used to calculate the comprehensive similarity between visual features and average memory features.

[0033] S104: Based on the fusion similarity and the comprehensive similarity, determine the target recognition result that matches the image to be recognized with the category to be recognized.

[0034] Specifically, based on the fusion similarity and the comprehensive similarity, a target recognition result matching the to-be-recognized category of the target to be recognized in the to-be-recognized image is obtained.

[0035] In one application, the fused similarity and the comprehensive similarity are fused to obtain at least one final similarity, and the final similarities are compared to obtain a target recognition result that matches the target category in the image to be recognized.

[0036] In another application, the fused similarity and the comprehensive similarity are fused to obtain at least one final similarity, and the largest similarity value is selected to compare with a preset similarity threshold to obtain a target recognition result that matches the category of the target to be recognized in the image to be recognized.

[0037] The above scheme obtains an image to be identified and an identification instruction text corresponding to the image to be identified, wherein the identification instruction text includes the category of the target to be identified, obtains the visual features of the image to be identified and the text features of the identification instruction text, and obtains the average memory features corresponding to the image to be identified based on the visual features and the text features, obtains the fused similarity based on the acquired visual features and the text features, and obtains the comprehensive similarity based on the acquired visual features and the average memory features, and obtains the target recognition result that matches the category of the target to be identified in the image to be identified based on the fused similarity and the comprehensive similarity. On the basis of using the similarity of visual features and text features for target recognition, by combining the similarity of visual features and the average memory features, it is possible to better alleviate the problem that the gap between the two modalities is large, making the fused features non-discriminative and resulting in matching failure, thereby improving the accuracy of target recognition.

[0038] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the target recognition method of the present application, the method comprising:

[0039] S201: Acquire an image to be identified and its corresponding identification indication text; wherein the identification indication text includes the category of the target to be identified.

[0040] Specifically, an image to be recognized is obtained, and recognition indication text corresponding to the image to be recognized is obtained, wherein the recognition indication text includes the category of the target to be recognized.

[0041] S202: Obtain visual features of the image to be recognized and text features of the recognition instruction text, and obtain average memory features corresponding to the image to be recognized based on the visual features and the text features.

[0042] Specifically, the visual features of the image to be recognized and the text features of the recognition instruction text are obtained, and based on the visual features and the text features, the average memory features corresponding to the image to be recognized are obtained.

[0043] In one embodiment, the target recognition result is obtained using a target recognition model, which includes a visual feature extraction module, a text feature extraction module and an average feature memory module. The average feature memory module stores average memory features corresponding to preset categories of targets. Step S202 specifically includes: inputting the recognition indication text into the text feature extraction module to obtain text features, and inputting the image to be recognized into the visual feature extraction module to obtain visual features; based on the text features, visual features, the category to be recognized and the preset category in the average feature memory module, obtaining the average memory features corresponding to the image to be recognized.

[0044] Specifically, the recognition indication text is input into the text feature extraction module to obtain the text features output by the text feature extraction module, the image to be recognized is input into the visual feature extraction module to obtain the visual features output by the visual feature extraction module, and based on the extracted text features, visual features, the category of the target to be recognized and the preset category stored in the average feature memory module, the average memory features corresponding to the image to be recognized are obtained.

[0045] It should be noted that the target recognition model is obtained through pre-training, and the average memory features corresponding to the preset categories of targets stored in the average feature memory module are constructed during the training process. The specific training process and construction process will be explained later.

[0046] In one implementation scenario, the step of obtaining the average memory feature corresponding to the image to be identified based on text features, visual features, the category to be identified and the preset category in the average feature memory module specifically includes: in response to the category to be identified matching the preset category, obtaining the average memory feature corresponding to the target of the preset category stored in the average feature memory module; in response to the category to be identified being a new category different from the preset category, generating the average memory feature corresponding to the target of the new category based on text features and visual features.

[0047] Specifically, when the category to be identified matches the preset category, it means that the category to be identified has appeared in the training process. Therefore, the average memory feature corresponding to the target of the preset category stored in the average feature memory module will be directly obtained, thereby reducing the time consumption and memory usage during the use of the target recognition model.

[0048] Furthermore, when the category to be identified is a new category different from the preset category, it means that the category to be identified has not appeared in the training process. For these new categories, the corresponding average memory features are missing in the average feature memory module. Therefore, it is necessary to establish the average memory features corresponding to the targets of these new categories and update them in real time. Based on text features and visual features, the average memory features corresponding to the targets of the new categories are generated.

[0049] In an application scenario, the text features correspond to the first candidate text features and the second candidate text features obtained after being input into the text feature extraction module. The step of generating the average memory features corresponding to the targets of the newly added category based on the text features and the visual features specifically includes: obtaining the first screening similarity based on the first candidate text features and the visual features, and obtaining the second screening similarity based on the second candidate text features and the visual features; in response to the first screening similarity and the second screening similarity satisfying the preset screening conditions, generating the average memory features corresponding to the targets of the newly added category based on the visual features.

[0050] Specifically, the first candidate text feature represents the text feature with the highest similarity matched by the visual feature, and the second candidate text feature represents the text feature with the second highest similarity matched by the visual feature. Based on the first candidate text feature and the visual feature, a first screening similarity is obtained, and based on the second candidate text feature and the visual feature, a second screening similarity is obtained. When the first screening similarity and the second candidate fusion similarity meet the preset screening conditions, the visual feature is used as the first average memory feature corresponding to the target of the newly added category.

[0051] In a specific application scenario, the following strategy is used to select the first average memory feature of a newly added category k:

[0052] (1)

[0053] in, It represents the first average memory feature of the newly added category k, I t represents the tth visual feature, T k It represents the text feature with the highest similarity matched by the t-th visual feature, T e It represents the second most similar text feature matched by the t-th visual feature. λ1 and λ2 are two threshold parameters used to control the strictness of the screening. t ·T k Represents the first screening similarity, I t ·T e Indicates the second screening similarity. When the first screening similarity I t ·T k Greater than the threshold parameter λ2 or when the first screening similarity I t ·T k Similarity with the second screening I t ·T e When the difference is greater than the threshold parameter λ1, the visual feature I t As the first average memory feature of the newly added category k Through the above strict screening requirements, the first average memory feature of the newly added category k can be selected as accurately as possible.

[0054] In one application scenario, after the step of generating the average memory features corresponding to the targets of the newly added category based on the text features and the visual features, it also includes: adding the average memory features corresponding to the targets of the newly added category to the average feature memory module; in response to obtaining the new images and new texts corresponding to the new categories again, updating the average memory features corresponding to the targets of the newly added category based on the new images, the new texts and the average memory features corresponding to the targets of the newly added category.

[0055] Specifically, after obtaining the average memory features corresponding to the targets of the newly added category, the average memory features are added to the average feature memory module and updated in real time. When the new images and new texts corresponding to the new category are obtained again, the average memory features corresponding to the targets of the newly added category are updated based on the average memory features corresponding to the new images, new texts and targets of the newly added category, thereby enriching the semantic discriminability of the average memory features.

[0056] In a specific application scenario, the following update strategy is used for the average memory features corresponding to the newly added category k:

[0057] (2)

[0058] Among them, k represents the newly added category k, n represents the average memory feature updated for the nth time, represents the average memory feature after update, represents the average memory feature before updating, It represents the visual features of the nth newly added image of the newly added category k, sim k It represents the similarity between the text features of the newly added text and the visual features of the newly added image. Through the above strategy, the average memory features corresponding to the targets of the newly added category k can be updated according to the similarity of real-time recognition, thereby improving the stability and accuracy of subsequent recognition of the newly added category.

[0059] In one implementation scenario, the average memory feature corresponding to a target of a preset category is obtained based on the following steps: obtaining a training sample set corresponding to a target of a preset category; wherein the training sample set includes multiple training images; obtaining visual training features of the multiple training images, and obtaining the average memory feature corresponding to the target of the preset category based on the multiple visual training features.

[0060] Specifically, a training sample set corresponding to a target of a preset category is obtained, the training sample set includes multiple training images, and each preset category is matched with multiple training images, and visual training features of the multiple training images matching each preset category are extracted. Based on the extracted multiple visual training features, an average memory feature corresponding to the target of the preset category is obtained. As the training process proceeds, the average memory feature will gather more and more visual semantic information of the same type of target, and have better semantic discriminability of the target, thereby improving the accuracy of target recognition.

[0061] In a specific implementation scenario, the specific updating process of the average memory feature is as follows:

[0062] (3)

[0063] Among them, i represents the i-th category, n represents the total number of training images that appear in the i-th category at the current stage of training, represents the average memory feature after update, represents the average memory feature before updating, It represents the visual features of the nth training image of the i-th category.

[0064] Optionally, before updating the average feature memory of the current stage, the similarity sim1 between the visual features of the current training target and the average memory features is calculated:

[0065] (4)

[0066] After obtaining the similarity sim1, it is combined with the similarity sim1 calculated between the text feature and the visual feature. m Fusion is performed to obtain the comprehensive similarity sim2. The specific process is as follows:

[0067] (5)

[0068] (6)

[0069] in, represents the visual features of the nth training image, It represents the text features corresponding to the training image, and k represents a balance parameter used to average the relationship between the two similarities.

[0070] Furthermore, the calculated sim2 is sent as the final recognition probability into the contrast loss function to calculate the recognition loss function of the current training image, and is added to the cross-modal contrast loss function to jointly optimize the visual feature extraction module and the text feature extraction module. The cross-modal contrast loss function also includes the loss function between the text training features and the visual training features, thereby completing the training of the target recognition model.

[0071] S203: Based on the visual features and the text features, obtain a first candidate fusion similarity and a second candidate fusion similarity; wherein the first candidate fusion similarity matches a first category, and the second candidate fusion similarity matches a second category.

[0072] Specifically, the visual features are matched with the text features of all categories to obtain the matching matrix , where j represents the jth category, It represents the similarity between the visual feature and the text feature of the jth category, from the matching matrix Select the two similarities with the highest similarity values ​​as the first candidate fusion similarity Fusion similarity with the second candidate ,in, Greater than , the first candidate fusion similarity Matches the first category m, the second candidate fusion similarity There is a second category n for the match.

[0073] S204: Obtain a first average feature corresponding to the first category, and obtain a second average feature corresponding to the second category.

[0074] Specifically, the first average feature corresponding to the first category m is obtained from the average feature memory module. , and obtain the second average feature corresponding to the second category n .

[0075] S205: Obtain a first candidate comprehensive similarity based on the visual feature and the first average feature, and obtain a second candidate comprehensive similarity based on the visual feature and the second average feature.

[0076] Specifically, based on visual features and the first average characteristic , get the first candidate comprehensive similarity sim3 m ,in, , based on visual features and the second average feature , get the second candidate comprehensive similarity sim3 n ,in, .

[0077] S206: Obtain a first recognition similarity based on the first candidate fusion similarity and the first candidate comprehensive similarity, and obtain a second recognition similarity based on the second candidate fusion similarity and the second candidate comprehensive similarity.

[0078] Specifically, the first candidate fusion similarity The comprehensive similarity with the first candidate Fusion is performed to obtain the first recognition similarity , and the second candidate fusion similarity The comprehensive similarity with the second candidate Fusion is performed to obtain the second recognition similarity .

[0079] S207: Determine a target recognition result that matches the image to be recognized with the category to be recognized based on the first recognition similarity and the second recognition similarity.

[0080] Specifically, the first recognition similarity and the second recognition similarity When comparing, first identify the similarity Greater than the second recognition similarity When the target recognition result that the image to be recognized matches the category to be recognized is determined to be the first category m, when the first recognition similarity Less than the second recognition similarity When , it is determined that the target recognition result that matches the image to be recognized and the category to be recognized is the second category n. Since the average memory feature collects a large amount of visual semantic information of the category corresponding to the target, it is applied to the cross-modal recognition task, which can greatly improve the accuracy of target recognition.

[0081] See also Figure 3 , Figure 3: is a structural diagram of an embodiment of an electronic device of the present application, the electronic device 30 comprises a memory 300 and a processor 302 coupled to each other, wherein the memory 300 stores program instructions (not shown), and the processor 302 is used to execute the program instructions to implement the method in any of the above embodiments. For the description of the relevant content, please refer to the detailed description of the above method embodiment, which will not be repeated here. In addition, the processor 302 can also be called a CPU (Center Processing Unit). The processor 302 may be an integrated circuit chip with signal processing capabilities. The processor 302 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 302 may be implemented by an integrated circuit chip.

[0082] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0083] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., and other media that can store program codes.

[0086] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target recognition method, characterized in that: include: Acquire an image to be identified and its corresponding identification indication text; wherein the identification indication text includes the category of the target to be identified; Acquire the visual features of the image to be recognized and the text features of the recognition instruction text, and obtain the average memory features corresponding to the image to be recognized based on the visual features and the text features; Based on the visual features and the text features, a fusion similarity is obtained, and based on the visual features and the average memory features, a comprehensive similarity is obtained; wherein the fusion similarity includes candidate fusion similarities matched by all the categories to be identified, and the fusion similarity is obtained based on the visual features and the text features, and the comprehensive similarity is obtained based on the visual features and the average memory features, including: obtaining a first candidate fusion similarity and a second candidate fusion similarity based on the visual features and the text features; wherein the first candidate fusion similarity matches a first category, and the second candidate fusion similarity matches a second category; obtaining a first average feature corresponding to the first category, and obtaining a second average feature corresponding to the second category; obtaining a first candidate comprehensive similarity based on the visual features and the first average feature, and obtaining a second candidate comprehensive similarity based on the visual features and the second average feature; Based on the fused similarity and the comprehensive similarity, a target recognition result that matches the image to be recognized with the category to be recognized is determined.

2. The method according to claim 1, characterized in that The target recognition result is obtained by using a target recognition model, wherein the target recognition model includes a visual feature extraction module, a text feature extraction module and an average feature memory module, wherein the average feature memory module stores average memory features corresponding to targets of a preset category; The acquiring of the visual features of the image to be recognized and the text features of the recognition instruction text, and obtaining the average memory features corresponding to the image to be recognized based on the visual features and the text features, includes: Inputting the recognition indication text into the text feature extraction module to obtain the text feature, and inputting the image to be recognized into the visual feature extraction module to obtain the visual feature; Based on the text features, the visual features, the category to be identified and the preset category in the average feature memory module, the average memory feature corresponding to the image to be identified is obtained.

3. The method according to claim 2, characterized in that The obtaining the average memory feature corresponding to the image to be identified based on the text feature, the visual feature, the category to be identified and the preset category in the average feature memory module includes: In response to the to-be-identified category matching the preset category, acquiring the average memory feature corresponding to the target of the preset category stored in the average feature memory module; In response to the to-be-identified category being a newly added category different from the preset category, an average memory feature corresponding to a target of the newly added category is generated based on the text feature and the visual feature.

4. The method according to claim 3, characterized in that The text features correspond to the first candidate text features and the second candidate text features obtained after being input into the text feature extraction module; The step of generating an average memory feature corresponding to the target of the newly added category based on the text feature and the visual feature includes: Obtaining a first screening similarity based on the first candidate text feature and the visual feature, and obtaining a second screening similarity based on the second candidate text feature and the visual feature; In response to the first screening similarity and the second screening similarity satisfying a preset screening condition, an average memory feature corresponding to the target of the newly added category is generated based on the visual feature.

5. The method according to claim 3, characterized in that: After generating the average memory feature corresponding to the target of the newly added category based on the text feature and the visual feature, the method further includes: Adding the average memory features corresponding to the newly added category of targets to the average feature memory module; In response to obtaining a new image and a new text corresponding to the new category again, the average memory feature corresponding to the target of the new category is updated based on the new image, the new text and the average memory feature corresponding to the target of the new category.

6. The method according to claim 2, characterized in that The average memory feature corresponding to the target of the preset category is obtained based on the following steps: Acquire a training sample set corresponding to the target of the preset category; wherein the training sample set includes a plurality of training images; Visual training features of the plurality of training images are obtained, and based on the plurality of visual training features, average memory features corresponding to the objects of the preset category are obtained.

7. The method according to claim 1, characterized in that The determining, based on the fusion similarity and the comprehensive similarity, a target recognition result matching the image to be recognized with the category to be recognized includes: Based on the first candidate fusion similarity and the first candidate comprehensive similarity, a first recognition similarity is obtained, and based on the second candidate fusion similarity and the second candidate comprehensive similarity, a second recognition similarity is obtained; Based on the first recognition similarity and the second recognition similarity, a target recognition result is determined in which the image to be recognized matches the category to be recognized.

8. An electronic device, characterized in that: The method comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the method according to any one of claims 1 to 7.

Images