Target identification method and device, electronic equipment and computer storage medium

By obtaining the feature and semantic feature information of the target image, generating correlation information, and dynamically adjusting parameters, the problem of low multi-objective recognition accuracy is solved, and higher recognition accuracy and adaptability are achieved.

CN118918316APending Publication Date: 2024-11-08LENOVO (BEIJING) LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411036001.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the multi-objective recognition task, the target recognition accuracy is low due to the complex image content and the difference in target size, and it is difficult to manually set fixed parameters to adapt to the proportion changes of different input images, resulting in low recognition accuracy.

Method used

By acquiring feature map information and semantic feature information of the target image, the first correlation information and the second correlation information are generated, and the parameters are dynamically adjusted using these information, thereby improving the accuracy of multi-objective recognition. The specific method includes using the feature detection head and the semantic feature detection head to extract features and semantic features, generating weights through the attention mechanism, fusing features and semantic information, and generating association information to identify multiple targets.

Benefits of technology

The accuracy of multi-object recognition is improved, adapting to the differences between different size targets, enhancing the accuracy and meaning of associated information, and alleviating the problem of low recognition accuracy under fixed parameter settings.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918316A_ABST
    Figure CN118918316A_ABST
Patent Text Reader

Abstract

The invention provides a target recognition method and device, electronic equipment and a computer storage medium, and can be applied to the technical field of image processing and artificial intelligence. The target recognition method comprises the steps that a to-be-detected target image is acquired, and the target image comprises a plurality of targets of a plurality of sizes; determining feature map information and semantic feature information of the target image according to the target image; according to the feature map information and the semantic feature information, first associated information and second associated information are generated, and the first associated information is different from the second associated information; and identifying a plurality of targets from the target image according to the first associated information and the second associated information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing and artificial intelligence technology, and in particular to a target recognition method, device, electronic device and computer storage medium. Background Art

[0002] With the rapid development of artificial intelligence and computer technology, the application of target detection / recognition tasks in image processing tasks is becoming more and more extensive. For example, through the target recognition task, one or more targets can be identified from an image. The target can be a person, animal, plant, object, etc.

[0003] Since the image content in multi-target recognition tasks is usually complex, such as the image includes multiple targets or there are size differences between multiple targets, the accuracy of target recognition is low. Summary of the invention

[0004] One aspect of the present disclosure provides a target recognition method, including: acquiring a target image to be detected, wherein the target image includes multiple targets of multiple sizes; determining feature map information and semantic feature information of the target image based on the target image; generating first association information and second association information based on the feature map information and the semantic feature information, the first association information being different from the second association information; and identifying multiple targets from the target image based on the first association information and the second association information.

[0005] Optionally, first association information and second association information are generated based on feature graph information and semantic feature information, including: determining first weights and second weights based on the feature graph information and semantic feature information; generating first association information based on the first weights and semantic feature information; generating second association information based on the second weights and feature graph information.

[0006] Optionally, a first weight and a second weight are determined based on feature graph information and semantic feature information; first association information is generated based on the first weight and the semantic feature information; and second association information is generated based on the second weight and feature graph information, including: determining the first weight based on the feature graph information and first semantic feature sub-information segmented from the semantic feature information; generating the first association information based on the first weight and second semantic feature sub-information different from the first semantic feature sub-information; wherein the second weight is obtained after normalization processing of at least one third semantic feature sub-information segmented from the semantic feature information, and the at least one third semantic feature sub-information is different from the first semantic feature sub-information.

[0007] Optionally, a first weight and a second weight are determined based on feature graph information and semantic feature information; first association information is generated based on the first weight and the semantic feature information; and second association information is generated based on the second weight and the feature graph information, including: determining a second weight based on first feature subgraph information segmented from the feature graph information and the semantic feature information; generating second association information based on the second weight and second feature subgraph information different from the first feature subgraph information; wherein the first weight is obtained after normalization processing of at least one third feature subgraph information segmented from the feature graph information, and the at least one third feature subgraph information is different from the first feature subgraph information.

[0008] Optionally, the method further includes at least one of the following: updating the feature map information according to the first association information, and obtaining updated first association information according to the updated feature map information and the semantic feature information;

[0009] The semantic feature information is updated according to the second association information, and the updated second association information is obtained according to the updated semantic feature information and the feature map information.

[0010] Optionally, based on the target image, feature map information and semantic feature information of the target image are determined, including: using a feature detection head to generate feature map information based on image features of the target image, wherein the image features are obtained by extracting features of the target image using a backbone network; and using a semantic feature detection head to generate semantic feature information based on the image features of the target image.

[0011] Optionally, multiple targets are identified from the target image based on the first association information and the second association information, including: generating new feature map information based on the first association information and the second association information; and identifying multiple targets from the target image based on the new feature map information through a target recognition algorithm.

[0012] Another aspect of the present disclosure provides a target recognition device, including: an acquisition module, used to acquire a target image to be detected, wherein the target image includes candidate targets of multiple sizes; a feature determination module, used to determine feature map information and semantic feature information of the target image based on the target image; a generation module, used to generate first association information and second association information based on the feature map information and the semantic feature information, the first association information being different from the second association information; and an identification module, used to identify multiple targets from the target image based on the first association information and the second association information.

[0013] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute any of the above methods.

[0014] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement any of the above methods when executed by a processor.

[0015] Another aspect of the present disclosure provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement any of the above methods when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0017] Figure 1 The application scenario of the target recognition method and device according to the embodiment of the present disclosure is schematically shown;

[0018] Figure 2 A flowchart of a target recognition method according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 3 A flowchart of a method for generating first association information and second association information according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 4A A flowchart of generating first association information and second association information according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 4B A flowchart of generating first association information and second association information according to another embodiment of the present disclosure is schematically shown;

[0022] Figure 5 A flowchart of generating first association information and second association information according to another embodiment of the present disclosure is schematically shown;

[0023] Figure 6 A flowchart of a method for determining feature map information and semantic feature information of a target image according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 7 A block diagram schematically shows a target recognition device according to an embodiment of the present disclosure; and

[0025] Figure 8 A block diagram of an electronic device for target recognition according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0030] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transmit instructions. For example, a computer-readable medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tape or hard disk (HDD); optical storage devices, such as compact disk (CD-ROM); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0031] For multi-target recognition tasks, related technologies can use target recognition / detection algorithms based on multiple feature maps to identify multiple targets of multiple sizes in an image, such as the YOLO algorithm. However, for different input images, the proportion of multi-sized targets inside them varies. For example, some images have more large-sized targets, while some images have more small-sized targets. The density of targets with different information in different areas of the image varies, and the importance of multiple sizes between multiple feature maps and within a single feature map also varies. Therefore, if multi-target recognition is performed by manually setting fixed parameters in the prior art, there will be a technical problem of low target recognition accuracy.

[0032] Therefore, the embodiment of the present disclosure provides a target recognition method, including: obtaining a target image to be detected, wherein the target image includes multiple targets of multiple sizes; determining feature map information and semantic feature information of the target image according to the target image; generating first association information and second association information according to the feature map information and the semantic feature information, wherein the first association information is different from the second association information; and identifying multiple targets from the target image according to the first association information and the second association information. Through the embodiment of the present disclosure, for each target image to be detected, the corresponding parameters can be used to perform multi-target recognition, thereby improving the accuracy of multi-target recognition.

[0033] Figure 1 The application scenarios of the target recognition method and device according to the embodiments of the present disclosure are schematically illustrated.

[0034] like Figure 1 As shown, the target image 101 to be detected may include multiple targets of multiple sizes, such as cars, trucks, street lamps, etc. After target recognition / detection processing, a target image 103 after multiple targets are recognized is obtained. In the target image 103 after multiple targets are recognized, multiple targets can be marked by identification boxes.

[0035] The target recognition method of the disclosed embodiment can be executed by the terminal device 1021, and the target recognition device is also set in the terminal device 1021. For example, the user can use the terminal device 1021 to shoot the target image 101 to be detected, and use the terminal device 1021 to perform the following operations: according to the target image, determine the feature map information and semantic feature information of the target image; according to the feature map information and the semantic feature information, generate first association information and second association information, the first association information is different from the second association information; according to the first association information and the second association information, identify multiple targets from the target image.

[0036] Alternatively, the target recognition method of the embodiment of the present disclosure may also be executed by the server 1022, and the target recognition device is also disposed in the server 1022. For example, a user may use the terminal device 1021 to shoot the target image 101 to be detected, and the server 1022 may obtain the target image 101 to be detected from the terminal device 1021, and determine the feature map information and semantic feature information of the target image according to the target image; generate first association information and second association information according to the feature map information and the semantic feature information, and the first association information is different from the second association information; and identify multiple targets from the target image according to the first association information and the second association information.

[0037] Figure 2 The flowchart schematically shows the target recognition method according to the embodiment of the present disclosure.

[0038] like Figure 2 As shown, the method includes operations S210 to S240.

[0039] In operation S210, an object image to be detected is acquired, wherein the object image includes a plurality of objects of a plurality of sizes.

[0040] In operation S220, feature map information and semantic feature information of the target image are determined according to the target image.

[0041] In operation S230, first association information and second association information are generated according to the feature graph information and the semantic feature information, and the first association information is different from the second association information.

[0042] In operation S240, a plurality of objects are recognized from the object image according to the first association information and the second association information.

[0043] According to an embodiment of the present disclosure, the target image may be acquired by a variety of image acquisition devices, such as a mobile terminal device, a camera, a gimbal, etc. The target image to be detected may be acquired from a local, cached, external terminal device or image acquisition device.

[0044] The target image may include targets of multiple sizes. In the embodiments of the present disclosure, the size may be understood as the size of the target in the target image, or the proportion of pixels occupied by the target in the target image. It should be noted that for two objects with obvious size differences in the real world, due to the difference in distance between the two objects and the shooting point when the target image is shot, the two objects may have the same size in the target image.

[0045] The feature map information of the target image represents the feature information related to the target recognition task, and can be obtained by performing a feature extraction operation on the target image. For multi-target recognition tasks, the feature map information usually includes at least one feature map corresponding to multiple sizes. Among them, one size can correspond to one feature map or multiple feature maps. For example, different convolution kernels can be used to perform convolution operations on the target image to obtain feature extraction results for targets of different sizes, that is, feature maps.

[0046] For example, a multi-target recognition task may include targets of three sizes: large, medium, and small, and the feature map information may include three layers of feature maps for the above three sizes.

[0047] The semantic feature information represents feature information extracted from the target image from a semantic perspective, and can be obtained by performing semantic feature extraction on the target image. Similar to the feature map information, for multi-target recognition tasks, the semantic feature information generally includes at least one semantic feature map corresponding to multiple sizes. In an illustrative embodiment, the number of semantic feature maps is the same as the number of feature maps.

[0048] The semantic feature information and the feature map information can reflect the features of the target image from different dimensions. Therefore, different first association information and second association information can be generated using the feature map information and the semantic feature information. The first association information and the second association information are both used to characterize the degree of association between multiple targets in the target image and the target image.

[0049] In a specific embodiment, the first association information may be generated using feature graph information, and the second association information may be generated using semantic feature information. For example, the feature graph information and the semantic feature information may be processed separately through an attention mechanism to generate the corresponding first association information and second association information.

[0050] In another specific embodiment, the feature map information and the semantic feature information may be fused, and the first association information and the second association information may be generated respectively. For example, the feature map information / semantic feature information may be processed by an attention mechanism to generate the first association information; and the information after the feature map information and the semantic feature information are fused may be processed by an attention mechanism to generate the second association information. The information after the feature map information and the semantic feature information are fused may be the information after the feature map information and the semantic feature information are added together.

[0051] In another specific embodiment, different processing may be performed on the information after the fusion of the feature map information and the semantic feature information to generate the first association information and the second association information respectively. Alternatively, the feature map information and the semantic feature information may be fused in different fusion methods, such as using different weights for weighted fusion, and using the attention mechanism to process the information obtained by different fusion methods to generate the first association information and the second association information.

[0052] According to an embodiment of the present disclosure, after the first association information and the second association information are determined, multiple targets can be identified from the target image according to the first association information and the second association information using an existing target recognition algorithm.

[0053] For example, the first association information and the second association information may be fused to identify multiple targets from the target image. Alternatively, the second association information may be adjusted according to the first association information or the first association information may be adjusted according to the second association information, and then multiple targets may be identified from the target image according to the adjusted association information.

[0054] In the embodiments of the present disclosure, not only feature map information of the feature dimension is generated based on the target image to be detected, but also semantic feature information of the semantic dimension is generated, and the target image is interpreted in the newly added dimension. Different first association information and second association information are generated based on the semantic feature information and the feature map information, and both the first association information and the second association information integrate the feature map information and the newly added semantic feature information, thereby improving the accuracy of the first association information / second association information itself, thereby improving the recognition accuracy of multi-sized targets. In addition, multiple targets are identified from the target image based on different first association information and second association information, which not only increases the number of association information, but also enriches the meaning of the association information, thereby further improving the recognition accuracy of multi-sized targets.

[0055] In addition, it should be noted that for multiple target images with different size ratios and target numbers, their corresponding semantic feature information and feature map information are different. However, the embodiments of the present disclosure can adaptively generate first association information and second association information based on their corresponding semantic feature information and feature map information to reflect the differences between multiple targets in each target image, thereby realizing the execution of multi-target recognition tasks based on dynamic parameters.

[0056] Figure 3 The flowchart of the method for generating first association information and second association information according to an embodiment of the present disclosure is schematically shown.

[0057] like Figure 3As shown, the embodiment 300 includes operations S331 to S333, which are a specific embodiment of the above operation S230, generating the first association information and the second association information. In addition, the embodiment 300 may also include the above operations S210, S220 and S240, which will not be repeated here.

[0058] In operation S331, a first weight and a second weight are determined based on the feature map information and the semantic feature information.

[0059] In operation S332, first association information is generated according to the first weight and the semantic feature information.

[0060] In operation S333, second association information is generated based on the second weight and the feature map information.

[0061] According to an embodiment of the present disclosure, similar to the attention weight in the attention mechanism, the first weight is used to characterize semantic features with differences in semantic feature information, and the second weight is used to characterize image features with differences in feature map information.

[0062] After applying the first weight to the semantic feature information, the obtained first association information can reflect the differences between objects of multiple sizes in the target image from a semantic dimension; after applying the second weight to the feature map information, the obtained second association information can reflect the differences between objects of multiple sizes in the target image from a feature dimension.

[0063] For example, the first weight may be applied to the semantic feature information and the second weight may be applied to the feature map information by element-wise multiplication.

[0064] It should also be noted that, in the case where the feature map information includes multiple feature maps and the semantic feature information includes multiple semantic feature maps, the feature map information (semantic feature information) can include multiple feature maps (semantic feature maps) in the form of a vector matrix, that is, the vector matrix includes information of multiple positions within each feature map in the multiple feature maps. Thus, the first association information can not only reflect the differences between targets of multiple sizes in the target image, but also reflect the differences between multiple semantic feature maps corresponding to multiple sizes; similarly, the second association information can also reflect the differences between multiple feature maps corresponding to multiple sizes.

[0065] In a specific embodiment, the feature map information and the semantic feature information may be used as two data sources respectively, and the first weight and the second weight may be generated respectively according to a cross attention mechanism.

[0066] For example, the semantic feature information is used as the Q (query) of the cross-attention mechanism, and the feature map information is used as the K (Key) and V (Value) of the cross-attention mechanism, and the first weight is generated according to the cross-attention mechanism; or, the feature map information is used as the Q of the cross-attention mechanism, and the semantic feature information is used as the K and V of the cross-attention mechanism, and the second weight is generated according to the cross-attention mechanism.

[0067] Alternatively, only the cross-attention mechanism is used to generate the first weight (or the second weight), and other methods different from the cross-attention mechanism are used to generate the second weight (or the first weight), so as to subsequently generate different first association information and second association information. When the second weight (or the first weight) is generated by other methods, only the feature map information (or the semantic feature information) may be used, or both the feature map information and the semantic feature information may be used.

[0068] In the embodiments of the present disclosure, since the generated first weight and second weight both fuse the feature map information and the semantic feature information at the same time, the first association information obtained after combining the first weight can fuse the information of the two dimensions to more completely and comprehensively focus on some features in the semantic feature information, and the second association information obtained after combining the second weight can fuse the information of the two dimensions to more completely and comprehensively focus on some features in the feature map information, thereby improving the recognition accuracy of targets of various sizes in different target images.

[0069] In a specific example, the first weight and the second weight can be used as a forgetting weight and a memory weight, respectively, to forget some features in the semantic feature information and enhance some features in the feature map information. Alternatively, the first weight and the second weight can be used as a memory weight and a forgetting weight, respectively; or, both the first weight and the second weight can be used as a forgetting weight or a memory weight.

[0070] In order to more clearly describe the generation process of the first weight and the second weight, two specific embodiments are used for explanation below. The following two specific embodiments are only used to describe the generation process of the first weight and the second weight, but it should be understood that the present solution is not limited to the following two embodiments.

[0071] In the first embodiment, a first weight and a second weight are determined based on feature graph information and semantic feature information; first association information is generated based on the first weight and the semantic feature information; and second association information is generated based on the second weight and feature graph information, including: determining the first weight based on the feature graph information and first semantic feature sub-information segmented from the semantic feature information; generating the first association information based on the first weight and second semantic feature sub-information different from the first semantic feature sub-information; wherein the second weight is obtained after normalization processing based on at least one third semantic feature sub-information segmented from the semantic feature information, and the at least one third semantic feature sub-information is different from the semantic feature sub-information.

[0072] Specifically, the semantic feature information may be segmented into a plurality of different semantic feature sub-information in an equal-channel segmentation manner, so that each semantic feature sub-information can reflect the semantic features of the target image and there are differences between the plurality of semantic feature sub-information.

[0073] In this embodiment, determining the first weight includes: performing element-wise multiplication of the feature map information and the first semantic feature sub-information obtained by segmenting the semantic feature information to obtain first feature map information that preliminarily fuses the feature map information and the first semantic feature sub-information; and normalizing the first feature map information to obtain the first weight.

[0074] For example, a normalization function may be used for normalization processing. The normalization function may be a function that normalizes the first feature map information to between 0 and 1 (including the boundaries 0 and 1), such as a sigmoid function.

[0075] It should be noted that the first weight and the second weight in the embodiment of the present disclosure are not a single weight value, but a weight value of each position inside the image in the field of image processing. The first weight can also be called a first weight matrix or a first weight vector.

[0076] The first association information may be obtained by multiplying the first weight by the second semantic feature sub-information different from the first semantic feature sub-information. Since the first weight is a value between 0 and 1, the first association information may be information obtained by forgetting some features in the second semantic feature sub-information based on the first weight.

[0077] In this embodiment, the second weight may be obtained by normalizing one or more third semantic feature sub-information using a normalization function.

[0078] For example, if the semantic feature information is divided into three parts, such as the first semantic feature sub-information, the second semantic feature sub-information, and the third semantic feature sub-information, the first weight can be determined based on the feature graph information and the first semantic feature sub-information; the first association information is generated based on the first weight and the second semantic feature sub-information. The second weight is obtained by normalizing the third semantic feature sub-information using a normalization function.

[0079] Alternatively, for the plurality of third semantic feature sub-information obtained by segmentation, each third semantic feature sub-information may be normalized, and then the average value of the weights of the plurality of third semantic feature sub-information is taken as the second weight.

[0080] In the embodiment of the present disclosure, since the semantic feature sub-information used in calculating the first weight, generating the first association information, and calculating the second weight is different, each operation in the process of determining the first weight can integrate part of the newly added information in the semantic feature sub-information; and the second association information generated by the second weight can integrate different semantic feature sub-information from the first. Therefore, different first association information and second association information can make full use of information in the semantic dimension and feature dimension to improve the accuracy of target recognition.

[0081] Figure 4A The flowchart of generating the first association information and the second association information according to an embodiment of the present disclosure is schematically shown.

[0082] like Figure 4A As shown, for a target image 401 including targets of three sizes: large, medium and small, its corresponding semantic feature information 402 may include multiple semantic feature maps of three sizes: large, medium and small, and feature map information 403 may include multiple feature maps of three sizes: large, medium and small.

[0083] For example, the feature map information may include N feature maps, each of which includes d channels; the semantic feature information of the target image includes N semantic feature maps, each of which includes 3d channels. The semantic feature information may be divided into three parts according to equal channels, such as the first semantic feature sub-information 4041, the second semantic feature sub-information 4042, and the third semantic feature sub-information 4043, each of which includes N semantic feature maps of the same channel.

[0084] like Figure 4A As shown, the semantic feature information 402, the feature map information 403, the first semantic feature sub-information 4041, the second semantic feature sub-information 4042 and the third semantic feature sub-information 4043 each include three data blocks, which correspond to the feature maps / semantic feature maps of large, medium and small objects from left to right. Among them, one data block can be one feature map / semantic feature map or multiple feature maps / semantic feature maps.

[0085] The first semantic feature sub-information 4041 and the feature graph information 403 are processed using the A operator to obtain a first weight 405. The A operator includes element multiplication and normalization processing, and the processing steps using the A operator include: performing element multiplication on the first semantic feature sub-information 4041 and the feature graph information 403, and then normalizing the information after the element multiplication.

[0086] The second semantic feature sub-information 4042 and the first weight 405 are processed using the B operator to obtain the first association information 406. The B operator includes element-wise multiplication. The processing steps using the B operator include: performing element-wise multiplication on the second semantic feature sub-information 4042 and the first weight 405.

[0087] The third semantic feature sub-information 4043 and the feature graph information 403 are processed using the C operator to obtain the second association information 407. The C operator includes normalization processing and element multiplication, and the processing steps using the C operator include: performing element multiplication of the feature graph information 403 with the second weight obtained after normalization processing of the third semantic feature sub-information 4043.

[0088] In addition, after determining the first association information 406 and the second association information 407, the first association information 406 and the second association information 407 may be added using a D operator, and the added information is sent to the output layer 408 to identify multiple targets of multiple sizes.

[0089] The first semantic feature sub-information 4041 and the second semantic feature sub-information 4042 can be understood as K and V in the cross-attention mechanism, and the feature map information is Q in the cross-attention mechanism.

[0090] In a second embodiment, a first weight and a second weight are determined based on feature graph information and semantic feature information; first association information is generated based on the first weight and the semantic feature information; and second association information is generated based on the second weight and the feature graph information, including: determining a second weight based on first feature subgraph information segmented from the feature graph information and the semantic feature information; generating second association information based on the second weight and second feature subgraph information different from the first feature subgraph information; wherein the first weight is obtained after normalization processing of at least one third feature subgraph information segmented from the feature graph information, and the at least one third feature subgraph information is different from the first feature subgraph information.

[0091] Similar to segmenting semantic feature information, the feature map information can be segmented into multiple feature sub-map information in an equal-channel segmentation manner, so that each feature sub-map information can reflect the characteristics of the target image, and there are differences between the multiple feature sub-map information.

[0092] In this embodiment, determining the second weight includes: performing element-wise multiplication of the semantic feature information and the first feature subgraph information obtained by segmenting the feature graph information to obtain second feature graph information that preliminarily fuses the feature graph information and the first semantic feature subgraph information; performing normalization processing on the second feature graph information to obtain a second weight. Then, multiplying the second weight by the second feature subgraph information that is different from the first feature subgraph information to obtain second association information.

[0093] In this embodiment, the first weight of another branch may be obtained by normalizing one or more third feature subgraph information different from the first feature subgraph information using a normalization function.

[0094] In the embodiment of the present disclosure, since the characteristic subgraph information used in calculating the second weight, generating the second association information, and calculating the first weight is different, each step of the operation in determining the second weight can integrate part of the newly added information in the characteristic subgraph information; and the first association information generated by using the first weight can integrate characteristic subgraph information different from that in the second weight. Therefore, different first association information and second association information can make full use of information in the semantic dimension and the characteristic dimension to improve the accuracy of target recognition.

[0095] Figure 4B The flowchart of generating first association information and second association information according to another embodiment of the present disclosure is schematically shown. Figure 4B The target image 401, semantic feature information 402 and feature map information 403 in Figure 4A The same is true in , so I will not go into details here.

[0096] The feature graph information 403 may be divided into three parts according to equal channels, such as first feature sub-graph information 4091, second feature sub-graph information 4092 and third feature sub-graph information 4093, and each feature sub-graph information includes N feature graphs of the same channel.

[0097] The first feature subgraph information 4091 and the semantic feature information 402 are processed using the A operator to obtain the second weight 410. The A operator includes element multiplication and normalization processing, and the processing steps using the A operator include: performing element multiplication on the first feature subgraph information 4091 and the semantic feature information 402, and normalizing the information after the element multiplication.

[0098] The second characteristic subgraph information 4092 and the second weight 410 are processed using the B operator to obtain the second association information 407. The B operator includes element-wise multiplication. The processing steps using the B operator include: performing element-wise multiplication on the second characteristic subgraph information 4092 and the second weight 410.

[0099] The third feature subgraph information 4093 and the semantic feature information 402 are processed using the C operator to obtain the first association information 406. The C operator includes normalization processing and element multiplication. The processing steps using the C operator include: performing element multiplication of the semantic feature information 402 with the first weight obtained after normalization processing on the third feature subgraph information 4093.

[0100] In addition, after determining the first association information 406 and the second association information 407, the first association information 406 and the second association information 407 may be added using a D operator, and the added information is sent to the output layer 408 to identify multiple targets of multiple sizes.

[0101] The first feature subgraph information 4091 and the second feature subgraph information 4092 can be understood as K and V in the cross-attention mechanism, and the semantic feature information is Q in the cross-attention mechanism.

[0102] According to an embodiment of the present disclosure, the target recognition method further includes at least one of the following:

[0103] Update the feature graph information according to the first association information, and obtain updated first association information according to the updated feature graph information and the semantic feature information;

[0104] The semantic feature information is updated according to the second association information, and the updated second association information is obtained according to the updated semantic feature information and the feature map information.

[0105] In the embodiments of the present disclosure, only the first association information may be updated, only the second association information may be updated, or both the first association information and the second association information may be updated simultaneously.

[0106] After the first association information is generated, the first association information may be used as updated feature map information; and then, the updated feature map information is combined with the semantic feature information to generate new first association information.

[0107] In a specific embodiment, the first association information can be used as updated feature graph information, and a new first weight can be determined based on the updated feature graph information and the first semantic feature sub-information obtained by segmenting the semantic feature information; and new first association information can be generated based on the updated first weight and the second semantic feature sub-information different from the first semantic feature sub-information. In this embodiment, the first semantic feature sub-information / second semantic feature sub-information used in the process of updating the first association information can be the same as or different from the first semantic feature sub-information / second semantic feature sub-information corresponding to the process of generating the first association information.

[0108] In addition, it should be noted that the process of generating the first association information can be used as an attention gate, such as a forget gate. Therefore, the process of updating the first association information is to connect at least one forget gate in series on the basis of the original forget gate, and the input of each forget gate includes the semantic feature sub-information and the first association information output by the previous forget gate. Among them, for the first forget gate, its input is the feature map information and the semantic feature information.

[0109] Similarly, after the second association information is generated, the second association information may be used as new semantic feature information, and the new semantic feature information may be combined with the feature map information to update the second association information.

[0110] In a specific embodiment, the second association information can be used as updated semantic feature information, and a new second weight can be determined based on the updated semantic feature information and the first feature subgraph information obtained by segmenting the feature graph information; and the updated second association information can be generated based on the new second weight and the second feature subgraph information different from the first feature subgraph information. In this embodiment, the first feature subgraph information / second feature subgraph information used in the process of generating the updated second association information can be the same as or different from the first feature subgraph information / second feature subgraph information corresponding to the process of generating the second association information.

[0111] The process of generating the second association information can be regarded as an attention gate, such as a memory gate. Therefore, the process of updating the second association information can be regarded as connecting at least one memory gate in series on the basis of the original memory gate, and the input of each memory gate includes the feature subgraph information and the second association information output by the previous memory gate. Among them, for the first memory gate, its input is the semantic feature information and the feature graph information.

[0112] Figure 5 The following schematically shows a flowchart of generating first association information and second association information according to yet another embodiment of the present disclosure. Figure 5 The target image 501, semantic feature information 502 and feature map information 503, output layer 508 and Figure 4A The same is true in , so I will not go into details here.

[0113] The semantic feature information may be divided into five parts according to equal channels, such as first semantic feature sub-information 5041 , second semantic feature sub-information 5042 , fourth semantic feature sub-information 5043 , fifth semantic feature sub-information 5044 and third semantic feature sub-information 5045 .

[0114] The first association information and the new first association information are generated respectively by two forget gates connected in series, such as the forget gate 51 - 1 and the forget gate 51 - 2 , and the second association information is generated by a memory gate 52 - 1 .

[0115] For the forget gate 51-1, the first semantic feature sub-information 5041 and the feature map information 503 are processed using the A operator to obtain the first weight 5051. The second semantic feature sub-information 5042 and the first weight 5051 are processed using the B operator to obtain the first association information 5061. The first association information 5061 is input into the forget gate 51-2 as new feature map information.

[0116] For the forget gate 51-2, the fourth semantic feature sub-information 5043 and the first association information 5061 are processed by the A operator to obtain a new first weight 5052. The fifth semantic feature sub-information 5044 and the new first weight 5052 are processed by the B operator to obtain new first association information 5062. The fourth semantic feature sub-information 5043 is different from the first semantic feature sub-information 5041, and the fifth semantic feature sub-information 5044 is different from the second semantic feature sub-information 5042.

[0117] The memory gate 52 - 1 uses the C operator to process the third semantic feature sub-information 5045 and the feature graph information 503 to obtain the second association information 507 .

[0118] Figure 5 A, B, C, D operators and Figure 4A The operator operations in are the same and will not be repeated here.

[0119] In the embodiments of the present disclosure, by taking the first association information as new feature map information and generating new first association information together with the semantic feature information; and / or taking the second association information as new semantic feature information and generating new second association information together with the feature map information, the architectural depth of the first association information / second association information generation branch is increased, so that the final first association information / second association information can better learn the information of multiple targets of multiple sizes in the target image, thereby improving the recognition accuracy of multi-size targets.

[0120] According to an embodiment of the present disclosure, for operation S220, determining feature map information and semantic feature information of the target image based on the target image includes: using a feature detection head to generate feature map information based on image features of the target image, wherein the image features are obtained after feature extraction of the target image using a backbone network; and using a semantic feature detection head to generate semantic feature information based on the image features of the target image.

[0121] In this embodiment, a target image may be acquired through operation S210; after generating feature map information and semantic feature information in the above manner, multiple targets may still be identified from the target image through operations S230 and S240.

[0122] Backbone network is also called backbone network, which is used to extract image features in image processing tasks. The backbone network can be a convolutional neural network (CNN) or a residual neural network such as ResNeXt, ResNet, AlexNet, MobileNet, etc.

[0123] The feature detection head is used to extract feature map information related to the predicted target in the target recognition / target detection task. Specifically, the feature detection head can include convolutional layers, pooling layers, etc. to improve the richness of feature representation. The backbone network and feature detection head can use the existing target recognition / detection framework.

[0124] The semantic feature detection head (Head_w) is used to extract semantic features from the image features output by the backbone network. The semantic feature detection head can be a Transformer, a convolutional neural network, a semantic extraction network, etc.

[0125] The embodiments of the present disclosure add a semantic feature detection head to the existing target recognition / detection framework, which is used to extract semantic feature information from the image features of the target image, so that the target recognition / detection framework adds a semantic dimension and enriches the feature expression of the target image, which helps to better identify multi-size targets from the target image based on the feature map information and the newly added semantic feature information.

[0126] Figure 6 The flowchart of the method for determining feature map information and semantic feature information of a target image according to an embodiment of the present disclosure is schematically shown.

[0127] like Figure 6 As shown, the target image 601 is input into the backbone network 602 to obtain the output image features; the image features are then input into the semantic feature detection head 603 and the feature detection head 604 respectively, and the semantic feature information 605 and the feature map information 606 are output respectively.

[0128] like Figure 6 As shown, the semantic feature information 605 and the feature map information 606 may include multiple data blocks.

[0129] Taking the case where the target image includes targets of three sizes, large, medium and small, as an example, the semantic feature information 605 and the feature map information 606 may include three data blocks, and each data block may focus more on a feature map of a size feature.

[0130] For example, the semantic feature detection head 603 may perform convolution operations on image features using three convolution kernels of different sizes to obtain three semantic feature maps. Similarly, the feature detection head 604 may perform convolution operations on image features using three convolution kernels of different sizes to obtain three feature maps.

[0131] When a larger convolution kernel is used for convolution operation, its receptive field is larger, it tends to be more about features of larger size, and the amount of feature map data generated is smaller; conversely, when a smaller convolution kernel is used for convolution operation, its receptive field is smaller, it tends to be more about features of smaller size, and the amount of feature map data generated is larger. Therefore, Figure 6 The smallest data block on the left tends to focus on large-sized targets, the data block in the middle tends to focus on medium-sized targets, and the smallest data block on the right tends to focus on small-sized targets.

[0132] According to an embodiment of the present disclosure, for operation S240, multiple targets are identified from the target image based on the first association information and the second association information, including: generating new feature map information based on the first association information and the second association information; and identifying multiple targets from the target image based on the new feature map information through a target recognition algorithm.

[0133] The first association information and the second association information may be added together to generate richer and newer feature map information.

[0134] The target recognition algorithm can use existing target recognition algorithms to predict the types and locations of multi-size targets (Bounding boxes). For example, classification algorithms, detection box positioning algorithms, etc. can be used. The target recognition algorithm can be reflected in the form of an output layer, such as Figure 4A , Figure 4B and Figure 5 Specifically, the output layer may include a convolutional layer for predicting the types and positions of multi-size objects.

[0135] In the existing target recognition / detection framework, the target recognition algorithm usually directly uses the feature map information output by the feature detection head to identify multiple targets from the target image.

[0136] The embodiments of the present disclosure add semantic feature information, and use the semantic feature information and feature map information to generate different first attention information and second attention information. Therefore, the first attention information and the second attention information are fused into new feature map information, and then a target detection algorithm is used to identify multiple targets of multiple sizes from a target image according to the new feature map information, thereby improving the detection accuracy of targets of various sizes.

[0137] In summary, the embodiments of the present disclosure can dynamically generate optimal dynamic attention degrees (different first association information and second association information) between multiple feature maps and within feature maps for an input target image, so that the target recognition task can recognize targets of various sizes according to the optimal dynamic attention degrees adapted to each target image, thereby alleviating the problem that the proportion of multi-sized targets within the manually set feature map is inconsistent with the actual proportion of multi-sized targets in the input image, and the fixed proportion between the manually set feature maps is inconsistent with the density of targets with different information in different areas of the image, thereby improving the detection accuracy of targets of various sizes.

[0138] Figure 7 A block diagram of a target recognition device according to an embodiment of the present disclosure is schematically shown.

[0139] like Figure 7 As shown, the target recognition device 700 includes an acquisition module 710 , a feature determination module 720 , a generation module 730 and a recognition module 740 .

[0140] The acquisition module 710 is used to acquire a target image to be detected, wherein the target image includes candidate targets of multiple sizes.

[0141] The feature determination module 720 is used to determine target image feature map information and semantic feature information according to the target image.

[0142] The generating module 730 is used to generate first association information and second association information according to the feature graph information and the semantic feature information, where the first association information is different from the second association information.

[0143] The recognition module 740 is used to recognize multiple targets from the target image according to the first association information and the second association information.

[0144] According to an embodiment of the present disclosure, the generation module 730 includes a weight determination unit, a first generation unit, and a second generation unit. The weight determination unit is used to determine the first weight and the second weight according to the feature map information and the semantic feature information. The first generation unit is used to generate the first association information according to the first weight and the semantic feature information. The second generation unit is used to generate the second association information according to the second weight and the feature map information.

[0145] According to an embodiment of the present disclosure, the generation module 730 includes a first determination unit and a third generation unit. The first determination unit is used to determine a first weight based on feature graph information and first semantic feature sub-information obtained from segmentation of semantic feature information. The third generation unit is used to generate first association information based on the first weight and second semantic feature sub-information different from the first semantic feature sub-information. The second weight is obtained after normalization processing based on at least one third semantic feature sub-information obtained from segmentation of semantic feature information, and at least one third semantic feature sub-information is different from the first semantic feature sub-information.

[0146] According to an embodiment of the present disclosure, the generation module 730 includes a second determination unit and a fourth generation unit. The second determination unit is used to determine the second weight according to the first feature subgraph information obtained by segmenting the feature graph information and the semantic feature information. The fourth generation unit is used to generate the second association information according to the second weight and the second feature subgraph information different from the first feature subgraph information. The first weight is obtained after normalization processing based on at least one third feature subgraph information obtained by segmenting the feature graph information, and the at least one third feature subgraph information is different from the first feature subgraph information.

[0147] According to an embodiment of the present disclosure, the generating module 730 further includes one of the following:

[0148] The fifth generating unit is used to update the feature map information according to the first association information, and obtain the updated first association information according to the updated feature map information and the semantic feature information.

[0149] The sixth generating unit is used to update the semantic feature information according to the second association information, and obtain the updated second association information according to the updated semantic feature information and the feature map information.

[0150] According to an embodiment of the present disclosure, the feature determination module 720 includes a feature generation unit and a semantic generation unit. The feature generation unit is used to generate feature map information based on the image features of the target image using a feature detection head, wherein the image features are obtained after the target image is subjected to feature extraction using a backbone network. The semantic generation unit is used to generate semantic feature information based on the image features of the target image using a semantic feature detection head.

[0151] According to an embodiment of the present disclosure, the recognition module 720 includes: a seventh generation unit and a recognition unit. The seventh generation unit is used to generate new feature map information according to the first association information and the second association information. The recognition unit is used to recognize multiple targets from the target image according to the new feature map information through a target recognition algorithm.

[0152] The operation of the device part of the embodiment of the present disclosure is similar to that of the above-mentioned target recognition method, and will not be repeated here.

[0153] It is understandable that the acquisition module 710, the feature determination module 720, the generation module 730 and the identification module 740 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 710, the feature determination module 720, the generation module 730 and the identification module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in a proper combination of software, hardware and firmware. Alternatively, at least one of the acquisition module 710, the feature determination module 720, the generation module 730 and the identification module 740 can be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module can be executed.

[0154] Figure 8 A block diagram of an electronic device for target recognition according to an embodiment of the present disclosure is schematically shown. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0155] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0156] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0157] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The system 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0158] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0159] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0160] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0161] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0162] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiments of the present disclosure.

[0163] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0164] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0165] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0166] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0167] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A target recognition method, comprising: Acquire a target image to be detected, wherein the target image includes multiple targets of multiple sizes; Determining feature map information and semantic feature information of the target image according to the target image; generating first association information and second association information according to the feature graph information and the semantic feature information, wherein the first association information is different from the second association information; The multiple objects are identified from the object image according to the first association information and the second association information.

2. The method according to claim 1, wherein: The generating the first association information and the second association information according to the feature graph information and the semantic feature information includes: Determining a first weight and a second weight according to the feature map information and the semantic feature information; generating the first association information according to the first weight and the semantic feature information; The second association information is generated according to the second weight and the feature map information.

3. The method according to claim 2, wherein the first weight and the second weight are determined according to the feature map information and the semantic feature information; and the first association information is generated according to the first weight and the semantic feature information; Generating the second association information according to the second weight and the feature map information; comprising: Determining a first weight according to the feature graph information and first semantic feature sub-information obtained by segmenting the semantic feature information; generating the first association information according to the first weight and second semantic feature sub-information different from the first semantic feature sub-information; The second weight is obtained by normalizing at least one third semantic feature sub-information obtained by segmenting the semantic feature information, and the at least one third semantic feature sub-information is different from the first semantic feature sub-information.

4. The method according to claim 2, wherein the first weight and the second weight are determined according to the feature map information and the semantic feature information; and the first association information is generated according to the first weight and the semantic feature information; Generating the second association information according to the second weight and the feature map information; comprising: Determining a second weight according to first feature subgraph information obtained by segmenting the feature graph information and the semantic feature information; generating the second association information according to the second weight and second characteristic subgraph information different from the first characteristic subgraph information; The first weight is obtained after normalization processing is performed on at least one third feature sub-graph information obtained by segmenting the feature graph information, and the at least one third feature sub-graph information is different from the first feature sub-graph information.

5. The method according to claim 1, further comprising at least one of the following: updating the feature graph information according to the first association information, and obtaining updated first association information according to the updated feature graph information and the semantic feature information; The semantic feature information is updated according to the second association information, and the updated second association information is obtained according to the updated semantic feature information and the feature map information.

6. The method according to claim 1, wherein: The step of determining feature map information and semantic feature information of the target image according to the target image includes: Using a feature detection head, generating the feature map information according to image features of the target image, wherein the image features are obtained by extracting features of the target image using a backbone network; The semantic feature information is generated according to the image features of the target image by using a semantic feature detection head.

7. The method according to claim 1, wherein: The identifying the plurality of targets from the target image according to the first association information and the second association information comprises: generating new feature map information according to the first association information and the second association information; and The multiple targets are identified from the target image according to the new feature map information through a target recognition algorithm.

8. A target recognition device, comprising: An acquisition module, used for acquiring a target image to be detected, wherein the target image includes candidate targets of multiple sizes; A feature determination module, used to determine the target image feature map information and semantic feature information according to the target image; a generating module, configured to generate first association information and second association information according to the feature graph information and the semantic feature information, wherein the first association information is different from the second association information; and A recognition module is used to recognize the multiple targets from the target image according to the first association information and the second association information.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, The one or more processors are caused to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method and device for quickly identifying fine-grained small target

    CN112418345A

  • Target recognition model training method and device, electronic equipment and storage medium

    CN115471810A

  • Action recognition method and device, electronic equipment and readable storage medium

    CN117953581A