Method and apparatus for identifying target object in image, and device and medium

Abstract

Disclosed in the embodiments of the present disclosure are a method and apparatus for identifying a target object in an image, and a device and a medium. The method comprises: performing spatial position coding on an initial position of a target object in an image, so as to obtain a plurality of first spatial position codes; processing the plurality of first spatial position codes, a plurality of query vectors corresponding to the plurality of first spatial position codes, an image feature of the image, and a second spatial position code that matches the image feature, so as to obtain an appearance feature and a position feature of the target object on the image; on the basis of the position feature and the plurality of first spatial position codes, determining a position offset of the target object relative to the initial position; on the basis of the appearance feature, determining size information of the target object in the image; and on the basis of the initial position and the position offset of the target object, determining a final position of the target object in the image. The embodiments of the present disclosure can realize decoupling of an appearance feature of a target object in an image from a position feature thereof, thereby realizing high-precision positioning for the target object in the image.

Description

Method, device, equipment and medium for identifying target object in image

[0001] The present disclosure claims priority to the Chinese patent application No. CN202410452568.0, filed on April 15, 2024, and entitled "Method, device, equipment and medium for identifying target object in image", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to computer vision technology, and in particular, to a method, device, equipment and medium for identifying target object in image. BACKGROUND

[0003] Attention networks are increasingly widely used in target detection tasks of computer vision due to their strong global perception capabilities.

[0004] When training a target detection model using a convolutional neural network, if anchor points or anchor boxes are set according to feature maps, and then the anchor points or anchor boxes are divided into different true values according to preset rules to add different constraints, the positive and negative sample distributions need to be balanced, and a large amount of post-processing such as non-maximum suppression needs to be performed, resulting in a slow convergence speed of the target detection model.

[0005] In a detection model for target objects in an image based on an attention network, if multiple learnable encoding vectors are used as templates to obtain the target to be detected through querying the templates on a feature map, and the template prediction is matched with the true value box one by one in the training process to balance the positive and negative samples, the detection accuracy of the target objects in the image will be reduced in the case that the template coverage scene is limited. SUMMARY

[0006] Embodiments of the present disclosure provide a method, device, equipment and medium for identifying target objects in an image.

[0007] According to some aspects of embodiments of the present disclosure, a method for identifying target objects in an image is provided, comprising:

[0008] performing spatial position encoding based on initial positions of the target objects in the image to obtain a plurality of first spatial position encodings;

[0009] processing the plurality of first spatial position encodings, a plurality of query vectors corresponding to the plurality of first spatial position encodings, and image features of the image and second spatial position encodings matched with the image features, to obtain apparent features and position features of the target objects on the image;

[0010] determine a position offset of the target object relative to the initial position based on the position feature and the plurality of first spatial position encodings;

[0011] determine size information of the target object in the image based on the appearance feature;

[0012] determine a final position of the target object in the image based on the initial position of the target object and the position offset.

[0013] According to still another aspect of the embodiments of the present disclosure, there is provided a device for identifying a target object in an image, comprising:

[0014] a spatial position encoding module configured to perform spatial position encoding based on an initial position of a target object in an image to obtain a plurality of first spatial position encodings;

[0015] a first processing module configured to process the plurality of first spatial position encodings, a plurality of query vectors corresponding to the plurality of first spatial position encodings, and an image feature of the image and a second spatial position encoding matched with the image feature, to obtain an appearance feature and a position feature of the target object on the image, wherein the query vector is a vector used for query when detecting the target object in the image;

[0016] a position offset determination module configured to determine a position offset of the target object relative to the initial position based on the position feature and the plurality of first spatial position encodings;

[0017] a size information determination module configured to determine size information of the target object in the image based on the appearance feature;

[0018] a second processing module configured to determine a final position of the target object in the image based on the initial position of the target object and the position offset.

[0019] According to still another aspect of the embodiments of the present disclosure, there is provided a computer readable storage medium storing a computer program, the computer program being configured to execute the above-mentioned method for identifying a target object in an image.

[0020] According to still another aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising:

[0021] a processor;

[0022] a memory configured to store executable instructions of the processor;

[0023] the processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-mentioned method for identifying a target object in an image.

[0024] The method, device, equipment and medium for identifying the target object in the image provided by the above embodiments of the present disclosure can encode the initial position of the target object in the image to obtain a plurality of first spatial position encodings, and then can process the plurality of first spatial position encodings, a plurality of query vectors corresponding to the plurality of first spatial position encodings, and image features of the image and second spatial position encodings matched with the image features according to a preset attention mechanism to obtain the apparent feature and the position feature of the target object on the image, thereby realizing the decoupling of the apparent feature and the position feature. Since the apparent feature of the target object on the image can represent the feature of the appearance size of the target object on the image, the size information of the target object in the image can be accurately determined according to the apparent feature of the target object on the image. Since the position feature of the target object on the image can represent the feature of the final spatial position of the target object on the image, and the plurality of first spatial position encodings are obtained according to the initial position encoding of the target object on the image, the position offset relative to the initial position of the target object can be obtained according to the position feature of the target object on the image and the plurality of spatial position encodings, and the final position of the target object can be obtained in combination with the initial position of the target object, thereby realizing high-precision positioning of the target object in the image.

[0025] The technical solutions of the present disclosure will be described in further detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0026] FIG. 1 is a system block diagram to which some embodiments of the present disclosure are applicable;

[0027] FIG. 2 is a flowchart of a method for identifying a target object in an image in some embodiments of the present disclosure;

[0028] FIG. 3 is a flowchart of step S2 in some embodiments of the present disclosure;

[0029] FIG. 4 is a working principle diagram of a method for identifying a target object in an image in some examples of the present disclosure;

[0030] FIG. 5 is a flowchart of step S2-3 in some embodiments of the present disclosure;

[0031] FIG. 6 is a working principle diagram of a method for identifying a target object in an image in some examples of the present disclosure;

[0032] FIG. 7 is a flowchart of step S2-4 in some embodiments of the present disclosure;

[0033] FIG. 8 is a flowchart of step S2-2 in some embodiments of the present disclosure;

[0034] FIG. 9 is a structural block diagram of a device for identifying a target object in an image in some embodiments of the present disclosure;

[0035] FIG. 10 is a structural block diagram of the first processing module 200 in some embodiments of the present disclosure;

[0036] FIG. 11 is a structural block diagram of the first processing module 200 in some other embodiments of the present disclosure;

[0037] FIG. 12 is a structural block diagram of an apparatus for identifying a target object in an image in some other embodiments of the present disclosure;

[0038] FIG. 13 is a structural block diagram of an electronic device in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0039] For the purpose of explaining the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, obviously, the described embodiments are only part of the embodiments of the present disclosure, but not all the embodiments, it should be understood that the present disclosure is not limited by the example embodiments.

[0040] It should be noted that: unless otherwise specified, the relative arrangement, numerical expression and numerical value of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0041] SUMMARY

[0042] In the process of implementing the present disclosure, the inventors found that at least the following problems exist when using an attention network-based detection model to detect a target object in an image: the attention network-based detection model usually includes an encoder and a decoder, the encoder uses a self-attention network to deeply encode image features, and the input of the decoder is a plurality of pre-specified learnable feature encodings. However, the attention mechanism usually does not have permutation invariance, in order to make the attention network in the detection model have permutation invariance, the spatial position encoding and the appearance feature of the target object in the image are usually superimposed, resulting in coupling of the appearance feature and the position feature of the target object in the image, and using the superimposed features will greatly increase the difficulty of the bias regression task of the target object relative to the query position point, and further result in low positioning accuracy of the target object in the image.

[0043] Example system

[0044] FIG. 1 is a system block diagram to which some embodiments of the present disclosure are applicable. As shown in FIG. 1, the system to which the embodiments of the present disclosure are applicable can include an image acquisition apparatus 1 and an electronic device 2.

[0045] Among them, the image acquisition apparatus 1 can include a camera for acquiring an image including a target object. The image acquired by the camera can include a color image, or can include a black and white image. The target object can include pedestrians, vehicles, cyclists and other objects for identification or behavior analysis.

[0046] The electronic device 2 can include a stationary electronic device such as a desktop computer and a server, and can also include a mobile electronic device disposed on a vehicle or other type of movable carrier.

[0047] The electronic device 2 can receive an image collected by the image collection device 1. The electronic device 2 can identify a target object in the image using a pre-trained first detection model, and preliminarily locate the target object in the image to obtain an initial position of the target object in the image. The first detection model can be a neural network model, such as a convolutional neural network model, a recurrent neural network model, etc., which is obtained by model training based on sample images with target objects.

[0048] The electronic device 2 can encode the initial position of the target object in the image according to a preset spatial position encoding rule by using a second detection model to obtain a plurality of first spatial position encodings, and generate a plurality of query vectors required by an attention network according to the plurality of first spatial position encodings. The second detection model is a detection model used by the embodiments of the present disclosure to identify the target object in the image, such as an attention network-based detection model. The electronic device 2 can extract image features of the image by using a feature extraction network independent of the attention network, and encode the spatial position according to the image features of the image to obtain second spatial position encodings matched with the image features. The electronic device 2 inputs the plurality of first spatial position encodings and the plurality of query vectors, as well as the image features of the image and the second spatial position, into the attention network for processing by the attention network, and outputs the apparent features and the position features of the target object on the image, which can realize decoupling of the apparent features and the position features. The electronic device 2 can accurately determine the size information of the target object in the image according to the apparent features of the target object on the image. The second detection model can obtain a position offset relative to the initial position of the target object according to the position features of the target object on the image and the plurality of spatial position encodings, and can obtain the final position of the target object in combination with the initial position of the target object, thereby realizing high-precision positioning of the target object in the image.

[0049] It should be noted that when the electronic device 2 is disposed on a vehicle or other type of movable carrier, the movable carrier can realize real-time high-precision positioning of a nearby target object, which helps to improve the collision prediction accuracy and trajectory planning rationality of the movable carrier in an automatic driving mode.

[0050] Exemplary method

[0051] FIG. 2 is a flowchart of a method for identifying a target object in an image according to some embodiments of the present disclosure. As shown in FIG. 2, the method includes the following steps:

[0052] S1: encode the initial position of the target object in the image to obtain a plurality of first spatial position encodings.

[0053] In some optional embodiments, after the image acquisition device acquires the image, the first processor in the electronic device can preliminarily locate the target object in the image by using the pre-trained first detection model to obtain the initial position of the target object in the image.

[0054] It should be noted that, in the process of the first detection model extracting the image features of the sample image, the appearance features and the position features of the target object in the sample image are coupled. The appearance features of the target object refer to the observed features of the target object. For example, when the target object includes a face, the appearance features of the face can include the positions of the key feature points of the face, the distance information of the key positions, the area ratios of the organs of the face, and the texture features of the face, etc. The position features of the target object refer to the features corresponding to the position information (e.g., spatial position encoding) of the target object.

[0055] In some optional embodiments, after obtaining the initial position of the target object in the image, in order to ensure the arrangement variability of the attention mechanism, the initial position of the target object is encoded according to a preset encoding rule to obtain a plurality of spatial position encodings, which are defined as a plurality of first spatial position encodings. In addition, a plurality of query vectors can be set for the attention mechanism. The query vector is a vector used for querying when detecting the target object in the image.

[0056] In an optional example, this step S1 can be performed by the processor invoking the corresponding instructions stored in the memory, or by the spatial position encoding module running on the processor.

[0057] S2: processing the plurality of first spatial position encodings, the plurality of query vectors corresponding to the plurality of first spatial position encodings, and the image features of the image and the second spatial position encoding matched with the image features, to obtain the appearance features and the position features of the target object on the image.

[0058] In some optional embodiments, the second processor in the electronic device can execute the computer instructions corresponding to the second detection model, which can use the attention mechanism to process the plurality of first spatial position encodings, the plurality of query vectors corresponding to the plurality of first spatial position encodings, and the image features of the image and the second spatial position encoding based on the attention mechanism, to obtain the appearance features and the position features of the target object on the image. Since the image features of the image can be obtained by using a feature extraction network independent of the attention mechanism, the electronic device can generate decoupled appearance features and position features when performing the processing based on the attention mechanism by using the second detection model.

[0059] In an optional example, the step S2 can be performed by the processor invoking corresponding instructions stored in the memory, or by a first processing module run by the processor.

[0060] S3: determining a position offset of the target object relative to the initial position based on the position feature and the plurality of first spatial position encodings.

[0061] In some optional embodiments, a preset position offset determination formula can be used to calculate the position offset of the target object relative to the initial position, with the position feature and the first spatial position encodings as input parameters.

[0062] In some optional examples of the present disclosure, the position offset of the target object relative to the initial position can be calculated by the following formula: Dxy=W d (P′-P)+b

[0063] wherein Dxy represents the position offset of the target object relative to the initial position, W d represents a learnable first weight matrix, P′ represents the position feature, P represents the first spatial position encoding, and b represents a learnable first bias.

[0064] In some other optional examples of the present disclosure, the position offset of the target object relative to the initial position can be calculated by the following formula: Dxy=W e (P′-P)

[0065] wherein Dxy represents the position offset of the target object relative to the initial position, W e () represents a learnable function for determining the position offset and related to the vector difference between the position feature and the first spatial position encoding.

[0066] In an optional example, the step S3 can be performed by the processor invoking corresponding instructions stored in the memory, or by a position offset determination module run by the processor.

[0067] S4: determining size information of the target object in the image based on the appearance feature.

[0068] In some optional embodiments, a preset size regression formula can be used to calculate the size information of the target object in the image, with the appearance feature as an input parameter.

[0069] In some optional examples of the present disclosure, the size information of the target object in the image can be calculated by the following formula: WH=W s V′+b s

[0070] wherein WH represents the size of the target object in the image, W srepresents a second learnable weight matrix, V' represents an apparent feature, and b represents a second learnable bias. s represents a second learnable bias.

[0071] In some optional embodiments, the size information of the target object in the image can be calculated by the following formula: WH = W o (V')

[0072] wherein, WH represents the size of the target object in the image, W o () represents a learnable function for determining the size information and related to the apparent feature, and V' represents the apparent feature.

[0073] In one optional embodiment, the step S4 can be performed by a processor invoking corresponding instructions stored in a memory or by a size information determination module running on the processor.

[0074] S5: determining a final position of the target object in the image based on the initial position of the target object and the position offset.

[0075] In some optional embodiments, the final position of the target object in the image can be obtained by adding the initial position of the target object to the position offset.

[0076] It should be noted that the electronic device in the embodiments of the present disclosure can include one processor or multiple processors, and the electronic device can determine the final position of the target object in the image and obtain the size information of the target object in the image through the one processor or the multiple processors.

[0077] In some optional embodiments, in the case where the electronic device includes one processor, the processor can preliminarily locate the initial position of the target object in the image through the first detection model to obtain the initial position of the target object in the image. The processor can also determine the size information of the target object in the image and the position offset of the target object relative to the initial position through the second detection model. The processor can further determine the final position of the target object in the image according to the initial position of the target object in the image and the position offset of the target object relative to the initial position.

[0078] In some optional embodiments, in the case where the electronic device includes multiple processors, for example, in the case where the electronic device includes a first processor, a second processor and a third processor, the first processor can preliminarily locate the initial position of the target object in the image through the first detection model to obtain the initial position of the target object in the image. The second processor can determine the size information of the target object in the image and the position offset of the target object relative to the initial position through the second detection model. The third processor can determine the final position of the target object in the image according to the initial position of the target object in the image and the position offset of the target object relative to the initial position.

[0079] In some optional embodiments, the efficiency of real-time or multiple accurate positioning of the target object in the image is higher when the electronic device comprises multiple processors, which is suitable for scenarios of real-time or multiple detection of the target object around the vehicle or other types of movable carriers in an automatic driving mode.

[0080] In the embodiment, after the initial position of the target object in the image is spatially encoded to obtain a plurality of first spatial position encodings, the plurality of first spatial position encodings, the plurality of query vectors corresponding to the plurality of first spatial position encodings, and the image features of the image and the second spatial position encodings matched with the image features can be processed according to the preset attention mechanism to obtain the apparent feature and the position feature of the target object on the image, and the decoupling of the apparent feature and the position feature is realized. Since the apparent feature of the target object on the image can represent the feature of the appearance size of the target object on the image, the size information of the target object in the image can be accurately determined according to the apparent feature. Since the position feature of the target object on the image can represent the feature of the final spatial position of the target object on the image, and the plurality of first spatial position encodings are obtained according to the initial position encoding of the target object on the image, the position offset relative to the initial position of the target object can be obtained according to the position feature of the target object on the image and the plurality of spatial position encodings, and the final position of the target object can be obtained in combination with the initial position of the target object, thereby realizing high-precision positioning of the target object in the image.

[0081] In some optional embodiments, the identification result obtained by identifying the target object in the image can include the size information of the target object and the final position of the target object in the image.

[0082] In an optional example, the step S5 can be performed by the processor invoking the corresponding instructions stored in the memory, or by the second processing module running on the processor.

[0083] In some optional embodiments of the present disclosure, the first detection model can be trained in the following manner:

[0084] A plurality of sample images comprising different types of target objects are obtained. Each sample image has a position label of target object position information.

[0085] The plurality of sample images are divided into a training set and a test set. The sample quantity ratio of the training set to the test set can be 9:1, 8:2, and the like.

[0086] The sample images in the training set are respectively input into the initial neural network model for model training, and the sample images in the test set are used to test the position prediction accuracy of the neural network model for the target objects in the sample images. In the training process of the neural network model, the image features of the sample images are extracted by using the neural network model, the image features of the sample images are processed, the predicted positions of the target objects in the sample images are output, and the parameters of the neural network model are updated by using the backward propagation method according to the difference between the predicted positions of the target objects in the sample images and the accurate positions provided in the position labels of the sample images. When the position prediction accuracy of the neural network model for the target objects in the sample images reaches a preset accuracy threshold, or the iteration number of the neural network model reaches a preset number, the training of the neural network model is ended, and the first detection model is obtained.

[0087] In this embodiment, according to a plurality of sample images including different types of target objects, a first detection model for initial positioning of target objects in images can be trained by machine learning, and the method for identifying target objects in images according to the embodiments of the present disclosure can obtain the final positions of the target objects in the images, which is helpful for accurate positioning of the target objects in the images.

[0088] FIG. 3 is a flowchart of step S2 in some embodiments of the present disclosure. As shown in FIG. 3, step S2 can include:

[0089] S2-1: using the appearance feature encoding layer in the attention network, encoding a plurality of query vectors by using learnable encoding parameters to obtain a query item, a key-value item, and a value item.

[0090] In some optional embodiments, a network layer for encoding the appearance features can be arranged in the attention network, and the network layer can be defined as an appearance feature encoding layer. The appearance feature encoding layer can encode a plurality of query vectors by using learnable encoding parameters to obtain a query item Q, a key-value item K, and a value item V required by the self-attention network. The learnable encoding parameters can be parameters for encoding the plurality of query vectors in the attention network, and the parameters can be learned in the processing process of the attention network.

[0091] In an optional example, this step S2-1 can be executed by a processor invoking corresponding instructions stored in a memory, or can be executed by a query vector encoding unit running on the processor.

[0092] S2-2: using the multi-head self-attention network layer in the attention network, processing based on the query item, the key-value item, the value item, and a plurality of first spatial position encodings to obtain first fusion features.

[0093] In some optional embodiments, a network layer based on a self-attention mechanism can be arranged in the attention network, and the network layer can be defined as a multi-head self-attention network layer. The multi-head self-attention network layer takes the query item Q, the key value item K, the value item V, and the first spatial position encoding P as multi-head input, performs feature fusion processing on the query item Q, the key value item K, the value item V, and the first spatial position encoding P of the multi-head input, and outputs the first fused feature.

[0094] In an optional example, the step S2-2 can be performed by a processor invoking corresponding instructions stored in a memory, or can be performed by a multi-head self-attention processing unit running on the processor.

[0095] S2-3: Based on the first fused feature, sampling the image feature of the image and the second spatial position encoding to obtain a plurality of apparent feature sampling values and a plurality of position feature sampling values of the target object on the image.

[0096] In some optional embodiments, a feature sampling module can be arranged in the second detection model, and the feature sampling module can perform feature sampling on the image feature of the image and the second spatial position encoding based on the first fused feature according to a preset feature sampling rule, to correspondingly obtain the plurality of apparent feature sampling values and the plurality of position feature sampling values of the target object on the image.

[0097] In an optional example, the step S2-3 can be performed by a processor invoking corresponding instructions stored in a memory, or can be performed by a feature sampling unit running on the processor.

[0098] S2-4: Using a multi-head cross-attention network layer in the attention network, processing the plurality of apparent feature sampling values, the plurality of position feature sampling values, the first fused feature, and the plurality of first spatial position encodings to obtain the apparent feature and the position feature.

[0099] In some optional embodiments, a network layer that processes input partially derived from within the attention network and another part of the input derived from outside the attention network can be arranged in the attention network, and the network layer can be defined as a multi-head cross-attention network layer. The multi-head cross-attention network layer takes the first fused feature and the plurality of first spatial position encodings derived from within the attention network as part of the input, and takes the plurality of apparent feature sampling values and the plurality of position feature sampling values derived from outside the attention network as another part of the input, processes the two parts of the input as multi-head input, and can obtain the apparent feature and the position feature of the target object on the image.

[0100] It should be noted that the multi-head self-attention network layer in the embodiments of the present disclosure is a network layer in which input information is derived from inside the attention network and the input information is processed. The multi-head cross-attention network layer in the embodiments of the present disclosure is a network layer in which part of the input information (for example, multiple appearance feature sampling values and multiple position feature samplings) is derived from inside the attention network, another part of the input information (for example, appearance feature sampling values and position feature sampling values) is derived from outside the attention network, and the input information is processed.

[0101] In an optional example, the step S2-4 can be performed by a processor invoking corresponding instructions stored in a memory, or by a multi-head cross-attention processing unit running on the processor.

[0102] FIG. 4 is a working principle diagram of a method for identifying a target object in an image in some examples of the present disclosure. As shown in FIG. 4, the appearance feature encoding layer in the attention network is used to encode multiple query vectors from the channels of the image to obtain a query item Q, a key value item K, and a value item V. The query item Q, the key value item K, the value item V, and a first spatial position encoding P are input into a multi-head self-attention network layer, and each input is divided into h groups, referred to as h vector heads, by the multi-head self-attention network layer, where h is an integer greater than 0. The multi-head self-attention network layer performs the same preset operation on the h vector heads to output a first fused feature Q'. According to the first fused feature Q', the image features of the image are sampled according to a preset sampling rule to obtain multiple appearance feature sampling values V k , and multiple position feature sampling values P k corresponding to each appearance feature sampling value V k are obtained from the second spatial position encoding. In FIG. 4, the feature sampling module is used to sample the image features of the image and the second spatial position encoding according to the first fused feature Q' and a preset sampling rule to obtain multiple appearance feature sampling values V k , and multiple position feature sampling values P k corresponding to each appearance feature sampling value V k are obtained from the second spatial position encoding. The multiple appearance feature sampling values V k , the multiple position feature sampling values P k , the first fused feature Q', and the first spatial position encoding P are input into a multi-head cross-attention network layer, and the multi-head cross-attention network layer processes each appearance feature sampling value V k , the position feature sampling value P k corresponding to each appearance feature sampling value V k , the first fused feature Q', and the first spatial position encoding P to output an appearance feature V' and a position feature P' of the target object in the image.

[0103] In the embodiment, the attention network is used to process the plurality of first spatial position encodings, the plurality of query vectors corresponding to the plurality of first spatial position encodings, and the image feature of the image and the second spatial position encoding matched with the image feature, so that the apparent feature and the position feature of the target object on the image can be quickly and accurately obtained, and the size regression and the position regression can be performed according to the apparent feature and the position feature, so that the size information and the position offset of the target object in the image can be accurately obtained, and the high-precision positioning of the target object in the image can be further achieved.

[0104] FIG. 5 is a flowchart of step S2-3 in some embodiments of the present disclosure. As shown in FIG. 5, step S2-3 can include:

[0105] S2-3-1: using a sampling bias network layer of the attention network, processing the first fusion feature based on the plurality of query positions of the plurality of query vectors to obtain a plurality of sampling biases corresponding to the plurality of query positions.

[0106] In some optional embodiments, a network layer for determining the sampling bias can be arranged in the attention network, and the network layer can be defined as a sampling bias network layer. After the sampling bias network layer obtains the plurality of query positions of the plurality of query vectors, the first fusion feature is processed according to the plurality of query positions of the plurality of query vectors, and a plurality of sampling biases corresponding to the plurality of query positions can be obtained.

[0107] FIG. 6 is a working principle diagram of a method for identifying a target object in an image in another example of the present disclosure. As shown in FIG. 6, the sampling bias network layer generates an N×L×m×2 tensor according to the first fusion feature Q', and the tensor represents m sampling biases of N query vectors on L image features. Wherein, N represents the number of query vectors, which is the same as the number of the plurality of query vectors in step S2, L represents the number of layers of the feature pyramid graph in the case that the image feature of the image includes the feature pyramid graph, m represents the number of sampling biases, and N, L and m are all integers greater than 0. The sampling bias here is the sampling bias of the sampling point relative to the position of the query vector itself.

[0108] S2-3-2: based on the plurality of sampling biases and the plurality of query positions, sampling the image feature of the image to obtain a plurality of apparent feature sampling values, and obtaining a plurality of position feature sampling values corresponding to positions of the plurality of apparent feature sampling values.

[0109] Wherein, the plurality of position feature sampling values corresponding to the positions of the plurality of apparent feature sampling values can be obtained from the second spatial position encoding according to the matching relationship between the image feature and the second spatial position encoding.

[0110] In some optional embodiments, when sampling the appearance features, the features of m positions in the L-layer feature pyramid graph can be respectively sampled according to the sampling bias of the sampling bias network layer output and the plurality of query positions corresponding to the plurality of query vectors, that is, a plurality of appearance feature sampling values V k , and the position feature sampling values P k corresponding to each appearance feature sampling value V k are obtained.

[0111] In an optional example, the steps S2-3-1 and S2-3-2 can be executed by a processor invoking corresponding instructions stored in a memory, or by a feature sampling unit running on the processor.

[0112] In the embodiment, the sampling bias network layer of the attention network is used to process the first fusion features according to the plurality of query positions corresponding to the plurality of query vectors, so that a plurality of sampling biases corresponding to the plurality of query positions can be obtained, and then the image features and the first spatial position encoding of the image are sampled according to the plurality of sampling biases and the plurality of query positions, so that the sampling of the appearance features and the position features is realized, which is helpful for decoupling the appearance features and the position features, and further helps to improve the positioning accuracy of the target object in the image.

[0113] FIG. 7 is a flowchart of step S2-4 in some embodiments of the present disclosure. As shown in FIG. 7, step S2-4 can include:

[0114] S2-4-1: using a sampling feature weight network layer in the attention network, learning-based encoding of the first fusion features based on the plurality of query vectors is performed to obtain the sampling feature weight.

[0115] In some optional embodiments, a network layer for determining the sampling feature weight can be provided in the attention network, which can be defined as a sampling feature weight network layer. The sampling feature weight network layer in the attention network is used to learning-based encode the first fusion features Q' according to the plurality of query vectors to generate an N x L x m tensor, which represents the sampling feature weight W of the N query vectors on the L x m sampling points. Wherein, the learning-based encoding is to encode the first fusion features Q' in the attention network processing process, and the encoding manner is learning-based.

[0116] In an optional example, the step S2-4-1 can be executed by a processor invoking corresponding instructions stored in a memory, or by a sampling feature weight determination unit running on the processor.

[0117] S2-4-2: The multi-head cross-attention network layer in the attention network is used to perform multi-head cross-attention processing on the plurality of appearance feature sampling values, the plurality of position feature sampling values, the first fusion feature, the sampling feature weight, and the plurality of first spatial position encodings, to obtain a second fusion feature.

[0118] In some optional embodiments, the plurality of appearance feature sampling values V k , the plurality of position feature sampling values P k , the first fusion feature Q', the sampling feature weight W, and the plurality of first spatial position encodings P are input into the multi-head cross-attention network layer, and the multi-head cross-attention network layer is used to process the input, to output a second fusion feature Q ″″ .

[0119] In an optional example, the step S2-4-2 can be executed by a processor invoking corresponding instructions stored in a memory, or can be executed by a multi-head self-attention processing unit running on the processor.

[0120] S2-4-3: Based on the second fusion feature and the plurality of appearance feature sampling values, an appearance feature is determined.

[0121] In some optional embodiments, the plurality of appearance feature sampling values V k may be cross-layer linked and normalized according to the second fusion feature Q" to obtain an appearance feature V'.

[0122] In some optional embodiments, the appearance feature V' can be obtained by the following formula:

[0123] Wherein, softmax() represents normalizing each row of the matrix.

[0124] It should be noted that the formula for determining the appearance feature herein is only an example, and in actual applications, other applicable formulas can be selected to determine the appearance feature.

[0125] In an optional example, the step S2-4-3 can be executed by a processor invoking corresponding instructions stored in a memory, or can be executed by an appearance feature determination unit running on the processor.

[0126] S2-4-4: Based on the second fusion feature and the plurality of position feature sampling values, a position feature is determined.

[0127] In some optional embodiments, the plurality of position feature sampling values P k may be cross-layer linked and normalized according to the second fusion feature Q" to obtain a position feature P'.

[0128] In some optional embodiments, the position feature P' can be obtained by the following formula:

[0129] It should be noted that the formula for determining the position feature here is only an example, and other applicable formulas can be selected to determine the position feature in actual applications.

[0130] In an optional example, the step S2-4-4 can be executed by a processor invoking corresponding instructions stored in a memory, or by a position feature determination unit running on the processor.

[0131] In the embodiment, the first fusion feature is processed by the sampling feature weight network layer of the attention network corresponding to the plurality of query vectors and the plurality of sampling points, so that the sampling feature weight can be obtained, the second fusion feature is obtained by the multi-head cross-attention network layer of the attention network performing multi-head cross-attention processing on the plurality of appearance feature sampling values, the plurality of position feature sampling values, the first fusion feature and the plurality of first spatial position encodings, and then the decoupled appearance feature and position feature can be accurately obtained according to the second fusion feature, the plurality of appearance feature sampling values and the plurality of position feature sampling values, which is helpful to improve the positioning accuracy of the target object in the image.

[0132] FIG. 8 is a flowchart of the step S2-2 in some embodiments of the present disclosure. As shown in FIG. 8, the step S2-2 can include:

[0133] S2-2-1: processing the query item, the key value item, the value item and the plurality of first spatial position encodings by the multi-head self-attention network layer to obtain a third fusion feature.

[0134] In some optional examples of the present disclosure, the third fusion feature can be obtained by the multi-head self-attention network layer of the attention network through the following formula:

[0135] Wherein, Q”’ represents the third fusion feature, Q, K, V and P represent the query item, the key value item, the value item and the first spatial position encoding respectively, and C represents the channel number of the feature V.

[0136] It should be noted that the formula for determining the third fusion feature here is only an example, and other applicable formulas can be selected to determine the third fusion feature in actual applications.

[0137] S2-2-2: performing element sum operation and normalization processing on the third fusion feature and the value item to obtain the first fusion feature.

[0138] In an optional example, the steps S2-2-1 and S2-2-2 can be executed by a processor invoking corresponding instructions stored in a memory, or by a multi-head self-attention processing unit running on the processor.

[0139] In the embodiment, the multi-head self-attention network layer of the attention network processes the query item, the key-value item, the value item and the plurality of first spatial position encodings to obtain the third fusion feature, and then performs element sum operation and normalization processing on the third fusion feature and the value item to quickly and accurately obtain the first fusion feature, which helps to quickly decouple the appearance feature and the position feature, and further helps to improve the positioning accuracy of the target object in the image.

[0140] In some embodiments of the present disclosure, step S2-2-1 can include: splicing the first spatial position encoding with the query item and the key-value item respectively to obtain a query splicing item and a key-value splicing item; performing similarity calculation on the query splicing item and the key-value splicing item, and weighting the value item by taking the similarity calculation result as the weight, and determining the third fusion feature based on the weighting result.

[0141] In some optional embodiments, the vector of the first spatial position encoding P can be spliced with the vector of the query item Q to obtain a query splicing item [P, Q], and the vector of the first spatial position encoding P can be spliced with the vector of the key-value item K to obtain a key-value splicing item [P, K], and then similarity calculation and weighting processing can be performed in sequence by using a preset similarity calculation formula and a preset weighting formula to finally obtain the third fusion feature.

[0142] In the embodiment, the multi-head self-attention network layer of the attention network can be used to splice the first spatial position encoding representing the position feature of the target object in the image with the query item and the value item representing the appearance feature of the target object in the image to obtain a query splicing item and a key-value splicing item, and then similarity calculation and weighting processing can be performed on the query splicing item and the key-value splicing item to obtain the third fusion feature, which realizes separate processing of the appearance feature and the position feature, helps to decouple the appearance feature and the position feature, and further helps to improve the positioning accuracy of the target object in the image.

[0143] In some embodiments of the present disclosure, before step S2-3, further comprising: determining a feature pyramid graph of a preset number of layers corresponding to the image based on a preset image feature extraction rule; determining spatial position encodings matched with image features of each layer in the feature pyramid graph.

[0144] Using the feature extraction network, the image is subjected to image feature extraction and down-sampling based on a preset image feature extraction rule, and a feature pyramid graph of a preset number of layers can be obtained. For the image features of each layer in the feature pyramid graph of the preset number of layers, the matched spatial position encodings are obtained. Wherein, all the spatial position encodings extracted from the feature pyramid graph are the second spatial position encodings.

[0145] In an optional example, the step of determining the feature pyramid map can be executed by a processor invoking corresponding instructions stored in a memory, or by a feature pyramid map obtaining module run by the processor.

[0146] In an optional example, the step of determining the spatial position encoding matched with the image features in each layer of the feature pyramid map can be executed by a processor invoking corresponding instructions stored in a memory, or by a spatial position encoding module run by the processor.

[0147] In some examples of the present disclosure, the preset number of layers is 3, and the process of obtaining the feature pyramid map and the spatial position encoding matched with the image features in each layer of the feature pyramid map can include:

[0148] Feature extraction is performed on the image to obtain a first image feature with a dimension of HxWxC, and a spatial position encoding X matched with the first image feature and with a dimension of HxWxC is obtained. H and W respectively represent the number of pixels corresponding to the width and height of the first image feature, and C represents the number of channels of the image.

[0149] Down-sampling is performed on the first image feature to obtain a second image feature with a dimension of (H / 2)x(W / 2)x(C / 2), and a spatial position encoding Y matched with the second image feature and with a dimension of (H / 2)x(W / 2)x(C / 2) is obtained.

[0150] Down-sampling is performed on the second image feature to obtain a third image feature with a dimension of (H / 4)x(W / 4)x(C / 4), and a spatial position encoding Z matched with the third image feature and with a dimension of (H / 4)x(W / 4)x(C / 4) is obtained.

[0151] In the present example, the feature pyramid map of the image includes the first image feature with a dimension of HxWxC, the second image feature with a dimension of (H / 2)x(W / 2)x(C / 2), and the third image feature with a dimension of (H / 4)x(W / 4)x(C / 4). The second spatial position encoding includes the spatial position encoding X with a dimension of HxWxC, the spatial position encoding Y with a dimension of (H / 2)x(W / 2)x(C / 2), and the spatial position encoding Z with a dimension of (H / 4)x(W / 4)x(C / 4).

[0152] In the present embodiment, according to the preset image feature extraction rule, the feature pyramid map corresponding to the preset number of layers of the image can be determined, and the spatial position encoding matched with the image features in each layer of the feature pyramid map can be obtained, which is helpful to quickly obtain the apparent feature sampling value and the position feature sampling value, and further helpful to improve the efficiency of identifying the target object in the image.

[0153] Any one of the methods for identifying a target object in an image provided by the embodiments of the present disclosure can be executed by any appropriate electronic device with data processing capability, including but not limited to: terminal devices and servers, etc. Alternatively, any one of the methods for identifying a target object in an image provided by the embodiments of the present disclosure can be executed by a processor, such as a processor executing any one of the methods for identifying a target object in an image mentioned by the embodiments of the present disclosure by calling corresponding instructions stored in a memory. Details are omitted below.

[0154] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and various storage medium that can store program codes.

[0155] Exemplary apparatus

[0156] FIG. 9 is a structural block diagram of an apparatus for identifying a target object in an image in some embodiments of the present disclosure. As shown in FIG. 9, the apparatus includes:

[0157] A spatial position encoding module 100 is configured to encode a spatial position of an initial position of a target object in an image to obtain a plurality of first spatial position encodings;

[0158] A first processing module 200 is configured to process the plurality of first spatial position encodings, a plurality of query vectors corresponding to the plurality of first spatial position encodings, and a second spatial position encoding matched with an image feature of the image, to obtain an apparent feature and a position feature of the target object on the image, wherein the query vector is a vector used for query when detecting the target object in the image;

[0159] A position offset determination module 300 is configured to determine a position offset of the target object relative to the initial position based on the position feature and the plurality of first spatial position encodings;

[0160] A size information determination module 400 is configured to determine size information of the target object in the image based on the apparent feature;

[0161] A second processing module 500 is configured to determine a final position of the target object in the image based on the initial position of the target object and the position offset.

[0162] FIG. 10 is a structural block diagram of the first processing module 200 in some embodiments of the present disclosure. As shown in FIG. 10, the first processing module 200 includes:

[0163] The query vector encoding unit 210 is configured to encode the plurality of query vectors by using an appearance feature encoding layer in the attention network, to obtain query items, key value items and value items.

[0164] The multi-head self-attention processing unit 220 is configured to process the query items, the key value items, the value items and the plurality of first spatial position encodings by using a multi-head self-attention network layer in the attention network, to obtain first fusion features.

[0165] The feature sampling unit 230 is configured to sample image features of the image and the second spatial position encodings based on the first fusion features, to obtain a plurality of appearance feature sampling values and a plurality of position feature sampling values of the target object on the image.

[0166] The multi-head cross-attention processing unit 240 is configured to process the plurality of appearance feature sampling values, the plurality of position feature sampling values, the first fusion features and the plurality of first spatial position encodings by using a multi-head cross-attention network layer in the attention network, to obtain the appearance features and the position features.

[0167] In some embodiments of the present disclosure, the feature sampling unit 230 is configured to process the first fusion features based on a plurality of query positions of the plurality of query vectors by using a sampling bias network layer of the attention network, to obtain a plurality of sampling biases corresponding to the plurality of query positions; and the feature sampling unit 230 is further configured to sample the image features of the image based on the plurality of sampling biases and the plurality of query positions, to obtain the plurality of appearance feature sampling values, and to obtain the plurality of position feature sampling values corresponding to positions of the plurality of appearance feature sampling values.

[0168] FIG. 11 is a structural block diagram of the first processing module 200 in some other embodiments of the present disclosure. As shown in FIG. 11, the first processing module 200 further includes:

[0169] The sampling feature weight determination unit 250 is configured to learnably encode the first fusion features based on the plurality of query vectors by using a sampling feature weight network layer in the attention network, to obtain sampling feature weights.

[0170] The appearance feature determination unit 260 is configured to determine the appearance features based on second fusion features and the plurality of appearance feature sampling values.

[0171] The position feature determination unit 270 is configured to determine the position features based on the second fusion features and the plurality of position feature sampling values.

[0172] The multi-head self-attention processing unit 220 is configured to perform multi-head cross-attention processing on the plurality of appearance feature sample values, the plurality of position feature sample values, the first fusion feature, the sample feature weight, and the plurality of first spatial position encodings by using a multi-head cross-attention network layer in the attention network, to obtain a second fusion feature.

[0173] In some embodiments of the present disclosure, the multi-head self-attention processing unit 220 is configured to perform processing on the query item, the key-value item, the value item, and the plurality of first spatial position encodings, to obtain a third fusion feature; and the multi-head self-attention processing unit 220 is further configured to perform element sum operation and normalization processing on the third fusion feature and the value item, to obtain the first fusion feature.

[0174] In some embodiments of the present disclosure, the multi-head self-attention processing unit 220 is configured to concatenate the first spatial position encodings with the query item and the key-value item respectively, to obtain a query concatenation item and a key-value concatenation item; and the multi-head self-attention processing unit 220 is further configured to perform similarity calculation on the query concatenation item and the key-value concatenation item, to perform weighting processing on the value item by using the similarity calculation result as a weight, and to determine the third fusion feature based on the weighting processing result.

[0175] FIG. 12 is a structural block diagram of an apparatus for identifying a target object in an image according to some embodiments of the present disclosure. As shown in FIG. 12, the apparatus for identifying a target object in an image further includes:

[0176] The feature pyramid graph acquisition module 600 is configured to determine a feature pyramid graph of a preset number of layers corresponding to the image based on a preset image feature extraction rule.

[0177] The spatial position encoding module 100 is further configured to determine a second spatial position encoding matched with each layer of image features in the feature pyramid graph.

[0178] It should be noted that the specific implementation of the apparatus for identifying a target object in an image according to the embodiments of the present disclosure is similar to the specific implementation of the method for identifying a target object in an image according to the embodiments of the present disclosure. For details, refer to the method for identifying a target object in an image. In order to reduce redundancy, no further description is given.

[0179] Exemplary electronic device

[0180] FIG. 13 is a structural diagram of an electronic device according to an embodiment of the present disclosure, which includes at least one processor 11 and a memory 12.

[0181] The processor 11 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities and can control other components in the electronic device 10 to perform desired functions.

[0182] The memory 12 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, which the processor 11 can execute to implement the method of identifying a target object in an image and / or other desired functions of various embodiments of the present disclosure described above.

[0183] In one example, the electronic device 10 can further include an input device 13 and an output device 14, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0184] The input device 13 can further include, for example, a keyboard, a mouse, and / or the like.

[0185] The output device 14 can output various information to the outside, which can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0186] Of course, for simplicity, only some of the components in the electronic device 10 related to the present disclosure are shown in FIG. 13, and components such as buses, input / output interfaces, and / or the like are omitted. In addition to this, the electronic device 10 can further include any other appropriate components depending on specific application cases.

[0187] Exemplary computer program product and computer-readable storage medium

[0188] In addition to the above-described method and device, embodiments of the present disclosure can further provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method of identifying a target object in an image described in the above "Exemplary Method" section of various embodiments of the present disclosure.

[0189] A computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the present disclosure can be a computer program product, which can include a computer program tangibly embodied in a machine readable storage medium.

[0190] In addition, embodiments of the present disclosure can also be a computer readable storage medium, which stores computer program instructions, and the computer program instructions, when executed by a processor, cause the processor to perform the steps of the method of identifying a target object in an image described in the above "Exemplary Method" section.

[0191] The computer readable storage medium can be any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium, for example, but not limited to, includes an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0192] The above describes the basic principles of the present disclosure in combination with specific embodiments, but the advantages, benefits, effects and the like mentioned in the present disclosure are only examples and are not limiting, and it cannot be considered that each embodiment of the present disclosure must have them. In addition, the above specific details are only for the purpose of example and for the purpose of understanding, and are not limiting, and the above details do not limit the present disclosure to be necessarily implemented with the above specific details.

[0193] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for identifying a target object in an image, comprising: spatially positionally encoding an initial position of the target object in the image to obtain a plurality of first spatial positional encodings; processing the plurality of first spatial positional encodings, a plurality of query vectors corresponding to the plurality of first spatial positional encodings, and an image feature of the image and a second spatial positional encoding matching the image feature to obtain an apparent feature and a position feature of the target object on the image; determining a position offset of the target object relative to the initial position based on the position feature and the plurality of first spatial positional encodings; determining size information of the target object in the image based on the apparent feature; determining a final position of the target object in the image based on the initial position of the target object and the position offset.

2. The method of claim 1, wherein, The processing the plurality of first spatial positional encodings, a plurality of query vectors corresponding to the plurality of first spatial positional encodings, and an image feature of the image and a second spatial positional encoding matching the image feature to obtain an apparent feature and a position feature of the target object on the image comprises: using an apparent feature encoding layer in an attention network to encode the plurality of query vectors through learnable encoding parameters to obtain a query item, a key value item, and a value item; using a multi-head self-attention network layer in the attention network to process the query item, the key value item, the value item, and the plurality of first spatial positional encodings to obtain a first fusion feature; based on the first fusion feature, sampling the image feature of the image and the second spatial positional encoding to obtain a plurality of apparent feature sampling values and a plurality of position feature sampling values of the target object on the image; using a multi-head cross-attention network layer in the attention network to process the plurality of apparent feature sampling values, the plurality of position feature sampling values, the first fusion feature, and the plurality of first spatial positional encodings to obtain the apparent feature and the position feature.

3. The method of claim 2, wherein, The based on the first fusion feature, sampling the image feature of the image and the second spatial positional encoding to obtain a plurality of apparent feature sampling values and a plurality of position feature sampling values of the target object on the image comprises: using a sampling bias network layer of the attention network to process the first fusion feature based on a plurality of query positions of the plurality of query vectors to obtain a plurality of sampling biases corresponding to the plurality of query positions; based on the plurality of sampling biases and the plurality of query positions, sampling the image feature of the image to obtain the plurality of apparent feature sampling values, and obtaining the plurality of position feature sampling values corresponding to positions of the plurality of apparent feature sampling values.

4. The method of claim 2, wherein, The using a multi-head cross-attention network layer in the attention network to process the plurality of apparent feature sampling values, the plurality of position feature sampling values, the first fusion feature, and the plurality of first spatial positional encodings to obtain the apparent feature and the position feature comprises: The sampling feature weight network layer in the attention network is used to perform learnable coding on the first fusion feature based on the plurality of query vectors, to obtain a sampling feature weight; The multi-head cross-attention network layer in the attention network is used to perform multi-head cross-attention processing on the plurality of appearance feature sample values, the plurality of position feature sample values, the first fusion feature, the sampling feature weight, and the plurality of first spatial position encodings, to obtain a second fusion feature; The appearance feature is determined based on the second fusion feature and the plurality of appearance feature sample values; The position feature is determined based on the second fusion feature and the plurality of position feature sample values.

5. The method according to any of claims 2-4, wherein, The multi-head self-attention network layer in the attention network is used to process the query item, the key-value item, the value item, and the plurality of first spatial position encodings, to obtain a first fusion feature, including: The multi-head self-attention network layer is used to process the query item, the key-value item, the value item, and the plurality of first spatial position encodings, to obtain a third fusion feature; The third fusion feature and the value item are subjected to element sum operation and normalization processing, to obtain the first fusion feature.

6. The method of claim 5, wherein, The multi-head self-attention network layer is used to process the query item, the key-value item, the value item, and the plurality of first spatial position encodings, to obtain a third fusion feature, including: The first spatial position encodings are concatenated with the query item and the key-value item respectively, to obtain a query concatenation item and a key-value concatenation item; Similarity calculation is performed on the query concatenation item and the key-value concatenation item, and the similarity calculation result is used as a weight to perform weighted processing on the value item, and the third fusion feature is determined based on the weighted processing result.

7. The method of any one of claims 2-4, wherein, Before the plurality of appearance feature sample values and the plurality of position feature sample values of the target object on the image are obtained by sampling the image feature of the image and the second spatial position encoding based on the first fusion feature, the method further includes: A feature pyramid graph of a preset number of layers corresponding to the image is determined based on a preset image feature extraction rule; Second spatial position encodings matching each layer of image features in the feature pyramid graph are determined.

8. An apparatus for identifying a target object in an image, comprising: a spatial position encoding module configured to perform spatial position encoding on an initial position of a target object in an image to obtain a plurality of first spatial position encodings; a first processing module configured to process the plurality of first spatial position encodings, a plurality of query vectors corresponding to the plurality of first spatial position encodings, an image feature of the image, and second spatial position encodings matching the image feature, to obtain an appearance feature and a position feature of the target object on the image, wherein the query vector is a vector used for querying when detecting the target object in the image; a position offset determination module configured to determine a position offset of the target object relative to the initial position based on the position feature and the plurality of first spatial position encodings; a size information determination module, configured to determine size information of the target object in the image based on the apparent feature; a second processing module, configured to determine a final position of the target object in the image based on the initial position of the target object and the position offset. 9.A computer readable storage medium, the storage medium storing a computer program, the computer program being used to execute the method of any one of claims 1-7. 10.An electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor, configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-7.

Images