Image feature acquisition method, image retrieval method and related equipment

Abstract

The present invention provides an image feature acquisition method, an image retrieval method, and related equipment. For situations where certain parts of an object in an image are invisible, the image feature acquisition method provided by the present invention first determines the visible key point information of the object in the image, then locates the visible key area based on the visible key point information, then obtains the features of the visible key area, and then predicts the features of the invisible key area based on the features of the visible key area. In this way, the features of all key areas, namely local features, can be obtained. On this basis, the local features are fused with the global features to obtain comprehensive features that can better characterize the object in the image. Based on the image feature acquisition method provided by the present invention, the present invention also provides an image retrieval method. Since the image features used in image retrieval are image features that can better characterize the object in the image obtained by using the image feature acquisition method provided by the present invention, a better retrieval effect can ultimately be obtained.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image feature acquisition method, an image retrieval method and related equipment. Background Art

[0002] In some image-related application scenarios, it is necessary to obtain image features from the images in the scene and then perform certain operations based on the obtained image features. It is understandable that the acquisition of image features is a relatively basic and critical operation, and the quality of the obtained image features will directly affect the subsequent operation results.

[0003] For example, in a vehicle re-identification scenario, it is necessary to obtain image features for the target vehicle image, and then retrieve images with similar characteristics to the target vehicle image in the vehicle image library based on the obtained image features. If the obtained image features can better represent the vehicle, a better retrieval effect can be obtained, otherwise a better retrieval result cannot be obtained.

[0004] In actual application scenarios, some parts of the object in the image may be invisible for some reason (such as being blocked), which results in the image features obtained for the image being unable to well represent the object in the image, that is, it is impossible to obtain better image features. Summary of the Invention

[0005] In view of this, the present invention provides an image feature acquisition method, an image retrieval method, and related devices to solve the problem of being unable to obtain good image features due to the invisible parts of certain objects in the image. The technical solution is as follows:

[0006] A method for acquiring image features, comprising:

[0007] Get the target image of the specified scene;

[0008] According to the target image, obtaining global features and visible key point information of the target object in the target image;

[0009] Determining a visible key area of ​​the target object based on the visible key point information;

[0010] Acquiring features of the visible key area, and predicting features of the invisible key area of ​​the target object based on the features of the visible key area, so as to obtain local features of the target object;

[0011] The global features are fused with the local features, and the fused features are used as image features corresponding to the target image.

[0012] Optionally, the visible key point information is position information of a visible key point among a plurality of key points of the target object;

[0013] Obtaining visible key point information of a target object in the target image, including:

[0014] Acquiring shallow features of the target image;

[0015] For each key point among the plurality of key points: predicting a probability that the key point is a visible key point based on the shallow features, and generating a key point heat map corresponding to the key point based on the shallow features and the probability that the key point is a visible key point;

[0016] According to the key point heat maps respectively corresponding to the key points, position information of visible key points among the key points is determined.

[0017] Optionally, acquiring the features of the visible key area and predicting the features of the invisible key area of ​​the target object based on the features of the visible key area includes:

[0018] Acquire features of the visible key area based on a pre-trained feature acquisition model;

[0019] Based on the feature acquisition model, and taking the features of the visible key areas as a basis, predicting the features of the invisible key areas of the target object;

[0020] The feature acquisition model is trained using training images marked with object identifiers in the specified scene.

[0021] Optionally, acquiring the features of the visible key area based on a pre-trained feature acquisition model includes:

[0022] Inputting the target image into the first feature extraction module of the feature acquisition model to obtain shallow features of the target image;

[0023] Acquire shallow features of the visible key area from shallow features of the target image;

[0024] Inputting the shallow features of the visible key area into the second feature extraction module corresponding to the visible key area in the feature acquisition model, and obtaining the features of the visible key area output by the second feature extraction module corresponding to the visible key area;

[0025] Predicting features of the invisible key areas of the target object based on the feature acquisition model and the features of the visible key areas includes:

[0026] The features output by the second feature extraction module corresponding to the visible key area are input into the second feature extraction module corresponding to the invisible key area in the feature acquisition model to obtain the features of the invisible key area output by the second feature extraction module corresponding to the invisible key area.

[0027] Optionally, the training objectives of the feature acquisition model include:

[0028] Making the object identification predicted based on the local features of the object in the training image consistent with the object identification annotated in the training image, wherein the local features of the object in the training image include features of a visible key area and features of an invisible key area of ​​the object in the training image;

[0029] And / or, making the object identification predicted based on the image features corresponding to the training image consistent with the object identification annotated in the training image, wherein the image features corresponding to the training image are fused features of the global features and local features of the object in the training image.

[0030] Optionally, the training process of the feature acquisition model includes:

[0031] Acquire global features of objects in the training image based on a feature acquisition model;

[0032] Acquire features of visible key areas of the object in the training image based on the feature acquisition model, and predict features of invisible key areas of the object in the training image based on the features of the visible key areas of the object in the training image, so as to obtain local features of the object in the training image;

[0033] fusing the global features and local features of the object in the training image, and using the fused features as the image features corresponding to the training image;

[0034] Predicting the probability that the object identifier of the training image is each set identifier based on the local features of the object in the training image and / or the image features corresponding to the training image, and the global features of the object in the training image, to obtain a prediction result;

[0035] A prediction loss is determined based on the prediction result and the object identifier annotated on the training image, and parameters of a feature acquisition model are updated based on the prediction loss.

[0036] Optionally, there are multiple local features, each of which is a feature of a key area, and the prediction result includes a first prediction result, a second prediction result, and a third prediction result, wherein the first prediction result is a probability predicted based on the global feature, the second prediction result is a probability predicted based on each local feature, and the third prediction result is a probability predicted based on the image feature;

[0037] Determining the prediction loss according to the prediction result and the object identifier annotated on the training image includes:

[0038] determining a first prediction loss based on the first prediction result and the object identifier annotated on the training image, determining a second prediction loss based on the second prediction result and the object identifier annotated on the training image, and determining a third prediction loss based on the third prediction result and the object identifier annotated on the training image, wherein the second prediction loss includes prediction losses corresponding to the plurality of local features respectively;

[0039] The first prediction loss, the second prediction loss and the third prediction loss are fused, and the fused loss is used as the final prediction loss.

[0040] An image retrieval method, comprising:

[0041] Using any of the above-mentioned image feature acquisition methods, obtain image features corresponding to the target image in the specified scene;

[0042] Calculating the similarity between the image features corresponding to the target image and the image features corresponding to each image in the image library in the specified scene;

[0043] According to the calculated similarity, a retrieval result corresponding to the target image is determined.

[0044] An image feature acquisition device, comprising: an image acquisition module, a global feature acquisition module, a visible key point information acquisition module, a visible key area determination module, a local feature acquisition module, and a feature fusion module;

[0045] The image acquisition module is used to acquire a target image of a specified scene;

[0046] The global feature acquisition module is used to acquire the global features of the target object in the target image according to the target image;

[0047] The visible key point information acquisition module is used to acquire the visible key point information of the target object in the target image according to the target image;

[0048] The visible key area determination module is configured to determine the visible key area of ​​the target object based on the visible key point information;

[0049] The local feature acquisition module is used to acquire features of the visible key area and predict features of the invisible key area of ​​the target object based on the features of the visible key area to obtain local features of the target object;

[0050] The feature fusion module is used to fuse the global feature with the local feature, and the fused feature is used as the image feature corresponding to the target image.

[0051] An image retrieval device, comprising: the above-mentioned image feature acquisition device, as well as a feature similarity calculation module and a retrieval result determination module;

[0052] The image feature acquisition device is used to acquire image features corresponding to the target image in the specified scene;

[0053] The feature similarity calculation module is used to calculate the similarity between the image features corresponding to the target image acquired by the image feature acquisition device and the image features corresponding to each image in the image library in the specified scene;

[0054] The retrieval result determination module is used to determine the retrieval result corresponding to the target image according to the similarity calculated by the feature similarity calculation module.

[0055] An image processing device, comprising: a memory and a processor;

[0056] The memory is used to store programs;

[0057] The processor is used to execute the program to implement each step of any one of the above-mentioned image feature acquisition methods, or to implement each step of the above-mentioned image retrieval method.

[0058] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the various steps of any of the above-mentioned image feature acquisition methods, or implements the various steps of the above-mentioned image retrieval method.

[0059] The image feature acquisition method provided by the present invention, after obtaining a target image of a specified scene, first obtains the global features and visible key point information of the target object in the target image based on the target image, then determines the visible key area of ​​the target object based on the visible key point information, then obtains the features of the visible key area, and based on the features of the visible key area, predicts the features of the invisible key area of ​​the target object to obtain the local features of the target object (features of the visible key area and features of the invisible key area), and finally fuses the global features with the local features, and the fused features are used as the image features corresponding to the target image. For the case where some parts of the object in the image are invisible, the present invention first determines the features of the visible key area, and then further predicts the features of the invisible key area based on the features of the visible key area, so that the features of all key areas can be obtained. On this basis, the features of all key areas (i.e., local features) are fused with the global features to obtain comprehensive features that can better characterize the object in the image. Based on the image feature acquisition method provided by the present invention, the present invention also provides an image retrieval method. Since the image features used in the image retrieval process are image features that can better characterize the object in the image obtained by the image feature acquisition method provided by the present invention, a better retrieval effect can be ultimately achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0061] Figure 1 A schematic diagram of the hardware architecture involved in the present invention;

[0062] Figure 2 A schematic diagram of a flow chart of an image feature acquisition method provided by an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of the structure of a key point detection model provided by an embodiment of the present invention;

[0064] Figure 4 A schematic diagram of key points of a vehicle provided by an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of a flow chart of the training process of a feature acquisition model provided in an embodiment of the present invention;

[0066] Figure 6 A schematic diagram of the structure of a feature acquisition model provided in an embodiment of the present invention;

[0067] Figure 7 A schematic diagram of the structure of an image feature acquisition device provided by an embodiment of the present invention;

[0068] Figure 8 A schematic diagram of the structure of an image retrieval device provided by an embodiment of the present invention;

[0069] Figure 9 A schematic structural diagram of an image processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] In order to obtain features that can better characterize the objects in the target image for the target image in the specified field, the applicant conducted research. The initial idea was: obtain the target image; perform key point detection on the target image to obtain the key point information of the object in the target image and its corresponding confidence level; divide the object in the target image into N parts according to the obtained key point information and its corresponding confidence level as local images; after enlarging or reducing the local image by a set multiple, merge it with the original local image as a preprocessed local image; extract features from the target image to obtain global features, and extract features from the preprocessed local image to obtain local features; fuse the global features with the local features, and use the fused features as the comprehensive features of the object in the target image, which are used as the image features finally obtained for the target image.

[0073] The applicants studied the above approach and discovered that, while dividing the target image object into N parts based on keypoint information and its corresponding confidence scores, this approach failed to account for situations where keypoints might be invisible due to occlusion, angle, and other issues. Consequently, the resulting features still failed to effectively represent the object in the image. In light of this, the applicants continued their research and ultimately developed a highly effective method for acquiring image features. Furthermore, they proposed an image retrieval method based on this method.

[0074] Before introducing the image feature acquisition method and image retrieval method provided by the present invention, the hardware architecture involved in the present invention is first described.

[0075] In one possible implementation, Figure 1 As shown, the hardware architecture involved in the present invention may include: an electronic device 101 and a server 102.

[0076] Exemplarily, the electronic device 101 may be any electronic product capable of performing human-computer interaction with a user, such as a PC, a laptop computer, a tablet computer, a PDA, or the like.

[0077] It should be noted that Figure 1 This is just an example. There are many types of electronic devices, not limited to Figure 1 Laptop in the.

[0078] For example, the server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. The server 102 may include a processor, a memory, a network interface, and the like.

[0079] Exemplarily, the electronic device 101 may establish a connection and communicate with the server 102 via a wireless communication network; exemplarily, the electronic device 101 may establish a connection and communicate with the server 102 via a wired communication network.

[0080] The electronic device 101 can obtain a target image in a specified scene and send the target image to the server 102. The server 102 obtains the image features corresponding to the target image according to the image feature acquisition method provided by the present invention. Furthermore, the server 102 can also perform image retrieval in the image library based on the image features corresponding to the obtained target image and send the retrieval results to the electronic device 101.

[0081] In another possible implementation, the hardware architecture of the present invention may include an electronic device. The electronic device is an electronic product with strong data processing capabilities, such as a PC, laptop computer, or mobile phone. The electronic device can obtain image features corresponding to a target image in a specified scene using the image feature acquisition method provided by the present invention. Furthermore, the electronic device can perform image retrieval in an image library based on the obtained image features corresponding to the target image to obtain retrieval results.

[0082] Those skilled in the art should understand that the above-mentioned electronic devices and servers are only examples, and other existing or future electronic devices or servers that are applicable to the present invention should also be included in the scope of protection of the present invention and are included here by reference.

[0083] Next, the image feature acquisition method and image retrieval method provided by the present invention are introduced through the following embodiments.

[0084] First embodiment

[0085] See also Figure 2 , which shows a flow chart of the image feature acquisition method provided by an embodiment of the present invention, which may include:

[0086] Step S201: Acquire a target image of a specified scene.

[0087] The target image in the specified scene is the image whose image features are to be obtained in the specified scene.

[0088] Exemplarily, the designated scene is a vehicle re-identification scene, and the target image is a vehicle image.

[0089] Step S202: According to the target image, global features and visible key point information of the target object in the target image are obtained.

[0090] The visible key point information is the position information of visible key points among the multiple key points of the target object in the target image. For example, the target image is a vehicle image, the target object in the target image is a vehicle, the global features of the target object in the target image are the global features of the vehicle in the vehicle image, and the visible key point information of the target object in the target image is the position information of the visible key points among the multiple key points of the vehicle in the vehicle image. For example, if the vehicle has a total of 20 key points, the visible key point information of the target object is the position information of the visible key points among the 20 key points.

[0091] Specifically, according to the target image, the process of obtaining visible key point information of the target object in the target image may include:

[0092] Step S2021: Obtain shallow features of the target image.

[0093] In one possible implementation, Figure 3 As shown, the shallow features of the target image can be obtained based on the feature extraction module 301 in the key point detection model obtained by pre-training.

[0094] Among them, the key point detection model uses training images marked with key point positions, and is trained with the goal of making the key point positions determined based on the training images consistent with the key point positions marked on the training images.

[0095] Step S2022: For each key point among the key points of the target object in the target image, predict the probability that the key point is a visible key point based on the shallow features of the target image, and generate a key point heat map corresponding to the key point based on the shallow features and the probability that the key point is a visible key point.

[0096] Specifically, for each key point of the target object in the target image, such as Figure 3 As shown, the probability that the key point is a visible key point can be predicted based on the key point classification module 302 in the key point detection model, and the key point heat map generation module 303 in the key point detection model, the shallow features of the target image and the probability that the key point is a visible key point, can generate a key point heat map corresponding to the key point to obtain key point heat maps corresponding to several key points of the target object in the target image.

[0097] In one possible implementation, the above-mentioned process of "generating a key point heat map corresponding to the key point based on the key point heat map generation module 303 in the key point detection model, the shallow features of the target image and the probability that the key point is a visible key point" may include: inputting the shallow features of the target image and the probability that the key point is a visible key point into the key point heat map generation module 303, the key point heat map generation module 303 further extracts features from the shallow features of the target image, weights the extracted features using the probability that the key point is a visible key point as a weight, and generates a key point heat map based on the weighted features.

[0098] Step S2023: Determine position information of visible key points among the key points according to the key point heat maps corresponding to the key points of the target object in the target image.

[0099] In one possible implementation, for each key point among several key points of the target object in the target image, the point with the largest value in the key point heat map is determined as the target point. If the value of the target point is greater than a preset value K, the key point is determined to be a visible key point, and the position of the target point is determined to be the position of the key point. If the value of the target point is less than or equal to the preset value K, the key point can be determined to be an invisible key point.

[0100] Step S203: Determine the visible key area of ​​the target object based on the visible key point information.

[0101] After obtaining the position information of the visible key points of the target object in the target image, the visible key area of ​​the target object in the target image can be located according to the position information of the visible key points.

[0102] Next, we will introduce key points and key areas with a specific example: for vehicle images in vehicle re-identification application scenarios, such as Figure 4 As shown in the figure, the vehicle in the vehicle image includes 20 key points, including the left front wheel, right front wheel, left rear wheel, right rear wheel, front logo, rear logo, front license plate, and rear license plate. According to the positions of the key points, several key areas can be determined. For example, based on Figure 4The positions of the key points 13, 14, 15, and 16 in the figure can determine the roof area (the quadrilateral area formed by connecting the key points 13, 14, 15, and 16), the left side area of ​​the car can be determined according to the positions of the key points 11, 12, 14, and 15 (the quadrilateral area formed by connecting the key points 11, 12, 14, and 15), the right side area of ​​the car can be determined according to the positions of the key points 12, 13, 16, and 17 (the quadrilateral area formed by connecting the key points 12, 13, 16, and 17), the front face area of ​​the car can be determined according to the positions of the key points 1, 2, 13, and 14 (the quadrilateral area formed by connecting the key points 1, 2, 13, and 14), and the rear face area of ​​the car can be determined according to the positions of the key points 15, 16, 17, and 18 (the quadrilateral area formed by connecting the key points 15, 16, 17, and 18).

[0103] Step S204: Acquire features of the visible key region, and predict features of the invisible key region of the target object based on the features of the visible key region to obtain local features of the target object.

[0104] It should be noted that the local features of the target object include features of the visible key area and features of the invisible key area predicted based on the features of the visible key area.

[0105] Exemplarily, the designated scene is a vehicle re-identification scene, the target image is a vehicle image, and there are a total of 5 key areas of the vehicle in the vehicle image, namely key area 1 (such as the roof area), key area 2 (such as the left area of ​​the vehicle), key area 3 (such as the right area of ​​the vehicle), key area 4 (the front face area of ​​the vehicle), and key area 5 (such as the rear face area of ​​the vehicle). Among them, it can be seen that the key areas are key area 1, key area 2, and key area 3. The embodiment of the present invention first obtains the features of key area 1, key area 2, and key area 3, and then predicts the features of key area 4 and key area 5 based on the features of key area 1, key area 2, and key area 3. In this way, the features of all key areas can be obtained.

[0106] In one possible implementation, the process of obtaining features of the visible key areas and predicting features of the invisible key areas of the target object based on the features of the visible key areas may include: obtaining features of the visible key areas based on a pre-trained feature acquisition model; and predicting features of the invisible key areas of the target object based on the features of the visible key areas based on the feature acquisition model.

[0107] It should be noted that the global features of the target object in the target image can also be acquired based on the feature acquisition model.

[0108] In which, the feature acquisition model is trained using training images marked with object identification in a specified scene. The training objectives of the feature acquisition model may include a first objective and / or a second objective, and a third objective. The first objective is to make the object identification predicted based on the local features of the object in the training image tend to be consistent with the object identification marked in the training image. The local features of the object in the training image include features of the visible key area of ​​the object in the training image and features of the invisible key area predicted based on features of the visible key area of ​​the object in the training image. The second objective is to make the object identification predicted based on the image features corresponding to the training image tend to be consistent with the object identification marked in the training image. The image features corresponding to the training image are fusion features of the global features of the object in the training image and the local features of the object in the training image. The third objective is to make the object identification predicted based on the global features of the object in the training image tend to be consistent with the object identification marked in the training image.

[0109] Step S205: Fusing the global features with the local features, and using the fused features as image features corresponding to the target image.

[0110] Exemplarily, the designated scene is a vehicle re-identification scene, the target image is a vehicle image, and there are a total of 5 key areas of the vehicle in the vehicle image, namely key area 1 (such as the roof area), key area 2 (such as the left area of ​​the vehicle), key area 3 (such as the right area of ​​the vehicle), key area 4 (the front face area of ​​the vehicle), and key area 5 (such as the rear face area of ​​the vehicle). After obtaining the global features of the vehicle and the features of the 5 key areas, the global features of the vehicle can be fused with the features of the 5 key areas. The fused features are comprehensive features that can better characterize the vehicle in the vehicle image, and are used as the image features corresponding to the vehicle image.

[0111] In one possible implementation, the global features and the local features (i.e., the features of each key area) can be spliced ​​together, and the spliced ​​features are used as the image features corresponding to the target image. It should be noted that when splicing the global features and the local features, they can be spliced ​​in a set order.

[0112] The image feature acquisition method provided by the embodiment of the present invention, after obtaining the target image of the specified scene, first obtains the global features and visible key point information of the target object in the target image based on the target image, then determines the visible key area of ​​the target object based on the visible key point information, then obtains the features of the visible key area, and based on the features of the visible key area, predicts the features of the invisible key area of ​​the target object to obtain the local features of the target object (i.e., the features of the visible key area and the features of the invisible key area), and finally fuses the global features with the local features, and the fused features are used as the image features corresponding to the target image. In response to the situation where certain parts of the object in the image are invisible, the embodiment of the present invention first determines the features of the visible key area, and then further predicts the features of the invisible key area based on the features of the visible key area, so that the features of all key areas can be obtained. On this basis, the features of all key areas (i.e., local features) are fused with the global features to obtain comprehensive features that can better characterize the object in the image.

[0113] Second embodiment

[0114] The above embodiment mentioned that the global features of the target object in the target image, the features of the visible key areas and the features of the invisible key areas can be obtained based on the pre-trained feature acquisition model. This embodiment focuses on introducing the training process of the feature acquisition model.

[0115] See also Figure 5 , which shows a flow chart of the training process of the feature acquisition model, which may include:

[0116] Step S501: Obtain visible key point information of an object in the training image according to the training image, and determine a visible key area of ​​the object in the training image according to the visible key point information of the object in the training image.

[0117] The specific implementation process of step S501 is similar to the specific implementation process of obtaining the visible key point information of the target object in the target image according to the target image, and determining the visible key area of ​​the target object in the target image according to the visible key point information of the target object in the target image in the target image. For details, please refer to the above embodiment, and this embodiment will not be repeated here.

[0118] Step S502: Based on the feature acquisition model, global features of the object in the training image are acquired.

[0119] In one possible implementation, Figure 6 As shown, the feature acquisition model may include a first feature extraction module, and the global features of the object in the training image may be acquired based on the first feature acquisition module in the feature acquisition model.

[0120] Step S503: Based on the feature acquisition model, the features of the visible key areas of the objects in the training image are acquired, and based on the features of the visible key areas of the objects in the training image, the features of the invisible key areas of the objects in the training image are predicted to obtain the local features of the objects in the training image.

[0121] like Figure 6 As shown, the feature acquisition model may include, in addition to the first feature extraction module, several second feature extraction modules. The number of second feature extraction modules can be set according to the specific scenario. For example, if the specified scenario is a vehicle re-identification scenario, and there are a total of 5 key areas of the vehicle in the vehicle image in the vehicle re-identification scenario, then the feature acquisition model may include 5 second feature extraction modules. It should be noted that Figure 6 The number of feature acquisition models shown in is 5, which is only an example.

[0122] based on Figure 6 The process of the feature acquisition model shown in the figure acquiring features of the visible key areas and invisible key areas of the objects in the training image may include: acquiring shallow features of the training image (the shallow features of the training image may be acquired based on the first feature extraction module of the feature acquisition model. It should be noted that the global features are obtained by further processing the shallow features), acquiring shallow features of the visible key areas of the objects in the training image from the shallow features of the training image; inputting the shallow features of the visible key areas of the objects in the training image into the second feature extraction module corresponding to the visible key areas in the feature acquisition model to obtain features of the visible key areas output by the second feature extraction module corresponding to the visible key areas; inputting the features output by the second feature extraction module corresponding to the visible key areas into the second feature extraction module corresponding to the invisible key areas in the feature acquisition model to obtain features of the invisible key areas output by the second feature extraction module corresponding to the invisible key areas. Optionally, the second feature extraction module may use a 3*3 convolution.

[0123] For example, the designated scene is a vehicle re-identification scene, the training image is a vehicle image, and there are 5 key areas of the vehicle in the training image, namely key area 1 (such as the roof area), key area 2 (such as the left area of ​​the vehicle), key area 3 (such as the right area of ​​the vehicle), key area 4 (the front face area of ​​the vehicle), and key area 5 (such as the rear face area of ​​the vehicle). Among them, key area 1, key area 2, and key area 3 are visible key areas, and key area 4 and key area 5 are invisible key areas. The feature acquisition model includes 5 second feature extraction modules, namely second feature extraction module 1, second feature extraction module 2, second feature extraction module 3, second feature extraction module 4, and second feature extraction module 5. Extraction module 4, in which 3 second feature extraction modules serve as second feature extraction modules corresponding to visible key areas, and 2 second feature extraction modules serve as second feature extraction modules corresponding to invisible key areas. Features of key area 1 can be obtained based on second feature extraction module 1, features of key area 2 can be obtained based on second feature extraction module 2, and features of key area 3 can be obtained based on second feature extraction module 3. Features of key area 1, key area 2 and key area 3 are input into second feature extraction module 4 to obtain features of key area 4. Features of key area 1, key area 2 and key area 3 are input into second feature extraction module 5 to obtain features of key area 5.

[0124] Step S504: fusing the global features of the object in the training image with the local features of the object in the training image, and using the fused features as the image features corresponding to the training image.

[0125] Specifically, the global features of the object in the training image may be spliced ​​with the local features of the object in the training image, and the spliced ​​features are used as the image features corresponding to the training image.

[0126] Step S505: predicting the probability that the object identification of the training image is each set identification based on the global features of the object in the training image, the local features of the object in the training image, and the image features corresponding to the training image, and obtaining a prediction result.

[0127] It should be noted that there are multiple local features, each of which is a feature of a key area. When predicting the probability that the object identification of the training image is each set identification based on the local features of the object in the training image, the probability that the object identification of the training image is each set identification is predicted based on each local feature of the object in the training image.

[0128] Exemplarily, the designated scene is a vehicle re-identification scene, the training image is a vehicle image, and there are a total of 5 key areas of the vehicle in the training image, namely key area 1 (such as the roof area), key area 2 (such as the left side area of ​​the vehicle), key area 3 (such as the right side area of ​​the vehicle), key area 4 (the front face area of ​​the vehicle), and key area 5 (such as the rear face area of ​​the vehicle). After obtaining the global features of the vehicle in the training image and the features of the 5 key areas, the probability of the vehicle identification of the training image being each set identification can be predicted based on the global features of the vehicle in the training image, the probability of the vehicle identification of the training image being each set identification can be predicted based on the features of key area 1 (such as the roof area) of the vehicle in the training image, the probability of the vehicle identification of the training image being each set identification can be predicted based on the features of key area 2 (such as the left side area) of the vehicle in the training image,..., the probability of the vehicle identification of the training image being each set identification can be predicted based on the features of key area 5 (such as the rear face area) of the vehicle in the training image.

[0129] Step S506: Determine the prediction loss based on the prediction result and the object identifier annotated on the training image, and update the parameters of the feature acquisition model based on the prediction loss.

[0130] Specifically, a first prediction loss is determined based on the first prediction result and the object identifier annotated on the training image, a second prediction loss is determined based on the second prediction result and the object identifier annotated on the training image, and a third prediction loss is determined based on the third prediction result and the object identifier annotated on the training image. The first, second, and third prediction losses are fused, and the fused loss is used as the final prediction loss. The parameters of the feature acquisition model are updated based on the final prediction loss. Optionally, a cross-entropy loss function can be used to determine the first, second, and third prediction losses.

[0131] The first prediction result is the probability predicted based on the global features of the object in the training image, the second prediction result is the probability predicted based on the local features of the object in the training image, and the third prediction result is the probability predicted based on the image features corresponding to the training image. It should be noted that the third prediction loss includes prediction losses corresponding to multiple local features.

[0132] For example, the designated scene is a vehicle re-identification scene, the training image is a vehicle image, and there are a total of 5 key areas of the vehicle in the training image, namely key area 1 (such as the roof area), key area 2 (such as the left area of ​​the vehicle), key area 3 (such as the right area of ​​the vehicle), key area 4 (the front face area of ​​the vehicle), and key area 5 (such as the rear face area of ​​the vehicle): the probability P1 (P1={p 11 ,…,p 1N}, N is the number of set identifiers), and the probability P2 (P2 = {p 21 ,…,p 2N}), predict the probability P3 (P3 = {p 31 ,…,p 3N}), predict the probability P4 (P4 = {p 41 ,…,p 4N}), predict the probability P5 (P5 = {p 51 ,…,p 5N}), predict the probability P6 (P6 = {p 61 ,…,p 6N}), predict the probability P7 (P7 = {p 71 ,…,p 7N}), determine the prediction loss L1 based on P1 and the vehicle identification annotated on the training image, determine the prediction loss L2 based on P2 and the vehicle identification annotated on the training image, determine the prediction loss L3 based on P3 and the vehicle identification annotated on the training image, ..., determine the prediction loss L7 based on P7 and the vehicle identification annotated on the training image, fuse L1 to L7, and optionally, perform a weighted sum of L1 to L7, and use the weighted sum loss as the final prediction loss L:

[0133] L=α1*L1+α2*L2+α3*L3+α4*L4+α5*L5+α6*L6+α7*L7

[0134] Optionally, taking into account the high discriminability of certain key areas (such as the roof area and the front face area of ​​the vehicle), the prediction loss corresponding to the key areas with higher discriminability can be given a larger weight. For example, the weights of L2 (corresponding to the prediction loss of the roof area) and L5 (corresponding to the prediction loss of the front face area of ​​the vehicle) in the above formula can be set to 3, and the weights of L1, L3, L4, L6 and L7 can be set to 1.

[0135] It should be noted that, in addition to the above-mentioned weighted summation method, other methods can also be used to fuse L1 to L7, for example, directly summing L1 to L7.

[0136] According to the process of steps S501 to S506 above, multiple iterations of training are performed using different training images until the training end condition is met.

[0137] After the training results, the feature acquisition model obtained through training can be used to obtain the global features, features of visible key areas and features of invisible key areas of the target object in the target image. Specifically: the target image is input into the first feature extraction module of the feature acquisition model to obtain the shallow image features and global features of the target image, wherein the global features are obtained by processing the shallow image features; the shallow features of the visible key areas of the target object in the target image are obtained from the shallow image features of the target image; the shallow features of the visible key areas of the target object in the target image are input into the second feature extraction module corresponding to the visible key areas in the feature acquisition model to obtain the features of the visible key areas of the target object in the target image output by the second feature extraction module corresponding to the visible key areas; the features output by the second feature extraction module corresponding to the visible key areas are input into the second feature extraction module corresponding to the invisible key areas in the feature acquisition model to obtain the features of the invisible key areas of the target object in the target image output by the second feature extraction module corresponding to the invisible key areas.

[0138] Third embodiment

[0139] Based on the image feature acquisition method provided in the above embodiment, this embodiment provides an image retrieval method, which may include:

[0140] Step a1: Obtain image features corresponding to a target image in a specified scene.

[0141] The image features corresponding to the target image in the specified scene are acquired using the image feature acquisition method provided in the above embodiment.

[0142] Optionally, the designated scene may be but is not limited to a vehicle re-identification scene. When the designated scene is a vehicle re-identification scene, the target image is a vehicle image, and the image features corresponding to the target image are features that can better characterize the vehicle in the target image.

[0143] Step a2: Calculate the similarity between the image features corresponding to the target image and the image features corresponding to each image in the image library in the specified scene.

[0144] The image features corresponding to the respective images in the image library in the specified scene are also acquired using the image feature acquisition method provided in the above embodiment.

[0145] Step a3: Determine the retrieval result corresponding to the target image based on the calculated similarity.

[0146] Optionally, the M images whose corresponding image features in the image library have the greatest similarity with the image features corresponding to the target image can be used as the retrieval results corresponding to the target image. In addition to this method, images whose corresponding image features in the image library have a similarity with the image features corresponding to the target image that is greater than a preset similarity threshold can also be used as the retrieval results corresponding to the target image.

[0147] Since the image features used in image retrieval are image features that are obtained by the image feature acquisition method provided by the embodiment of the present invention and can better represent the objects in the image, a better retrieval effect can be finally obtained.

[0148] Fourth embodiment

[0149] An embodiment of the present invention further provides an image feature acquisition device. The image feature acquisition device provided by the embodiment of the present invention is described below. The image feature acquisition device described below and the image feature acquisition method described above can be referred to in correspondence with each other.

[0150] See also Figure 7 , shows a structural schematic diagram of the image feature acquisition device provided by an embodiment of the present invention, which may include an image acquisition module 701, a global feature acquisition module 702, a visible key point information acquisition module 703, a visible key area determination module 704, a local feature acquisition module 705 and a feature fusion module 706.

[0151] Image acquisition module 701, used to acquire a target image of a specified scene;

[0152] The global feature acquisition module 702 is configured to acquire global features of a target object in the target image based on the target image.

[0153] The visible key point information acquisition module 703 is configured to acquire the visible key point information of the target object in the target image according to the target image.

[0154] The visible key area determination module 704 is configured to determine the visible key area of ​​the target object based on the visible key point information.

[0155] The local feature acquisition module 705 is configured to acquire features of the visible key region and predict features of the invisible key region of the target object based on the features of the visible key region to obtain local features of the target object.

[0156] The feature fusion module 706 is used to fuse the global feature with the local feature, and use the fused feature as the image feature corresponding to the target image.

[0157] Optionally, the visible key point information is position information of a visible key point among a plurality of key points of the target object. When acquiring the visible key point information of the target object in the target image, the visible key point information acquiring module 703 is specifically configured to:

[0158] Acquiring shallow features of the target image;

[0159] For each key point among the plurality of key points: predicting a probability that the key point is a visible key point based on the shallow features, and generating a key point heat map corresponding to the key point based on the shallow features and the probability that the key point is a visible key point;

[0160] According to the key point heat maps respectively corresponding to the key points, position information of visible key points among the key points is determined.

[0161] Optionally, when the local feature acquisition module 705 acquires the features of the visible key area and predicts the features of the invisible key area of ​​the target object based on the features of the visible key area, it is specifically configured to:

[0162] Acquire features of the visible key area based on a pre-trained feature acquisition model;

[0163] Based on the feature acquisition model, and taking the features of the visible key areas as a basis, predicting the features of the invisible key areas of the target object;

[0164] The feature acquisition model is trained using training images marked with object identifiers in the specified scene.

[0165] Optionally, when the local feature acquisition module 705 acquires the features of the visible key area based on the pre-trained feature acquisition model, it is specifically configured to:

[0166] Inputting the target image into the first feature extraction module of the feature acquisition model to obtain shallow features of the target image;

[0167] Acquire shallow features of the visible key area from shallow features of the target image;

[0168] Inputting the shallow features of the visible key area into the second feature extraction module corresponding to the visible key area in the feature acquisition model, and obtaining the features of the visible key area output by the second feature extraction module corresponding to the visible key area;

[0169] Predicting features of the invisible key areas of the target object based on the feature acquisition model and the features of the visible key areas includes:

[0170] The features output by the second feature extraction module corresponding to the visible key area are input into the second feature extraction module corresponding to the invisible key area in the feature acquisition model to obtain the features of the invisible key area output by the second feature extraction module corresponding to the invisible key area.

[0171] Optionally, the training objectives of the feature acquisition model include:

[0172] Making the object identification predicted based on the local features of the object in the training image consistent with the object identification annotated in the training image, wherein the local features of the object in the training image include features of a visible key area and features of an invisible key area of ​​the object in the training image;

[0173] And / or, making the object identification predicted based on the image features corresponding to the training image consistent with the object identification annotated in the training image, wherein the image features corresponding to the training image are fused features of the global features and local features of the object in the training image.

[0174] Optionally, the image feature acquisition device provided in this embodiment may further include: a model training module. The model training module is used to train the feature acquisition model. When training the feature acquisition model, the model training module is specifically used to:

[0175] Acquire global features of objects in the training image based on a feature acquisition model;

[0176] Acquire features of visible key areas of the object in the training image based on the feature acquisition model, and predict features of invisible key areas of the object in the training image based on the features of the visible key areas of the object in the training image, so as to obtain local features of the object in the training image;

[0177] fusing the global features and local features of the object in the training image, and using the fused features as the image features corresponding to the training image;

[0178] Predicting the probability that the object identifier of the training image is each set identifier based on the local features of the object in the training image and / or the image features corresponding to the training image, and the global features of the object in the training image, to obtain a prediction result;

[0179] A prediction loss is determined based on the prediction result and the object identifier annotated on the training image, and parameters of a feature acquisition model are updated based on the prediction loss.

[0180] Optionally, there are multiple local features, each of which is a feature of a key area, and the prediction result includes a first prediction result, a second prediction result, and a third prediction result, wherein the first prediction result is a probability predicted based on the global feature, the second prediction result is a probability predicted based on each local feature, and the third prediction result is a probability predicted based on the image feature;

[0181] When determining the prediction loss based on the prediction result and the object identifier annotated on the training image, the model training module is specifically configured to:

[0182] determining a first prediction loss based on the first prediction result and the object identifier annotated on the training image, determining a second prediction loss based on the second prediction result and the object identifier annotated on the training image, and determining a third prediction loss based on the third prediction result and the object identifier annotated on the training image, wherein the second prediction loss includes prediction losses corresponding to the plurality of local features respectively;

[0183] The first prediction loss, the second prediction loss and the third prediction loss are fused, and the fused loss is used as the final prediction loss.

[0184] The image feature acquisition device provided by the embodiment of the present invention, after obtaining the target image of a specified scene, first obtains the global features and visible key point information of the target object in the target image based on the target image, then determines the visible key area of ​​the target object based on the visible key point information, then obtains the features of the visible key area, and based on the features of the visible key area, predicts the features of the invisible key area of ​​the target object to obtain the local features of the target object (i.e., the features of the visible key area and the features of the invisible key area), and finally fuses the global features with the local features, and the fused features are used as the image features corresponding to the target image. In case that some parts of the object in the image are invisible, the embodiment of the present invention first determines the features of the visible key area, and then further predicts the features of the invisible key area based on the features of the visible key area, so that the features of all key areas can be obtained. On this basis, the features of all key areas (i.e., local features) are fused with the global features to obtain comprehensive features that can better characterize the object in the image.

[0185] Fifth embodiment

[0186] Based on the image feature acquisition device provided in the above embodiment, this embodiment provides an image retrieval device, see Figure 8 , shows a schematic structural diagram of the image retrieval device, which may include: the image feature acquisition device 801 provided in the above embodiment, as well as a feature similarity calculation module 802 and a retrieval result determination module 803.

[0187] The image feature acquisition device 801 is used to acquire image features corresponding to a target image in a specified scene.

[0188] The feature similarity calculation module 802 is used to calculate the similarity between the image features corresponding to the target image acquired by the image feature acquisition device and the image features corresponding to each image in the image library in the specified scene.

[0189] The retrieval result determination module 803 is configured to determine the retrieval result corresponding to the target image according to the similarity calculated by the feature similarity calculation module.

[0190] Since the image features used in image retrieval are image features that can better represent objects in the image and are obtained by the image feature acquisition device provided by the embodiment of the present invention, a better retrieval effect can be finally obtained.

[0191] Sixth embodiment

[0192] The embodiment of the present invention also provides an image processing device, see Figure 9 , shows a schematic structural diagram of the image processing device, which may include: at least one processor 901, at least one communication interface 902, at least one memory 903 and at least one communication bus 904.

[0193] In the embodiment of the present invention, there is at least one processor 901 , communication interface 902 , memory 903 , and communication bus 904 , and the processor 901 , communication interface 902 , and memory 903 communicate with each other via the communication bus 904 .

[0194] The processor 901 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0195] The memory 903 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0196] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0197] Get the target image of the specified scene;

[0198] According to the target image, obtaining global features and visible key point information of the target object in the target image;

[0199] Determining a visible key area of ​​the target object based on the visible key point information;

[0200] Acquiring features of the visible key area, and predicting features of the invisible key area of ​​the target object based on the features of the visible key area, so as to obtain local features of the target object;

[0201] The global features are fused with the local features, and the fused features are used as image features corresponding to the target image.

[0202] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0203] Seventh embodiment

[0204] An embodiment of the present invention further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0205] Get the target image of the specified scene;

[0206] According to the target image, obtaining global features and visible key point information of the target object in the target image;

[0207] Determining a visible key area of ​​the target object based on the visible key point information;

[0208] Acquiring features of the visible key area, and predicting features of the invisible key area of ​​the target object based on the features of the visible key area, so as to obtain local features of the target object;

[0209] The global features are fused with the local features, and the fused features are used as image features corresponding to the target image.

[0210] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0211] Eighth embodiment

[0212] An embodiment of the present invention further provides an image processing device, the structure of which is similar to that of the image processing device provided in the sixth embodiment, and may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0213] In an embodiment of the present invention, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0214] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0215] The memory 803 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk storage.

[0216] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0217] Using the image feature acquisition method provided in the above embodiment, image features corresponding to the target image in the specified scene are acquired;

[0218] Calculating the similarity between the image features corresponding to the target image and the image features corresponding to each image in the image library in the specified scene;

[0219] According to the calculated similarity, a retrieval result corresponding to the target image is determined.

[0220] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0221] Ninth embodiment

[0222] An embodiment of the present invention further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0223] Using the image feature acquisition method provided in the above embodiment, image features corresponding to the target image in the specified scene are acquired;

[0224] Calculating the similarity between the image features corresponding to the target image and the image features corresponding to each image in the image library in the specified scene;

[0225] According to the calculated similarity, a retrieval result corresponding to the target image is determined.

[0226] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0227] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0228] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0229] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for acquiring image features, characterized in that: include: Get the target image of the specified scene; According to the target image, obtaining global features and visible key point information of the target object in the target image; Determining a visible key area of ​​the target object based on the visible key point information; Acquiring features of the visible key area, and predicting features of the invisible key area of ​​the target object based on the features of the visible key area, so as to obtain local features of the target object, the local features including features of the visible key area and features of the invisible key area predicted based on the features of the visible key area; fusing the global features with the local features, and using the fused features as image features corresponding to the target image; The acquiring the features of the visible key area and predicting the features of the invisible key area of ​​the target object based on the features of the visible key area includes: Inputting the target image into a first feature extraction module of a feature acquisition model to obtain shallow features of the target image; Acquire shallow features of the visible key area from shallow features of the target image; Inputting the shallow features of the visible key area into the second feature extraction module corresponding to the visible key area in the feature acquisition model, and obtaining the features of the visible key area output by the second feature extraction module corresponding to the visible key area; The features output by the second feature extraction module corresponding to the visible key area are input into the second feature extraction module corresponding to the invisible key area in the feature acquisition model to obtain the features of the invisible key area output by the second feature extraction module corresponding to the invisible key area.

2. The image feature acquisition method according to claim 1, characterized in that: The visible key point information is position information of visible key points among a plurality of key points of the target object; Obtaining visible key point information of a target object in the target image, including: Acquiring shallow features of the target image; For each key point among the plurality of key points: predicting a probability that the key point is a visible key point based on the shallow features, and generating a key point heat map corresponding to the key point based on the shallow features and the probability that the key point is a visible key point; According to the key point heat maps respectively corresponding to the key points, position information of visible key points among the key points is determined.

3. The image feature acquisition method according to claim 1, wherein: The feature acquisition model is trained using training images marked with object identifiers in the specified scene.

4. The image feature acquisition method according to claim 3, characterized in that: The training objectives of the feature acquisition model include: Making the object identification predicted based on the local features of the object in the training image consistent with the object identification annotated in the training image, wherein the local features of the object in the training image include features of a visible key area and features of an invisible key area of ​​the object in the training image; And / or, making the object identification predicted based on the image features corresponding to the training image consistent with the object identification annotated in the training image, wherein the image features corresponding to the training image are fused features of the global features and local features of the object in the training image.

5. The image feature acquisition method according to claim 4, characterized in that: The training process of the feature acquisition model includes: Acquire global features of objects in the training image based on a feature acquisition model; Acquire features of visible key areas of the object in the training image based on the feature acquisition model, and predict features of invisible key areas of the object in the training image based on the features of the visible key areas of the object in the training image, so as to obtain local features of the object in the training image; fusing the global features and local features of the object in the training image, and using the fused features as the image features corresponding to the training image; Predicting the probability that the object identifier of the training image is each set identifier based on the local features of the object in the training image and / or the image features corresponding to the training image, and the global features of the object in the training image, to obtain a prediction result; A prediction loss is determined based on the prediction result and the object identifier annotated on the training image, and parameters of a feature acquisition model are updated based on the prediction loss.

6. The image feature acquisition method according to claim 5, characterized in that: There are multiple local features, each of which is a feature of a key area. The prediction results include a first prediction result, a second prediction result, and a third prediction result. The first prediction result is a probability predicted based on the global feature, the second prediction result is a probability predicted based on each local feature, and the third prediction result is a probability predicted based on the image feature. Determining the prediction loss according to the prediction result and the object identifier annotated on the training image includes: determining a first prediction loss based on the first prediction result and the object identifier annotated on the training image, determining a second prediction loss based on the second prediction result and the object identifier annotated on the training image, and determining a third prediction loss based on the third prediction result and the object identifier annotated on the training image, wherein the second prediction loss includes prediction losses corresponding to the plurality of local features respectively; The first prediction loss, the second prediction loss and the third prediction loss are fused, and the fused loss is used as the final prediction loss.

7. An image retrieval method, characterized in that: include: Acquire image features corresponding to a target image in a specified scene using the image feature acquisition method according to any one of claims 1 to 6; Calculating the similarity between the image features corresponding to the target image and the image features corresponding to each image in the image library in the specified scene; According to the calculated similarity, a retrieval result corresponding to the target image is determined.

8. An image feature acquisition device, characterized in that: include: Image acquisition module, global feature acquisition module, visible key point information acquisition module, visible key area determination module, local feature acquisition module and feature fusion module; The image acquisition module is used to acquire a target image of a specified scene; The global feature acquisition module is used to acquire the global features of the target object in the target image according to the target image; The visible key point information acquisition module is used to acquire the visible key point information of the target object in the target image according to the target image; The visible key area determination module is configured to determine the visible key area of ​​the target object based on the visible key point information; The local feature acquisition module is configured to acquire features of the visible key area and predict features of the invisible key area of ​​the target object based on the features of the visible key area to obtain local features of the target object, wherein the local features include features of the visible key area and features of the invisible key area predicted based on the features of the visible key area; The feature fusion module is used to fuse the global feature with the local feature, and the fused feature is used as the image feature corresponding to the target image; The local feature acquisition module is specifically used for: Inputting the target image into a first feature extraction module of a feature acquisition model to obtain shallow features of the target image; Acquire shallow features of the visible key area from shallow features of the target image; Inputting the shallow features of the visible key area into the second feature extraction module corresponding to the visible key area in the feature acquisition model, and obtaining the features of the visible key area output by the second feature extraction module corresponding to the visible key area; The features output by the second feature extraction module corresponding to the visible key area are input into the second feature extraction module corresponding to the invisible key area in the feature acquisition model to obtain the features of the invisible key area output by the second feature extraction module corresponding to the invisible key area.

9. An image retrieval device, characterized in that: include: The image feature acquisition device according to claim 8, and a feature similarity calculation module and a retrieval result determination module; The image feature acquisition device is used to acquire image features corresponding to the target image in the specified scene; The feature similarity calculation module is used to calculate the similarity between the image features corresponding to the target image acquired by the image feature acquisition device and the image features corresponding to each image in the image library in the specified scene; The retrieval result determination module is used to determine the retrieval result corresponding to the target image according to the similarity calculated by the feature similarity calculation module.

10. An image processing device, characterized in that: include: memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the image feature acquisition method according to any one of claims 1 to 6, or to implement the various steps of the image retrieval method according to claim 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the image feature acquisition method according to any one of claims 1 to 6, or implements the steps of the image retrieval method according to claim 7.

Images