Image feature extraction method, device, computer equipment and readable storage medium
By rotating the input image at different angles, amplified image set is generated and used to process multiple initial subnetworks, combining loss function and parallel operation, the problem of insufficient robustness and accuracy in image feature extraction in traditional depth measurement networks is solved. Improved feature extraction effect.
Patent Information
- Application Number
- CN202010668643.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-13
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2040-07-13
AI Technical Summary
Traditional depth measurement networks are poorly robust and have low accuracy when extracting image features, and cannot effectively distinguish the rotated sample pictures.
By rotating the input image at different angles, amplified image collection is generated and inputted to the pre-trained depth metric network for feature extraction. Multiple initial subnets use to process sample rotation pictures with the same rotation angle, and adjust network parameters using loss function , combine parallel operation to obtain the picture characteristics of the input picture.
It improves the accuracy of picture features and the robustness of the depth measurement network, can better distinguish picture features of different rotation angles, and enhances the comprehensiveness and accuracy of feature extraction.
Smart Images
Figure CN111898619B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment and readable storage medium for extracting image features. Background Art
[0002] Deep metric learning is a method of metric learning. Its goal is to learn a mapping from original features to a low-dimensional dense vector space (called embedding space) (which can be understood as a feature extraction process) so that the distance (metric) of similar objects in the embedding space is closer, while the distance of different objects is farther.
[0003] In the process of deep metric network training, a large number of sample images are usually required. Traditional technology will rotate the obtained sample images to achieve data augmentation, and then use the original sample images and the rotated images, as well as the labels corresponding to the original sample images as training data sets to train the deep metric network; and use the trained deep metric network to extract features from the input images.
[0004] However, the deep measurement network obtained by traditional technology has poor robustness and the extracted image features have low accuracy. Summary of the invention
[0005] Based on this, it is necessary to provide a method, apparatus, computer device and readable storage medium for extracting image features to address the problem of low accuracy of image features extracted by the deep metric network of traditional technology.
[0006] A method for extracting image features, the method comprising:
[0007] Obtaining an augmented picture set of the input picture, the augmented picture set including a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0008] Input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0009] The image features of the input image are determined according to the image features corresponding to each rotated image.
[0010] In one embodiment, the training method of the deep metric network includes:
[0011] For each sample image, perform rotation operations on the sample image at different angles to obtain multiple sample rotated images, and obtain labels corresponding to the multiple sample rotated images respectively;
[0012] Input the sample rotation image into the initial depth metric network for feature extraction to obtain the predicted image features;
[0013] The loss function is used to calculate the loss between the predicted image features and the labels corresponding to the sample rotated images. The network parameters of the initial deep metric network are adjusted according to the loss to obtain the deep metric network.
[0014] In one embodiment, the initial depth measurement network includes multiple initial sub-networks, and the network structures of the multiple initial sub-networks are the same; the sample rotation image is input into the initial depth measurement network for feature extraction to obtain predicted image features, including:
[0015] The sample rotated images with the same rotation angle are input into the same initial sub-network for feature extraction to obtain the predicted image features corresponding to each sample rotated image.
[0016] In one embodiment, a loss function is used to calculate the loss between the predicted image features and the labels corresponding to the sample rotated images, and the network parameters of the initial deep metric network are adjusted according to the loss to obtain the deep metric network, including:
[0017] For each initial sub-network, a loss function is used to calculate the sub-loss between the predicted image features and the label of the input sample rotation image;
[0018] The sub-loss of each initial sub-network is summed to obtain the above loss;
[0019] The network parameters of each initial sub-network are adjusted according to the loss to obtain multiple sub-networks, and the multiple sub-networks are used as depth measurement networks.
[0020] In one of the embodiments, the above loss function includes a triplet loss function.
[0021] In one embodiment, determining the image features of the input image according to the image features corresponding to each rotated image includes:
[0022] The image features corresponding to each rotated image are operated in parallel to obtain the image features of the input image.
[0023] In one embodiment, obtaining an augmented picture set of an input picture includes:
[0024] The input image is rotated by 0°, 90°, 180° and 270° respectively to obtain rotated images with four rotation angles;
[0025] The rotated images at four rotation angles are used as augmented image sets.
[0026] A device for extracting image features, comprising:
[0027] An acquisition module is used to acquire an augmented picture set of an input picture, where the augmented picture set includes a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0028] A feature extraction module is used to input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0029] The determination module is used to determine the image features of the input image according to the image features corresponding to each rotated image.
[0030] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0031] Obtaining an augmented picture set of the input picture, the augmented picture set including a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0032] Input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0033] The image features of the input image are determined according to the image features corresponding to each rotated image.
[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0035] Obtaining an augmented picture set of the input picture, the augmented picture set including a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0036] Input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0037] The image features of the input image are determined according to the image features corresponding to each rotated image.
[0038] The above-mentioned image feature extraction method, device, computer equipment and readable storage medium can input the augmented image set of the input image into a preset deep metric network for feature extraction, obtain the image features corresponding to each rotated image, and then determine the image features of the input image based on the image features of each rotated image. Thus, the obtained image features comprehensively consider the image features of different rotated images, making the obtained image features more comprehensive and improving the accuracy of the image features. In addition, during the training process of the deep metric network, the labels of the sample rotated images corresponding to the same sample image are different, so that the deep metric network can distinguish how many degrees each image is rotated. The robustness of the deep metric network can be improved, and the corresponding image features can be better extracted for different rotated images, which further improves the accuracy of the obtained image features. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is an internal structure diagram of a computer device in one embodiment;
[0040] Figure 2 Schematic diagram of a process of extracting image features in one embodiment;
[0041] Figure 3 is a flow chart of a method for extracting image features in another embodiment;
[0042] Figure 4 Schematic diagram of a process of extracting image features in another embodiment;
[0043] Figure 4a Schematic diagram of a process flow of a deep metric network training process in one embodiment;
[0044] Figure 5 Schematic diagram of a process of extracting image features in another embodiment;
[0045] Figure 6 4 is a structural block diagram of an image feature extraction device in an embodiment. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0047] The image feature extraction method provided in the embodiment of the present application can be applied to Figure 1The computer device shown. The computer device includes a processor and a memory connected by a system bus, and a computer program is stored in the memory. When the processor executes the computer program, the steps of the following method embodiment can be executed. Optionally, the computer device may also include a communication interface, a display screen and an input device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory, and the non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal through a network connection. Optionally, the computer device can be a personal computer (personal computer, referred to as PC), a personal digital assistant, or other terminal devices, such as a tablet computer (portable android device, referred to as PAD), a mobile phone, etc., or a cloud or remote server. The embodiment of the present application does not limit the specific form of the computer device.
[0048] In one embodiment, Figure 2 As shown, a method for extracting image features is provided. This embodiment involves a specific process of extracting features from an input image to obtain its image features. Figure 1 The computer device in the example is used to illustrate, including the following steps:
[0049] S101, obtaining an augmented picture set of an input picture, where the augmented picture set includes a plurality of rotated pictures obtained by rotating the input picture at different angles.
[0050] Specifically, in order to improve the accuracy of the features of the extracted input image, the present embodiment can process and analyze the rotated images of the input image at multiple angles. Therefore, the computer device first needs to obtain multiple rotated images corresponding to the input image, that is, an augmented image set. Among them, the augmented image set includes multiple rotated images obtained by rotating the input image at different angles. Optionally, the computer device can rotate the input image at angles such as 45°, 90°, 135°, and 180°, that is, viewing the input image from multiple different angles to obtain multiple rotated images. It can be understood that the rotated images at different angles are images in different spatial domains.
[0051] Optionally, the computer device can perform rotation operations of 0° (i.e., no rotation), 90°, 180°, and 270° on the input image to obtain rotated images at four rotation angles; and the rotated images at the four rotation angles are used as the augmented image set. The four rotation angles in this embodiment can include 360 degrees of the input image as much as possible, that is, the various angle features of the input image are fully considered.
[0052] S102, inputting the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is trained based on the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the labels of the sample rotated pictures corresponding to the same sample picture are different.
[0053] Specifically, the computer device inputs the above-mentioned augmented picture set into a preset deep metric network, and performs convolution operations (i.e., feature extraction process) on each rotated picture therein through the convolution layer in the deep metric network to obtain the picture features corresponding to each rotated picture. Among them, the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set. The sample augmented picture set includes multiple sample rotated pictures obtained by rotating each sample picture at different angles, and each sample rotated picture corresponds to a label. For the same sample picture, the label of the corresponding sample rotated picture is different. Exemplarily, for a handwritten digit recognition data set including ten digits from 0 to 9, taking the number 2 as an example, 2 is the original picture (i.e., the picture that has not been rotated), then the label corresponding to this picture is "2-0°"; For a picture rotated 90°, the label corresponding to this picture is "2-90°"; For an image rotated 180°, the label corresponding to this image is "2-180°", which can distinguish how many degrees each image is rotated. Based on this training data set, the deep metric network is trained to improve the robustness of the deep metric network, and the corresponding image features can be better extracted for different rotated images.
[0054] However, in traditional technology, for the same sample image, the labels of its corresponding sample rotated images are the same. For example, the image of the number 2, no matter how much it is rotated, has the corresponding label "2". In this case, for some special numbers, such as the number 6 is the same as the number 9 after rotating 180°, and the number 1 is the same as the original number after rotating 180°, it is impossible to distinguish which type of image it actually belongs to, resulting in poor robustness of the trained deep metric network.
[0055] Among them, when training the above-mentioned deep metric network, the entire training set is recorded as The label of each sample image is recorded as Y = {y1, ..., y n}. The convolutional layer and embedding layer of the deep metric network map an image into high-dimensional (d-dimensional) features. Denoted as f(·; θ f ): X→R d ,θ fNetwork parameters. Metric learning f(·;θ f ) is usually normalized to unit length (i.e. ||f(x j )||2=1), the purpose of metric learning is to learn a feature extractor f°(x;θ f ), so that the feature distances of samples of the same type are close and the distances of samples of different types are far. In metric learning, the distance between the features of two images is generally defined as the cosine distance d f (x i ,x j )=1-f(x i )·f(x j ), (where || f(x i )||2=||f(x j )||2=1).
[0056] S103: Determine the image features of the input image according to the image features corresponding to each rotated image.
[0057] Specifically, after the computer device obtains the image features corresponding to each rotated image, it can rotate each image feature at an opposite angle to obtain image features at the same angle as the original input image, and then fuse the rotated image features to obtain the image features of the input image.
[0058] Optionally, the computer device may also perform a concat operation on the image features corresponding to each rotated image in parallel to obtain the image features of the input image.
[0059] In the image feature extraction method provided in this embodiment, the computer device inputs the augmented image set of the input image into a preset deep metric network for feature extraction, obtains the image features corresponding to each rotated image, and then determines the image features of the input image based on the image features of each rotated image. Thus, the obtained image features comprehensively consider the image features of different rotated images, making the obtained image features more comprehensive and improving the accuracy of the image features. Moreover, during the training process of the deep metric network, the labels of the sample rotated images corresponding to the same sample image are different, so that the deep metric network can distinguish how many degrees each image is rotated as a sample rotated image, which can improve the robustness of the deep metric network, better extract the corresponding image features for different rotated images, and further improve the accuracy of the obtained image features.
[0060] Generally, before using the deep metric network to extract features from the augmented image set, the deep metric network needs to be trained to converge. Figure 3 As shown, the specific process of training the deep metric network is involved. Optionally, the training method of the deep metric network may include:
[0061] S201, for each sample image, performing rotation operations on the sample image at different angles to obtain multiple sample rotated images, and obtaining labels corresponding to the multiple sample rotated images.
[0062] Specifically, the computer device may first obtain a large number of sample images. Considering the limited number of sample images, each sample image may be rotated at different angles to obtain multiple sample rotated images, thereby increasing the number of sample images and the comprehensiveness of the sample images. After obtaining multiple sample rotated images, each sample rotated image may be labeled. The labeling method may refer to the description of the above embodiment to obtain a label corresponding to each sample rotated image.
[0063] S202, input the sample rotated image into the initial depth metric network for feature extraction to obtain predicted image features.
[0064] Specifically, the computer device can input the sample rotation images obtained above into the initial depth metric network, and perform convolution operations (i.e., feature extraction process) on each sample rotation image through the convolution layer in the initial depth metric network to obtain the predicted image features corresponding to each sample rotation image. It should be noted that the initial depth metric network at this time is a network in training.
[0065] S203, using a loss function to calculate the loss between the predicted image features and the label corresponding to the sample rotated image, and adjusting the network parameters of the initial depth metric network according to the loss to obtain a depth metric network.
[0066] Specifically, the computer device may use a loss function to calculate the loss between the predicted image features and the label corresponding to the sample rotation image. Optionally, the loss function may be a cross entropy loss function, a triplet loss function, etc. This embodiment does not limit the specific type of the loss function. Then, the network parameters of the initial deep metric network are adjusted according to the calculated loss reverse gradient propagation, thereby iteratively training to obtain a deep metric network.
[0067] In the image feature extraction method provided in this embodiment, a computer device performs rotation operations on each sample image at different angles to obtain multiple sample rotated images; each sample rotated image is input into an initial depth metric network for feature extraction to obtain predicted image features; a loss function is used to calculate the loss between the predicted image features and the labels corresponding to the sample rotated images, and the initial depth metric network is trained according to the loss to obtain a converged depth metric network. Since the training data set is a data set that is amplified for the sample images, and considering the different rotation angles of the sample images, the depth metric network can learn the image features at different angles, thereby improving the robustness of the depth metric network, and the accuracy of the image features obtained using the network is also high.
[0068] In the above embodiment, each sample rotation image is processed by the same initial depth metric network for feature extraction. With a handwritten digit recognition data set, including ten digits from 0 to 9, assuming that the sample image is rotated at a total of 4 angles, each digit image obtains 4 rotation images, each rotation image is a label, and the ten digits will eventually correspond to 40 category labels; in this way, the depth metric network needs to learn 40 classification image features, the learning amount is large, and the efficiency is correspondingly reduced. Therefore, in this embodiment, the initial depth metric network is divided into a plurality of initial subnetworks, the number of initial subnetworks is related to the number of rotation angles, and the network structure of each initial subnetwork is the same, the network parameters are shared, and each initial subnetwork processes only sample rotation images with the same rotation angle. That is to say, assuming that the above rotation angles include 45°, 90°, 135°, and 180°, then one initial subnetwork only processes sample rotation images corresponding to 45°, one initial subnetwork only processes sample rotation images corresponding to 90°, and so on. On this basis, optionally, the above S202 may include: inputting sample rotated images with the same rotation angle into the same initial sub-network for feature extraction to obtain predicted image features corresponding to each sample rotated image. Thus, each initial sub-network only needs to learn the features of sample rotated images with the same rotation angle, which greatly reduces the amount of learning and improves the learning efficiency.
[0069] In one embodiment, Figure 4 As shown, the above S203 may include:
[0070] S301, for each initial sub-network, a loss function is used to calculate a sub-loss between the predicted image features and the label of the input sample rotated image.
[0071] S302, summing up the sub-losses of each initial sub-network to obtain the above loss.
[0072] S303, adjusting the network parameters of each initial sub-network according to the loss to obtain multiple sub-networks, and using the multiple sub-networks as depth measurement networks.
[0073] Specifically, for each initial sub-network, there will be a corresponding predicted image feature, and the computer device will calculate the sub-loss between each predicted image feature and the label respectively, and each initial sub-network will correspond to a sub-loss. The computer device then sums the sub-losses of each initial sub-network, such as direct summation, weighted summation, or average summation, to obtain the loss corresponding to the above-mentioned initial deep metric network. Next, the computer device can adjust the network parameters of each initial sub-network according to the loss, thereby iteratively training to obtain multiple sub-networks; these multiple sub-networks are combined into a deep metric network. For a flowchart of the deep metric network training process, please refer to Figure 4a shown.
[0074] On this basis, after the computer device obtains an input image, it rotates it at different angles to obtain multiple rotated images. By inputting each rotated image into the corresponding sub-network, the image features corresponding to each rotated image can be obtained.
[0075] In this embodiment, each sub-network performs feature extraction on the corresponding rotated image. Since each sub-network has fully learned the image features corresponding to the rotation angle, the processing accuracy of the sub-network can be greatly improved, thereby improving the accuracy of the obtained image features.
[0076] In order to better understand the process of the entire image feature extraction method, a specific embodiment is described below. Figure 5 As shown, the method includes:
[0077] S401, for each sample image, performing rotation operations on the sample image at different angles to obtain multiple sample rotated images, and obtaining labels corresponding to the multiple sample rotated images respectively;
[0078] S402, inputting sample rotated images with the same rotation angle into the same initial sub-network for feature extraction to obtain predicted image features corresponding to each sample rotated image;
[0079] S403, for each initial sub-network, using a loss function to calculate a sub-loss between the predicted image features and the label of the input sample rotation image;
[0080] S404, summing up the sub-losses of each initial sub-network to obtain the loss;
[0081] S405, adjusting the network parameters of each initial sub-network according to the loss to obtain multiple sub-networks, and using the multiple sub-networks as a depth measurement network;
[0082] S406, performing rotation operations at different angles on the input image to obtain an augmented image set;
[0083] S407, inputting the augmented image set into a preset depth metric network for feature extraction to obtain image features corresponding to each rotated image;
[0084] S408, performing parallel operation on the image features corresponding to each rotated image to obtain the image features of the input image.
[0085] Regarding the implementation process of each step in this embodiment, please refer to the description of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.
[0086] It should be understood that although Figure 2-Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0087] In one embodiment, Figure 6 As shown, a device for extracting image features is provided, including: an acquisition module 11, a feature extraction module 12 and a determination module 13.
[0088] Specifically, the acquisition module 11 is used to acquire an augmented picture set of the input picture, where the augmented picture set includes a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0089] The feature extraction module 12 is used to input the augmented picture set into a preset deep metric network for feature extraction to obtain the picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0090] The determination module 13 is used to determine the image features of the input image according to the image features corresponding to each rotated image.
[0091] The image feature extraction device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0092] In one embodiment, the above-mentioned device also includes a training module, which is used to rotate the sample image at different angles for each sample image to obtain multiple sample rotated images, and obtain labels corresponding to the multiple sample rotated images; input the sample rotated images into the initial depth measurement network for feature extraction to obtain predicted image features; use the loss function to calculate the loss between the predicted image features and the labels corresponding to the sample rotated images, and adjust the network parameters of the initial depth measurement network according to the loss to obtain the depth measurement network.
[0093] In one embodiment, the initial depth measurement network includes multiple initial sub-networks, and the network structures of the multiple initial sub-networks are the same; the training module is specifically used to input sample rotated images with the same rotation angle into the same initial sub-network for feature extraction to obtain the predicted image features corresponding to each sample rotated image.
[0094] In one embodiment, the training module is specifically used to calculate the sub-loss between the predicted image features and the label of the input sample rotated image for each initial sub-network using a loss function; summing the sub-loss of each initial sub-network to obtain the above loss; adjusting the network parameters of each initial sub-network according to the loss to obtain multiple sub-networks, and using the multiple sub-networks as depth measurement networks.
[0095] In one embodiment, the above loss function includes a triplet loss function.
[0096] In one embodiment, the determination module 13 is specifically configured to perform parallel operations on the image features corresponding to each rotated image to obtain the image features of the input image.
[0097] In one embodiment, the acquisition module 11 is specifically used to perform rotation operations on the input image by 0°, 90°, 180° and 270° respectively to obtain rotated images with four rotation angles; and use the rotated images with four rotation angles as the augmented image set.
[0098] For the specific definition of the image feature extraction device, please refer to the definition of the image feature extraction method above, which will not be repeated here. Each module in the above-mentioned image feature extraction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0099] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 1As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for extracting image features is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0100] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0101] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0102] Obtaining an augmented picture set of the input picture, the augmented picture set including a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0103] Input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0104] The image features of the input image are determined according to the image features corresponding to each rotated image.
[0105] The implementation principle and technical effects of the computer device provided in this embodiment are similar to those of the above method embodiments, and will not be repeated here.
[0106] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0107] For each sample image, perform rotation operations on the sample image at different angles to obtain multiple sample rotated images, and obtain labels corresponding to the multiple sample rotated images respectively;
[0108] Input the sample rotation image into the initial depth metric network for feature extraction to obtain the predicted image features;
[0109] The loss function is used to calculate the loss between the predicted image features and the labels corresponding to the sample rotated images. The network parameters of the initial deep metric network are adjusted according to the loss to obtain the deep metric network.
[0110] In one embodiment, the initial depth metric network includes multiple initial sub-networks, and the network structures of the multiple initial sub-networks are the same; when the processor executes the computer program, the following steps are also implemented:
[0111] The sample rotated images with the same rotation angle are input into the same initial sub-network for feature extraction to obtain the predicted image features corresponding to each sample rotated image.
[0112] In one embodiment, when the processor executes the computer program, the following steps are also implemented:
[0113] For each initial sub-network, a loss function is used to calculate the sub-loss between the predicted image features and the label of the input sample rotation image;
[0114] The sub-loss of each initial sub-network is summed to obtain the above loss;
[0115] The network parameters of each initial sub-network are adjusted according to the loss to obtain multiple sub-networks, and the multiple sub-networks are used as depth measurement networks.
[0116] In one embodiment, the above loss function includes a triplet loss function.
[0117] In one embodiment, when the processor executes the computer program, the following steps are also implemented:
[0118] The image features corresponding to each rotated image are operated in parallel to obtain the image features of the input image.
[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented:
[0120] The input image is rotated by 0°, 90°, 180° and 270° respectively to obtain rotated images with four rotation angles;
[0121] The rotated images at four rotation angles are used as augmented image sets.
[0122] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0123] Obtaining an augmented picture set of the input picture, the augmented picture set including a plurality of rotated pictures obtained by rotating the input picture at different angles;
[0124] Input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different;
[0125] The image features of the input image are determined according to the image features corresponding to each rotated image.
[0126] The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiments, and will not be described in detail here.
[0127] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0128] For each sample image, perform rotation operations on the sample image at different angles to obtain multiple sample rotated images, and obtain labels corresponding to the multiple sample rotated images respectively;
[0129] Input the sample rotation image into the initial depth metric network for feature extraction to obtain the predicted image features;
[0130] The loss function is used to calculate the loss between the predicted image features and the labels corresponding to the sample rotated images. The network parameters of the initial deep metric network are adjusted according to the loss to obtain the deep metric network.
[0131] In one embodiment, the initial depth metric network includes multiple initial sub-networks, and the network structures of the multiple initial sub-networks are the same; when the computer program is executed by the processor, the following steps are also implemented:
[0132] The sample rotated images with the same rotation angle are input into the same initial sub-network for feature extraction to obtain the predicted image features corresponding to each sample rotated image.
[0133] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0134] For each initial sub-network, a loss function is used to calculate the sub-loss between the predicted image features and the label of the input sample rotation image;
[0135] The sub-loss of each initial sub-network is summed to obtain the above loss;
[0136] The network parameters of each initial sub-network are adjusted according to the loss to obtain multiple sub-networks, and the multiple sub-networks are used as depth measurement networks.
[0137] In one embodiment, the above loss function includes a triplet loss function.
[0138] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0139] The image features corresponding to each rotated image are operated in parallel to obtain the image features of the input image.
[0140] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:
[0141] The input image is rotated by 0°, 90°, 180° and 270° respectively to obtain rotated images with four rotation angles;
[0142] The rotated images at four rotation angles are used as augmented image sets.
[0143] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0145] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A method for extracting image features, characterized in that: The method comprises: Acquire an augmented picture set of an input picture, wherein the augmented picture set includes a plurality of rotated pictures obtained by rotating the input picture at different angles; Input the augmented picture set into a preset deep metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the deep metric network is trained according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the label of the sample rotated picture corresponding to the same sample picture is different, and the label is used to distinguish the rotation angles of different sample rotated pictures; According to the image features corresponding to each rotated image, each image feature is rotated to obtain the image features at the same angle as the original input image, and then the rotated image features are merged to obtain the image features of the input image; The training method of the depth metric network includes: For each sample image, perform rotation operations on the sample image at different angles to obtain multiple sample rotated images, and obtain labels corresponding to the multiple sample rotated images respectively; Inputting sample rotated images with the same rotation angle into the same initial sub-network for feature extraction to obtain predicted image features corresponding to each sample rotated image; the initial depth measurement network includes multiple initial sub-networks, and the network structures of the multiple initial sub-networks are the same; For each initial sub-network, a loss function is used to calculate the sub-loss between the predicted image features and the label of the input sample rotation image; Summing the sub-losses of each initial sub-network to obtain the loss; The network parameters of each initial sub-network are adjusted according to the loss to obtain multiple sub-networks, and the multiple sub-networks are used as the depth metric network.
2. The method according to claim 1, characterized in that The loss function includes a triplet loss function.
3. The method according to claim 1, characterized in that The step of determining the image features of the input image according to the image features corresponding to each rotated image includes: The image features corresponding to each rotated image are operated in parallel to obtain the image features of the input image.
4. The method according to claim 1, characterized in that: The step of obtaining an augmented picture set of an input picture includes: The input image is rotated by 0°, 90°, 180° and 270° respectively to obtain rotated images at four rotation angles; The rotated images at the four rotation angles are used as the augmented image set.
5. A device for extracting image features, characterized in that: The device comprises: An acquisition module, used to acquire an augmented picture set of an input picture, wherein the augmented picture set includes a plurality of rotated pictures obtained by rotating the input picture at different angles; A feature extraction module, used for inputting the augmented picture set into a preset depth metric network for feature extraction to obtain picture features corresponding to each rotated picture; wherein the depth metric network is obtained by training according to the sample augmented picture set and the label corresponding to each sample rotated picture in the sample augmented picture set, and the sample rotated pictures corresponding to the same sample picture have different labels, and the labels are used to distinguish different rotation angles of the sample rotated pictures; A determination module, configured to rotate each of the image features corresponding to each rotated image to obtain the image features at the same angle as the original input image, and then fuse the rotated image features to obtain the image features of the input image; The training module is used to rotate each sample image at different angles to obtain multiple sample rotated images, and obtain labels corresponding to the multiple sample rotated images; input sample rotated images with the same rotation angle into the same initial sub-network for feature extraction to obtain predicted image features corresponding to each sample rotated image; the initial depth measurement network includes multiple initial sub-networks, and the network structures of the multiple initial sub-networks are the same; for each initial sub-network, a loss function is used to calculate the sub-loss between the predicted image features and the labels of the input sample rotated images; the sub-loss of each initial sub-network is summed to obtain the loss; the network parameters of each initial sub-network are adjusted according to the loss to obtain multiple sub-networks, and the multiple sub-networks are used as the depth measurement network.
6. The device according to claim 5, characterized in that The loss function includes the triplet loss function.
7. The device according to claim 5, characterized in that The acquisition module is specifically used for: performing rotation operations on the input image by 0°, 90°, 180° and 270° respectively to obtain rotated images at four rotation angles; and using the rotated images at the four rotation angles as the augmented image set.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.