A feature detection method, device, computer equipment and storage medium

Abstract

The present disclosure provides a feature detection method, device, computer equipment and storage medium, wherein the method comprises: acquiring a target image; extracting a basic image feature of the target image; determining a local feature point of the target image, local feature information corresponding to the local feature point and global feature information of the target image based on the basic image feature. The embodiment of the present disclosure can overcome the problem of large time consumption caused by step-by-step acquisition of the above three kinds of feature information by synchronously outputting the local feature point of the target image, the local feature information corresponding to the local feature point and the global feature information, thereby improving the efficiency of feature detection. In addition, the preset queue is used to iteratively train the feature detection neural network, so that the local feature point, the local feature information corresponding to the local feature point and the global feature information output by the trained feature detection neural network have high accuracy.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of deep learning, and in particular, to a feature detection method and device, a computer device and a storage medium. BACKGROUND

[0002] Local feature points, local feature information and global feature information of a target image play an important role in image retrieval, visual positioning and other scenarios. In the prior art, local feature points, local feature information and global feature information of a target image need to be extracted in three steps using three different detection neural networks.

[0003] The above-mentioned three-step extraction method consumes a large amount of time and is inefficient. SUMMARY

[0004] The present disclosure provides at least a feature detection method and device, a computer device and a storage medium.

[0005] In a first aspect, the present disclosure provides a feature detection method, comprising:

[0006] obtaining a target image;

[0007] extracting basic image features of the target image;

[0008] determining local feature points of the target image, local feature information corresponding to the local feature points and global feature information of the target image based on the basic image features.

[0009] Here, one neural network is used to first extract basic image features, and then based on the extracted basic image features, local feature points, local feature information and global feature information are simultaneously determined, avoiding the separate extraction of the above-mentioned three types of information, effectively improving the information extraction efficiency. Since the basic image features are the feature information required for extracting the above-mentioned three types of feature information, the acquisition of the above-mentioned three types of feature information is realized by using the basic image features extracted once, which can avoid the repeated extraction of basic image features when acquiring each of the above-mentioned feature information, reducing the amount of calculation in information extraction.

[0010] In a possible implementation, the determining of the local feature points of the target image, the local feature information corresponding to the local feature points and the global feature information of the target image based on the basic image features comprises:

[0011] using a feature detection neural network to output the local feature points of the target image, the local feature information corresponding to the local feature points and the global feature information of the target image based on the basic image features.

[0012] The local feature points, the local feature information corresponding to the local feature points, and the global feature information of the target image can be extracted by using one feature detection neural network, without using multiple detection neural networks to implement respectively, not only the structure optimization of the feature detection neural network is realized, but also the synchronous output of the above three kinds of feature information is realized, and the information detection efficiency is improved.

[0013] In a possible implementation, the feature detection neural network is trained, including:

[0014] A plurality of sample images are acquired;

[0015] For each sample image, at least one sample derived image having a co-view relationship with the sample image is determined, and the sample image is combined with each sample derived image respectively to obtain at least one sample image pair;

[0016] The feature detection neural network is trained by using the sample image pair.

[0017] The co-view relationship can reflect the change of the view angle of the camera when two images are captured, and by taking the sample image pair including two images having a co-view relationship as the sample data for training the feature detection neural network, the ability of the trained feature detection neural network to adapt to the change of the view angle of the camera when performing feature detection can be improved, thereby improving the precision and reliability of the three kinds of information output by the trained feature detection neural network.

[0018] In a possible implementation, the sample images include first sample scene images, and the sample image pair includes a first sample scene image pair.

[0019] The determination of at least one sample derived image having a co-view relationship with the sample image for each sample image, and the combination of the sample image with each sample derived image to obtain at least one sample image pair, includes:

[0020] A plurality of first sample scene images under a target scene are acquired;

[0021] Based on the plurality of first sample scene images, three-dimensional scene information corresponding to the target scene is determined;

[0022] Based on the three-dimensional scene information, at least one pair of first sample scene images having a co-view relationship is selected from the first sample scene images to obtain at least one first sample scene image pair.

[0023] With the determined three-dimensional scene information, the correspondence relationship between multiple first sample scene images can be accurately determined, and then multiple first sample scene image pairs with the correspondence relationship can be accurately determined. Moreover, both images included in the first sample scene image pair are images under the target scene, and the feature detection neural network is trained by using the multiple first sample scene image pairs, which can not only improve the ability of the trained feature detection neural network to adapt to the perspective change of the camera when performing feature detection, but also improve the adaptability of the trained feature detection neural network under the specific target scene, thereby improving the detection accuracy under the specific target scene.

[0024] In a possible implementation, the sample images include second sample scene images, and the sample image pairs include second sample scene image pairs.

[0025] The determining, for each sample image, at least one sample derived image having a correspondence relationship with the sample image, and combining the sample image with each sample derived image to obtain at least one sample image pair includes:

[0026] Obtaining multiple second sample scene images under a non-target scene;

[0027] For each second sample scene image, performing a first preprocessing operation on the second sample scene image to obtain at least one target derived image having a correspondence relationship with the second sample scene image, and combining the second sample scene image with each target derived image to obtain at least one second sample scene image pair.

[0028] Through the first preprocessing operation, the target derived image having a correspondence relationship with the second sample scene image can be obtained, and the second sample scene image and the target derived image are both images under the non-target scene. Therefore, the feature detection neural network is trained by using the second sample scene image pair composed of the second sample scene image and the target derived image, which can not only improve the ability of the trained feature detection neural network to adapt to the perspective change of the camera when performing feature detection, but also improve the adaptability of the trained feature detection neural network under various scenes, thereby improving the prediction accuracy of the feature detection neural network under various scenes.

[0029] In a possible implementation, the training, by using the sample image pairs, the feature detection neural network includes:

[0030] Performing a second preprocessing operation on at least part of the sample image pairs to obtain at least one preprocessed sample image pair; the sample image pairs include first sample image pairs and / or second sample image pairs;

[0031] The feature detection neural network is trained by using the at least part of the sample image pairs and the preprocessed sample image pairs.

[0032] The images in the preprocessed sample image pairs after the second preprocessing operation are different from the images in the original sample image pairs, for example, having different brightness, noise, and other information, so that the feature detection neural network trained by using the sample image pairs and the preprocessed sample image pairs as sample data can have better adaptability to small distortions of images, for example, better adaptability to changes in brightness and noise of images, and the reliability of the detection information output by the feature detection neural network is improved.

[0033] In a possible implementation, the second preprocessing operation includes at least one of the following: a random noise adding operation and a brightness changing operation.

[0034] The random noise adding operation can change the noise information of the images in the sample image pairs, and the brightness changing operation can change the brightness information of the images in the sample image pairs. The feature detection neural network trained by using the images in the preprocessed sample image pairs after the second preprocessing operation can have better adaptability to changes in brightness and noise of images, and the reliability of the detection information output by the feature detection neural network is improved.

[0035] In a possible implementation, the training of the feature detection neural network by using the sample image pairs includes:

[0036] Local labeled feature points are determined for each sample image in the sample image pairs.

[0037] Positive sample feature points and negative sample feature points corresponding to the local labeled feature points are screened from sample-derived images in the sample image pairs.

[0038] The feature detection neural network is trained by using the local labeled feature points, the positive sample feature points, and the negative sample feature points corresponding to the local labeled feature points.

[0039] The feature detection neural network is trained by using positive and negative samples, so that the prediction accuracy of the feature detection neural network in predicting local feature information of images is improved.

[0040] In a possible implementation, the screening of the positive sample feature points and the negative sample feature points corresponding to the local labeled feature points from the sample-derived images in the sample image pairs includes:

[0041] corresponding to the local labeled feature point in the sample-derived image as a positive sample feature point;

[0042] From the feature points in the sample-derived image other than the positive sample feature points, a feature point with the highest similarity to the local labeled feature point is screened, and the screened feature point with the highest similarity is taken as the negative sample feature point.

[0043] The training modes of the positive sample feature points and the negative sample feature points are different. The training mode of the positive sample feature point is to narrow the similarity between the original sample and the positive sample. There is a positive sample overlapping with the original sample in the sample-derived image having a co-view relationship with the sample image. Therefore, the local feature information of the positive sample feature point selected from the sample-derived image is closest to the local feature information corresponding to the local labeled feature point. The positive sample feature point selected from the sample-derived image can better achieve the training effect of the positive sample feature point. The training mode of the negative sample feature point is to widen the similarity between the original sample and the negative sample. Therefore, the feature point with the highest similarity to the local labeled feature point is selected as the negative sample feature point. The local feature information of the selected negative sample feature point is also closest to the local feature information corresponding to the local labeled feature point. Therefore, by widening the similarity between the original sample and the negative sample, the training effect of the negative sample feature point can be better achieved, thereby improving the prediction accuracy of the feature detection neural network in predicting local feature information.

[0044] In a possible implementation, the training of the feature detection neural network by using the local labeled feature point, the positive sample feature point corresponding to the local labeled feature point, and the negative sample feature point comprises:

[0045] inputting the sample image pair into the feature detection neural network to obtain a local predicted feature point, first predicted feature information corresponding to the local labeled feature point, second predicted feature information corresponding to the positive sample feature point, and third predicted feature information corresponding to the negative sample feature point;

[0046] generating a first loss by using the local predicted feature point and the local labeled feature point;

[0047] generating a second loss by using the first predicted feature information corresponding to the local labeled feature point, the second predicted feature information corresponding to the positive sample feature point, and the third predicted feature information corresponding to the negative sample feature point;

[0048] training the feature detection neural network by using the first loss and the second loss.

[0049] The first loss is a loss between the local predicted feature points and the local labeled feature points. Training the feature detection neural network by using the first loss can improve the prediction accuracy of the feature detection neural network in predicting local feature points. The second loss is a loss between the first predicted feature information, the second predicted feature information and the third predicted feature information of the local feature information. Training the feature detection neural network by using the second loss can improve the prediction accuracy of the feature detection neural network in predicting local feature information.

[0050] In a possible implementation, the inputting the sample image pair into the feature detection neural network to obtain the local predicted feature points, the first predicted feature information corresponding to the local labeled feature points, the second predicted feature information corresponding to the positive sample feature points and the third predicted feature information corresponding to the negative sample feature points comprises:

[0051] The sample image pair is input into the feature detection neural network, and the predicted basic image features corresponding to the sample image pair are obtained by a basic feature extraction branch in the feature detection neural network.

[0052] The local predicted feature points, the first predicted feature information corresponding to the local labeled feature points, the second predicted feature information corresponding to the positive sample feature points and the third predicted feature information corresponding to the negative sample feature points are obtained by a feature prediction branch in the feature detection neural network based on the predicted basic image features.

[0053] Through the training of the basic feature extraction branch, the accuracy of the predicted basic image features output by the basic feature extraction branch can be improved. Therefore, the feature prediction branch can output the local predicted feature points, the first predicted feature information, the second predicted feature information and the third predicted feature information with higher accuracy by using the predicted basic image features with higher accuracy. Then, the feature prediction branch is trained by using the local predicted feature points, the first predicted feature information, the second predicted feature information and the third predicted feature information, and finally the prediction accuracy of the trained feature detection neural network can be improved.

[0054] In a possible implementation, the training of the feature detection neural network by using the local labeled feature points, the positive sample feature points corresponding to the local labeled feature points and the negative sample feature points comprises:

[0055] The plurality of first sample image pairs are input into the feature detection neural network to obtain the global fourth predicted feature information corresponding to each image in each first sample image pair, and the global fourth predicted feature information is stored in a preset queue.

[0056] The feature detection neural network is trained based on the plurality of second sample image pairs and the fourth predicted feature information in the preset queue.

[0057] The fourth predicted feature information obtained by using the first sample image pair is stored in the preset queue as a negative sample, which can increase the number of negative samples for training and improve the accuracy of the feature detection neural network in predicting global feature information.

[0058] In a possible implementation, the training of the feature detection neural network based on the plurality of second sample image pairs and the fourth predicted feature information in the preset queue comprises:

[0059] The plurality of second sample image pairs are input into the feature detection neural network to obtain fifth predicted feature information of each image in each second sample image pair.

[0060] The third loss is generated based on the fourth predicted feature information in the preset queue and the fifth predicted feature information, and the feature detection neural network is trained by using the third loss.

[0061] The third loss is the loss between the fifth predicted feature information and the fourth predicted feature information, and the loss corresponding to the global feature information is established. The feature detection neural network is trained by using the third loss, which can improve the accuracy of the feature detection neural network in predicting global feature information.

[0062] In a possible implementation, the fifth predicted feature information comprises sixth predicted feature information and seventh predicted feature information.

[0063] The third loss is generated based on the fourth predicted feature information in the preset queue and the fifth predicted feature information, and the feature detection neural network is trained by using the third loss.

[0064] For each second sample image pair, the sixth predicted feature information of the sample image in the second sample image pair is determined, the sample derived image in the second sample image pair is determined as the positive sample image corresponding to the sample image, and the seventh predicted feature information of the positive sample image is determined.

[0065] The fourth predicted feature information with the highest similarity to the sixth predicted feature information of the sample image in the second sample image pair is screened from the preset queue, and the screened fourth predicted feature information is determined as the eighth predicted feature information corresponding to the negative sample image.

[0066] The third loss is generated by using the sixth predicted feature information, the seventh predicted feature information, and the eighth predicted feature information.

[0067] The sixth prediction feature information and the seventh prediction feature information are used to determine the loss corresponding to the positive sample pairs between the global feature information, the fourth prediction feature information with the highest similarity to the sixth prediction feature information of the sample image is used to determine the loss corresponding to the negative sample pairs between the global feature information, and then the determined loss corresponding to the positive sample pairs and the loss corresponding to the negative sample pairs are used to improve the rationality of the third loss.

[0068] In a possible implementation, after obtaining the global fifth prediction feature information of each image in each second sample image pair respectively, the method further includes:

[0069] Updating the preset queue based on the global fifth prediction feature information of each image.

[0070] The queue is updated by using the newly generated fifth prediction feature information, that is, the update of the negative sample can be realized, the accuracy of the global fifth prediction feature information generated in each round of training is higher, and the rationality of the obtained third loss can be further improved by using the negative sample with higher accuracy.

[0071] In a second aspect, the embodiments of the present disclosure further provide a feature detection device, including:

[0072] An acquisition module is configured to acquire a target image.

[0073] An extraction module is configured to extract a basic image feature of the target image.

[0074] A determination module is configured to determine, based on the basic image feature, a local feature point of the target image, local feature information corresponding to the local feature point, and global feature information of the target image.

[0075] In a possible implementation, the determination module is configured to output, based on the basic image feature, the local feature point of the target image, the local feature information corresponding to the local feature point, and the global feature information of the target image by using a feature detection neural network.

[0076] In a possible implementation, the device further includes a training module configured to train the feature detection neural network.

[0077] The training module is configured to acquire a plurality of sample images.

[0078] For each sample image, at least one sample derived image having a co-view relationship with the sample image is determined, and the sample image is combined with each sample derived image respectively to obtain at least one sample image pair.

[0079] The feature detection neural network is trained by using the sample image pair.

[0080] In a possible implementation, the sample images include first sample scene images, and the sample image pairs include first sample scene image pairs.

[0081] The training module is configured to acquire a plurality of first sample scene images under a target scene.

[0082] The training module is configured to determine three-dimensional scene information corresponding to the target scene based on the plurality of first sample scene images.

[0083] The training module is configured to filter at least one pair of first sample scene images having a co-view relationship from the first sample scene images based on the three-dimensional scene information, to obtain at least one first sample scene image pair.

[0084] In a possible implementation, the sample images include second sample scene images, and the sample image pairs include second sample scene image pairs.

[0085] The training module is configured to acquire a plurality of second sample scene images under a non-target scene.

[0086] The training module is configured to perform a first preprocessing operation on each second sample scene image to obtain at least one target derived image having a co-view relationship with the second sample scene image, and to combine the second sample scene image with each target derived image respectively to obtain at least one second sample scene image pair.

[0087] In a possible implementation, the training module is configured to perform a second preprocessing operation on at least part of the sample image pairs to obtain at least one preprocessed sample image pair; the sample image pairs include first sample image pairs and / or second sample image pairs.

[0088] The training module is configured to train the feature detection neural network by using the at least part of the sample image pairs and the preprocessed sample image pairs.

[0089] In a possible implementation, the second preprocessing operation includes at least one of the following: a random noise adding operation, an image brightness changing operation.

[0090] In a possible implementation, the training module is configured to determine local labeled feature points of each sample image in the sample image pairs respectively.

[0091] The training module is configured to filter positive sample feature points and negative sample feature points corresponding to the local labeled feature points from sample derived images in the sample image pairs.

[0092] The feature detection neural network is trained by using the local labeled feature point, the positive sample feature point corresponding to the local labeled feature point, and the negative sample feature point.

[0093] In a possible implementation, the training module is configured to take the feature point corresponding to the local labeled feature point in the sample derived image as a positive sample feature point.

[0094] The feature point with the highest similarity to the local labeled feature point is screened from the feature points other than the positive sample feature point in the sample derived image, and the feature point with the highest similarity screened is taken as a negative sample feature point.

[0095] In a possible implementation, the training module is configured to input the sample image pair into the feature detection neural network to obtain a local predicted feature point, first predicted feature information corresponding to the local labeled feature point, second predicted feature information corresponding to the positive sample feature point, and third predicted feature information corresponding to the negative sample feature point.

[0096] A first loss is generated by using the local predicted feature point and the local labeled feature point.

[0097] A second loss is generated by using the first predicted feature information corresponding to the local labeled feature point, the second predicted feature information corresponding to the positive sample feature point, and the third predicted feature information corresponding to the negative sample feature point.

[0098] The feature detection neural network is trained by using the first loss and the second loss.

[0099] In a possible implementation, the training module is configured to input the sample image pair into the feature detection neural network, and obtain a predicted basic image feature corresponding to the sample image pair by a basic feature extraction branch in the feature detection neural network.

[0100] The local predicted feature point, the first predicted feature information corresponding to the local labeled feature point, the second predicted feature information corresponding to the positive sample feature point, and the third predicted feature information corresponding to the negative sample feature point are obtained by a feature prediction branch in the feature detection neural network based on the predicted basic image feature.

[0101] In a possible implementation, the training module is configured to input a plurality of first sample image pairs into the feature detection neural network to respectively obtain global fourth predicted feature information corresponding to each image in each of the first sample image pairs, and store the global fourth predicted feature information in a preset queue.

[0102] The feature detection neural network is trained based on the fourth predicted feature information in the preset queue and the second sample image pairs.

[0103] In a possible implementation, the training module is configured to input the second sample image pairs into the feature detection neural network to obtain fifth predicted feature information of each image in each second sample image pair.

[0104] The third loss is generated based on the fourth predicted feature information in the preset queue and the fifth predicted feature information, and the feature detection neural network is trained by using the third loss.

[0105] In a possible implementation, the training module is configured to determine sixth predicted feature information of a sample image in each second sample image pair, and determine seventh predicted feature information of a positive sample image corresponding to the sample image, by taking a sample derived image in the second sample image pair as the positive sample image.

[0106] The fourth predicted feature information with the highest similarity to the sixth predicted feature information of the sample image in the second sample image pair is filtered from the preset queue, and the filtered fourth predicted feature information is taken as eighth predicted feature information corresponding to a negative sample image.

[0107] The third loss is generated by using the sixth predicted feature information, the seventh predicted feature information, and the eighth predicted feature information.

[0108] In a possible implementation, the training module is further configured to update the preset queue based on the fifth predicted feature information of each image after obtaining the fifth predicted feature information of each image in each second sample image pair.

[0109] In a third aspect, an optional implementation of the present disclosure further provides a computer device, a processor and a memory, the memory stores machine readable instructions executable by the processor, and the processor is configured to execute the machine readable instructions stored in the memory, and the machine readable instructions are executed by the processor to execute the steps of the first aspect or any possible implementation of the first aspect.

[0110] In a fourth aspect, an optional implementation of the present disclosure further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed to execute the steps of the first aspect or any possible implementation of the first aspect.

[0111] Effects of the feature detection apparatus, the computer device, and the computer-readable storage medium described above can be seen from the description of the feature detection method described above, and thus will not be repeated here.

[0112] In order to make the above objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are used for detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0113] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the description and form a part of the description, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the description. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0114] Figure 1 A flow chart of a feature detection method provided by an embodiment of the present disclosure is shown;

[0115] Figure 2 A schematic diagram of a feature detection neural network outputting local feature points of a target image, local feature information corresponding to the local feature points, and global feature information is shown;

[0116] Figure 3 A flow chart of a method for training a feature detection neural network provided by an embodiment of the present disclosure is shown;

[0117] Figure 4 A schematic diagram of iteratively training a feature detection neural network based on a queue is shown;

[0118] Figure 5 A schematic diagram of a feature detection apparatus provided by an embodiment of the present disclosure is shown;

[0119] Figure 6 A structural schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0120] To make the purposes, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings in the embodiments of the present disclosure to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The components of the embodiments of the present disclosure described and shown herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present disclosure.

[0121] In addition, the terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0122] "Multiple or several" mentioned herein refers to two or more than two. "And / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. The character " / " generally represents that the front and rear associated objects are in an "or" relationship.

[0123] It is found through research that the local feature points, local feature information, and global feature information of a target image play an important role in image retrieval, visual positioning, and the like. In the prior art, the local feature points, local feature information, and global feature information of a target image need to be extracted in steps by using three different detection neural networks. The above-mentioned step-by-step extraction method will consume a large amount of time and is low in information extraction efficiency.

[0124] Based on the above research, the present disclosure provides a feature detection method and device, computer equipment, and storage medium, which first extracts basic image features by using one neural network, and then based on the extracted basic image features, can simultaneously determine the local feature points, local feature information, and global feature information, avoiding the separate extraction of the above-mentioned three kinds of information, effectively improving the information extraction efficiency, and since the basic image features are the feature information needed for extracting the above-mentioned three kinds of feature information, the acquisition of the above-mentioned three kinds of feature information is realized by sharing the basic image features extracted once, which can avoid the problem of repeated extraction of basic image features caused by extracting basic image features for acquiring each of the above-mentioned feature information, and reduce the amount of calculation in information extraction.

[0125] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0126] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0127] To facilitate understanding of this embodiment, a feature detection method disclosed in this disclosure will first be described in detail. The execution subject of the feature detection method provided in this disclosure is generally a computer device with certain computing power. In some possible implementations, the feature detection method can be implemented by the processor calling computer-readable instructions stored in the memory.

[0128] The feature detection method provided in this disclosure is illustrated below using a computer device as an example.

[0129] like Figure 1 The diagram shown is a flowchart of a feature detection method provided in an embodiment of this disclosure, including the following steps:

[0130] S101: Acquire the target image.

[0131] Here, the target image can be an image of the target scene captured by the camera device, or it can be an image of a non-target scene. When there are local feature points in the target image, the local feature points can be the key points with the most obvious features within a preset range at the corresponding pixel points in the target image.

[0132] Local feature points can reflect the essential features of a local part of a target image and can be used to identify target objects in the target image. By matching the local feature points of the target image with the local feature points of other images, it is possible to match the target image with other images.

[0133] S102: Extract the basic image features of the target image.

[0134] Here, based on the acquired target image, the basic image features of the target image can be extracted. These basic image features are image features that are needed when determining the local feature points of the target image, the local feature information corresponding to the local feature points, and the global feature information of the target image.

[0135] S103: Based on the basic image features, determine the local feature points of the target image, the local feature information corresponding to the local feature points, and the global feature information of the target image.

[0136] Here, the local feature information is feature information used to describe the overall feature in a preset range at a pixel point corresponding to a local feature point, and one local feature information can exist for each local feature point. The global feature information is feature information used to describe the overall feature of the target image, and one unique global image feature exists for one target image.

[0137] Here, the local feature information and the global feature information can be represented by a multi-dimensional feature vector.

[0138] In this step, after obtaining the basic image feature, the basic image feature and the target image can be used to determine the local feature points of the target image, and each local feature point of the target image and the local feature information corresponding to each local feature point can be determined simultaneously. In the case of determining the local feature points of the target image, at least one local feature point can exist in one target image, i.e., at least one local feature information can exist for one target image. In addition, based on the basic image feature and the target image, the global feature information of the target image can also be determined simultaneously.

[0139] In specific implementation, the determination of each local feature information needs to rely on the basic image feature, the target image, and the local feature point corresponding to the local feature information. The local feature information can also be referred to as a local descriptor, and the global feature information can also be referred to as a global descriptor.

[0140] In this way, the basic image feature is first extracted by using one neural network, and then the local feature points, the local feature information, and the global feature information can be determined simultaneously based on the extracted basic image feature, which avoids the separate extraction of the above three kinds of information, effectively improves the information extraction efficiency, and avoids the repeated extraction of the basic image feature when extracting each of the above feature information, thereby reducing the calculation amount in information extraction.

[0141] In one embodiment, for S103, a feature detection neural network can be used to output the local feature points of the target image, the local feature information corresponding to the local feature points, and the global feature information of the target image based on the basic image feature.

[0142] Here, the feature detection method provided by the embodiment of the present disclosure can be applied in a feature detection neural network, and S101-S103 can all be performed by using the feature detection neural network. Specifically, the feature detection neural network can be used to process the obtained target image, output the basic image features of the target image, and then use the basic image features and the target image to output the local feature points of the target image, the local feature information corresponding to the local feature points, and the global feature information of the target image.

[0143] In a specific implementation, the feature detection neural network can include a basic feature extraction branch and a feature prediction branch, and the feature prediction branch can include a local feature point detection branch, a local feature information detection branch, and a global feature information detection branch. As shown in Figure 2 FIG. 1 is a schematic diagram of a feature detection neural network outputting local feature points, local feature information corresponding to the local feature points, and global feature information of a target image provided by an embodiment of the present disclosure. The basic feature extraction branch is used to extract the basic image features of the target image, the local feature point detection branch is used to output the local feature points of the target image, the local feature information detection branch is used to output the local feature information corresponding to the local feature points, and the global feature information detection branch is used to output the global feature information of the target image.

[0144] In addition, the feature detection neural network needs to be trained before it can output prediction information with high reliability and accuracy. Therefore, the embodiment of the present disclosure further provides a method for training the feature detection neural network, as shown in Figure 3 FIG. 2 is a flowchart of a method for training a feature detection neural network provided by an embodiment of the present disclosure, which can include the following steps:

[0145] S301: Obtain a plurality of sample images.

[0146] S302: For each sample image, determine at least one sample derived image having a co-visibility relationship with the sample image, and combine the sample image with each sample derived image respectively to obtain at least one sample image pair.

[0147] The sample image pair having the co-visibility relationship includes an image pair having partially overlapping pixel points.

[0148] S303: Train the feature detection neural network by using the sample image pair.

[0149] Here, the sample image can be an image in any scene, and the epipolar relationship can reflect the change in the perspective of the camera when the two images are captured, as well as the number and position of overlapping pixels between the two images. The trained feature detection neural network can output local feature points of the sample image, local feature information corresponding to the local feature points, and global feature information of the target image based on processing of the sample image, with high precision and reliability.

[0150] After the sample image pairs are screened, each sample image pair in the sample image pairs can be input to the feature detection neural network, and then based on the output result of the feature detection neural network, a loss for iterative training of the feature detection neural network is constructed, and the feature detection neural network is iteratively trained using the constructed loss to obtain the trained feature detection neural network. Here, the sample image pairs include multiple.

[0151] In addition, after obtaining the multiple sample image pairs, at least part of the sample image pairs can be subjected to a second preprocessing operation to obtain at least one preprocessed sample image pair. The second preprocessing operation can include at least one of the following: a random noise adding operation, an image brightness changing operation.

[0152] In specific implementation, at least part of the sample image pairs can be screened from the multiple sample image pairs, and for the screened at least part of the sample image pairs, both the random noise adding operation and the image brightness changing operation can be performed on each sample image pair in the at least part of the sample image pairs, or the random noise adding operation can be performed on part of the sample image pairs in the at least part of the sample image pairs, and the image brightness changing operation can be performed on part of the sample image pairs in the at least part of the sample image pairs. Here, the sample image pairs subjected to the random noise adding operation and the sample image pairs subjected to the image brightness changing operation can partially overlap, and then multiple preprocessed sample image pairs can be obtained. Subsequently, the obtained multiple sample image pairs and the preprocessed sample image pairs can be used to iteratively train the feature detection neural network.

[0153] In this way, the images in the preprocessed sample image pairs subjected to the second preprocessing operation are different from the images in the original sample image pairs, for example, having different brightness, noise, and other information, so that using the sample image pairs and the preprocessed sample image pairs as sample data can improve the adaptability of the trained feature detection neural network to some small distortions of images, for example, improving the adaptability of the trained feature detection neural network to changes in brightness and noise of images, and improving the reliability of the detection information output by the feature detection neural network.

[0154] In an embodiment, the sample image can include a first sample scene image, and the sample image pair can include a first sample scene image pair.

[0155] For step S302, the sample image pair can be determined as follows:

[0156] Step one, obtaining a plurality of first sample scene images under a target scene;

[0157] Step two, determining three-dimensional scene information corresponding to the target scene based on the plurality of first sample scene images;

[0158] Step three, based on the three-dimensional scene information, at least one pair of first sample scene images having a co-view relationship is selected from the first sample scene images to obtain at least one first sample scene image pair.

[0159] Here, in order to improve the prediction accuracy of the trained feature detection neural network for images under the target scene, the feature detection neural network can be trained using images under the target scene.

[0160] The first sample scene image can be an image obtained under the target scene, and the first sample scene image pair includes two first sample scene images having a co-view relationship selected from the plurality of first sample scene images.

[0161] In specific implementation, after obtaining a plurality of different sample scene images, the target scene corresponding to the sample scene images can be three-dimensionally modeled based on the plurality of sample scene images using a motion reconstruction method to determine the three-dimensional scene information corresponding to the target scene. Then, for each first sample scene image in the plurality of first sample scene images, other first sample scene images having a co-view relationship with the first sample scene image can be selected from the plurality of first sample scene images using the three-dimensional scene information. Then, each of the other first sample scene images having a co-view relationship with the first sample scene image can be combined with the first sample scene image to obtain a first sample scene image pair corresponding to the first sample scene image. Then, the first sample scene image pair corresponding to each first sample scene image in the plurality of first sample scene images can be determined. Here, at least one pair of first sample scene image pairs can be determined.

[0162] Afterwards, the determined first sample scene image pair can be used to iteratively train the feature detection neural network to obtain a trained feature detection neural network. Here, since the two first sample scene images in the first sample scene image pair used for training not only have a co-view relationship, but also are images under a specific target scene, training the feature detection neural network using multiple first sample scene image pairs can not only improve the ability of the trained feature detection neural network to adapt to changes in the view angle of the camera when performing feature detection, but also enable the trained feature detection neural network to adapt to a specific target scene, thereby improving detection accuracy under the specific target scene.

[0163] In another embodiment, the sample images can also include second sample scene images, and similarly, the sample image pairs can also include second sample scene image pairs. In implementation, the second sample scene images are images under a non-target scene. After obtaining multiple second sample scene images, a first preprocessing operation can be performed on each second sample scene image, and then at least one target derived image having a co-view relationship with the second sample scene image can be determined. Then, the second sample scene image and each target derived image in the determined at least one target derived image are combined to obtain at least one second sample scene image pair corresponding to the second sample scene image. Further, based on the first preprocessing operation, a second sample scene image pair corresponding to each second sample scene image can be determined.

[0164] Here, the first preprocessing operation can be an affine transformation operation, which is an operation simulating translation, rotation, motion in distance, etc. of a camera device used to capture the second sample scene image. Based on the affine transformation operation, at least one target derived image having a co-view relationship with the second sample scene image can be determined.

[0165] After the second sample scene image pairs are determined, the first sample scene image pairs and the second sample scene image pairs can be used together to iteratively train the feature detection neural network. In implementation, a preset ratio can be used to determine a first number of first sample scene image pairs and a second number of second sample scene image pairs used for training, and then the first number of first sample scene image pairs and the second number of second sample scene image pairs are selected together to iteratively train the feature detection neural network.

[0166] In one embodiment, the preset ratio can be a ratio of 6:4 between the number of first sample scene image pairs and the number of second sample scene image pairs.

[0167] In this way, the first sample scene image pair and the second sample scene image pair are used together to train the feature detection neural network, which can improve the detection accuracy of the trained feature detection neural network in a specific target scene, and also improve the generalization ability of the trained feature detection neural network in other scenes, thereby improving the detection accuracy in other scenes.

[0168] In addition, for the selected first number of first sample scene image pairs and the second number of second sample scene image pairs, at least part of the first sample scene image pairs and / or the second sample scene image pairs can be selected to perform a second preprocessing operation to obtain preprocessed first sample scene image pairs and / or preprocessed second sample scene image pairs, and then the first sample scene image pairs and / or the second sample scene image pairs, the preprocessed first sample scene image pairs and / or the preprocessed second sample scene image pairs can be used together to iteratively train the feature detection neural network.

[0169] In this way, the trained feature detection network can adapt to changes in the angle of view of the camera device, changes in image brightness and noise changes, and can also have reliable prediction accuracy in a specific target scene and other non-target scenes.

[0170] Further, for S303, after determining the sample image pairs, the Homographic Adaptation (Homographic Adaptation) method can be used to determine the local feature points included in the images in each sample image pair, and then the local feature points included in the sample images in the determined sample image pairs can be used as the local labeled feature points corresponding to the sample image pairs. In addition, the positive sample feature points and negative sample feature points corresponding to the determined local labeled feature points can be selected from the sample derived images in the sample image pairs, and then the local labeled feature points, the positive sample feature points and the negative sample feature points can be used to iteratively train the feature detection neural network to obtain the trained feature detection neural network.

[0171] In addition, each sample image in the sample image pair can include a plurality of local feature points, i.e. the positive sample feature points and negative sample feature points corresponding to each local feature labeled point in the plurality of local feature points can be determined from the sample derived image corresponding to the sample image pair. In addition, each local feature labeled point and its corresponding positive sample feature points and negative sample feature points can be used to iteratively train the feature detection neural network.

[0172] In an embodiment, for determining the positive sample feature point and the negative sample feature point corresponding to any one of the local annotated feature points included in the sample image of the sample image pair, a local feature point corresponding to the local annotated feature point can be determined from the sample derived image and taken as the positive sample feature point corresponding to the local annotated feature point. Then, a negative sample feature point corresponding to the local annotated feature point can be determined from the local feature points included in the sample derived image except for the positive sample feature point. In a specific implementation, for each of the local feature points included in the sample derived image except for the positive sample feature point, a similarity between each of the local feature points and the local annotated feature point can be determined, and then the local feature point with the highest similarity can be selected as the negative sample feature point.

[0173] Further, after determining the local annotated feature points, the positive sample feature points and the negative sample feature points included in the sample image pair, the sample image pair can be input into the feature detection neural network. For each of the local annotated feature points and the corresponding positive sample feature points and negative sample feature points included in the sample image pair, the feature detection neural network can output, based on processing of the images in the sample image pair, the local predicted feature point corresponding to the local annotated feature point, the first predicted feature information corresponding to the local annotated feature point, the second predicted feature information of the positive sample feature point corresponding to the local annotated feature point, and the third predicted feature information of the negative sample feature point corresponding to the local annotated feature point. The first predicted feature information, the second predicted feature information and the third predicted feature information are all the output local feature information.

[0174] Then, the first loss for local feature point detection can be constructed by using the local annotated feature point and the local predicted feature point corresponding thereto, taking the local annotated feature point as the true value, and the local feature point detection branch can be iteratively trained by using the first loss. In this way, the local feature point detection branch in the trained feature detection neural network can output local predicted feature points with higher accuracy.

[0175] Further, the second loss can be generated by using the first predicted feature information, the second predicted feature information and the third predicted feature information corresponding to the local annotated feature point, and the second loss can be a hard sample mining loss function.

[0176] After that, the local feature information detection branch can be iteratively trained by using the second loss, so that the local feature information detection branch in the trained feature detection neural network can output local predicted feature information with higher accuracy.

[0177] Here, the first loss and the second loss can be generated synchronously, and then the feature detection neural network can be iteratively trained by using the first loss and the second loss.

[0178] The feature detection neural network is iteratively trained to obtain a trained feature detection neural network.

[0179] In addition, the feature detection method provided by the embodiment of the present disclosure can also be used to synchronously train the global feature information detection branch.

[0180] In a specific implementation, the plurality of first sample image pairs can be image pairs selected from the plurality of determined sample image pairs, and then, for each of the plurality of input first sample image pairs, the base feature extraction branch in the feature detection neural network can extract the predicted base feature of the image included in the first sample image pair, the local feature point detection branch in the feature detection neural network can determine the local predicted feature point corresponding to each image in the first sample image pair based on the determined predicted base feature and the image included in the first sample image pair, the local feature information detection branch in the feature detection neural network can output the first predicted feature information, the second predicted feature information and the third predicted feature information corresponding to each image in the first sample image pair based on the local predicted feature point corresponding to the first sample image pair, the predicted base feature and the image included in the first sample image pair, and the global feature information detection branch in the feature detection neural network can output the global fourth predicted feature information corresponding to each image in the first sample image pair based on the predicted base feature and the image included in the first sample image pair. The fourth predicted feature information is the output global feature information.

[0181] Further, the second loss and the third loss can be constructed by using the local predicted feature point, the first predicted feature information, the second predicted feature information and the third predicted feature information, the local feature point detection branch and the local feature information detection branch can be iteratively trained, and the third loss can be constructed by using the fourth predicted feature information, and the global feature information detection branch can be synchronously iteratively trained.

[0182] It should be noted that in the process of iteratively training the global feature information detection branch by using the sample image pairs, for any sample image, the corresponding positive sample image can be an image having a co-view relationship with the sample image, and the negative sample image is all images that do not have a co-view relationship with the current image, so for each input sample image pair, the sample image and the sample derived image included therein have a co-view relationship, so for the sample image included in each input sample image pair, there is only a positive sample image and no negative sample image, but in order to improve the detection accuracy of the global predicted feature information output by the trained global feature information detection branch, the embodiment of the present disclosure proposes a queue-based training method, which stores the global predicted feature information generated by each iteration training in a preset queue as the global predicted feature information corresponding to the negative sample image of the next round of iteration training, thereby improving the detection accuracy of the global predicted feature information output by the trained global feature information detection branch.

[0183] In specific implementation, after obtaining the fourth predicted feature information, it can be stored in the preset queue as the global predicted feature information corresponding to the negative sample image of the next round of iteration training, and the generation time of each fourth predicted feature information is recorded, but at this time, the fourth predicted feature information is not used to generate the third loss, that is, the global feature information detection branch is not trained, because the number of global predicted feature information corresponding to the negative sample image at this time is small, and the training effect is poor. After completing the training of the current round, the new first sample image pair selected from the determined plurality of sample image pairs is re-input into the feature detection neural network, and the global feature information detection branch outputs the fourth predicted feature information corresponding to the new first sample image pair, which is stored in the preset queue, and the generation time of the newly generated fourth predicted feature information is recorded, and the same is true for the subsequent. When the number of rounds of training of the feature detection neural network reaches a preset value, the initialization of the queue is completed, at this time, a certain number of fourth predicted feature information has been stored in the preset queue, and the training using a certain number of fourth predicted feature information can achieve good training effect, therefore, the new round of input sample image pairs will be used for training the feature detection neural network.

[0184] In specific implementation, a second sample image pair different from the first sample image pair can be selected from the determined plurality of sample image pairs and input into the feature detection neural network, and the global fifth predicted feature information of each image in each second sample image pair can be obtained respectively, and then the fifth predicted feature information and the fourth predicted feature information in the preset queue can be used to generate the third loss for training the feature detection neural network.

[0185] In an embodiment, the fifth predicted feature information can include sixth predicted feature information corresponding to the sample image in the second sample image pair, and seventh predicted feature information corresponding to the sample derived image in the second sample image pair.

[0186] In implementation, for each of the plurality of second sample image pairs input to the feature detection neural network, the sixth predicted feature information corresponding to the sample image in each second sample image pair and the seventh predicted feature information corresponding to the sample derived image in the second sample image pair can be determined, and then the fourth predicted feature information with the highest similarity to the sixth predicted feature information can be determined from the fourth predicted feature information stored in the preset queue as the global predicted feature information corresponding to the negative sample image, and the fourth predicted feature information is taken as the global predicted feature information corresponding to the negative sample image for generating the third loss, i.e., the eighth predicted feature information, and then the third loss can be generated using the sixth predicted feature information, the seventh predicted feature information, and the eighth predicted feature information, and the global feature information detection branch is trained in the current round using the third loss, wherein the third loss can be a contrastive loss function.

[0187] In addition, after determining the eighth predicted feature information corresponding to the negative sample image in the current round, the fifth predicted feature information obtained in the current round can be stored in the preset queue, and the third quantity of fourth predicted feature information generated earlier in time in the preset queue is deleted based on the generation time of the fourth predicted feature information stored in the preset queue and the third quantity of the fifth predicted feature information, and the updating of the queue is completed. Then, using the new second sample image, the next round of training of the feature detection neural network can be completed according to the above steps, and finally, the trained feature detection neural network can be obtained.

[0188] As shown in Figure 4 Fig. 1 is a schematic diagram of the iterative training of the feature detection neural network based on the queue provided by the embodiment of the present disclosure.

[0189] In addition, when each round of iterative training of the feature detection neural network is performed, the training of the basic feature extraction branch is also performed, and when the trained feature detection neural network is obtained, the basic feature extraction branch has also been trained and can output predicted basic features with high accuracy.

[0190] In this way, the trained feature detection neural network obtained by using the synchronous training method can synchronously output the local feature points of the target image, the local feature information corresponding to the local feature points, and the global feature information of the target image based on the extracted basic image features, with high accuracy, thereby improving the speed and efficiency of information detection of the target image.

[0191] In an embodiment, the feature detection method provided by the embodiment of the present disclosure can also be used to iteratively train the basic feature extraction branch, the local feature point detection branch and the local feature information detection branch in the feature detection neural network using sample image pairs, and after the training is completed, iteratively train the global feature information detection branch in the feature detection neural network using sample image pairs, and finally obtain the trained feature detection neural network, which will not be described herein.

[0192] Those skilled in the art can understand that the sequence of writing each step in the above method of the specific embodiment does not mean a strict execution sequence and does not constitute any limitation on the implementation process. The specific execution sequence of each step should be determined by its function and possible internal logic.

[0193] Based on the same inventive concept, the embodiment of the present disclosure also provides a feature detection device corresponding to the feature detection method. Since the principle of solving problems of the device in the embodiment of the present disclosure is similar to the above-mentioned feature detection method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0194] As shown in FIG. 1, a schematic diagram of a feature detection device provided by the embodiment of the present disclosure includes: Figure 5

[0195] The acquisition module 501 is configured to acquire a target image.

[0196] The extraction module 502 is configured to extract a basic image feature of the target image.

[0197] The determination module 503 is configured to determine, based on the basic image feature, a local feature point of the target image, local feature information corresponding to the local feature point, and global feature information of the target image.

[0198] In a possible implementation, the determination module 503 is configured to use a feature detection neural network to output, based on the basic image feature, the local feature point of the target image, the local feature information corresponding to the local feature point, and the global feature information of the target image.

[0199] In a possible implementation, the device further includes a training module 504 configured to train the feature detection neural network:

[0200] The training module 504 is configured to acquire a plurality of sample images.

[0201] For each sample image, at least one sample derived image having a co-view relationship with the sample image is determined, and the sample image is combined with each sample derived image to obtain at least one sample image pair. ​

[0202] train the feature detection neural network using the sample image pairs.

[0203] In a possible implementation, the sample images include first sample scene images, and the sample image pairs include first sample scene image pairs.

[0204] The training module 504 is configured to obtain a plurality of first sample scene images under a target scene.

[0205] Based on the plurality of first sample scene images, three-dimensional scene information corresponding to the target scene is determined.

[0206] Based on the three-dimensional scene information, at least one pair of first sample scene images having a co-view relationship is screened from the first sample scene images, to obtain at least one first sample scene image pair.

[0207] In a possible implementation, the sample images include second sample scene images, and the sample image pairs include second sample scene image pairs.

[0208] The training module 504 is configured to obtain a plurality of second sample scene images under a non-target scene.

[0209] For each second sample scene image, a first preprocessing operation is performed on the second sample scene image, to obtain at least one target derived image having a co-view relationship with the second sample scene image, and the second sample scene image is combined with each target derived image respectively, to obtain at least one second sample scene image pair.

[0210] In a possible implementation, the training module 504 is configured to perform a second preprocessing operation on at least part of the sample image pairs, to obtain at least one preprocessed sample image pair; the sample image pairs include first sample image pairs and / or second sample image pairs.

[0211] The feature detection neural network is trained using the at least part of the sample image pairs and the preprocessed sample image pairs.

[0212] In a possible implementation, the second preprocessing operation includes at least one of the following: a random noise adding operation, an image brightness changing operation.

[0213] In a possible implementation, the training module 504 is configured to determine local labeled feature points of each sample image in the sample image pairs respectively.

[0214] From a sample derived image in the sample image pairs, positive sample feature points and negative sample feature points corresponding to the local labeled feature points are screened.

[0215] The feature detection neural network is trained by using the local labeled feature point, the positive sample feature point corresponding to the local labeled feature point, and the negative sample feature point.

[0216] In a possible implementation, the training module 504 is configured to take the feature point corresponding to the local labeled feature point in the sample derived image as a positive sample feature point.

[0217] The feature point with the highest similarity to the local labeled feature point is screened from the feature points other than the positive sample feature point in the sample derived image, and the feature point with the highest similarity screened is taken as a negative sample feature point.

[0218] In a possible implementation, the training module 504 is configured to input the sample image pair into the feature detection neural network to obtain a local predicted feature point, first predicted feature information corresponding to the local labeled feature point, second predicted feature information corresponding to the positive sample feature point, and third predicted feature information corresponding to the negative sample feature point.

[0219] A first loss is generated by using the local predicted feature point and the local labeled feature point.

[0220] A second loss is generated by using the first predicted feature information corresponding to the local labeled feature point, the second predicted feature information corresponding to the positive sample feature point, and the third predicted feature information corresponding to the negative sample feature point.

[0221] The feature detection neural network is trained by using the first loss and the second loss.

[0222] In a possible implementation, the training module 504 is configured to input the sample image pair into the feature detection neural network to obtain predicted basic image features corresponding to the sample image pair by a basic feature extraction branch in the feature detection neural network.

[0223] The local predicted feature point, the first predicted feature information corresponding to the local labeled feature point, the second predicted feature information corresponding to the positive sample feature point, and the third predicted feature information corresponding to the negative sample feature point are obtained by a feature prediction branch in the feature detection neural network based on the predicted basic image features.

[0224] In a possible implementation, the training module 504 is configured to input a plurality of first sample image pairs into the feature detection neural network to obtain global fourth predicted feature information corresponding to each image in each of the first sample image pairs respectively, and store the global fourth predicted feature information in a preset queue.

[0225] The feature detection neural network is trained based on the plurality of second sample image pairs and the fourth predicted feature information in the preset queue.

[0226] In a possible implementation, the training module 504 is configured to input the plurality of second sample image pairs into the feature detection neural network to obtain fifth predicted feature information of each image in each second sample image pair globally.

[0227] The third loss is generated based on the fourth predicted feature information in the preset queue and the fifth predicted feature information, and the feature detection neural network is trained by using the third loss.

[0228] In a possible implementation, the training module 504 is configured to determine, for each second sample image pair, sixth predicted feature information of a sample image in the second sample image pair, and determine seventh predicted feature information of a positive sample image corresponding to the sample image by taking a sample derived image in the second sample image pair as the positive sample image.

[0229] The fourth predicted feature information with the highest similarity to the sixth predicted feature information of the sample image in the second sample image pair is screened from the preset queue, and the screened fourth predicted feature information is taken as eighth predicted feature information corresponding to a negative sample image.

[0230] The third loss is generated by using the sixth predicted feature information, the seventh predicted feature information, and the eighth predicted feature information.

[0231] In a possible implementation, the training module 504 is further configured to update the preset queue based on the fifth predicted feature information of each image globally after obtaining the fifth predicted feature information of each image in each second sample image pair globally.

[0232] The description of the processing procedure of each module in the apparatus and the interaction procedure between the modules can refer to the related description in the above method embodiments, and will not be described in detail here.

[0233] The present disclosure also provides a computer device, as shown in the accompanying drawings, which is a structural schematic diagram of a computer device provided by the present disclosure, comprising: Figure 6

[0234] ​The processor 61 and the memory 62; the memory 62 stores machine readable instructions executable by the processor 61, and the processor 61 is configured to execute the machine readable instructions stored in the memory 62, and when the machine readable instructions are executed by the processor 61, the processor 61 performs the following steps: S101: obtaining a target image; S102: extracting a basic image feature of the target image; and S103: determining a local feature point of the target image, local feature information corresponding to the local feature point, and global feature information of the target image based on the basic image feature.

[0235] The memory 62 includes the memory 621 and the external memory 622; the memory 621 is also called the internal memory, and is used to temporarily store operation data in the processor 61 and exchange data with the external memory 622 such as a hard disk, and the processor 61 exchanges data with the external memory 622 through the memory 621.

[0236] The specific execution process of the instructions can refer to the steps of the feature detection method described in the embodiments of the present disclosure, which will not be described here.

[0237] The embodiments of the present disclosure also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the feature detection method described in the method embodiments. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0238] The computer program product of the feature detection method provided by the embodiments of the present disclosure includes a computer readable storage medium storing program codes, and the instructions included in the program codes can be used to perform the steps of the feature detection method described in the method embodiments. For details, refer to the above method embodiments, which will not be described here.

[0239] The computer program product can be specifically implemented by hardware, software or a combination thereof. In one optional embodiment, the computer program product is specifically embodied as a computer storage medium, and in another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

[0240] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the apparatus described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here. In several embodiments provided in the present disclosure, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division in actual implementation. For example, a plurality of units or components can be combined or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, and can be electrical, mechanical or other forms.

[0241] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0242] In addition, the functional units in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0243] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or say the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various program code storage media.

[0244] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit the present disclosure. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easy changes to the technical solutions described in the foregoing embodiments, or easily think of changes or equivalent replacements for some of the technical features; and these modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A feature detection method, characterized in that, include: Acquire the target image; Extract the basic image features of the target image; Using a feature detection neural network, based on the basic image features, outputs local feature points of the target image, local feature information corresponding to the local feature points, and global feature information of the target image; training the feature detection neural network includes: Acquire multiple sample images; for each sample image, determine at least one derived sample image that has a co-view relationship with the sample image, and combine the sample image with each derived sample image to obtain at least one sample image pair; use the sample image pair to train the feature detection neural network; The step of training the feature detection neural network using the sample image pairs includes: Locally labeled feature points are determined for each sample image in the sample image pair; positive and negative sample feature points corresponding to the locally labeled feature points are selected from the sample derived images in the sample image pair; the negative sample feature points are the feature points in the sample derived images that have the highest similarity to the locally labeled feature points, excluding the positive sample feature points; the feature detection neural network is trained using the locally labeled feature points, the corresponding positive sample feature points, and the negative sample feature points; the step of training the feature detection neural network using the locally labeled feature points, the corresponding positive sample feature points, and the negative sample feature points further includes: Multiple first sample image pairs are input into the feature detection neural network to obtain global fourth prediction feature information corresponding to each image in each first sample image pair, and the global fourth prediction feature information is stored in a preset queue. The feature detection neural network is trained based on multiple second sample image pairs and the fourth prediction feature information in the preset queue; the fourth prediction feature information in the preset queue is used as the global prediction feature information of the negative samples corresponding to the second sample image pairs when the number of iterations of training the feature detection network reaches a preset value.

2. The method according to claim 1, characterized in that, The sample image includes a first sample scene image, and the sample image pair includes a first sample scene image pair; For each sample image, at least one derived sample image that has a co-view relationship with the sample image is determined, and the sample image is combined with each derived sample image to obtain at least one sample image pair, including: Acquire multiple first-sample scene images of the target scene; Based on multiple first sample scene images, determine the three-dimensional scene information corresponding to the target scene; Based on the three-dimensional scene information, at least one pair of first sample scene images with a co-view relationship are selected from the first sample scene images to obtain at least one pair of first sample scene images.

3. The method according to claim 1 or 2, characterized in that, The sample image includes a second sample scene image, and the sample image pair includes a second sample scene image pair; For each sample image, at least one derived sample image that has a co-view relationship with the sample image is determined, and the sample image is combined with each derived sample image to obtain at least one sample image pair, including: Acquire multiple second-sample scene images from non-target scenes; For each second sample scene image, a first preprocessing operation is performed on the second sample scene image to obtain at least one target derived image that has a co-view relationship with the second sample scene image. The second sample scene image is then combined with each target derived image to obtain at least one pair of second sample scene images.

4. The method according to claim 3, characterized in that, The step of training the feature detection neural network using the sample image pairs includes: A second preprocessing operation is performed on at least a portion of the sample image pairs to obtain at least one preprocessed sample image pair; the sample image pair includes a first sample image pair and / or a second sample image pair. The feature detection neural network is trained using at least a portion of the sample image pairs and the preprocessed sample image pairs.

5. The method according to claim 4, characterized in that, The second preprocessing operation includes at least one of the following: adding random noise or changing image brightness.

6. The method according to claim 1, characterized in that, The step of filtering positive and negative sample feature points corresponding to the locally labeled feature points from the sample derived images in the sample image pair includes: The feature points in the derived image of the sample that correspond to the locally labeled feature points are taken as positive sample feature points; From the feature points in the derived image of the samples, excluding the positive sample feature points, select the feature points with the highest similarity to the locally labeled feature points, and use the selected feature points with the highest similarity as the negative sample feature points.

7. The method according to claim 1 or 6, characterized in that, The step of training the feature detection neural network using the locally labeled feature points, the corresponding positive sample feature points, and the negative sample feature points includes: The sample image pairs are input into the feature detection neural network to obtain local predicted feature points, first predicted feature information corresponding to the local labeled feature points, second predicted feature information corresponding to the positive sample feature points, and third predicted feature information corresponding to the negative sample feature points; A first loss is generated using the locally predicted feature points and the locally labeled feature points; A second loss is generated by using the first predicted feature information corresponding to the locally labeled feature points, the second predicted feature information corresponding to the positive sample feature points, and the third predicted feature information corresponding to the negative sample feature points; The feature detection neural network is trained using the first loss and the second loss.

8. The method according to claim 7, characterized in that, The step of inputting the sample image pairs into the feature detection neural network to obtain local predicted feature points, first predicted feature information corresponding to the locally labeled feature points, second predicted feature information corresponding to the positive sample feature points, and third predicted feature information corresponding to the negative sample feature points includes: The sample image pairs are input into the feature detection neural network, and the basic feature extraction branch in the feature detection neural network obtains the predicted basic image features corresponding to the sample image pairs. Based on the prediction base image features, the feature prediction branch in the feature detection neural network obtains local predicted feature points, first predicted feature information corresponding to the local labeled feature points, second predicted feature information corresponding to the positive sample feature points, and third predicted feature information corresponding to the negative sample feature points.

9. The method according to claim 1, characterized in that, The step of training the feature detection neural network based on multiple second sample image pairs and the fourth predicted feature information in the preset queue includes: Multiple pairs of second sample images are input into a feature detection neural network to obtain global fifth prediction feature information for each image in each pair of second sample images. Based on the fourth and fifth predicted feature information in the preset queue, a third loss is generated, and the feature detection neural network is trained using the third loss.

10. The method according to claim 9, characterized in that, The fifth predictive feature information includes the sixth predictive feature information and the seventh predictive feature information; The generation of the third loss based on the fourth and fifth predicted feature information in the preset queue includes: For each pair of second sample images, the sixth prediction feature information of the sample image in the pair of second sample images is determined, and the sample derived image in the pair of second sample images is taken as the positive sample image corresponding to the sample image, and the seventh prediction feature information of the positive sample image is determined. The fourth prediction feature information with the highest similarity to the sixth prediction feature information of the sample image in the second sample image pair is selected from the preset queue, and the selected fourth prediction feature information is used as the eighth prediction feature information corresponding to the negative sample image. The third loss is generated using the sixth, seventh, and eighth prediction feature information.

11. The method according to claim 9, characterized in that, After obtaining the global fifth predicted feature information for each image in each second sample image pair, the following is also included: The preset queue is updated based on the global fifth prediction feature information of each image.

12. A feature detection device, characterized in that, include: The acquisition module is used to acquire the target image; The extraction module is used to extract the basic image features of the target image; The determination module is used to utilize a feature detection neural network to output local feature points of the target image, local feature information corresponding to the local feature points, and global feature information of the target image based on the basic image features. The apparatus further includes a training module for training the feature detection neural network, configured to acquire multiple sample images; for each sample image, determine at least one derived sample image that has a co-view relationship with the sample image, and combine the sample image with each derived sample image to obtain at least one sample image pair; train the feature detection neural network using the sample image pair; and when training the feature detection neural network using the sample image pair, configured to: determine local labeled feature points for each sample image in the sample image pair; select positive and negative sample feature points corresponding to the local labeled feature points from the derived sample images in the sample image pair; the negative sample feature points are the feature points in the derived sample images that have the highest similarity to the local labeled feature points, excluding the positive sample feature points; train the feature detection neural network using the local labeled feature points, the corresponding positive sample feature points, and the negative sample feature points; the training module, when training the feature detection neural network using the local labeled feature points, the corresponding positive sample feature points, and the negative sample feature points, is further configured to: Multiple first sample image pairs are input into the feature detection neural network to obtain global fourth prediction feature information corresponding to each image in each first sample image pair, and the global fourth prediction feature information is stored in a preset queue. The feature detection neural network is trained based on multiple second sample image pairs and the fourth prediction feature information in the preset queue; the fourth prediction feature information in the preset queue is used as the global prediction feature information of the negative samples corresponding to the second sample image pairs when the number of iterations of training the feature detection network reaches a preset value.

13. A computer device, characterized in that, include: The processor and the memory, wherein the memory stores machine-readable instructions executable by the processor, the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the processor performs the steps of the feature detection method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer device, performs the steps of the feature detection method as described in any one of claims 1 to 11.

Images