Image feature detection method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image feature detection method and device, electronic equipment and storage medium, comprising: determining an external parameter matrix of a to-be-detected image relative to a corresponding stitching image, determining a target external parameter sub-matrix in which the external parameter matrix is located according to the external parameter matrix and a preset external parameter matrix threshold set, wherein a preset external parameter matrix threshold in the preset external parameter matrix threshold set divides a preset external parameter matrix into a plurality of external parameter sub-matrices, and the feature detection algorithms corresponding to the external parameter sub-matrices on both sides of the preset external parameter matrix threshold are different; and performing feature detection on the to-be-detected image through the feature detection algorithm corresponding to the target external parameter sub-matrix. By determining the target external parameter sub-matrix in which the external parameter matrix of the to-be-detected image relative to the corresponding stitching image is located, and performing feature detection on the to-be-detected image through the corresponding feature detection algorithm preset for the target external parameter sub-matrix, the accuracy and applicability of image feature detection in panoramic image stitching can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to an image stitching method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Panoramic image stitching is an image processing technology of merging multiple photos collected at different times, different perspectives, or different sensors to create a seamless panoramic image or a high-resolution image. Panoramic image stitching is widely used in the technical fields of VR (Virtual Reality), AR (Augmented Reality), and assisted driving.

[0003] In the related art, a specific feature detection algorithm is used to detect features of multiple images to be stitched, and then a panoramic image is obtained through feature matching and stitching. However, the multiple images have position differences, and using the same feature detection algorithm to detect features of all images may reduce the accuracy of image detection. SUMMARY

[0004] To overcome the problems in the related art, the present disclosure provides an image feature detection method, device, electronic device, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, an image feature detection method is provided, comprising:

[0006] In response to obtaining a to-be-detected image, determining an external parameter matrix of the to-be-detected image relative to a corresponding stitched image;

[0007] According to the external parameter matrix and a preset external parameter matrix threshold set, determining a target external parameter sub-matrix in which the external parameter matrix is located, wherein a preset external parameter matrix threshold in the preset external parameter matrix threshold set divides a preset external parameter matrix into multiple external parameter sub-matrices, and the external parameter sub-matrices on both sides of the preset external parameter matrix threshold correspond to different feature detection algorithms;

[0008] Performing feature detection on the to-be-detected image by using the feature detection algorithm corresponding to the target external parameter sub-matrix.

[0009] Optionally, the preset external parameter matrix threshold is determined by the following method:

[0010] Performing feature detection on images of a sample object by using multiple feature detection algorithms to obtain reference feature vectors corresponding to the feature detection algorithms;

[0011] Obtaining multiple to-be-detected sample images of the sample object in a panoramic shooting mode;

[0012] determine an extrinsic parameter matrix of each of the sample images relative to the corresponding stitched image;

[0013] divide the preset extrinsic parameter matrix into a plurality of extrinsic parameter sub-matrices by setting a reference extrinsic parameter matrix threshold;

[0014] determine a preset extrinsic parameter matrix threshold according to the plurality of extrinsic parameter sub-matrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrices corresponding to each of the sample images;

[0015] The minimum feature distance corresponding to the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold is calculated based on different feature detection algorithms.

[0016] Optionally, the determination of the preset extrinsic parameter matrix threshold according to the plurality of extrinsic parameter sub-matrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrices corresponding to each of the sample images comprises:

[0017] perform image set division on the sample images according to the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to each of the sample images;

[0018] perform feature detection on the sample images in the image sets by using the plurality of feature detection algorithms to obtain sample feature vectors corresponding to each feature detection algorithm for different image sets;

[0019] calculate feature vector distances between the sample feature vectors corresponding to each feature detection algorithm in different image sets and the reference feature vectors corresponding to the feature detection algorithm;

[0020] in a case where the feature detection algorithms corresponding to the minimum feature vector distances of adjacent extrinsic parameter sub-matrices are different, set the reference extrinsic parameter matrix threshold as the preset extrinsic parameter matrix threshold.

[0021] Optionally, the method comprises:

[0022] in a case where the feature detection algorithms corresponding to the minimum feature vector distances of adjacent extrinsic parameter sub-matrices are the same, repeatedly perform the following steps:

[0023] re-set the reference extrinsic parameter matrix threshold to obtain a new plurality of extrinsic parameter sub-matrices;

[0024] perform the steps from the image set division on the sample images according to the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to each of the sample images to the calculation of the feature vector distances between the sample feature vectors corresponding to each feature detection algorithm in different image sets and the reference feature vectors corresponding to the feature detection algorithm.

[0025] determine whether the feature detection algorithm corresponding to the minimum feature vector distance corresponding to the adjacent sub-matrix of the external parameter is same;

[0026] until the feature detection algorithm corresponding to the minimum feature vector distance corresponding to the adjacent sub-matrix of the external parameter is different, the reference external parameter matrix threshold set this time is taken as the preset external parameter matrix threshold.

[0027] Optionally, the image set division of the sample image to be detected according to the plurality of sub-matrices of external parameters and the external parameter matrix corresponding to each sample image to be detected comprises:

[0028] determining the external parameter sub-matrix in which the external parameter matrix corresponding to each sample image to be detected is located according to the plurality of sub-matrices of external parameters;

[0029] dividing the sample images to be detected of the external parameter matrix in the same external parameter sub-matrix into the same image set until the sample images to be detected are all divided, and obtaining the image set.

[0030] Optionally, the target external parameter sub-matrix in which the external parameter matrix is located is determined according to the external parameter matrix and the preset external parameter matrix threshold, comprising:

[0031] determining the scene type when the sample image to be detected is collected, and each scene type is provided with a preset external parameter matrix threshold;

[0032] determining the target external parameter sub-matrix in which the external parameter matrix is located according to the external parameter matrix and the preset external parameter matrix threshold corresponding to the scene type.

[0033] According to a second aspect of the embodiment of the present disclosure, an image feature detection device is provided, comprising:

[0034] a first determination module configured to determine the external parameter matrix of the sample image to be detected relative to the corresponding spliced image in response to obtaining the sample image to be detected;

[0035] a second determination module configured to determine the target external parameter sub-matrix in which the external parameter matrix is located according to the external parameter matrix and a preset external parameter matrix threshold set, wherein the preset external parameter matrix threshold in the preset external parameter matrix threshold set divides the preset external parameter matrix into a plurality of external parameter sub-matrices, and the feature detection algorithms corresponding to the external parameter sub-matrices on both sides of the preset external parameter matrix threshold are different;

[0036] a feature detection module configured to perform feature detection on the sample image to be detected through the feature detection algorithm corresponding to the target external parameter sub-matrix.

[0037] Optionally, the second determining module comprises:

[0038] a first detecting submodule configured to perform feature detection on the image of the sample object by using a plurality of feature detection algorithms to obtain a reference feature vector corresponding to each feature detection algorithm;

[0039] a obtaining submodule configured to obtain a plurality of to-be-detected sample images of the sample object in a panoramic shooting mode;

[0040] a first determining submodule configured to determine an extrinsic parameter matrix of each to-be-detected sample image relative to a corresponding stitched image;

[0041] a dividing submodule configured to divide a preset extrinsic parameter matrix into a plurality of extrinsic parameter submatrices by setting a reference extrinsic parameter matrix threshold;

[0042] a second determining submodule configured to determine the preset extrinsic parameter matrix threshold according to the plurality of extrinsic parameter submatrices, the reference feature vector corresponding to each feature detection algorithm, and the extrinsic parameter matrix corresponding to each to-be-detected sample image;

[0043] wherein the minimum feature distance corresponding to the extrinsic parameter submatrices on both sides of the preset extrinsic parameter matrix threshold is calculated based on different feature detection algorithms.

[0044] Optionally, the second determining submodule is configured to:

[0045] perform image set division on the to-be-detected sample images according to the plurality of extrinsic parameter submatrices and the extrinsic parameter matrix corresponding to each to-be-detected sample image;

[0046] perform feature detection on the to-be-detected sample images in the image set by using the plurality of feature detection algorithms to obtain a sample feature vector corresponding to each feature detection algorithm for different image sets;

[0047] calculate a feature vector distance between the sample feature vector corresponding to each feature detection algorithm in different image sets and the reference feature vector corresponding to the feature detection algorithm;

[0048] in a case where it is determined that the feature detection algorithms corresponding to the minimum feature vector distances of adjacent extrinsic parameter submatrices are different, set the reference extrinsic parameter matrix threshold as the preset extrinsic parameter matrix threshold.

[0049] Optionally, the second determining submodule is configured to:

[0050] in a case where it is determined that the feature detection algorithms corresponding to the minimum feature vector distances of adjacent extrinsic parameter submatrices are the same, repeatedly perform the following steps:

[0051] resetting the reference extrinsic parameter matrix threshold to obtain a plurality of new extrinsic parameter sub-matrices;

[0052] performing image set partitioning on the to-be-detected sample images from the according to the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to each of the to-be-detected sample images, to the step of calculating the feature vector distance between the sample feature vector corresponding to each feature detection algorithm in different image sets and the reference feature vector corresponding to the feature detection algorithm;

[0053] determining whether the feature detection algorithms corresponding to the minimum feature vector distances corresponding to adjacent extrinsic parameter sub-matrices are the same;

[0054] until the feature detection algorithms corresponding to the minimum feature vector distances corresponding to adjacent extrinsic parameter sub-matrices are different, setting the reference extrinsic parameter matrix threshold set this time as the preset extrinsic parameter matrix threshold.

[0055] Optionally, the partitioning module is configured to:

[0056] determining, according to the plurality of extrinsic parameter sub-matrices, the extrinsic parameter sub-matrices in which the extrinsic parameter matrices corresponding to each of the to-be-detected sample images are located;

[0057] partitioning the to-be-detected sample images corresponding to the extrinsic parameter matrices in the same extrinsic parameter sub-matrices into the same image set until the to-be-detected sample images are all partitioned, to obtain the image sets.

[0058] Optionally, the second determination module is configured to:

[0059] determining a scene type when the to-be-detected images are collected, and each scene type is set with a preset extrinsic parameter matrix threshold;

[0060] determining, according to the extrinsic parameter matrix and the preset extrinsic parameter matrix threshold corresponding to the scene type, the target extrinsic parameter sub-matrix in which the extrinsic parameter matrix is located.

[0061] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising:

[0062] a processor;

[0063] a memory for storing processor-executable instructions;

[0064] wherein the processor is configured to:

[0065] in response to obtaining a to-be-detected image, determining an extrinsic parameter matrix of the to-be-detected image relative to a corresponding stitched image;

[0066] According to the extrinsic parameter matrix and a preset extrinsic parameter matrix threshold set, a target extrinsic parameter sub-matrix in which the extrinsic parameter matrix is located is determined, wherein a preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides a preset extrinsic parameter matrix into a plurality of extrinsic parameter sub-matrices, and the feature detection algorithms corresponding to the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold are different.

[0067] The feature detection algorithm corresponding to the target extrinsic parameter sub-matrix is used to perform feature detection on the image to be detected.

[0068] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores computer program instructions. When the computer program instructions are executed by a processor, the steps of the image feature detection method provided in the first aspect of the present disclosure are implemented.

[0069] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects: in response to obtaining an image to be detected, an extrinsic parameter matrix of the image to be detected relative to a corresponding stitched image is determined, an extrinsic parameter sub-matrix in which the extrinsic parameter matrix is located is determined according to the extrinsic parameter matrix and a preset extrinsic parameter matrix threshold set, wherein a preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides a preset extrinsic parameter matrix into a plurality of extrinsic parameter sub-matrices, and the feature detection algorithms corresponding to the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold are different; and the feature detection algorithm corresponding to the target extrinsic parameter sub-matrix is used to perform feature detection on the image to be detected. By determining the target extrinsic parameter sub-matrix in which the extrinsic parameter matrix of the image to be detected relative to the corresponding stitched image is located, and performing feature detection on the image to be detected by the corresponding feature detection algorithm of the target extrinsic parameter sub-matrix, the accuracy and applicability of image feature detection in panoramic image stitching can be improved.

[0070] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0071] The accompanying drawings, which are incorporated into the specification and constitute part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0072] Figure 1 is a flowchart of an image feature detection method according to an exemplary embodiment.

[0073] Figure 2 is a schematic diagram of determining an extrinsic parameter matrix according to an exemplary embodiment.

[0074] Figure 3 is a schematic diagram of panoramic image stitching according to an exemplary embodiment.

[0075] Figure 4 is a flow chart of a preset extrinsic parameter matrix threshold determination method according to an example embodiment.

[0076] Figure 5 is a flow chart of a preset extrinsic parameter matrix threshold determination method according to an example embodiment.

[0077] Figure 6 is a flow chart of a step S45 in the method of Figure 4 according to an example embodiment.

[0078] Figure 7 is a block diagram of an image feature detection device according to an example embodiment.

[0079] Figure 8 is a block diagram of a second determination module according to an example embodiment.

[0080] Figure 9 is a block diagram of a device for image feature detection according to an example embodiment. DETAILED DESCRIPTION

[0081] The example embodiments will be described in detail herein with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like elements. The following description of example embodiments is not representative of all embodiments consistent with the present disclosure. Rather, it is merely an example of apparatus and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0082] Figure 1 is a flow chart of an image feature detection method according to an example embodiment, which can be applied to a mobile terminal, a tablet computer, etc., for image feature detection in, for example, panorama image stitching, in the present embodiment of the present disclosure, the image feature detection method is exemplarily illustrated as applied to a mobile terminal, as shown in Figure 1 comprises the following steps.

[0083] In step S11, in response to obtaining a to-be-detected image, an extrinsic parameter matrix of the to-be-detected image relative to a corresponding stitched image is determined.

[0084] In the step S11, the to-be-detected image is captured by an image capturing device such as a camera on the mobile terminal, and the extrinsic parameter matrix of the to-be-detected image relative to the corresponding stitched image is determined by a motion sensor on the mobile terminal. After the feature detection is completed, the to-be-detected image needs to be stitched to the stitched image to obtain a panorama stitched image.

[0085] For example, referring toFigure 2 As shown, Image1 is acquired at time t1 and Image2 is acquired at time t2. Image1 is a stitched image, and Image2 is the image to be detected. Simultaneously with the acquisition of Image1 and Image2, the extrinsic parameter matrix [RT] of Image2 relative to Image1 is determined using a motion sensor.

[0086] Since the sampling frequency of the motion sensor is usually much higher than the shooting frame rate of the camera, the extrinsic parameter matrix [RT] of image Image2 relative to image Image1 is determined by summing the extrinsic parameters of the motion sensor in each dimension during the process from time t1 to time t2.

[0087] For example, from time t1 to time t2, the extrinsic parameter matrices acquired by the motion sensor are [r1 t1], [r2 t2], ..., [r...]. n t n Then for r1, r2, ..., r n Summing yields the rotation R of image Image2 relative to image Image1, i.e. And for t1, t2, ..., t n Summing yields the translation T of image Image2 relative to image Image1, i.e. Then, based on the rotation amount R and translation amount T, the extrinsic parameter matrix [RT] of image Image2 relative to image Image1 is constructed.

[0088] It is understandable that motion sensors and cameras can be fixed on the same mobile terminal to ensure the accuracy of the acquired extrinsic parameter matrix.

[0089] In step S12, the target extrinsic submatrix containing the extrinsic matrix is ​​determined based on the extrinsic parameter matrix and the preset extrinsic parameter matrix threshold set.

[0090] The preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides the preset extrinsic parameter matrix into multiple extrinsic parameter sub-matrices, and the feature detection algorithms corresponding to the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold are different.

[0091] The preset extrinsic parameter matrix threshold set includes one or more preset extrinsic parameter matrix thresholds. For example, if the preset extrinsic parameter matrix threshold set includes only one preset extrinsic parameter matrix threshold, the preset extrinsic parameter matrix threshold divides the preset extrinsic parameter matrix into two extrinsic parameter sub-matrices, i.e., a first extrinsic parameter sub-matrix in which the extrinsic parameter matrix is less than the preset extrinsic parameter matrix threshold, and a second extrinsic parameter sub-matrix in which the extrinsic parameter matrix is greater than or equal to the preset extrinsic parameter matrix threshold. Moreover, the preset feature detection algorithm corresponding to the first extrinsic parameter sub-matrix is different from the preset feature detection algorithm corresponding to the second extrinsic parameter sub-matrix. For example, the preset feature detection algorithm corresponding to the first extrinsic parameter sub-matrix is an optical flow detection algorithm, and the preset feature detection algorithm corresponding to the second extrinsic parameter sub-matrix is an angle point detection algorithm.

[0092] Further, if the extrinsic parameter matrix [R T] of the image Image2 relative to the image Image1 is less than the preset extrinsic parameter matrix threshold, it is determined that the target extrinsic parameter sub-matrix in which the extrinsic parameter matrix [R T] of the image Image2 relative to the image Image1 is located is the first extrinsic parameter sub-matrix. If the extrinsic parameter matrix [R T] of the image Image2 relative to the image Image1 is greater than or equal to the preset extrinsic parameter matrix threshold, it is determined that the target extrinsic parameter sub-matrix in which the extrinsic parameter matrix [R T] of the image Image2 relative to the image Image1 is located is the second extrinsic parameter sub-matrix.

[0093] For another example, if the preset extrinsic parameter matrix threshold set includes multiple preset extrinsic parameter matrix thresholds, in the embodiments of the present disclosure, the preset extrinsic parameter matrix threshold set includes a first preset extrinsic parameter matrix threshold and a second preset extrinsic parameter matrix threshold, and the two preset extrinsic parameter matrix thresholds are taken as an example. The first preset extrinsic parameter matrix threshold is less than the second preset extrinsic parameter matrix threshold.

[0094] Moreover, the first preset extrinsic parameter matrix threshold and the second preset extrinsic parameter matrix threshold divide the preset extrinsic parameter matrix into three extrinsic parameter sub-matrices, i.e., a third extrinsic parameter sub-matrix in which the extrinsic parameter matrix is less than the first preset extrinsic parameter matrix threshold, a fourth extrinsic parameter sub-matrix in which the extrinsic parameter matrix is greater than or equal to the first preset extrinsic parameter matrix threshold and less than the second preset extrinsic parameter matrix threshold, and a fifth extrinsic parameter sub-matrix in which the extrinsic parameter matrix is greater than or equal to the second preset extrinsic parameter matrix threshold. Moreover, the preset feature detection algorithm corresponding to the third extrinsic parameter sub-matrix, the fourth extrinsic parameter sub-matrix, and the fifth extrinsic parameter sub-matrix is different. For example, the preset feature detection algorithm corresponding to the third extrinsic parameter sub-matrix is an optical flow detection algorithm, the preset feature detection algorithm corresponding to the fourth extrinsic parameter sub-matrix is a scale-invariant feature transform (SIFT) detection algorithm, and the preset feature detection algorithm corresponding to the fifth extrinsic parameter sub-matrix is an angle point detection algorithm.

[0095] In a possible implementation, the third external parameter sub-matrix and the fifth external parameter sub-matrix are not external parameter sub-matrices on both sides of the preset external parameter matrix threshold, and therefore the preset feature detection algorithm corresponding to the third external parameter sub-matrix and the preset feature detection algorithm corresponding to the fifth external parameter sub-matrix can be the same, as long as the feature detection algorithms corresponding to the external parameter sub-matrices on both sides of the preset external parameter matrix threshold are different.

[0096] In step S13, feature detection is performed on the image to be detected by using the feature detection algorithm corresponding to the target external parameter sub-matrix.

[0097] For example, if the external parameter matrix [R T] of the image Image2 relative to the image Image1 is less than the third preset external parameter matrix threshold, feature detection is performed on the image Image2 by using the optical flow detection algorithm corresponding to the third preset external parameter matrix threshold.

[0098] Further, referring to Figure 3 As shown in the figure, according to the feature detection result, feature matching is performed on the image to be detected to obtain a matching point set, and the feature points in the matching set are registered. After registration, the image to be detected is spliced and fused to the corresponding spliced image to obtain a panoramic spliced image, and the next image to be spliced is continuously acquired. In response to the acquisition of the next image to be spliced, the external parameter matrix of the next image to be spliced relative to the image to be detected that has been spliced is determined.

[0099] The above technical solution determines the external parameter matrix of the image to be detected relative to the corresponding spliced image in response to the acquisition of the image to be detected, determines the target external parameter sub-matrix in which the external parameter matrix is located according to the external parameter matrix and the preset external parameter matrix threshold set, wherein the preset external parameter matrix thresholds in the preset external parameter matrix threshold set divide the preset external parameter matrix into a plurality of external parameter sub-matrices, and the feature detection algorithms corresponding to the external parameter sub-matrices on both sides of the preset external parameter matrix threshold are different. Feature detection is performed on the image to be detected by using the feature detection algorithm corresponding to the target external parameter sub-matrix. By determining the target external parameter sub-matrix in which the external parameter matrix of the image to be detected relative to the corresponding spliced image is located, and performing feature detection on the image to be detected by using the corresponding feature detection algorithm preset for the target external parameter sub-matrix, the accuracy and applicability of image feature detection in panoramic image splicing can be improved.

[0100] On the basis of the above embodiment, Figure 4 is a flow chart of a preset external parameter matrix threshold determination method according to an example embodiment. The method can be executed on a mobile terminal or not, but offline. After determining the preset external parameter matrix threshold offline, the method is migrated to the mobile terminal. As shown in the figure, Figure 4 The method includes the following steps.

[0101] In step S41, feature detection is performed on the image of the sample object using multiple feature detection algorithms to obtain the reference feature vector corresponding to each feature detection algorithm.

[0102] In this embodiment, see Figure 5 As shown, three different feature detection algorithms are used to perform feature detection on the image of the sample object, resulting in reference feature vectors 0_1, 0_2, and 0_3 corresponding to the three algorithms. For example, the aforementioned optical flow detection algorithm, SIFT detection algorithm, and corner detection algorithm can be used to perform feature detection on the image of the sample object, obtaining the corresponding reference feature vectors.

[0103] In step S42, multiple images of the sample object to be detected are acquired in panoramic shooting mode.

[0104] For example, multiple images of the sample object to be detected are acquired by a camera in panoramic shooting mode from time t3 to time t4, and the extrinsic parameter matrix of each sample image to be detected relative to the reference feature vector is acquired by a motion sensor. The image acquired at time t3 is the same image as the image corresponding to the reference feature vector.

[0105] In step S43, the extrinsic parameter matrix of each sample image to be detected relative to the corresponding stitched image is determined.

[0106] For example, from time t3 to time t4, at t3 and t4 respectively 3.1 t 3.2 t 3.3 At times t1, t2, ..., t4, one image of the sample to be detected is acquired sequentially, resulting in 11 images of the sample to be detected. Based on the extrinsic parameter matrix of each image acquired by the motion sensor relative to the image acquired at time t3, the time step t is determined. 3.1 The extrinsic parameter matrix of the sample image to be detected acquired at time t3 relative to the image acquired at time t4 is t_t. 3.1 The extrinsic parameter matrix of the sample image to be detected at time t is relative to the corresponding stitched image.

[0107] Similarly, based on the data collected by the motion sensor, t 3.2 The extrinsic parameter matrix of the sample image to be detected at time t3 relative to the image acquired at time t3 is used to determine t. 3.2 The image of the sample to be detected acquired at time t is relative to t 3.1 The extrinsic parameter matrix of the image acquired at time t is t 3.2 The extrinsic parameter matrix of the sample image to be detected at time t4 relative to the corresponding stitched image is determined. Similarly, the extrinsic parameter matrix of the sample image to be detected at time t4 relative to the corresponding stitched image is determined.

[0108] In step S44, the preset extrinsic parameter matrix is divided into a plurality of extrinsic parameter sub-matrices by setting a reference extrinsic parameter matrix threshold.

[0109] In the embodiments of the present disclosure, the reference extrinsic parameter matrix threshold can be set according to an empirical value, and the preset extrinsic parameter matrix is divided into a plurality of extrinsic parameter sub-matrices.

[0110] The threshold corresponding to the rotation amount and the threshold corresponding to the translation amount are set respectively, and then the reference extrinsic parameter matrix threshold is obtained according to the threshold corresponding to the rotation amount and the threshold corresponding to the translation amount. In addition, the number of reference extrinsic parameter matrix thresholds can be determined according to the number of feature detection algorithms.

[0111] In step S45, the preset extrinsic parameter matrix threshold is determined according to the plurality of extrinsic parameter sub-matrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrices corresponding to each sample image to be detected.

[0112] The minimum feature distances corresponding to the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold are calculated based on different feature detection algorithms.

[0113] On the basis of the above embodiments, Figure 6 is a flowchart illustrating a process of determining a preset extrinsic parameter matrix threshold according to an example embodiment Figure 4 In step S45, the preset extrinsic parameter matrix threshold is determined according to the plurality of extrinsic parameter sub-matrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrices corresponding to each sample image to be detected.

[0114] In step S451, the sample images to be detected are divided into image sets according to the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to each sample image to be detected.

[0115] The extrinsic parameter sub-matrix in which the extrinsic parameter matrix corresponding to each sample image to be detected is located is determined according to the plurality of extrinsic parameter sub-matrices.

[0116] In the embodiments of the present disclosure, Figure 5 In the example embodiment, the threshold corresponding to the rotation amount and the threshold corresponding to the translation amount are both 1, that is, the threshold corresponding to the rotation amount is Thesh_R, the threshold corresponding to the translation amount is Thesh_T, and the reference extrinsic parameter matrix threshold is [Thesh_R Thesh_T]. The preset extrinsic parameter matrix is divided into four extrinsic parameter sub-matrices, as shown in Figure 5The first sub-matrix of the external parameter matrix has a rotation value greater than or equal to Thesh_R and a translation value greater than or equal to Thesh_T; the second sub-matrix of the external parameter matrix has a rotation value greater than or equal to Thesh_R and a translation value less than Thesh_T; the third sub-matrix of the external parameter matrix has a rotation value less than Thesh_R and a translation value greater than or equal to Thesh_T; and the fourth sub-matrix of the external parameter matrix has a rotation value less than Thesh_R and a translation value less than Thesh_T.

[0117] In a special case, the threshold value corresponding to the rotation value is Thesh_R, Thesh_R is not 0, and the threshold value corresponding to the translation value is 0, or the threshold value corresponding to the rotation value is 0, and the threshold value corresponding to the translation value is Thesh_T, Thesh_T is not 0. See Figure 4 In the example shown, when the threshold value corresponding to the translation value is 0, Thesh_R divides the preset external parameter matrix into a sub-matrix of the external parameter matrix greater than or equal to Thesh_R and a sub-matrix of the external parameter matrix less than Thesh_R; and when the threshold value corresponding to the rotation value is 0, Thesh_T divides the preset external parameter matrix into a sub-matrix of the external parameter matrix greater than or equal to Thesh_T and a sub-matrix of the external parameter matrix less than Thesh_T.

[0118] Further, the external parameter matrix is divided into the same image set for the same sub-matrix of the external parameter matrix, until all the sample images to be detected are divided, to obtain an image set. That is, the size relationship between the external parameter matrix corresponding to each sample image to be detected and the threshold value Thesh_R corresponding to the rotation value and the threshold value Thesh_T corresponding to the translation value is determined, and then the sample images to be detected are divided into an image set.

[0119] The above embodiment S43 is used for illustration, and the size relationship between the external parameter matrix corresponding to the time t1, t2, t3, t4 and the threshold value Thesh_R corresponding to the rotation value and the threshold value Thesh_T corresponding to the translation value is determined, and then the sample images to be detected at the time t1, t2, t3, t4 are divided into an image set. 3.1 , t 3.2 , t 3.3 , …, t4 are determined, and then the sample images to be detected at the time t1, t2, t3, t4 are divided into an image set. 3.1 , t 3.2 , t 3.3 , …, t4 are determined, and then the sample images to be detected at the time t1, t2, t3, t4 are divided into an image set.

[0120] For example, when the rotation value of the external parameter matrix corresponding to the time t1, t2, t3, t4 is greater than or equal to Thesh_R, and the translation value is greater than or equal to Thesh_T, the sample images to be detected at the time t1, t2, t3, t4 are divided into an image set. 3.1 , t 3.2 , t 3.1 , t 3.2The images of the samples to be detected at time t4 are grouped into the same image set to obtain the first image set.

[0121] t 3.3 t 3.4 In the extrinsic parameter matrix corresponding to time t, the rotation is equal to or greater than Thesh_R, and the translation is less than Thesh_T. 3.3 t 3.4 The sample images to be detected at each time point are grouped into the same image set to obtain the second image set.

[0122] t 3.5 t 3.6 If the rotation in the extrinsic parameter matrix at time t is less than Thesh_R and the translation is greater than or equal to Thesh_T, then t 3.5 t 3.6 The images of the samples to be detected at each time point are grouped into the same image set to obtain the third image set.

[0123] t 3.7 t 3.8 t 3.9 If the rotation in the extrinsic parameter matrix at time t is less than Thesh_R and the translation is greater than or equal to Thesh_T, then t 3.7 t 3.8 t 3.9 The sample images to be detected at each time point are grouped into the same image set, resulting in the fourth image set. Feature detection, matching, and feature distance calculation for the third and fourth image sets are performed in the same manner as described above, and will not be repeated here.

[0124] In step S452, feature detection is performed on the sample images to be detected in the image set using multiple feature detection algorithms to obtain sample feature vectors corresponding to each feature detection algorithm for different image sets.

[0125] The above embodiments will be used for illustration. See also: Figure 5 As shown, for t in the first image set 3.1 t 3.2 The sample image to be detected at time t4 is subjected to feature detection using detection algorithm 1 (optical flow detection algorithm), detection algorithm 2 (SIFT detection algorithm), and detection algorithm 3 (corner detection algorithm), respectively, to obtain t 3.1 t 3.2 The feature vector 1_1 of the sample image to be detected at time t4 under detection algorithm 1. 3.1 t 3.2 The sample feature vector 1_2 corresponding to the sample image to be detected at time t4 under detection algorithm 2, t 3.1 t 3.2, the sample feature vector 1_3 of the to-be-detected sample image corresponding to the time t4 under the detection algorithm 2. Similarly, for the to-be-detected sample image corresponding to the time t 3.3 , the sample feature vector 2_1 of the to-be-detected sample image corresponding to the time t 3.4 , the sample feature vector 2_2 of the to-be-detected sample image corresponding to the time t 3.3 , the sample feature vector 2_3 of the to-be-detected sample image corresponding to the time t 3.4 , the sample feature vector 2_3 of the to-be-detected sample image corresponding to the time t 3.3 , the sample feature vector 2_3 of the to-be-detected sample image corresponding to the time t 3.4 , the sample feature vector 2_3 of the to-be-detected sample image corresponding to the time t 3.3 , the sample feature vector 2_3 of the to-be-detected sample image corresponding to the time t 3.4 , the sample feature vector 2_3 of the to-be-detected sample image corresponding to the time t

[0126] In step S453, the feature vector distance between the sample feature vector corresponding to each feature detection algorithm in different image sets and the reference feature vector corresponding to the feature detection algorithm is calculated.

[0127] For example, in the first image set, the feature vector distance between the sample feature vector 1_1 corresponding to the to-be-detected sample image at the time t 3.1 , the time t 3.2 , and the reference feature vector corresponding to the detection algorithm 1 at the time t3 is calculated to obtain the feature vector distance; similarly, the feature vector distance between the sample feature vector 1_2 corresponding to the to-be-detected sample image at the time t 3.1 , the time t 3.2 , and the reference feature vector corresponding to the detection algorithm 2 at the time t3 is calculated to obtain the feature vector distance; the feature vector distance between the sample feature vector 1_3 corresponding to the to-be-detected sample image at the time t 3.1 , the time t 3.2 , and the reference feature vector corresponding to the detection algorithm 3 at the time t3 is calculated to obtain the feature vector distance.

[0128] Similarly, in the second image set, the feature vector distance between the sample feature vector 2_1 corresponding to the to-be-detected sample image at the time t 3.3 , the time t 3.4 , and the reference feature vector corresponding to the detection algorithm 1 at the time t3 is calculated to obtain the feature vector distance; similarly, the feature vector distance between the sample feature vector 2_2 corresponding to the to-be-detected sample image at the time t 3.3 , the time t 3.4The feature vector distance is obtained by calculating the feature vector distance between the sample feature vector corresponding to the to-be-detected sample image at the time t3 and the reference feature vector corresponding to the detection algorithm 2 at the time t3. 3.3 , the feature vector distance is obtained by calculating the feature vector distance between the sample feature vector corresponding to the to-be-detected sample image at the time t3 and the reference feature vector corresponding to the detection algorithm 3 at the time t3. 3.4 The feature vector distance is obtained by calculating the feature vector distance between the sample feature vector corresponding to the to-be-detected sample image at the time t3 and the reference feature vector corresponding to the detection algorithm 3 at the time t3.

[0129] In the embodiment, first, the feature detection algorithm corresponding to the minimum feature vector distance of the adjacent sub-matrix of the extrinsic parameter matrix is determined according to the feature detection algorithm corresponding to the minimum feature vector distance of the extrinsic parameter matrix of the first image set, the feature detection algorithm corresponding to the minimum feature vector distance of the extrinsic parameter matrix of the second image set, and the feature detection algorithm corresponding to the minimum feature vector distance of the extrinsic parameter matrix of the third image set. 3.1 , the feature vector distance is obtained by calculating the feature vector distance between the sample feature vector corresponding to the to-be-detected sample image at the time t3 and the reference feature vector corresponding to the detection algorithm 3 at the time t3. 3.2 The feature vector distance is obtained by calculating the feature vector distance between the sample feature vector corresponding to the to-be-detected sample image at the time t3 and the reference feature vector corresponding to the detection algorithm 3 at the time t3.

[0130] In step S454, in the case where the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent sub-matrices of the extrinsic parameter matrix are different, the reference extrinsic parameter matrix threshold is taken as the preset extrinsic parameter matrix threshold.

[0131] For example, in the first image set, the feature detection algorithm corresponding to the minimum feature vector distance of the sub-matrix of the extrinsic parameter matrix is the detection algorithm 1, while in the second image set, the feature detection algorithm corresponding to the minimum feature vector distance of the sub-matrix of the extrinsic parameter matrix is the detection algorithm 2, and since the sub-matrix of the extrinsic parameter matrix corresponding to the first image set is adjacent to the sub-matrix of the extrinsic parameter matrix corresponding to the second image set, the feature detection algorithm corresponding to the sub-matrix of the extrinsic parameter matrix corresponding to the first image set is different from the feature detection algorithm corresponding to the sub-matrix of the extrinsic parameter matrix corresponding to the second image set, so the reference extrinsic parameter matrix threshold at this time can continue to be determined according to the feature detection algorithms corresponding to the third image set and the fourth image set in the above manner.

[0132] In an embodiment, if the threshold value corresponding to the translation amount is 0, the threshold value corresponding to the rotation amount is Thesh_R, and a, b and c are a, b and c respectively, that is, a, b and c divide the preset extrinsic parameter matrix into the first to fourth sub-matrices of the extrinsic parameter matrix, wherein the first and second sub-matrices of the extrinsic parameter matrix are adjacent sub-matrices of the extrinsic parameter matrix, the first and third sub-matrices of the extrinsic parameter matrix are not adjacent sub-matrices of the extrinsic parameter matrix, the feature detection algorithms corresponding to the first and second sub-matrices of the extrinsic parameter matrix are different, the feature detection algorithms corresponding to the second and third sub-matrices of the extrinsic parameter matrix are different, and the feature detection algorithms corresponding to the first and third sub-matrices of the extrinsic parameter matrix can be different or the same.

[0133] Optionally, the method comprises:

[0134] In the case that the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent outer parameter sub-matrices are the same, the following steps are repeatedly executed:

[0135] The reference outer parameter matrix threshold is reset to obtain a new plurality of outer parameter sub-matrices.

[0136] The step of performing image set division on the sample images to be detected from the outer parameter matrices corresponding to the plurality of outer parameter sub-matrices and the sample images to be detected, to the step of calculating the feature vector distances between the sample feature vectors corresponding to different image sets and the reference feature vectors corresponding to the feature detection algorithms.

[0137] Determine whether the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent outer parameter sub-matrices are the same.

[0138] Until the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent outer parameter sub-matrices are different, the reference outer parameter matrix threshold set this time is taken as the preset outer parameter matrix threshold.

[0139] For example, if the first and second outer parameter sub-matrices are adjacent outer parameter sub-matrices, the first and second outer parameter sub-matrices are adjacent outer parameter sub-matrices, and the feature detection algorithms corresponding to the first and second outer parameter sub-matrices are the same, it indicates that the reference outer parameter matrix threshold for dividing the first and second outer parameter sub-matrices is not appropriate, and the reference outer parameter matrix threshold needs to be adjusted, and the calculation is performed again through the adjusted reference outer parameter matrix threshold, until the feature detection algorithms corresponding to the new first and second outer parameter sub-matrices adjacent outer parameter sub-matrices are different.

[0140] On the basis of the above embodiment, in step S12, the target outer parameter sub-matrix in which the outer parameter matrix is located is determined according to the outer parameter matrix and the preset outer parameter matrix threshold, including:

[0141] Determine the scene type when the sample images to be detected are collected, and a preset outer parameter matrix threshold is set for each scene type.

[0142] For example, the scene type can be divided according to light brightness, or can be divided according to the shooting object, for example, the shooting object can be landscape, person, combination of landscape and person.

[0143] The target outer parameter sub-matrix in which the outer parameter matrix is located is determined according to the outer parameter matrix and the preset outer parameter matrix threshold corresponding to the scene type.

[0144] The technical solution can determine the target extrinsic parameter sub-matrix in which the extrinsic parameter matrix is located according to the preset extrinsic parameter matrix threshold corresponding to the scene type, and can further improve the accuracy of the feature detection algorithm, thereby improving the accuracy of feature detection.

[0145] Based on the same concept, the present disclosure provides an image feature detection device for performing the steps of the image feature detection method provided by the above method embodiments. The device 700 can implement the image feature detection method in the form of software, hardware or a combination of both. Figure 7 is a block diagram of an image feature detection device 700 according to an exemplary embodiment, as shown in FIG. 7, the device 7 comprises: a first determination module 710, a second determination module 720 and an adjustment module 730.

[0146] The first determination module 710 is configured to determine the extrinsic parameter matrix of the to-be-detected image relative to the corresponding stitched image in response to obtaining the to-be-detected image.

[0147] The second determination module 720 is configured to determine the target extrinsic parameter sub-matrix in which the extrinsic parameter matrix is located according to the extrinsic parameter matrix and a preset extrinsic parameter matrix threshold set. The preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides the preset extrinsic parameter matrix into a plurality of extrinsic parameter sub-matrices, and the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold correspond to different feature detection algorithms.

[0148] The feature detection module 730 is configured to perform feature detection on the to-be-detected image through the feature detection algorithm corresponding to the target extrinsic parameter sub-matrix.

[0149] Optionally, Figure 8 is a block diagram of a second determination module 720 according to an exemplary embodiment, the second determination module 720 comprises:

[0150] The first detection sub-module 7201 is configured to perform feature detection on the image of the sample object through a plurality of feature detection algorithms to obtain a reference feature vector corresponding to each feature detection algorithm.

[0151] The acquisition sub-module 7202 is configured to acquire a plurality of to-be-detected sample images of the sample object in a panoramic shooting mode.

[0152] The first determination sub-module 7203 is configured to determine the extrinsic parameter matrix of each to-be-detected sample image relative to the corresponding stitched image.

[0153] The division sub-module 7204 is configured to divide the preset extrinsic parameter matrix into a plurality of extrinsic parameter sub-matrices by setting a reference extrinsic parameter matrix threshold.

[0154] The second determining sub-module 7205 is configured to determine a preset extrinsic parameter matrix threshold according to the plurality of extrinsic parameter sub-matrices, the reference feature vectors corresponding to the feature detection algorithms, and the extrinsic parameter matrices corresponding to the sample images to be detected.

[0155] The minimum feature distance corresponding to the extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold is calculated based on different feature detection algorithms.

[0156] Optionally, the second determining sub-module 7205 is configured to:

[0157] perform image set division on the sample images to be detected according to the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to the sample images to be detected;

[0158] perform feature detection on the sample images to be detected in the image sets by using the plurality of feature detection algorithms, to obtain sample feature vectors corresponding to the feature detection algorithms in different image sets;

[0159] calculate feature vector distances between the sample feature vectors corresponding to the feature detection algorithms in different image sets and the reference feature vectors corresponding to the feature detection algorithms;

[0160] in a case where the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent extrinsic parameter sub-matrices are different, the reference extrinsic parameter matrix threshold is taken as the preset extrinsic parameter matrix threshold.

[0161] Optionally, the second determining sub-module 7205 is configured to:

[0162] in a case where the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent extrinsic parameter sub-matrices are the same, repeatedly perform the following steps:

[0163] reset the reference extrinsic parameter matrix threshold to obtain a new plurality of extrinsic parameter sub-matrices;

[0164] perform the steps from the image set division on the sample images to be detected according to the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to the sample images to be detected to the calculation of the feature vector distances between the sample feature vectors corresponding to the feature detection algorithms in different image sets and the reference feature vectors corresponding to the feature detection algorithms;

[0165] determine whether the feature detection algorithms corresponding to the minimum feature vector distances of the adjacent extrinsic parameter sub-matrices are the same;

[0166] Until the minimum eigenvector distance corresponding to the adjacent outer parameter sub-matrix corresponds to different feature detection algorithms, the reference outer parameter matrix threshold set this time is taken as the preset outer parameter matrix threshold.

[0167] Optionally, the dividing sub-module 7204 is configured to:

[0168] According to the plurality of outer parameter sub-matrices, determine the outer parameter sub-matrix where the outer parameter matrix corresponding to each of the to-be-detected sample images is located;

[0169] Divide the to-be-detected sample images corresponding to the outer parameter matrix into the same image set in the same outer parameter sub-matrix until the to-be-detected sample images are all divided, and obtain the image set.

[0170] Optionally, the second determining module 720 is configured to:

[0171] Determine the scene type when the to-be-detected image is collected, and each scene type is correspondingly provided with a preset outer parameter matrix threshold;

[0172] According to the outer parameter matrix and the preset outer parameter matrix threshold corresponding to the scene type, determine the target outer parameter sub-matrix where the outer parameter matrix is located.

[0173] As to the apparatus in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in details in the embodiments related to the method, and thus will not be described in details here.

[0174] In addition, it is worth noting that, for the convenience and brevity of description, the embodiments described in the specification all belong to preferred embodiments, and the parts involved are not necessarily necessary for the present application. For example, the second determining module 720 and the adjusting module 730 can be independent apparatuses or the same apparatus in specific implementation, and the present disclosure does not limit this.

[0175] According to the embodiments of the present disclosure, an electronic device is provided, comprising:

[0176] a processor;

[0177] a memory for storing processor-executable instructions;

[0178] wherein the processor is configured to:

[0179] in response to obtaining a to-be-detected image, determine an outer parameter matrix of the to-be-detected image relative to a corresponding stitched image;

[0180] According to the extrinsic parameter matrix and a preset extrinsic parameter matrix threshold set, a target extrinsic parameter sub-matrix in which the extrinsic parameter matrix is located is determined, wherein a preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides a preset extrinsic parameter matrix into a plurality of extrinsic parameter sub-matrices, and extrinsic parameter sub-matrices on both sides of the preset extrinsic parameter matrix threshold correspond to different feature detection algorithms;

[0181] The target extrinsic parameter sub-matrix is used for feature detection on the image to be detected.

[0182] The present disclosure provides a computer readable storage medium, which stores computer program instructions, and the program instructions are executed by a processor to implement the steps of the image feature detection method provided by the present disclosure.

[0183] Figure 9 is a block diagram of an apparatus 900 for image feature detection according to an exemplary embodiment. For example, the apparatus 900 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0184] Referring to Figure 9 , the apparatus 900 can include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0185] The processing component 902 usually controls the overall operation of the apparatus 900, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 902 can include one or more processors 920 to execute instructions to complete all or part of the steps of the image feature detection method described above. In addition, the processing component 902 can include one or more modules to facilitate interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0186] The memory 904 is configured to store various types of data to support the operation of the device 900. Examples of these data include instructions for any application or method operating on the device 900, contact data, phonebook data, messages, pictures, videos, and the like. The memory 904 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0187] The power component 906 provides power to the various components of the device 900. The power component 906 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.

[0188] The multimedia component 908 includes a screen providing an output interface between the device 900 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensors can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. The front and rear cameras can receive external multimedia data when the device 900 is in an operation mode such as a photographing mode or a video mode. Each of the front and rear cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0189] The audio component 910 is configured to output and / or input an audio signal. For example, the audio component 910 includes a microphone (MIC) configured to receive an external audio signal when the device 900 is in an operation mode such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 includes a speaker for outputting an audio signal.

[0190] The I / O interface 912 provides an interface between the processing component 902 and peripheral interface modules such as a keyboard, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0191] The sensor component 914 includes one or more sensors for providing status assessments for various aspects of the device 900. For example, the sensor component 914 can detect an open / closed position of the device 900, relative positioning of components, such as a display and keypad of the device 900, a change in position of the device 900 or a component of the device 900, presence or absence of user contact with the device 900, orientation or acceleration / deceleration of the device 900, and temperature changes of the device 900. The sensor component 914 can include proximity sensor(s) configured to detect presence of nearby objects without any physical contact. The sensor component 914 can include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 914 can include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0192] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and another device. The device 900 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 916 includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0193] In an example embodiment, the device 900 can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements to perform the image feature detection method described above.

[0194] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 904 including instructions, is provided, which can be executed by the processor 920 of the device 900 to complete the image feature detection method described above. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0195] In another exemplary embodiment, a computer program product is provided, the computer program product comprising a computer program being executable by a programmable apparatus, the computer program having code portions for performing the image feature detection method described above when the computer program is executed by the programmable apparatus.

[0196] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0197] It is to be understood that the disclosure is not limited to the precise construction described above and shown in the attached drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the disclosure is limited only by the claims that follow.

Claims

1. An image feature detection method, characterized in that, include: In response to acquiring the image to be detected, the extrinsic parameter matrix of the image to be detected relative to the corresponding stitched image is determined; Based on the extrinsic parameter matrix and the preset extrinsic parameter matrix threshold set, the target extrinsic parameter submatrix in which the extrinsic parameter matrix is ​​located is determined. The preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides the preset extrinsic parameter matrix into multiple extrinsic parameter submatrices. The feature detection algorithms corresponding to the extrinsic parameter submatrices on both sides of the preset extrinsic parameter matrix threshold are different. The image to be detected is subjected to feature detection using a feature detection algorithm corresponding to the target extrinsic parameter submatrix.

2. The method according to claim 1, characterized in that, The preset extrinsic parameter matrix threshold is determined in the following way: The image of the sample object is subjected to feature detection by multiple feature detection algorithms, and the reference feature vectors corresponding to each feature detection algorithm are obtained. Acquire multiple images of the sample object to be detected in panoramic shooting mode; Determine the extrinsic parameter matrix of each of the sample images to be detected relative to the corresponding stitched image; By setting a threshold for the reference extrinsic parameter matrix, the preset extrinsic parameter matrix is ​​divided into multiple extrinsic parameter submatrices; Based on the multiple extrinsic parameter sub-matrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrices corresponding to each of the sample images to be detected, a preset extrinsic parameter matrix threshold is determined. The minimum feature distances corresponding to the extrinsic parameter submatrices on both sides of the preset extrinsic parameter matrix threshold are calculated based on different feature detection algorithms.

3. The method according to claim 2, characterized in that, The step of determining a preset extrinsic parameter matrix threshold based on the plurality of extrinsic parameter sub-matrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrix corresponding to each of the sample images to be detected includes: Based on the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to each of the sample images to be detected, the sample images to be detected are divided into image sets. The various feature detection algorithms are used to detect the features of the sample images to be detected in the image set, so as to obtain the sample feature vectors corresponding to each feature detection algorithm for different image sets. Calculate the feature vector distance between the sample feature vector corresponding to each feature detection algorithm and the reference feature vector corresponding to that feature detection algorithm in different image sets; When the feature detection algorithms corresponding to the minimum feature vector distance between adjacent extrinsic submatrices are different, the reference extrinsic matrix threshold is used as the preset extrinsic matrix threshold.

4. The method according to claim 3, characterized in that, The method includes: If the feature detection algorithms corresponding to the minimum eigenvector distance between adjacent extrinsic submatrices are the same, repeat the following steps: The threshold of the reference extrinsic parameter matrix is ​​reset to obtain multiple new extrinsic parameter submatrices; The process involves dividing the image set of the sample image to be detected based on the multiple extrinsic sub-matrices and the extrinsic matrix corresponding to each of the sample images to be detected, and calculating the feature vector distance between the sample feature vector corresponding to each feature detection algorithm and the reference feature vector corresponding to the feature detection algorithm in different image sets. Determine whether the feature detection algorithms corresponding to the minimum eigenvector distance between adjacent extrinsic submatrices are the same; Until the feature detection algorithms corresponding to the minimum feature vector distances of adjacent extrinsic submatrices are different, the threshold of the reference extrinsic matrix set this time will be used as the preset extrinsic matrix threshold.

5. The method according to claim 4, characterized in that, The step of partitioning the image set of the sample images to be detected based on the plurality of extrinsic parameter sub-matrices and the extrinsic parameter matrices corresponding to each of the sample images to be detected includes: Based on the plurality of extrinsic parameter sub-matrices, determine the extrinsic parameter sub-matrices in which the extrinsic parameter matrix corresponding to each of the sample images to be detected is located; The extrinsic parameter matrix is ​​divided into the same image set for the sample images to be detected within the same extrinsic parameter submatrix, until all the sample images to be detected are divided, thus obtaining the image set.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the target extrinsic parameter submatrix containing the extrinsic parameter matrix based on the extrinsic parameter matrix and a preset set of extrinsic parameter matrix thresholds includes: Determine the scene type when acquiring the image to be detected, and set a preset extrinsic parameter matrix threshold for each scene type; Based on the extrinsic parameter matrix and the preset extrinsic parameter matrix threshold corresponding to the scene type, the target extrinsic parameter submatrix containing the extrinsic parameter matrix is ​​determined.

7. An image feature detection device, characterized in that, include: The first determining module is configured to determine the extrinsic parameter matrix of the image to be detected relative to the corresponding stitched image in response to acquiring the image to be detected. The second determining module is configured to determine the target extrinsic submatrix where the extrinsic matrix is ​​located based on the extrinsic matrix and a preset extrinsic matrix threshold set, wherein the preset extrinsic matrix threshold in the preset extrinsic matrix threshold set divides the preset extrinsic matrix into multiple extrinsic submatrixes, and the feature detection algorithms corresponding to the extrinsic submatrixes on both sides of the preset extrinsic matrix threshold are different. The feature detection module is configured to perform feature detection on the image to be detected using a feature detection algorithm corresponding to the target extrinsic parameter submatrix.

8. The apparatus according to claim 7, characterized in that, The second determining module includes: The first detection submodule is configured to perform feature detection on the image of the sample object using multiple feature detection algorithms, and obtain the reference feature vectors corresponding to each feature detection algorithm. The acquisition submodule is configured to acquire multiple images of the sample object to be detected in panoramic shooting mode; The first determining submodule is configured to determine the extrinsic parameter matrix of each of the sample images to be detected relative to the corresponding stitched image; The sub-module is configured to divide a preset extrinsic matrix into multiple extrinsic sub-matrices by setting a threshold for the reference extrinsic matrix. The second determining submodule is configured to determine a preset extrinsic parameter matrix threshold based on the plurality of extrinsic parameter submatrices, the reference feature vectors corresponding to each feature detection algorithm, and the extrinsic parameter matrices corresponding to each of the sample images to be detected. The minimum feature distances corresponding to the extrinsic parameter submatrices on both sides of the preset extrinsic parameter matrix threshold are calculated based on different feature detection algorithms.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: In response to acquiring the image to be detected, the extrinsic parameter matrix of the image to be detected relative to the corresponding stitched image is determined; Based on the extrinsic parameter matrix and the preset extrinsic parameter matrix threshold set, the target extrinsic parameter submatrix in which the extrinsic parameter matrix is ​​located is determined. The preset extrinsic parameter matrix threshold in the preset extrinsic parameter matrix threshold set divides the preset extrinsic parameter matrix into multiple extrinsic parameter submatrices. The feature detection algorithms corresponding to the extrinsic parameter submatrices on both sides of the preset extrinsic parameter matrix threshold are different. The image to be detected is subjected to feature detection using a feature detection algorithm corresponding to the target extrinsic parameter submatrix.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-6.

Images