Feature extraction and descriptor generation method and system based on convolutional neural network

Abstract

The present application belongs to the technical field of computer vision and navigation positioning, and provides a method and system for feature extraction and descriptor generation based on convolutional neural network. The system includes an image data acquisition and calibration module, an image dedistortion module, a feature point extraction module, a descriptor generation and feature point matching module, a loss function calculation module, a loss function back propagation training module and a post-training extraction generation module. Feature point pairs are generated by extracting feature points and generating descriptors for dedistorted images. Reprojection errors are calculated based on the feature point pairs and camera intrinsic parameters. The reprojection errors are used to train the convolutional neural networks of the feature point extraction module and the descriptor generation and feature point matching modules, and weight parameters are adjusted to obtain a trained convolutional neural network. The trained convolutional neural network is used to directly extract feature points and generate descriptors for the dedistorted images. Data calibration is not required, and the system belongs to unsupervised learning with higher accuracy.

Description

Technical Field

[0001] The present application belongs to the field of computer vision and navigation positioning technology, and specifically relates to a feature extraction and descriptor generation method and system based on convolutional neural network. Background Art

[0002] Image feature points and descriptors are the basis of many practical applications, such as the registration of multiple images, 3D reconstruction using multiple 2D images, and restoration of scene 3D structures. These applications usually rely on an excellent local image feature point and descriptor to establish the correspondence between image points. With the development of dual-view geometry, the algorithms for feature point extraction and descriptor generation have an increasingly greater impact on the results of the final pose solution of dual-view geometry. Some of the algorithms currently used, such as FAST feature points, HARRIS feature points, SIFT descriptors, and BRIEF descriptors, are not as satisfactory in terms of algorithm time and accuracy.

[0003] With the emergence of large-scale labeled image datasets, neural networks have begun to be widely used to handle computer vision-related tasks. Among them, using them in feature point extraction and descriptor generation can bring good results. Deep convolutional neural networks have powerful expressive power in image feature extraction. Through learning and training with a large amount of labeled data, the trained deep convolutional neural network is used to extract feature points and generate descriptors.

[0004] However, the existing image feature point extraction and descriptor generation based on convolutional neural networks still have the following defects: generally, supervised learning methods are required, which requires a lot of data labeling work; convolutional neural networks generally take a long time to calculate, but the generation of feature points and descriptors requires real-time calculation during simultaneous positioning and mapping to meet application requirements. Once the calculation cannot be achieved in real time, it often leads to frame loss during simultaneous positioning and mapping, making the final positioning and mapping results inaccurate. Summary of the invention

[0005] The present application provides a method and system for feature extraction and descriptor generation based on a convolutional neural network, so as to provide a method and system for feature point extraction and descriptor generation for dual-view geometry calculation that has a higher accuracy and can be generated through unsupervised learning.

[0006] The first aspect of the present application provides a feature extraction and descriptor generation system based on a convolutional neural network, including an image data acquisition and calibration module, an image dedistortion module, a feature point extraction module, a descriptor generation and feature point matching module, a loss function calculation module, a loss function back propagation training module and a post-training extraction generation module;

[0007] The image data acquisition and calibration module is used to collect multiple groups of image sensor images and calibrate the collected images to obtain camera intrinsic parameters and distortion parameters, wherein each group of images collected includes two two-dimensional images of the same object taken from different viewing angles;

[0008] The image dedistortion module is used to receive the image, camera intrinsic parameters and distortion parameters collected by the image data acquisition and calibration module, and perform dedistortion processing on the collected image;

[0009] The feature point extraction module is connected to the image dedistortion module and is used to extract feature points from the dedistorted image using a convolutional neural network;

[0010] The descriptor generation and feature point matching module is connected to the feature point extraction module, and is used to use a convolutional neural network to describe the feature points, generate descriptors, and perform dual-view feature matching based on the minimum-norm criterion for each group of images according to the feature points and the descriptors to generate feature point pairs;

[0011] The loss function calculation module is connected to the descriptor generation and feature point matching module, and is used to obtain feature point pairs, use the epipolar geometry constraint reprojection error as the loss function, and calculate the reprojection error;

[0012] The loss function back propagation training module is connected to the loss function calculation module, and is used to back propagate the reprojection error, train the parameters of the convolutional neural network of the feature point extraction module, and obtain the first convolutional neural network; train the parameters of the convolutional neural network of the descriptor generation and feature point matching module, and obtain the second convolutional neural network;

[0013] The post-training extraction and generation module is used to extract feature points using the first convolutional neural network, and to generate descriptors based on the feature points and the second convolutional neural network.

[0014] Optionally, the image data acquisition and calibration module uses a camera that captures RGB images to collect images.

[0015] Optionally, the step of calibrating the captured image to obtain camera intrinsic parameters and distortion parameters includes:

[0016] The image is calibrated through the pinhole camera model or the fisheye camera model, and the camera intrinsic parameters and distortion parameters are output. The images collected by the camera with a small field of view angle are calibrated using the pinhole camera model, and the images collected by the camera with a large field of view angle are calibrated using the fisheye camera model.

[0017] Optionally, the image dedistortion module uses a pinhole camera model to dedistort images captured by a camera with a small viewing angle, and uses a fisheye camera model to dedistort images captured by a camera with a large viewing angle.

[0018] Optionally, each set of images collected has partial overlap.

[0019] Optionally, the convolutional neural network of the feature point extraction module and the convolutional neural network of the feature point matching module are both shallow convolutional neural networks.

[0020] Optionally, the using the epipolar geometry constrained reprojection error as a loss function to calculate the reprojection error includes:

[0021] Based on the feature point pairs, epipolar geometry is used to solve the relative pose, and the rotation matrix and translation matrix between the two views are calculated through epipolar geometry constraints;

[0022] According to the relative pose and camera intrinsic parameters, the pixel coordinates of the feature point pairs are reprojected, and the sum of the projection errors formed by projecting the feature points onto the matched image is calculated to obtain the reprojection error.

[0023] Optionally, the loss function back-propagation training module trains parameters of the convolutional neural network of the feature point extraction module and the convolutional neural network of the feature point matching module in sequence according to the reprojection errors of multiple groups of images.

[0024] Optionally, the parameters of the convolutional neural network of the feature point extraction module and the convolutional neural network of the feature point matching module are trained in sequence, wherein the parameters of the convolutional neural network of the feature point extraction module and the convolutional neural network of the feature point matching module are trained each time, including:

[0025] Receiving reprojection errors of a set of images in a loss function calculation module;

[0026] Determine whether the reprojection error is greater than a preset threshold. If the reprojection error is greater than the preset threshold, input the reprojection error to the feature point extraction module and the descriptor generation and feature point matching module. According to the reprojection error, adjust the convolutional neural network weights of the feature point extraction module and the convolutional neural network weights of the feature point matching module. According to the convolutional neural network with adjusted weights, re-perform feature point extraction and descriptor generation, regenerate feature point pairs, and calculate the reprojection error.

[0027] Repeat the above steps until the reprojection error is less than or equal to the preset threshold.

[0028] On the other hand, the present application provides a feature extraction and descriptor generation method based on a convolutional neural network, comprising the following steps:

[0029] Collecting multiple sets of image sensor images, and calibrating the collected images to obtain camera intrinsic parameters and distortion parameters, wherein each set of collected images includes two two-dimensional images of the same object taken from different viewing angles;

[0030] De-distort the captured image according to the camera's intrinsic parameters and distortion parameters;

[0031] A convolutional neural network is used to extract feature points from the dedistorted image.

[0032] A convolutional neural network is used to describe the extracted feature points and generate descriptors, and dual-view feature matching based on the minimum-norm criterion is performed on each group of images according to the feature points and descriptors to generate feature point pairs;

[0033] According to the feature point pairs, the reprojection error is calculated by using the epipolar geometry constrained reprojection error as the loss function;

[0034] The reprojection error is back-propagated, and the parameters of the convolutional neural network extracted by the feature points are trained to obtain the first convolutional neural network; the parameters of the convolutional neural network generated by the descriptor are trained to obtain the second convolutional neural network;

[0035] The first convolutional neural network and the second convolutional neural network are applied to dual-view geometry calculation, feature points are extracted using the first convolutional neural network, and descriptors are generated based on the feature points and the second convolutional neural network.

[0036] It can be seen from the above technical scheme that the present application provides a feature extraction and descriptor generation system based on a convolutional neural network, including an image data acquisition and calibration module, an image dedistortion module, a feature point extraction module, a descriptor generation and feature point matching module, a loss function calculation module, a loss function back propagation training module and a post-training extraction generation module. Feature point extraction and descriptor generation are performed on two dedistorted images of different viewing angles in each group of images, feature point matching is performed according to the descriptor, and feature point pairs are generated. The reprojection error is calculated according to the feature point pairs and the camera intrinsic parameters obtained by the image data acquisition and calibration module. Because the reprojection error is related to the feature point extraction and descriptor generation, the convolutional neural networks of the feature point extraction module and the descriptor generation and feature point matching module are trained using the size of the reprojection error, and the weight parameters are adjusted until the reprojection error of each group is less than or equal to a preset threshold, and the trained convolutional neural networks are obtained, which are respectively a first convolutional neural network for extracting feature points and a second convolutional neural network for generating descriptors. The whole process does not require data calibration, belongs to unsupervised learning, and has a higher accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 It is a structural diagram of a feature extraction and descriptor generation system based on a convolutional neural network provided in an embodiment of the present application.

[0039] Figure 2 It is a structural diagram of image calibration in a feature extraction and descriptor generation system based on a convolutional neural network provided in an embodiment of the present application.

[0040] Figure 3 It is a structural schematic diagram of a convolutional neural network of a feature point extraction module in a feature extraction and descriptor generation system based on a convolutional neural network provided in an embodiment of the present application.

[0041] Figure 4 It is a structural schematic diagram of a convolutional neural network of a descriptor generation and feature point matching module in a feature extraction and descriptor generation system based on a convolutional neural network provided in an embodiment of the present application.

[0042] Figure 5 It is a flowchart of the feature extraction and descriptor generation method based on convolutional neural network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0044] like Figure 1 As shown, it is a structural schematic diagram of the feature extraction and descriptor generation system based on convolutional neural network provided in an embodiment of the present application, wherein the feature extraction and descriptor generation system based on convolutional neural network includes an image data acquisition and calibration module 100, an image dedistortion module 200, a feature point extraction module 300, a descriptor generation and feature point matching module 400, a loss function calculation module 500, a loss function back propagation training module 600 and a post-training extraction generation module 700.

[0045] The image data acquisition and calibration module 100 is used to collect multiple groups of image sensor images and calibrate the collected images to obtain camera intrinsic parameters and distortion parameters, wherein each group of collected images includes two two-dimensional images of the same object taken from different viewing angles.

[0046] The embodiment of the present application uses a camera that shoots RGB images to collect multiple groups of images, each group of images includes two two-dimensional images of the same object shot from different perspectives, and each group of images has partial overlap, wherein some images can be selected for training and some images can be selected for testing. During training, the data of one group of images are first back-propagated and trained to adjust the convolutional neural network parameters, and then the data of the next group of images are back-propagated and trained to adjust the convolutional neural network parameters, and the process is repeated in sequence.

[0047] After acquiring the image, you first need to calibrate the image to obtain the camera intrinsic parameters and distortion parameters, see Figure 2 , which is a structural schematic diagram of image calibration in a feature extraction and descriptor generation system based on a convolutional neural network provided in an embodiment of the present application. In the embodiment of the present application, an image is calibrated through a pinhole camera model 101 or a fisheye camera model 102, and camera intrinsic parameters and distortion parameters are output. The image acquired by a camera with a small field of view angle is calibrated using the pinhole camera model 101, and the image acquired by a camera with a large field of view angle is calibrated using the fisheye camera model 102.

[0048] The image dedistortion module 200 is used to receive the image, camera intrinsic parameters and distortion parameters collected by the image data acquisition and calibration module 100, and perform dedistortion processing on the collected image.

[0049] Since the captured image has distortion parameters, it is not conducive to the extraction of feature points. Therefore, the image needs to be dedistorted to minimize the influence of distortion on subsequent feature extraction and descriptor generation. In the embodiment of the present application, a pinhole camera model 101 is used to dedistort the image captured by a camera with a small viewing angle, and a fisheye camera model 102 is used to dedistort the image captured by a camera with a large viewing angle.

[0050] The feature point extraction module 300 is connected to the image dedistortion module 200 and is used to extract feature points from the dedistorted image using a convolutional neural network.

[0051] After obtaining the dedistorted image data, the image is subjected to feature extraction using a convolutional neural network. Figure 3 , which is a structural schematic diagram of the convolutional neural network of the feature point extraction module in the feature extraction and descriptor generation system based on the convolutional neural network provided in the embodiment of the present application. Since the traditional deep convolutional neural network feature point extraction method is very time-consuming, the convolutional neural network used to extract image feature points in the embodiment of the present application is shallow, so as to ensure the effect of feature point extraction without reducing the performance, specifically including the convolutional layer 1, the maximum pooling layer, the convolutional layer 2, the maximum pooling layer, the fully connected layer, Relu, the fully connected layer and Softmax connected in sequence.

[0052] The descriptor generation and feature point matching module 400 is connected to the feature point extraction module 300, and is used to use a convolutional neural network to describe the feature points, generate descriptors, and perform dual-view feature matching based on the minimum-norm criterion for each group of images according to the feature points and descriptors to generate feature point pairs.

[0053] See also Figure 4, which is a schematic diagram of the structure of the convolutional neural network in the descriptor generation and feature point matching module in the feature extraction and descriptor generation system based on the convolutional neural network provided in the embodiment of the present application. Since the traditional descriptor generation method is very time-consuming, the present application uses a shallow convolutional neural network to calculate the descriptors corresponding to the corresponding feature points to ensure the effect of the descriptor calculation without reducing the performance. The dedistorted image and feature points are input into the convolutional neural network of the descriptor generation and feature point matching module 400, and the descriptor is generated after passing through the convolutional layer 1, the maximum pooling layer, the convolutional layer 2, the maximum pooling layer, the fully connected layer, the Relu, the fully connected layer and the Softmax in sequence. According to the descriptor, the same feature point represented by the two images in each group of images is matched and a feature point pair is generated.

[0054] The loss function calculation module 500 is connected to the descriptor generation and feature point matching module 400, and is used to obtain feature point pairs, use the epipolar geometry constraint reprojection error as the loss function, and calculate the reprojection error.

[0055] In the design of the loss function, the embodiment of the present application adopts the reprojection error in epipolar geometry. Epipolar geometry originally belongs to the method of multi-view geometry, which can be used to solve the relative pose of two images. Its reprojection error is defined as the sum of the projection errors formed by projecting the feature points of the two images onto the matched image based on the obtained relative pose. This sum is generally not 0. This value can be reduced by adjusting the relative pose. This value will have an absolute minimum value. The size of this minimum value is related to the selection of feature points and the matching of feature points. The matching of feature points is related to the descriptor of each feature point. The minimum value of the reprojection error is related to the extraction of feature points and the generation of feature point descriptors. Therefore, after the reprojection error is calculated, it can be used to update the generation of feature points and descriptors.

[0056] The epipolar geometry constrained reprojection error is used as a loss function to calculate the reprojection error, specifically including:

[0057] According to the feature point pairs, epipolar geometry is used to solve the relative pose, and the rotation matrix and translation matrix between the dual views are calculated through epipolar geometry constraints; according to the relative pose and camera intrinsic parameters, the pixel coordinates of the feature point pairs are reprojected, and the sum of the projection errors formed by projecting the feature points onto the matched image is calculated to obtain the reprojection error. Therefore, the whole process does not require data calibration and can be regarded as an unsupervised learning.

[0058] The loss function back propagation training module 600 is connected to the feature point extraction module 300, the descriptor generation and feature point matching module 400 and the loss function calculation module 500, and is used to back propagate the reprojection error, train the parameters of the convolutional neural network of the feature point extraction module 300, and obtain the first convolutional neural network; train the parameters of the convolutional neural network of the descriptor generation and feature point matching module 400, and obtain the second convolutional neural network.

[0059] The loss function back propagation training module 600 receives data from the loss function calculation module 500, the data including reprojection errors of multiple groups of images, and trains the parameters of the convolutional neural network of the feature point extraction module 300 and the convolutional neural network of the feature point matching module 400 in sequence according to the reprojection errors of the multiple groups of images, wherein each training specifically includes:

[0060] Determine whether the reprojection error is greater than a preset threshold. If the reprojection error is greater than the preset threshold, the reprojection error is input to the feature point extraction module 300 and the descriptor generation and feature point matching module 400. According to the reprojection error, adjust the convolutional neural network weights of the feature point extraction module 300 and the convolutional neural network weights of the feature point matching module 400. According to the convolutional neural network after the weight adjustment, re-perform feature point extraction and descriptor generation, regenerate feature point pairs and calculate reprojection errors. Then input the recalculated reprojection error to the loss function back propagation training module 600 to determine whether it is greater than the preset threshold. Repeat the above steps until the reprojection error is less than or equal to the preset threshold, and the training is completed.

[0061] The post-training extraction generation module 700 is used to extract feature points using the first convolutional neural network, and to generate descriptors based on the feature points and the second convolutional neural network.

[0062] After enough dual-view training, the first convolutional neural network for feature point extraction and the second convolutional neural network for descriptor generation are obtained. After the other dual views are dedistorted, there is no need to train through reprojection error calculation. The first convolutional neural network can be directly input for feature point extraction and the second convolutional neural network can be directly input for descriptor generation.

[0063] In order to more clearly illustrate the feature extraction and descriptor generation system based on convolutional neural network provided in the embodiment of the present application, the embodiment of the present application also provides a feature extraction and descriptor generation method based on convolutional neural network, see Figure 5 , is a flow chart of a feature extraction and descriptor generation method based on a convolutional neural network provided in an embodiment of the present application. The feature extraction and descriptor generation method based on a convolutional neural network includes steps 1 to 7.

[0064] Step 1: Collect multiple groups of image sensor images, and calibrate the collected images to obtain camera intrinsic parameters and distortion parameters, wherein each group of collected images includes two two-dimensional images of the same object taken from different perspectives.

[0065] Step 2: Dedistort the captured image based on the camera's intrinsic parameters and distortion parameters.

[0066] Step 3: Use a convolutional neural network to extract feature points from the dedistorted image.

[0067] Step 4: Use a convolutional neural network to describe the extracted feature points and generate descriptors, and perform dual-view feature matching based on the minimum-norm criterion for each group of images according to the feature points and descriptors to generate feature point pairs.

[0068] Step 5: Based on the feature point pairs, the epipolar geometry constrained reprojection error is used as the loss function to calculate the reprojection error.

[0069] Step 6: Back-propagate the reprojection error, train the parameters of the convolutional neural network extracted from the feature points, and obtain the first convolutional neural network; train the parameters of the convolutional neural network generated by the descriptor, and obtain the second convolutional neural network.

[0070] Step 7: Apply the first convolutional neural network and the second convolutional neural network to dual-view geometry calculations, use the first convolutional neural network to extract feature points, and generate descriptors based on the feature points and the second convolutional neural network.

[0071] It can be seen from the above technical solution that the embodiment of the present application provides a method and system for feature extraction and descriptor generation based on a convolutional neural network. The system includes an image data acquisition and calibration module, an image dedistortion module, a feature point extraction module, a descriptor generation and feature point matching module, a loss function calculation module, a loss function back propagation training module, and a post-training extraction generation module. Feature point extraction and descriptor generation are performed on two dedistorted images of different viewing angles in each group of images, and feature point matching is performed according to the descriptor to generate feature point pairs. The reprojection error is calculated according to the feature point pairs and the camera intrinsic parameters obtained by the image data acquisition and calibration module. Because the reprojection error is related to the feature point extraction and descriptor generation, the convolutional neural networks of the feature point extraction module and the descriptor generation and feature point matching module are trained using the size of the reprojection error, and the weight parameters are adjusted until the reprojection error of each group is less than or equal to a preset threshold, and the trained convolutional neural networks are obtained, which are respectively a first convolutional neural network for extracting feature points and a second convolutional neural network for generating descriptors. The whole process does not require data calibration, belongs to unsupervised learning, and has a higher accuracy rate.

[0072] The present application is described in detail above in conjunction with specific implementation methods and exemplary examples, but these descriptions cannot be understood as limiting the present application. Those skilled in the art understand that, without departing from the spirit and scope of the present application, a variety of equivalent replacements, modifications or improvements can be made to the technical solution of the present application and its implementation methods, all of which fall within the scope of the present application. The scope of protection of the present application shall be subject to the attached claims.

Claims

1. A feature extraction and descriptor generation system based on convolutional neural networks, characterized in that: It comprises an image data acquisition and calibration module (100), an image dedistortion module (200), a feature point extraction module (300), a descriptor generation and feature point matching module (400), a loss function calculation module (500), a loss function back propagation training module (600) and a post-training extraction generation module (700); The image data acquisition and calibration module (100) is used to collect multiple groups of image sensor images and calibrate the collected images to obtain camera internal parameters and distortion parameters, wherein each group of collected images includes two two-dimensional images of the same object taken from different viewing angles; The image dedistortion module (200) is used to receive the image, camera internal parameters and distortion parameters collected by the image data acquisition and calibration module (100), and perform dedistortion processing on the collected image; The feature point extraction module (300) is connected to the image dedistortion module (200) and is used to extract feature points from the dedistorted image using a convolutional neural network; The descriptor generation and feature point matching module (400) is connected to the feature point extraction module (300) and is used to use a convolutional neural network to perform feature description on the feature points to generate descriptors, and to perform dual-view feature matching based on a minimum-norm criterion on each group of images according to the feature points and the descriptors to generate feature point pairs; The loss function calculation module (500) is connected to the descriptor generation and feature point matching module (400), and is used to obtain feature point pairs, use epipolar geometry constraint reprojection errors as loss functions, and calculate reprojection errors; The loss function back propagation training module (600) is connected to the feature point extraction module (300), the descriptor generation and feature point matching module (400) and the loss function calculation module (500), and is used to back propagate the reprojection error, train the parameters of the convolutional neural network of the feature point extraction module (300), and obtain a first convolutional neural network; and train the parameters of the convolutional neural network of the descriptor generation and feature point matching module (400), and obtain a second convolutional neural network; The post-training extraction and generation module (700) is used to extract feature points using the first convolutional neural network, and to generate descriptors based on the feature points and the second convolutional neural network; The method of using the epipolar geometry constrained reprojection error as a loss function to calculate the reprojection error includes: Based on the feature point pairs, epipolar geometry is used to solve the relative pose, and the rotation matrix and translation matrix between the two views are calculated through epipolar geometry constraints; According to the relative pose and camera internal parameters, the pixel coordinates of the feature point pairs are reprojected, and the sum of the projection errors formed by projecting the feature points onto the matched image is calculated to obtain the reprojection error. The loss function back propagation training module (600) sequentially trains the parameters of the convolutional neural network of the feature point extraction module (300) and the convolutional neural network of the descriptor generation and feature point matching module (400) according to the reprojection errors of the multiple groups of images; The parameters of the convolutional neural network of the feature point extraction module (300) and the convolutional neural network of the descriptor generation and feature point matching module (400) are trained in sequence, wherein the parameters of the convolutional neural network of the feature point extraction module (300) and the convolutional neural network of the descriptor generation and feature point matching module (400) are trained each time, including: Receiving a reprojection error of a set of images in a loss function calculation module (500); Determining whether the reprojection error is greater than a preset threshold value, if the reprojection error is greater than the preset threshold value, inputting the reprojection error into a feature point extraction module (300) and a descriptor generation and feature point matching module (400), adjusting the convolutional neural network weights of the feature point extraction module (300) and the convolutional neural network weights of the descriptor generation and feature point matching module (400) according to the reprojection error, re-performing feature point extraction and descriptor generation according to the convolutional neural network after the weight adjustment, and re-generating feature point pairs and calculating the reprojection error; Repeat the above steps until the reprojection error is less than or equal to the preset threshold.

2. The feature extraction and descriptor generation system based on convolutional neural network according to claim 1, characterized in that: The image data acquisition and calibration module (100) uses a camera for taking RGB images to collect images.

3. The feature extraction and descriptor generation system based on convolutional neural network according to claim 1, characterized in that: The step of calibrating the captured image to obtain camera intrinsic parameters and distortion parameters includes: An image is calibrated using a pinhole camera model (101) or a fisheye camera model (102), and camera internal parameters and distortion parameters are output, wherein an image acquired by a camera with a small field of view angle is calibrated using the pinhole camera model (101), and an image acquired by a camera with a large field of view angle is calibrated using the fisheye camera model (102).

4. The feature extraction and descriptor generation system based on convolutional neural network according to claim 1, characterized in that: The image dedistortion module (200) uses a pinhole camera model (101) to dedistort images collected by a camera with a small viewing angle, and uses a fisheye camera model (102) to dedistort images collected by a camera with a large viewing angle.

5. The feature extraction and descriptor generation system based on convolutional neural network according to claim 1, characterized in that: Each group of images collected has partial overlap.

6. The feature extraction and descriptor generation system based on convolutional neural network according to claim 1, characterized in that: The convolutional neural network of the feature point extraction module (300) and the convolutional neural network of the descriptor generation and feature point matching module (400) are both shallow convolutional neural networks.

7. A feature extraction and descriptor generation method based on a convolutional neural network, characterized in that: Applied to the feature extraction and descriptor generation system based on convolutional neural network as claimed in claim 1, the method comprises: Collecting multiple sets of image sensor images, and calibrating the collected images to obtain camera intrinsic parameters and distortion parameters, wherein each set of collected images includes two two-dimensional images of the same object taken from different viewing angles; De-distort the captured image according to the camera's intrinsic parameters and distortion parameters; A convolutional neural network is used to extract feature points from the dedistorted image. A convolutional neural network is used to describe the extracted feature points and generate descriptors, and dual-view feature matching based on the minimum-norm criterion is performed on each group of images according to the feature points and descriptors to generate feature point pairs; According to the feature point pairs, the reprojection error is calculated by using the epipolar geometry constrained reprojection error as the loss function; The reprojection error is back-propagated, and the parameters of the convolutional neural network extracted by the feature points are trained to obtain the first convolutional neural network; the parameters of the convolutional neural network generated by the descriptor are trained to obtain the second convolutional neural network; The first convolutional neural network and the second convolutional neural network are applied to dual-view geometry calculation, feature points are extracted using the first convolutional neural network, and descriptors are generated based on the feature points and the second convolutional neural network.

Images