Discrete point matching method and platform under noise condition based on convolutional neural network

Abstract

The application is a discrete point matching method and platform under noise condition based on convolutional neural network, mainly breaks the traditional thinking barriers, and provides a new method for solving the problem of discrete point matching. Instead of using the traditional data-based matching method, the emerging convolutional neural network technology is introduced into the aspect of discrete point matching, and a discrete point matching network (PMNet) under noise condition based on convolutional neural network is designed, which effectively improves the simulation accuracy and speed. In particular, a discrete point matching platform is also designed, based on which users can realize the matching of image discrete points under noise condition based on PMNet.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, and particularly relates to a matching technology of discrete points in images under noise conditions based on a convolutional neural network. BACKGROUND

[0002] The matching technology of discrete points under noise conditions is widely used in fields such as aircraft homing and navigation, identification and guidance of military strategic weapons, and detection and monitoring of terrain. However, due to the large amount of data to be matched, the long distance, and the inevitable vibration amplitude and environmental noise in the local image shooting process, the matching accuracy and efficiency are seriously restricted.

[0003] At present, people often use data-based methods to realize the matching of discrete points, including triangle algorithm [1], pyramid algorithm [2] and grid algorithm [3]. However, the triangle algorithm and the pyramid algorithm are sensitive to focal plane noise and false points, which leads to a decrease in algorithm performance. Although the grid algorithm has lower sensitivity to focal plane noise and shorter time consumption, gray noise and false points may cause the grid algorithm to fail to identify adjacent points, thereby affecting the matching result. Therefore, simply using the above traditional data-based matching algorithm cannot meet the accuracy requirements of people in the field of discrete point matching.

[0004] Compared with traditional discrete point matching algorithms, convolutional neural networks can learn the internal rules of sample data from a large number of input samples by increasing the depth and iteration number of the network model, especially can learn the features of different levels of images and realize classification and recognition, so it is widely used in the field of image matching and has achieved good results.

[0005] The existing technology has the following disadvantages: at the algorithm level, the current data-based matching method is still the mainstream method in the field of discrete point matching. However, each data-based matching algorithm or its improved algorithm cannot overcome its own data dependency, resulting in large amount of calculation required for matching and slow matching speed. In particular, the noise and false points existing in the local image can seriously affect the matching accuracy. On the other hand, convolutional neural networks are widely used in the field of image matching and have good matching effect, but it is not clear whether this method is suitable for the matching of discrete points.

[0006] The matching algorithm based on convolutional neural network proposed in the present application can effectively solve the problem of false points and gray noise in images, and can efficiently and quickly realize the matching of discrete points. In particular, the present application also constructs a discrete point matching platform based on convolutional neural network, which can effectively assist researchers in this field to match and locate discrete points.

[0007] [1] Liebe CC,. Pattern recognition of star constellations for spacecraft applications. IEEE Aerosp Electron Syst Mag, 1992, 7(6): 34-41.

[0008] [2] Mortari D, Samaan MA, Bruccoleri C. The pyramid star identification technique. Navigation, 2004, 51(3): 171-183.

[0009] [3] Padgett C, Kreutz-Delgado K. A grid algorithm for autonomous star identification. IEEE Trans Aerosp Electron Syst, 1997, 33(1): 202-213. SUMMARY

[0010] In view of the various deficiencies of the prior art, the purpose of the present application is mainly to break through the traditional thinking barriers and provide a new method for solving the problem of discrete point matching. Instead of using the traditional data-based matching method, the emerging convolutional neural network technology is introduced into the field of discrete point matching, and a discrete point matching network (PMNet) under noisy conditions based on convolutional neural network is designed, which effectively improves the simulation accuracy and speed. In particular, a discrete point matching platform is also designed, based on which users can realize the matching of image discrete points under noisy conditions based on PMNet.

[0011] The technical scheme adopted by the present application to solve the technical problem is:

[0012] A discrete point matching method under noisy conditions based on convolutional neural network, characterized in that the method comprises the following contents:

[0013] A reference image is obtained, and after non-significant discrete points are removed according to the gray scale features and the density, the remaining E discrete points are retained; pseudo points with random position information and gray scale values are randomly added, and the E discrete points and 1 pseudo point have E+1 categories; a local image of each category point is formed by taking each point (including discrete points and pseudo points) as the center and taking a square field range as the screenshot;

[0014] A data set containing E+1 categories is obtained for each local image through data enhancement operations including Gaussian noise, random rotation, and flipping; the training set and the test set are divided in the ratio of 8:2, and are used to train the convolutional neural network;

[0015] constructing a convolutional neural network: the input of the convolutional neural network model is a local map, and the output is the class and confidence of the discrete points in the local map;

[0016] The convolutional neural network comprises 9 convolutional layers and 5 pooling layers, and then 3 fully connected layers are used to obtain the final probability of belonging to each class. The specific structure is that the first and second convolutional layers are connected in series and then connected to the first pooling layer, then the third and fourth convolutional layers and the second pooling layer are connected, and then the fifth, sixth and seventh convolutional layers and the third pooling layer are connected, and then the eighth convolutional layer, the fourth pooling layer, the ninth convolutional layer and the fifth pooling layer are connected to the three fully connected layers; the first two fully connected layers are composed of 4096 neurons, and the last fully connected layer is an output layer, which has E+1 neurons, representing E discrete point classes and one pseudo point class, and the probability of each class is calculated using a softmax function; thus the construction of the convolutional neural network is completed.

[0017] The convolutional neural network is trained using the constructed data set to obtain a trained convolutional neural network model for discrete point matching.

[0018] The square field of view range is 5-20°*5-20°, and the preferred field of view range is 10°*10°. In practice, the field of view range is related to the camera field of view. Too small field of view range will result in too few discrete points to match, and too many discrete points will result in low matching accuracy between discrete points. The reference map is applied in aircraft guidance and navigation, identification and guidance of military strategic weapons, and terrain field.

[0019] The platform for matching discrete points under image-based noise conditions comprises a display module, a data reading module, a local map acquisition module, and a noise-containing discrete point matching module based on a PMNet model. The above modules cooperate with each other, read local map information, set processing parameters, obtain the position of the local map based on the PMNet model, and finally display the matching result through the display module of the platform.

[0020] The data reading module can read picture files or picture files in a folder to obtain images to be matched.

[0021] The local map acquisition module is used for image preprocessing of the image to be matched, extracting discrete point position information of the image to be matched, and selecting reference points from the discrete points of the image to be matched to form a local map.

[0022] The noise-containing discrete point matching module is used for matching discrete points based on the PMNet model and outputting the matching result. When the confidence cannot reach the expected level, other discrete points of the image to be matched are selected as new reference points for re-matching until the expected level is reached.

[0023] The PMNet model takes a local graph as input and outputs the category and confidence score of discrete points in the local graph. The PMNet model has a total of E+1 outputs, where E is the number of discrete points in the baseline graph and 1 represents a pseudo-point class, which is used to match the location information of discrete points in the local graph.

[0024] The PMNet model creatively introduces a pseudo-point class in its output, which identifies a point as a pseudo-point if the discrete point to be matched does not belong to the discrete points in the baseline graph.

[0025] Compared with the prior art, the beneficial effects of the present invention are:

[0026] (1) The matching method of this invention uses the PMNet model to achieve discrete point matching under noisy conditions based on images. Since the matching process does not require a large data base, the matching efficiency is higher. At the same time, due to the pseudo-point class introduced in the network, it can be more robust to noise and pseudo-points, and the matching accuracy is higher.

[0027] (2) The present invention constructs a platform for realizing discrete point matching, which has a built-in PMNet model. Based on this platform, it can efficiently realize the matching of discrete points in images under noisy conditions based on the PMNet model, and has higher accuracy and efficiency than traditional matching methods. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the operation of the image-based discrete point matching platform under noisy conditions according to the present invention.

[0029] Figure 2 This is a schematic diagram of the structure of a convolutional neural network according to an embodiment of the present invention;

[0030] Figure 3 The figures show the variation curves of training accuracy and loss function on the training and test sets, according to an embodiment of the present invention.

[0031] Figure 3 Figure a shows the accuracy variation curve of PMNet on the training set; Figure b shows the loss function variation curve of PMNet on the training set; Figure c shows the accuracy variation curve of PMNet on the test set; and Figure d shows the loss function variation curve of PMNet on the test set. The accuracy and loss function stabilize more slowly on the test set than on the training set, which is due to the separation of the test and training sets. Both the training and test sets stabilize after 60 epochs of network operation, with the accuracy consistently remaining above 99.3%, demonstrating that the algorithm effectively guarantees matching accuracy.

[0032] Figure 4 This is a schematic diagram of the system platform interface of the present invention. Detailed Implementation

[0033] The technical solution of the present invention will be clearly and completely described and explained below with reference to the embodiments and accompanying drawings, but it is not intended to limit the scope of protection of this application.

[0034] This invention relates to a discrete point matching platform for images under noisy conditions. The platform mainly includes a display module, a data input module, a local map acquisition module, and a noisy discrete point matching module based on the PMNet model. The data input module reads files to obtain the image to be matched. The local map acquisition module preprocesses the image to be matched, extracts the location information of discrete points in the image, and selects reference points from the discrete points to form a local map. The noisy discrete point matching module matches discrete points based on the PMNet model and outputs the matching results. When the confidence level is insufficient, other discrete points in the image are selected as new reference points for rematching until the desired level is achieved. The display module allows for adjustment of image processing parameters and display of the PMNet model output. Image processing parameters include the image's grayscale threshold, Gaussian kernel standard deviation, and mean (using Gaussian filtering). Image preprocessing directly filters out some white noise and removes discrete points with insignificant grayscale features.

[0035] Considering that noise in images cannot be completely eliminated in real-world scenarios, noise interference is artificially introduced into each image during the training set preparation process. Therefore, the PMNet model trained on a dataset with artificially added false points can still accurately locate the centroid of discrete points even when the image to be matched contains noise that cannot be eliminated by filtering.

[0036] When a local image contains multiple discrete points, using grayscale features as a reference, the discrete points with the most significant grayscale features in the image are selected sequentially as reference points. The image is then moved around these reference points until they become the center of the local image. This image is then the processed discrete point map to be matched, which is essentially a local image of the actual image. Finally, this discrete point map is used as input to the network model to obtain the position information of the reference point in the baseline image. Based on the PMNet model, discrete point matching can be achieved, and the position information of the reference point in the baseline image can be output through a software platform.

[0037] In addition to their own positional information and relative positional information between discrete points, discrete points in a local image also contain grayscale features. The convolutional neural network can match the positions of discrete points in multi-scale images containing grayscale features. A pseudo-point class is creatively introduced into the PMNet model. After training the network model, when pseudo-points are present in local images acquired through photography or other methods, the algorithm can automatically identify their pseudo-point category. When the confidence level of the identification result is lower than the expected level, a new discrete point can be selected as a new reference point for re-identification.

[0038] See the platform operation process. Figure 1 .

[0039] S1: Acquire the raw input feature map data and display the image through the platform. It is worth noting that all images are read in grayscale format; in this embodiment, the image size is 128*128. S1 is implemented through the data reading module in the platform.

[0040] S2: Image preprocessing. Its main purpose is to prepare for the next step of extracting discrete point location information. In step S2, the image is filtered, binarized, and the discrete point region is dilated. Gaussian filtering is used for filtering, and binarization reveals the original discrete point regions in the image, finding the image boundaries. After binarization, the discrete point regions are black. The dilation process enlarges the discrete point regions to find the centroids of the discrete points. In particular, the preset grayscale threshold in the binarization process will affect the extraction of discrete points to a certain extent. When the grayscale threshold is too high, discrete points with insignificant grayscale features will be rejected. When the grayscale threshold is too low, noise points with significant grayscale features will be identified as discrete points. These pseudo-points will reduce the matching accuracy during the matching process. Experiments have verified that a grayscale threshold of 40 yields the best results. In real-world images, pseudo-points do not need to be added; pseudo-points are only added to the training set, and the obtained actual image is an image with noise pseudo-point interference.

[0041] S3: Extraction of discrete point location information. Obtain the centroids of discrete points in the dilated image, thus obtaining their location information. Record the number of discrete points and simultaneously acquire their grayscale features. Sort all discrete points according to their grayscale values.

[0042] S4: Selecting a reference point. The essence of PMNet's discrete point localization lies in the location of the reference point. In the initial iteration, the discrete point with the most significant gray-level features in step S3 is selected as the reference point. Using this reference point as a benchmark, the image is translated so that the reference point becomes the center of the local map, while ensuring that the obtained local map is the same size as the image in the dataset. At this point, the image is the processed discrete point map to be matched. If the reference point selected in the initial iteration cannot meet the matching accuracy requirements, the discrete point with the next lower gray-level value is selected as the new reference point, and the above process is repeated to obtain a local map of the new reference point. S2, S3, and S4 are implemented in the platform through the local map acquisition module.

[0043] S5: Input the discrete point map to be matched into the convolutional network model. The convolutional neural network structure diagram is as follows: Figure 2 As shown, the confidence level p of the reference point in the discrete point map to be matched at different positions in the baseline map is obtained.

[0044] The PMNet model employs a convolutional neural network. The input to the PMNet model is a local map (the discrete point map to be matched), and the network output is the confidence score p of a reference point in the discrete point map at different positions in the baseline map. The class corresponding to the highest confidence score is the class to which the reference point in the discrete point map belongs, along with its confidence score. For example, after reading a 1*128*128 original input feature map in step S1, the discrete point map to be matched is obtained after processing in steps S2, S3, and S4. The discrete point map to be matched serves as the input to the convolutional network model. In each convolutional computation layer, the sizes of the input and output feature maps are fixed. The convolutional neural network used in this embodiment contains 9 convolutional layers and 5 pooling layers, followed by 3 fully connected layers to obtain the final probability of belonging to each class.

[0045] The function of a convolutional layer is to filter the input signal in the spatial domain and generate a two-dimensional feature map. Specifically, the kernel size of each convolutional layer is 3x3. The convolutional layer is represented as follows:

[0046]

[0047] Where x m,n It is the input value at position (m,n) (0≤m≤128,0≤n≤128), w k and b k Let f(·) be the weights and biases of the k-th kernel, respectively, and f(·) be the activation function. The output value at position (m,n) (0≤m≤128, 0≤n≤128). The network uses the ReLU (Reduced Lack of Detail) function.

[0048]

[0049] α is set to 0.05.

[0050] In this embodiment, the spatial pooling of the convolutional neural network is performed by five max-pooling layers, which are typically located after several convolutional layers. Max pooling is performed on a 2x2 window with a stride of 2.

[0051] Fully connected layers. After convolution and pooling layers, a set of feature maps is obtained. Then, three fully connected layers combine these features to classify the input image. The first two fully connected layers consist of 4096 neurons each. The last fully connected layer, serving as the output layer, has 8241 neurons, representing the 8240 classes of the problem and one pseudopoint class. In this layer, the softmax function is used to calculate the probability of each class. The fully connected layer can be represented as:

[0052] y = f(W) T x+b)

[0053] Where x is the feature map set, y is the output vector, W is the weight matrix, and b is the bias vector. The Softmax function can be expressed as:

[0054]

[0055] Where p n It is the probability of the nth class, z n It is the output score of the nth class from the fully connected layer, where N is the total number of output classes.

[0056] Training set preparation: The baseline image obtained in the early stage contained 10,000 discrete points. After removing insignificant discrete points based on grayscale features and density, 8,240 discrete points remained. Pseudo-points with the same characteristics as the discrete points (i.e., random location information and grayscale features) were randomly added to improve the algorithm's ability to identify pseudo-points, resulting in 8,241 categories. For each discrete point, a local image with a field of view of 10°*10° was created. Gaussian noise, random rotation, and flipping were added to each local image to generate a dataset containing 82,410 images. The training and test sets were divided in an 8:2 ratio. The learning rate of the convolutional network was set to 0.001, and the cross-entropy loss function was used to update the network weights. The batch size was 32, and the network was trained for 100 epochs. The final network model achieved an accuracy of 99.7% on the test set. The fluctuations in accuracy and loss function values ​​during training are as follows: Figure 3 As shown, the upper figure represents the fluctuation of accuracy and loss function values ​​on the training set, and the lower figure represents the fluctuation of accuracy and loss function values ​​on the test set.

[0057] S6: Determine if the accuracy meets the requirements. Set the expected accuracy. The output of the PMNet model is the probability that the current reference point belongs to different categories. This probability is the confidence level. A confidence level of p indicates the probability of accepting that the coordinates of the reference point belong to that category. Select the category with the highest confidence level from among many confidence levels as the category of the reference point, and then obtain the location information of the reference point in the baseline map.

[0058] Determine whether the maximum confidence score has reached the expected level (expected accuracy, generally set above 95%). If the maximum confidence score cannot reach the expected level, select the discrete point with the second lowest gray level from the many discrete points obtained in step S3 as the reference point for the next iteration, and repeat steps S4 and S5 to obtain a local map of the new reference point, which is then input into the trained PMNet model until the result reaches the expected level.

[0059] Furthermore, to facilitate the matching of noisy discrete points, this invention designs a discrete point matching platform under noisy conditions based on the PMNet model. The platform interface is as follows: Figure 4 As shown in the figure, this platform is designed using the Python programming language and can achieve image-based matching of discrete points under noisy conditions through a simple human-computer interface.

[0060] Figure 4 In the platform interface diagram, the "Select" button on the left is mainly used to read files or folders, corresponding to the data import module. The "Image Preprocessing" button is used to preprocess images, corresponding to the local image acquisition module, and allows input of preprocessing parameters. The "Match" button is used to perform discrete point matching based on the PMNet model. When the matching confidence level does not meet the requirements, you can select another discrete point as a new reference point and re-match using the "Change Reference Point" option. The parameter information on the left mainly displays image information and discrete point information in the image (the actual path of the imported image and the number of discrete points in the image). The image window on the right provides users with an interface to view the selected images; the original image is the actual imported image, and the converted image is the discrete point map (local image) to be matched. The lower right side mainly displays the matching results (including the position of the reference point in the baseline image; the original position indicates the position of the reference point in the actual imported image).

[0061] This invention's platform acquires raw input feature map data, preprocesses the local image based on parameters in the platform's graphical interface, and obtains discrete point information from the feature map, including the number, grayscale, and relative position of the discrete points. Discrete points with significant grayscale features are selected and used as reference points to adjust the image position. The confidence level of the reference point in the baseline image is calculated using a trained discrete point matching model, thereby obtaining its position information. Experiments show that the method of this invention has good robustness to positional noise, white noise, grayscale noise, and false points, with a recognition time as short as 18.1 milliseconds. Under the current experimental environment, the recognition accuracy reaches 99.7%, which is higher than traditional data-based matching algorithms.

[0062] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A discrete point matching method based on convolutional neural networks under noisy conditions, characterized in that, The method includes the following: Obtain a baseline image. After removing insignificant discrete points based on grayscale features and density, retain the remaining E discrete points. Randomly add pseudo-points with random location information and grayscale values. There are a total of E+1 categories for the E discrete points and 1 pseudo-point. Sequentially, take each point as the center and obtain a local image of each category of points with a square field of view. For each local graph, a dataset containing E+1 classes is obtained through data augmentation operations. The training and test sets are then divided in an 8:2 ratio for training convolutional neural networks. Constructing a PMNet model for matching discrete points: The PMNet model is based on a convolutional neural network. The input of the model is a local graph, and the output is the category to which the discrete points in the local graph belong and their confidence scores. The PMNet model consists of 9 convolutional layers and 5 pooling layers, followed by 3 fully connected layers to obtain the final probability of belonging to each class. Specifically, the first and second convolutional layers are concatenated and then connected to the first pooling layer, followed by the third and fourth convolutional layers and the second pooling layer, then the fifth, sixth, and seventh convolutional layers and the third pooling layer, and finally the eighth, fourth, ninth, and fifth convolutional layers followed by the fifth pooling layer, and then connected to the three fully connected layers. The first two fully connected layers consist of 4096 neurons each, and the last fully connected layer serves as the output layer with E+1 neurons, representing E discrete point classes and one pseudo-point class. The softmax function is used to calculate the probability of each class. This completes the construction of the convolutional neural network. The PMNet model was trained using the constructed dataset, and the resulting discrete point matching model was used for discrete point matching in real images.

2. The discrete point matching method based on convolutional neural networks under noisy conditions according to claim 1, characterized in that, The square field of view is a field of view of (5-20)°*(5-20)°.

3. The discrete point matching method based on convolutional neural networks under noisy conditions according to claim 1, characterized in that, The learning rate of the convolutional network is set to 0.001, the cross-entropy loss function is used to update the network weights, the batch size is 32, and the training epochs are 100; E = 8240.

4. The discrete point matching method based on convolutional neural networks under noisy conditions according to claim 1, characterized in that, The actual image and the local image are the same size. The actual image is read in grayscale form, and after filtering and binary processing, the discrete point regions in the actual image are displayed. The discrete point regions are dilated to enlarge them, and the centroid and grayscale features of the discrete points are extracted. The discrete point with the most significant grayscale features is used as the reference point. The image is translated around the reference point to obtain a local image of the actual image with the same size as the images in the dataset. The local image of the actual image is input into a trained convolutional neural network model, which outputs the confidence of the reference point at different positions in the baseline image. The class corresponding to the class with the highest confidence is selected as the class of the reference point. Finally, the position information of the reference point in the baseline image is obtained, and matching is achieved.

5. A discrete point matching platform based on images under noisy conditions, characterized in that, The platform includes a display module, a data reading module, a local map acquisition module, and a noisy discrete point matching module based on the PMNet model. These modules work together to read local map information, set processing parameters, obtain the position of the local map based on the PMNet model, and finally display the matching results through the platform's display module. The data reading module can read image files or image files in a folder to obtain the image to be matched. The local map acquisition module is used to perform image preprocessing on the image to be matched, extract the discrete point position information of the image to be matched, and select reference points from the discrete points of the image to be matched to form a local map. The noisy discrete point matching module is used to match discrete points based on the PMNet model and output the matching results. When the confidence level cannot reach the expected level, other discrete points of the image to be matched are selected as new reference points for rematching until the expected level is reached. The PMNet model takes a local graph as input and outputs the category and confidence score of discrete points in the local graph. The PMNet model has a total of E+1 outputs, where E is the number of discrete points in the baseline graph and 1 represents a pseudo-point class, which is used to match the location information of discrete points in the local graph. The PMNet model is based on a convolutional neural network. It contains 9 convolutional layers and 5 pooling layers, followed by 3 fully connected layers to obtain the final probability of belonging to each class. Specifically, the first and second convolutional layers are concatenated and then connected to the first pooling layer, followed by the third and fourth convolutional layers and the second pooling layer, then the fifth, sixth, and seventh convolutional layers and the third pooling layer, and finally the eighth, fourth, ninth, and fifth convolutional layers followed by three fully connected layers. The first two fully connected layers consist of 4096 neurons each, and the last fully connected layer serves as the output layer with E+1 neurons, representing E discrete point classes and one pseudo-point class. The softmax function is used to calculate the probability of each class. This completes the construction of the convolutional neural network.

6. The image-based discrete point matching platform under noisy conditions according to claim 5, characterized in that, The reference points are selected based on the gray levels of discrete points in the image to be matched, starting from the discrete points corresponding to the maximum gray level, and the expected level is a confidence level of not less than 95%.

Images