A Multimode Fiber Spot Image Recognition Method Based on Similarity Calculation Network

Abstract

This invention discloses a method for multimode fiber spot image recognition based on a similarity calculation network. The invention acquires spot images of multimode fibers at different bending angles and improves the quality of the spot dataset through operations such as spot center cropping and scale normalization. After pairing the spot images in the training set, similarity scores are assigned based on the differences between corresponding angles in the spot images, constructing the training sample set required for the similarity calculation network. In the similarity calculation network, feature extraction modules first extract feature information from the spot images, then concatenate and fuse the feature information of paired spot images, and finally calculate the similarity of the feature information using a similarity measurement module. Finally, the similarity of the spot image to be tested with the reference set is calculated, and the angle corresponding to the image with the highest similarity score is the recognition result. This invention can fully mine the feature information of spot images with a small number of samples, achieving accurate recognition of spot images.

Description

Technical Field

[0001] This invention belongs to the field of fiber optic spot sensing and relates to a method for multimode fiber optic spot image recognition based on a similarity calculation network. Background Technology

[0002] In recent years, fiber optic spot sensors, as an important part of fiber optic sensing, have shown excellent performance and applications in various sensing fields, attracting widespread attention from researchers. Previous researchers have estimated the degree of disturbance by quantifying the similarity of spot pixel intensity, while others have explored the direct correlation between image features of the spot and the degree of external disturbance by analyzing the image features of the spot. However, methods that directly perceive external disturbances by calculating the similarity of spot pixel intensity suffer from low reliability, poor anti-interference ability, and limited dynamic range. On the other hand, methods that use manually set spot feature extraction operators to characterize the spot image and estimate disturbance changes based on the spot features suffer from insufficient manually designed feature extraction, complex steps, and poor applicability.

[0003] With the development of deep learning, many researchers have used deep learning methods to classify light spot images to detect external disturbances. Compared to previous methods that quantified the degree of light spot differences and manually set light spot feature extraction operators, deep learning methods, based on large amounts of light spot image data, build neural network models to learn the relationship between light spot changes and external disturbances, thereby achieving accurate light spot identification. The workflow generally includes data acquisition, light spot image preprocessing, light spot feature extraction and selection, and neural network model building and validation. Currently, most researchers use classification networks such as VGGNet, ResNet, and GoogleNet to sense light spot images, such as detecting whether a face is present in a light spot or the curvature of the multimode fiber corresponding to the light spot pattern. These methods often require large-scale light spot image data and are difficult to apply to unlabeled light spot image data. Summary of the Invention

[0004] The purpose of this invention is to provide a multimode fiber spot image recognition method based on a similarity calculation network to overcome the shortcomings of the prior art.

[0005] To fully extract feature information from light spot images with a limited sample size and achieve accurate image recognition, this invention proposes a deep learning-based similarity calculation network for recognizing multimode fiber light spot images at different bending angles. The network mainly consists of a light spot feature extraction module and a similarity calculation module. The feature extraction module extracts light spot features, while the similarity calculation module calculates the similarity between these features and provides a corresponding similarity score. By sequentially calculating the similarity between the light spot to be tested and the light spots in the reference set, the bending angle corresponding to the light spot with the highest similarity score is taken as the recognition result.

[0006] This invention includes the following steps:

[0007] Step 1: Acquire images of the light spot on the multimode fiber at different bending angles and assign them corresponding bending angle labels.

[0008] Step 2: Obtain the center position of the light spot, preprocess the light spot image to improve the quality of the light spot dataset; then divide it into training set and test set according to a certain ratio.

[0009] Step 3: Pair up the light spot images in the training set, label them with similarity scores, and construct the training sample set for the similarity calculation network.

[0010] Step 4: Divide the training sample set into multiple small batches, and use them as inputs to the similarity calculation network in sequence. Combine the input similarity score labels with the mean squared error loss function to train the network model.

[0011] Step 5: In the training set for each bending angle, randomly select one or more spot images as reference images for that bending angle category, and together they form a reference set for the bending angle category of multimode fiber.

[0012] Step 6: For any spot image with an unknown bending angle in the test set, combine it with all the spot images in the reference set and input them sequentially into the trained similarity calculation network. Based on the similarity score between the spot image to be tested and the spot images in the reference set, the bending angle corresponding to the spot image with the highest similarity score is the recognition result.

[0013] The beneficial effects of this invention are as follows: This invention designs a similarity calculation network that can identify the bending angle category of light spot images by recognizing the similarity between them; this invention can construct paired training data by cross-combining without changing the amount of light spot image data, which can greatly expand the training data and solve the problem of insufficient training data to a certain extent.

[0014] Furthermore, after extracting the features of the light spot image through CNN, this invention does not directly calculate the similarity between the light spot feature information using Euclidean distance. Instead, it constructs a similarity measurement module to deeply mine the similarity between the feature maps of the light spot image, making the similarity calculation between the light spot characteristic information more accurate.

[0015] Furthermore, this invention also labels different samples based on the degree of difference in the bending angles corresponding to the light spot images, namely the similarity scores between the light spot images, and trains the network model in combination with the mean squared error loss function, which enables the network to more accurately express the similarity between the light spot images, thereby achieving more accurate recognition.

[0016] Finally, the similarity calculation network proposed in this invention can achieve a certain accuracy in the bending angle of light spots that were not trained, indicating that the network can identify light spot images by learning the similarity between light spot images. It has good portability and generalization and can be applied to more real-world scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the multimode fiber spot experimental device in this invention;

[0018] Figure 2 This is a flowchart illustrating the similarity calculation network of the present invention.

[0019] Figure 3 This is a schematic diagram showing the detailed structure of each module of the network in this invention;

[0020] Figure 4 This is a schematic diagram of the process for identifying the bending angle of the light spot in this invention. Detailed Implementation

[0021] To describe the present invention in more detail, the technical solution of the present invention will be described below with reference to the accompanying drawings and specific implementation process.

[0022] This invention provides a method for recognizing multimode fiber spot images based on a similarity calculation network. By constructing a multimode fiber spot experimental device, spot images of multimode fibers at different bending angles are acquired. The quality of the spot dataset is improved through operations such as spot center cropping and scale normalization. The training set spot images are then paired and labeled with similarity scores based on the differences between corresponding angles, thus constructing the training sample set required for the similarity calculation network. In the similarity calculation network, a CNN-based feature extraction module first extracts the feature information of the spot images, then concatenates and fuses the feature information of paired spot images. A similarity measurement module then calculates the similarity of the feature information. Finally, the similarity between the spot image to be tested and the reference set is calculated, and the angle corresponding to the image with the highest similarity score is the recognition result.

[0023] This embodiment specifically includes the following steps:

[0024] Step 1: Build a multimode fiber spot experimental device, collect spot images of multimode fiber at different bending angles, and assign labels to the corresponding bending angles.

[0025] Step 2: Obtain the center position of the light spot through edge detection, crop the light spot image into an image of uniform size, and perform grayscale conversion, size adjustment and normalization operations on the cropped light spot image, and then divide it into training set and test set according to a certain ratio.

[0026] Step 3: Pair up the light spot images in the training set, label them with similarity scores, and construct the training sample set for the similarity calculation network.

[0027] Step 4: Randomly divide the training data into multiple small batches, and use them as input to the similarity calculation network in sequence. Combine the input similarity score labels with the mean squared error loss function to train the network model.

[0028] Step 5: In the training set for each bending angle, randomly select one or more spot images as reference images for that bending angle category, and together they form a reference set for the bending angle category of multimode fiber.

[0029] Step 6: For any spot image with an unknown bending angle in the test set, combine it with all the spot images in the reference set and input them into the trained similarity calculation network in sequence. Based on the similarity score between the spot image to be tested and the spot images in the reference set, the bending angle corresponding to the spot image with the highest similarity score is the recognition result.

[0030] In a preferred embodiment, during the acquisition of the multimode fiber spot image in step 1, the wavelength emitted by the laser should remain fixed; the length, material, and core diameter of the single-mode fiber and the multimode fiber should be consistent; and the parameters of the imaging device and the temperature of the surrounding environment should be consistent.

[0031] The experimental setup in this embodiment is as follows: Figure 1 As shown, a bend point is set at one point in the multimode fiber as the bend angle detection point. A laser, using a single wavelength as the light source, emits a laser beam. The laser beam is coupled through a section of single-mode fiber and then enters the multimode fiber. The beam output from the multimode fiber is projected onto a projection screen, with the screen direction kept perpendicular to the fiber output end and at a constant distance. A camera is then used to capture the image, which is fixed to the multimode fiber output end. The captured image has a pixel size of 1920*1080. To ensure data robustness, each bend angle is captured multiple times, with a 3° interval between bend angles, and the captured bend angles range from 0 to 90°.

[0032] In a preferred embodiment, the specific implementation process of the algorithm for obtaining the spot center in step 2 is as follows:

[0033] Step 2.1: Convert the spot image into a grayscale image and smooth the image using median filtering to reduce noise interference in the spot image.

[0034] Step 2.2: Apply a dilation processing algorithm to the light spot image to reduce the gap between the light spot pixels and connect the scattered light spots into a whole.

[0035] Step 2.3: Use the Canny operator to extract the edge of the light spot in the light spot image, and obtain the minimum bounding box that completely surrounds the light spot, as well as the center position of the bounding box.

[0036] Step 2.4: Based on the center position of the smallest border of the light spot, crop the light spot into a fixed size of 800*800 light spot image, and compress the size to 224*224.

[0037] In a preferred embodiment, the specific sub-steps for constructing the training sample set of the similarity calculation network in step 3 are as follows:

[0038] Step 3.1: Select the similarity score as the sample label. The similarity score ranges from 0 to 1. When two light spot images belong to the same bending angle, the sample label is 1. When two light spot images do not belong to the same bending angle, the similarity score is set to 0.4 when the angle difference is 3°, 0.3 when the angle difference is 6°, 0.2 when the angle difference is 9°, 0.1 when the angle difference is 12°, and 0 when the angle difference is greater than 12°.

[0039] Step 3.2: In the training set of light spot images, each light spot image under the same bending angle forms a positive sample with a similarity score of 1 with another light spot image, and forms a negative sample with a similarity score of less than 0.5 with any three light spot images under different bending angles. The ratio of positive to negative samples is 1:3.

[0040] Traverse all the light spot images in the training samples and combine them into pairs of light spot image data to conform to the input and output format of the similarity calculation network. The maximum number of positive samples is N(N-1) / 2, where N is the number of samples for each type of bending angle.

[0041] In a preferred embodiment, the specific structure of the similarity calculation network in step 4 is as follows: Figure 2 Specifically, it includes two modules: a feature extraction module based on a Convolutional Neural Network (CNN) and a similarity measurement module. The feature extraction module extracts feature information of the light spot image, and the similarity measurement module calculates the similarity of the feature information of the light spot image. The network model is optimized by calculating the mean squared error loss function between the similarity score and the sample label, and finally the recognition of the light spot image is achieved.

[0042] In this embodiment, the specific structure of each module in the similarity calculation network is as follows: Figure 3 As shown, a convolutional block is defined in the similarity calculation network. This convolution consists of a convolutional layer, a batch normalization layer, and an activation function layer (Conv-BN-ReLU, CBR). The convolutional layer uses a 3*3 kernel size, 64 filters, a stride of 1, and identical padding. The input spot image size is 224*224*1. The CNN-based feature extraction module includes two branches. Each branch, from input to output, consists of CBR, max pooling layer, CBR, max pooling layer, and CBR. The CNN extracts the feature information of the spot image, and the parameters of the two branches are shared.

[0043] In this embodiment, the similarity measurement module consists of, from input to input, a CBR, a max pooling layer, another CBR, another max pooling layer, a global average pooling layer, a fully connected layer FC1, and a fully connected layer FC2. FC1 has 256 nodes and uses ReLU activation; FC2 has 1 node and uses Sigmoid activation. Furthermore, all max pooling layers use a 2x2 kernel size.

[0044] The feature maps output by the two branches of the spot feature extraction module are input into the similarity measurement module through channel splicing. Finally, the network model is optimized by calculating the mean squared error loss function of the sample labels of the input training data and the similarity scores output by the similarity measurement module.

[0045] In a preferred embodiment, in step 5, the number of reference images for each angle in the reference set can be one or more. When the number of reference images for each bending angle is 1, the similarity of the light spot image to be tested with each reference image is calculated sequentially, and the image with the highest similarity score is the recognition result. The recognition process is as follows: Figure 4 As shown. When the number of reference images for each bending angle is greater than 1, the test spot image is sequentially compared with all the reference images for each bending angle. The average value is taken as the similarity score with the spot image at that angle, and the one with the highest average similarity score is the recognition result.

[0046] In a preferred embodiment, the robustness of the similarity calculation network is verified by cross-validation. Specifically, all data are first randomly divided into 5 groups, one of which is used as the validation set and the others as the training set. The model parameters under different training data are obtained through training, and the performance of the model is verified on the validation set.

[0047] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. It will be apparent to those skilled in the art that various modifications can be made to the above embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A method for recognizing multimode fiber spot images based on a similarity calculation network, characterized in that... The method includes the following steps: Step 1: Acquire images of the light spot on the multimode fiber at different bending angles and assign them corresponding bending angle labels; Step 2: Obtain the center position of the light spot, preprocess the light spot image to improve the quality of the light spot dataset; then divide it into training set and test set according to a certain ratio; Step 3: Pair up the light spot images in the training set, label them with similarity scores, and construct the training sample set for the similarity calculation network; Step 4: Divide the training sample set into multiple small batches, and use them as input to the similarity calculation network in turn. Combine the input similarity score labels with the mean squared error loss function to train the network model. Step 5: In the training set for each bending angle, randomly select one or more spot images as reference images for that bending angle category, and together form a reference set for the multimode fiber bending angle category; Step 6: For any spot image with an unknown bending angle in the test set, combine it with all the spot images in the reference set, and input them into the trained similarity calculation network in sequence. Based on the similarity score between the spot image to be tested and the spot images in the reference set, the bending angle corresponding to the spot image with the highest similarity score is the recognition result. The similarity calculation network in step 4 includes two modules: a feature extraction module based on a convolutional neural network and a similarity measurement module. The feature extraction module extracts feature information of the light spot image, and the similarity measurement module calculates the similarity of the feature information of the light spot image. The network model is optimized by calculating the mean squared error loss function between the similarity score and the sample label.

2. The multimode fiber spot image recognition method based on a similarity calculation network according to claim 1, characterized in that: In step 1, during the acquisition of multimode fiber spot images, the wavelength emitted by the laser remains fixed; the length, material, and core diameter of the single-mode and multimode fibers remain consistent; and the parameters of the imaging equipment and the temperature of the surrounding environment remain consistent.

3. The multimode fiber spot image recognition method based on a similarity calculation network according to claim 2, characterized in that: Step 2, obtaining the center position of the light spot, is as follows: Step 2.1: Convert the spot image into a grayscale image and smooth the image using median filtering to reduce noise interference in the spot image; Step 2.2: Apply a dilation processing algorithm to the light spot image to reduce the gap between light spot pixels and connect the scattered light spots into a whole; Step 2.3: Use edge detection to extract the edges of the light spot in the image, and obtain the minimum bounding box that completely surrounds the light spot, as well as the center position of the bounding box; Step 2.4: Based on the center position of the smallest border of the light spot, crop the light spot into a fixed-size light spot image.

4. The multimode fiber spot image recognition method based on a similarity calculation network according to claim 3, characterized in that: The preprocessing of the light spot image in step 2 specifically involves cropping the light spot image into an image of uniform size, and then performing grayscale conversion, size adjustment, and normalization operations on the cropped light spot image.

5. The multimode fiber spot image recognition method based on a similarity calculation network according to claim 1, characterized in that: The specific details of constructing the training sample set for the similarity calculation network in step 3 are as follows: Step 3.1: Select the similarity score as the sample label. The similarity score ranges from 0 to 1. When two light spot images belong to the same bending angle, the sample label is 1. When two light spot images do not belong to the same bending angle, the similarity score is set to 0.4 when the angle difference is 3°, 0.3 when the angle difference is 6°, 0.2 when the angle difference is 9°, 0.1 when the angle difference is 12°, and 0 when the angle difference is greater than 12°. Step 3.2: In the training set of light spot images, each light spot image under the same bending angle forms a positive sample with a similarity score of 1 with another light spot image, and forms a negative sample with a similarity score of no more than 0.5 with any light spot image under different bending angles. The ratio of positive to negative samples does not exceed 1:

3. Step 3.3: Traverse all the light spot images in the training samples and combine them into pairs of light spot image data to conform to the input and output format of the similarity calculation network. The maximum number of positive samples is N(N-1) / 2, where N is the number of samples for each type of bending angle.

6. The multimode fiber spot image recognition method based on a similarity calculation network according to claim 1, characterized in that: In step 5, when the number of reference images for each bending angle is 1, the similarity of the test spot image with each reference image is calculated sequentially, and the one with the highest similarity score is the recognition result; when the number of reference images for each bending angle is greater than 1, the similarity of the test spot image with all the reference images for each bending angle is calculated sequentially, and the average value is taken as the similarity score with the spot image at that angle, and the one with the highest average similarity score is the recognition result.

7. The multimode fiber spot image recognition method based on a similarity calculation network according to claim 1, characterized in that: It also includes verifying the robustness of the similarity calculation network through cross-validation. Specifically, all spot image data are first randomly divided into 5 groups, one of which is used as the test set and the others as the training set. The network model is trained using different training sets, and the performance of the model is verified on the test set.

Images