A method for matching degree analysis of fragmented images based on boundary line features

Abstract

The present invention discloses a method for analyzing the matching degree of fragmented images based on boundary straight line features: step 1, constructing a fragmented image data set; step 2, dividing the fragmented images in the fragmented image data set according to relative positions; step 3, performing Gaussian denoising preprocessing operation on an input fragmented image; step 4, calculating the matching degree of edge straight line features between fragmented images based on the position classification result and edge straight line information in step 3, arranging the matching results in descending order, and taking the image with the maximum matching degree as the matching result; step 5, performing fragmented image splicing restoration on the optimal splicing result in step 4, and calculating the accuracy rate by comparing with the original image. The method solves the splicing problem of fragmented images rich in straight line features and realizes the restoration of fragmented table images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, and in particular relates to a method for performing matching degree analysis of fragmented images based on boundary straight line features. Background Art

[0002] With the rapid development of big data and the Internet of Things, the data that people can use is no longer just simple numerical values. Text, images, voice, and video are the main channels for data dissemination. The amount of data contained in them is immeasurable and urgently needs to be developed and utilized. On the one hand, in-depth analysis of various types of data information and information combination can enhance human understanding of natural information; on the other hand, further analysis of data can fully explore potential correlations and enhance the economic value of information.

[0003] At present, a large amount of fragmented information has not been recycled. How to conduct correlation analysis on fragmented information and fully tap potential information is a major problem. The complexity of the problem scale, the correlation between information, and the degree of information fragmentation are important factors in solving such problems. In 2020, the new crown pneumonia swept the world. Accurately finding and locating contacts, that is, finding people who overlap with the infected in time and space, is a major problem. In the current society, personnel across the country immediately took action to integrate video image resources from various places, establish a big data platform, use information mining technology to splice fragmented information, and mine valuable data from massive data in real time to further track contacts. At the same time, in order to fully explore the correlation between images and effectively respond to the all-round challenges brought by the integration of fragmented information, many countries have also laid out relevant strategic plans.

[0004] However, there are still many challenges in the research field of fragmented information processing. Among them, the biggest problem is how to efficiently extract fragmented image features and how to measure the correlation between image feature information. The image is dynamic and random in the process of being fragmented, so the distribution of the generated fragmented images is unpredictable. The amount of information contained in the fragmented image is greatly reduced. Therefore, in order to fully explore the potential information contained in the image, we need to combine and restore the fragmented information. The connection between the fragmented images and the degree of feature density directly affect the accuracy of the restored image.

[0005] Image fragmentation splicing and restoration technology also relies on image edge information. The degree of image edge feature loss directly affects the accuracy of positioning and restoration. In this invention, the main problem we solve is to determine the connection relationship between fragmented images based on edge straight line features and measure the accuracy of fragmented image information restoration. Summary of the invention

[0006] The purpose of the present invention is to provide a method for analyzing the matching degree of fragmented images based on boundary straight line features, which solves the splicing problem of fragmented images rich in straight line features and realizes the restoration of fragmented table-like images.

[0007] The technical solution adopted by the present invention is a method for analyzing the matching degree of fragmented images based on boundary straight line features, which is specifically implemented according to the following steps:

[0008] Step 1: Segment several original images to construct a fragmented image dataset A = {a1, a2...a n},

[0009] Step 2: divide the fragmented images in the fragmented image dataset A into four categories according to their relative positions, and put the divided dataset into the Alexnet classification model for training, so that when any fragmented image is input, the classification model can predict the distribution position information of the fragmented image;

[0010] Step 3: Perform Gaussian denoising preprocessing on an input fragmented image a1 to obtain an image After preprocessing, the image Edge line detection: the detected edge line feature information is filtered and saved in array arr1;

[0011] Step 4: The fragmented images in A except a1 form the image dataset A to be matched. L ={a2,a3,...a n}, for A L The fragmented images in the image are used to predict the distribution position, and the A is reduced according to the prediction results. L range, and the reduced image dataset to be matched The image to be matched in i Perform edge line detection in sequence and put the detection information into array arr i Store in; calculate a1 and a respectively i The matching degree between them is taken as the maximum matching degree as the matching result;

[0012] Step 5: restore the fragmented images by stitching the optimal stitching result in step 4, and compare it with the original image to calculate the accuracy.

[0013] The present invention is also characterized in that

[0014] Step 2 is as follows:

[0015] Step 2.1, in the fragmented image dataset A, the fragmented images with the same relative position are put into the same class and saved, which are divided into left_up class, left_bottom class, right_up class, and right_bottom class.

[0016] Step 2.2, put the divided data set into the Alexnet classification model for training, the input image size is 42*42, batch_size is set to 32, the learning rate is set to 0.0001, and the cross entropy loss function CategoricalCrossentropy is used to calculate the loss during training. The convolution kernel size in the convolution operation is 5*5, and a total of 20 epochs are trained; after the training is completed, save the trained Alexnet classification model.

[0017] Step 3:

[0018] Step 3.1, use the Alexnet classification model trained in step 2.2 to predict the distribution position of the input fragmented image a1;

[0019] Step 3.2: Preprocess the input fragmented image a1 and perform Gaussian denoising. Then, Use HoughLinesP method to extract edge straight line feature information and save it;

[0020] Step 3.3, determine whether the edge straight line feature information is empty, if not, confirm the splicing of the input fragmented image in combination with the distribution position information in step 3.1, and filter out the edge straight line feature information arr1 of the splicing in the straight line data information and save it.

[0021] Step 4 The specific process is as follows:

[0022] Step 4.1: The fragmented images in the fragmented image dataset A except the input fragmented image a1 constitute the image dataset A to be matched. L ={a2,a3,...a n};

[0023] Step 4.2: In all the image datasets A to be matched L Use the Alexnet classification model trained in step 2.2 to predict the image dataset A to be matched L The distribution positions of the images to be matched are determined, and the images to be matched are screened to narrow the range of the image data set to be matched; the narrowed image data set to be matched is The image to be stitched a in i Execute steps 3.2 and 3.3 to obtain the edge straight line feature information arr at the splicing point i, stored in an array;

[0024] Step 4.3: Combine the input fragmented image a1 with the image to be stitched i The edge straight line feature information at the splicing point is matched, the position matching range is controlled within -5 to 5 pixels, and the quotient of the number of successful matches and the total number of straight line features of the input fragmented image is used as the quotient of the input fragmented image a1 and the image to be spliced ​​a i If the edge straight line feature information of the input fragmented image a1 is arr1, the image to be spliced ​​a i The edge line feature information is arr i , the calculation of matching degree is expressed in mathematical language as formula (1):

[0025]

[0026] The initial value count = 0, len(arr1) is the length of arr1, len(arr i ) is arr i length, and compare the data in arr1 with the data in arr i The data in the array are counted in turn. If the formula (2) is satisfied, the count value is increased by 1. Once the formula (2) is satisfied, b will no longer be taken down. a and b represent the subscript of the characteristic element in the array arr1 and the subscript of the characteristic element in the array arr2, respectively. i Subscripts of characteristic elements in:

[0027]

[0028] a∈{0,1...len(arr1)-1} b∈{0,1...len(arr i )-1} (2)

[0029] Step 4.4, calculate the matching degree between the input fragmented image and each image to be stitched in the image data set to be matched, and the maximum value of the matching degree is taken as the best matching value result, and return the input fragmented image and the image to be stitched corresponding to the best matching value, and the best matching value. The best matching value is expressed in mathematical language as shown in formula (3):

[0030]

[0031] In step 5, the fragmented image corresponding to the best matching value in step 4 is restored by splicing the fragmented image with the image to be spliced. The restoration result is used as the spliced ​​image obtained by this method. The accuracy D is expressed in mathematical language as formula (4):

[0032]

[0033] Where T is the stitched image obtained by this method, S represents the original image, the size of the stitched image is set to w*h, the original image is compared with the stitched image pixel by pixel, and the absolute value of the difference is summed as the accuracy calculation result.

[0034] The beneficial effects of the present invention are:

[0035] (1) The method of the present invention processes table-type images and uses the rich straight line features of table-type images for analysis and processing, which has a novel starting point.

[0036] (2) The method of the present invention is mainly divided into four parts: data set construction, image edge straight line feature detection, line feature matching between images, and image restoration. It can restore fragmented table images, make full use of potential big data information, and play a role in security verification, image restoration, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the present invention for analyzing the matching degree of fragmented images based on boundary straight line features;

[0038] Figure 2 It is a flow chart of edge straight line feature detection and storage for each fragmented image in the present invention;

[0039] Figure 3 It is a flow chart of performing image matching analysis based on edge line detection results and screening the best matching results in the present invention;

[0040] Figure 4 is an example of an original image in the embodiment;

[0041] Figure 5 yes Figure 4 A fragmented image located at the left_up position in ;

[0042] Figure 6 It is a restoration result image obtained by performing mosaic restoration of fragmented images according to the optimal mosaic result in the embodiment. DETAILED DESCRIPTION

[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The present invention provides a method for analyzing the matching degree of fragmented images based on boundary straight line features. Figure 1 As shown, the specific implementation steps are as follows:

[0045] Step 1: Segment several original images to construct a fragmented image dataset A = {a1, a2...a n},

[0046] Step 2: divide the fragmented images in the fragmented image dataset A into four categories according to their relative positions, and put the divided dataset into the Alexnet classification model for training, so that when any fragmented image is input, the classification model can predict the distribution position information of the fragmented image;

[0047] Step 2 is as follows:

[0048] Step 2.1, in the fragmented image dataset A, the fragmented images with the same relative position are put into the same class and saved, which are divided into left_up class, left_bottom class, right_up class, and right_bottom class.

[0049] Step 2.2, put the divided data set into the Alexnet classification model for training, the input image size is 42*42, batch_size is set to 32, the learning rate is set to 0.0001, and the cross entropy loss function CategoricalCrossentropy is used to calculate the loss during training. The convolution kernel size in the convolution operation is 5*5, and a total of 20 epochs are trained; after the training is completed, save the trained Alexnet classification model.

[0050] Step 3: Perform Gaussian denoising preprocessing on an input fragmented image a1 to obtain an image After preprocessing, the image Edge line detection: the detected edge line feature information is filtered and saved in array arr1;

[0051] Step 3: Specific process as follows Figure 2 The detailed description is as follows:

[0052] Step 3.1, use the Alexnet classification model trained in step 2.2 to predict the distribution position of the input fragmented image a1;

[0053] Step 3.2: Preprocess the input fragmented image a1 and perform Gaussian denoising. Then, Use HoughLinesP method to extract edge straight line feature information and save it;

[0054] Step 3.3, determine whether the edge straight line feature information is empty, if not, confirm the splicing of the input fragmented image in combination with the distribution position information in step 3.1, and filter out the edge straight line feature information arr1 of the splicing in the straight line data information and save it.

[0055] Step 4: The fragmented images in A except a1 form the image dataset A to be matched. L ={a2,a3,...an}, for A L The fragmented images in the image are used to predict the distribution position, and the A is reduced according to the prediction results. L range, and the reduced image dataset to be matched The image to be matched in i Perform edge line detection in sequence and put the detection information into array arr i Store in; calculate a1 and a respectively i The matching degree between them is taken as the maximum matching degree as the matching result;

[0056] Step 4: Specific process Figure 3 The detailed description is as follows:

[0057] Step 4.1: The fragmented images in the fragmented image dataset A except the input fragmented image a1 constitute the image dataset A to be matched. L ={a2,a3,...a n};

[0058] Step 4.2: In all the image datasets A to be matched L Use the Alexnet classification model trained in step 2.2 to predict the image dataset A to be matched L The distribution positions of the images to be matched are determined, and the images to be matched are screened to narrow the range of the image data set to be matched; the narrowed image data set to be matched is The image to be stitched a in i Execute steps 3.2 and 3.3 to obtain the edge straight line feature information arr at the splicing point i , stored in an array;

[0059] Step 4.3: Combine the input fragmented image a1 with the image to be stitched i The edge straight line feature information at the splicing point is matched, the position matching range is controlled within -5 to 5 pixels, and the quotient of the number of successful matches and the total number of straight line features of the input fragmented image is used as the quotient of the input fragmented image a1 and the image to be spliced ​​a i If the edge straight line feature information of the input fragmented image a1 is arr1, the image to be spliced ​​a i The edge line feature information is arr i , the calculation of matching degree is expressed in mathematical language as formula (1):

[0060]

[0061] The initial value count = 0, len(arr1) is the length of arr1, len(arr i ) is arri length, and compare the data in arr1 with the data in arr i The data in the array are counted in turn. If the formula (2) is satisfied, the count value is increased by 1. Once the formula (2) is satisfied, b will no longer be taken down. a and b represent the subscript of the characteristic element in the array arr1 and the subscript of the characteristic element in the array arr2, respectively. i Subscripts of characteristic elements in:

[0062]

[0063] a∈{0,1...len(arr1)-1} b∈{0,1...len(arr i )-1} (2)

[0064] Step 4.4, calculate the matching degree between the input fragmented image and each image to be stitched in the image data set to be matched, and the maximum value of the matching degree is taken as the best matching value result, and return the input fragmented image and the image to be stitched corresponding to the best matching value, and the best matching value. The best matching value is expressed in mathematical language as shown in formula (3):

[0065]

[0066] Step 5: restore the fragmented images by stitching the optimal stitching result in step 4, and compare it with the original image to calculate the accuracy.

[0067] In step 5, the fragmented image corresponding to the best matching value in step 4 is restored by splicing the fragmented image with the image to be spliced. The restoration result is used as the spliced ​​image obtained by this method. The accuracy D is expressed in mathematical language as formula (4):

[0068]

[0069] Where T is the stitched image obtained by this method, S represents the original image, the size of the stitched image is set to w (width) * h (height), the original image is compared with the stitched image pixel by pixel, and the absolute value of the difference is summed as the accuracy calculation result.

[0070] Example

[0071] 300 structural images rich in straight line features are segmented into 2*2 to construct a fragmented image dataset. There are 1200 images in the fragmented image dataset. Figure 4 An example of the original image.

[0072] The fragmented images in the fragmented image dataset are divided into four categories according to their relative positions: left_up, left_bottom, right_up, and right_bottom. The divided datasets are put into the Alexnet classification model for training, with batch_size set to 32 and epochs set to 20.

[0073] by Figure 4 As an example, input Figure 4 A fragmented image at the left_up position in Figure 5 , perform Gaussian denoising on the image, detect edge lines on the processed image, and only retain the edge line feature information at the splicing point based on the image position classification results.

[0074] The other fragmented images constitute the image data set to be matched, and the distribution position of the fragmented images is predicted. The range of the image data set to be matched is reduced according to the prediction results, and edge straight line detection is performed on the reduced images to be matched in turn, retaining the edge straight line feature information at the splicing point; the matching degree between the input fragmented image and the image to be matched is calculated, and the maximum matching degree is taken as the matching result.

[0075] According to the optimal stitching result, the fragmented image is stitched and restored. The restoration result is as follows Figure 6 .

[0076] Conclusion: By using the method of the present invention, the optimal matching result of a fragmented image can be found in the fragmented image data set and restored and displayed, thus solving the splicing problem of fragmented images rich in straight line features.

Claims

1. A method for analyzing the matching degree of fragmented images based on boundary straight line features, characterized in that: Follow the steps below to implement it: Step 1: Segment several original images to construct a fragmented image dataset A = {a1, a2...a n }, Step 2: divide the fragmented images in the fragmented image dataset A into four categories according to their relative positions, and put the divided dataset into the Alexnet classification model for training, so that when any fragmented image is input, the classification model can predict the distribution position information of the fragmented image; Step 3: Perform Gaussian denoising preprocessing on an input fragmented image a1 to obtain an image After preprocessing, the image Edge line detection: the detected edge line feature information is filtered and saved in array arr1; Step 4: The fragmented images in A except a1 form the image dataset A to be matched. L ={a2,a3,...a n }, for A L The fragmented images in the image are used to predict the distribution position, and the A is reduced according to the prediction results. L range, and the reduced image dataset to be matched The image to be matched in i Perform edge line detection in sequence and put the detection information into array arr i Store in; calculate a1 and a respectively i The matching degree between them is taken as the maximum matching degree as the matching result; Step 5: restore the fragmented images by stitching the optimal stitching result in step 4, and compare it with the original image to calculate the accuracy.

2. The method for analyzing the matching degree of fragmented images based on boundary straight line features according to claim 1, characterized in that: Step 2 is as follows: Step 2.1, in the fragmented image dataset A, the fragmented images with the same relative position are saved in the same class, which are divided into left_up class, left_bottom class, right_up class, and right_bottom class; Step 2.2, put the divided data set into the Alexnet classification model for training, the input image size is 42*42, batch_size is set to 32, the learning rate is set to 0.0001, and the cross entropy loss function CategoricalCrossentropy is used to calculate the loss during training. The convolution kernel size in the convolution operation is 5*5, and a total of 20 epochs are trained; after the training is completed, save the trained Alexnet classification model.

3. The method for analyzing the matching degree of fragmented images based on boundary straight line features according to claim 2, characterized in that: Step 3: Step 3.1, use the Alexnet classification model trained in step 2.2 to predict the distribution position of the input fragmented image a1; Step 3.2, perform image preprocessing on the input fragmented image a1, perform Gaussian denoising, and then use the HoughLinesP method to extract edge straight line feature information from the preprocessed fragmented image a1 and save it; Step 3.3, determine whether the edge straight line feature information is empty, if not, confirm the splicing of the input fragmented image in combination with the distribution position information in step 3.1, and filter out the edge straight line feature information arr1 of the splicing in the straight line data information and save it.

4. The method for analyzing the matching degree of fragmented images based on boundary straight line features according to claim 3, characterized in that: Step 4 The specific process is as follows: Step 4.1: The fragmented images in the fragmented image dataset A except the input fragmented image a1 constitute the image dataset A to be matched. L ={a2,a3,...a n }; Step 4.2: In all the image datasets A to be matched L Use the Alexnet classification model trained in step 2.2 to predict the image dataset A to be matched L The distribution positions of the images to be matched are determined, and the images to be matched are screened to narrow the range of the image data set to be matched; the narrowed image data set to be matched is The image to be stitched a in i Execute steps 3.2 and 3.3 to obtain the edge straight line feature information arr at the splicing point i , stored in an array; Step 4.3: Combine the input fragmented image a1 with the image to be stitched i The edge straight line feature information at the splicing point is matched, the position matching range is controlled within -5 to 5 pixels, and the quotient of the number of successful matches and the total number of straight line features of the input fragmented image is used as the quotient of the input fragmented image a1 and the image to be spliced ​​a i If the edge straight line feature information of the input fragmented image a1 is arr1, the image to be spliced ​​a i The edge line feature information is arr i , the calculation of matching degree is expressed in mathematical language as formula (1): The initial value count = 0, len(arr1) is the length of arr1, len(arr i ) is arr i length, and compare the data in arr1 with the data in arr i The data in the array are counted in turn. If the formula (2) is satisfied, the count value is increased by 1. Once the formula (2) is satisfied, b will no longer be taken down. a and b represent the subscript of the characteristic element in the array arr1 and the subscript of the characteristic element in the array arr2, respectively. i Subscripts of characteristic elements in: a∈{0,1...len(arr1)-1}b∈{0,1...len(arr i )-1} (2) Step 4.4, calculate the matching degree between the input fragmented image and each image to be stitched in the image data set to be matched, and the maximum value of the matching degree is taken as the best matching value result, and return the input fragmented image and the image to be stitched corresponding to the best matching value, and the best matching value. The best matching value is expressed in mathematical language as shown in formula (3):

5. The method for analyzing the matching degree of fragmented images based on boundary straight line features according to claim 4, characterized in that: In step 5, the fragmented image corresponding to the best matching value in step 4 is restored by splicing the fragmented image with the image to be spliced. The restoration result is used as the spliced ​​image obtained by this method. The accuracy D is expressed in mathematical language as formula (4): Where T is the stitched image obtained by this method, S represents the original image, the size of the stitched image is set to w*h, the original image is compared with the stitched image pixel by pixel, and the absolute value of the difference is summed as the accuracy calculation result.

Images