Deformable object similarity detection method and electronic device based on Radon transform

Abstract

The present invention relates to a deformable object similarity detection method and electronic device based on Radon transform, the method comprising: step S1: obtaining a first image of a flexible object before deformation and a second image of the flexible object after deformation; step S2: converting the first image and the second image respectively by Radon transform to obtain a first Radon transform feature map and a second Radon transform feature map; step S3: inputting the first Radon transform feature map and the second Radon transform feature map into a deep neural network, and calculating the similarity score between the first Radon transform feature map and the second Radon transform feature map by the deep neural network; step S4: judging the similarity between the flexible object before deformation and the flexible object after deformation by the similarity score. The present invention can effectively identify flexible objects before and after deformation.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a deformable object similarity detection method based on Radon transform and electronic equipment. Background Art

[0002] Automatic recognition, description and classification of images are important research topics in the field of computer vision. Among them, object recognition based on feature descriptors is a basic and important task with a wide range of applications, such as gesture recognition, target tracking, pedestrian detection, etc. Object descriptors for rigid deformations have been explored in depth and effectively. Typical studies include geometric shape-based methods and spectral analysis-based methods. However, in larger non-rigid deformation scenarios, the study of object descriptors is still a difficult topic. In addition, similarity learning of two-dimensional shapes also needs further exploration. It is very challenging to solve the structural obstacles between shape features and similarity learning, such as rigid or non-rigid transformations, irregular topological structures, etc.

[0003] Traditionally, shape classification is usually considered as a basic classification problem, including three steps: feature description, evaluation index and classification optimization. Among them, the most difficult is shape expression, which directly affects the efficiency and accuracy of the final recognition. Therefore, this part needs to be studied. In recent decades, many local and global feature descriptors have been proposed to extract the features of objects. Local descriptors achieve matching between fragments or point-by-point correspondence by constructing local features. However, descriptions based on local features are limited by the form of expression. They cannot express the internal content of the shape and are not applicable to disconnected and porous shapes. Compared with local shape description, global shape description is widely used because of its simple implementation. Its advantage is that it does not require preprocessing and additional segmentation. However, it requires that the description of the shape must be sufficient and its internal content cannot be missed. Therefore, it is very difficult and important to choose a suitable description method to adapt to various situations. At the same time, what kind of shape description features are used determines what matching algorithm is used to achieve the best performance. Neural networks seem to be a good choice for matching algorithms because they have good cross-scenario generalization capabilities. However, it has the problem of overfitting, so the parameters can be constrained through l2 regularization, which plays an indispensable role in deep learning and is more like an adjustment parameter that helps optimize the model.

[0004] The feature description of the shape is usually divided into two categories: (1) description based on local features; (2) description based on global features. Then, the matching algorithm is used according to the feature description.

[0005] Descriptions based on local features usually use features to describe edge information, mainly based on robustness to noise and tolerance to small distortions. In general, the most common edge descriptor is Fourier Descriptors (FD). Contour information can also be polygonal primitives or curvature information. However, any polygonal approximation algorithm will more or less lead to information loss. Lui et al. proposed a method to map objects into the unit circle, obtain a set of differential homomorphisms of the unit circle, and define shape features together with conformal moduli. This method is robust to polygons. However, its main disadvantage is its high time complexity. Rasche proposed a decomposition that parameterizes the geometry and appearance of contours and regions of grayscale images for the purpose of fast classification. In order to express contour geometry, the contour is converted into a local / global space, parameters are derived from the space, its global geometry (arc, inflection point or alternation) is classified, and its local aspects (curvature, edge, symmetry) are described.

[0006] Other methods, such as shape context (SC) or skeleton methods, are also based on contour information. Shape context is highly robust to small perturbations, but it is difficult to guarantee scale invariance. Skeleton extraction is also widely used in pattern matching. Bai matches skeleton graphs by comparing geodesic paths between skeleton endpoints. Wei et al. introduced a method to quantify skeleton properties using Bayesian probability weights and take a single parameter as the weight factor for this property.

[0007] In order to avoid the limitation of edge feature description by the form of expression, related studies adopt the description method based on regional features, taking into account all pixels in a certain area. The commonly used method is based on moment theory. Hu proposed the invariant moment theory of image recognition and proposed the moment invariant based on algebraic invariants for the first time. In fact, the so-called invariant moment can be regarded as a statistical feature of the image. It mainly uses the moments of each order of the grayscale distribution of the image to describe the characteristics of the grayscale distribution of the image. Zhang and Lu proposed a region-based Generic Fourier descriptor (GFD) to overcome the limitations of the contour-based FD. On the other hand, this method uses a two-dimensional Fourier transform (FT) on shape sampling to avoid the rotation problem in the Fourier spectrum. Experimental results show that GFD is superior to traditional contour-based and region-based methods. The description based on regional features usually adopts normalization methods such as centroid position and resampling to meet the geometric invariance characteristics of the graphics.

[0008] In early methods, content-based image similarity matching searches and sorts by comparing the differences in the visual content of images, that is, for a given image, the images in the collection are sorted according to their similarity to the given image. The typical method is to represent its visual content through a set of hand-crafted features (such as SIFT). Since a single image may contain hundreds of such features, aggregation techniques such as bag-of-words (BoW) can be used to encode local descriptors into a compact vector, thereby improving computational efficiency and scalability. Garcia et al. proposed a new shape descriptor based on unsupervised deep learning and BoW to discriminate and learn compact shape features.

[0009] Recently, due to the rapid development of deep learning, obtaining features from convolutional neural networks has become a new direction for image similarity learning. Some popular similarity learning works, such as OASIS and MLR, perform linear metric learning by optimizing the weights of linear transformation matrices. For example, Yang et al. proposed a framework for sorting elements in retrieval tasks by solving linear optimization problems.

[0010] Nonlinear similarity learning based on deep learning has recently been applied to many different visual environments. Usually, Siamese or Triple architecture is used to match patches by training CNN, which is suitable for face recognition, retrieval, classification and product search. For example, Sergey improved the traditional Siamese network and proposed Pseudo-siamese, 2-channel and other network structures. Albert proposed a new instance-level image retrieval method. It generates a global and compact fixed-length representation for each image by aggregating multiple region descriptors.

[0011] Shape descriptors based on local features are efficient because they are based on global optimization. However, they have high complexity. Descriptions based on local features are limited by the form of expression and cannot express the internal content of the shape. Therefore, this type of method is not applicable to disconnected and porous shapes. Shape descriptors based on global features are usually represented by a single feature vector, which may not ensure that all shape information is obtained, and thus the reliability of the final decision cannot be guaranteed.

[0012] Although the descriptors of early shape matching methods have high compactness and robustness, their parameter estimation is complex, resulting in high time complexity. Linear methods are easier to optimize and less prone to overfitting, but the accuracy of nonlinear methods is significantly better than linear methods. Deep learning-based methods usually rely on learning the properties of image pixels themselves, without analyzing the features of the original image. This often leads to errors in the network's similarity judgment of objects when the objects undergo large deformations. Summary of the invention

[0013] Therefore, the technical problem to be solved by the present invention is to overcome the problem of low accuracy in detecting the similarity of a flexible object before and after deformation in the prior art.

[0014] In order to solve the above technical problems, the present invention provides a deformable object similarity detection method based on Radon transform, comprising:

[0015] Step S1: acquiring a first image of the flexible object before deformation and a second image of the flexible object after deformation;

[0016] Step S2: transforming the first image and the second image respectively by Radon transform to obtain a first Radon transform feature map and a second Radon transform feature map;

[0017] Step S3: inputting the first Radon transform feature map and the second Radon transform feature map into a deep neural network, and calculating a similarity score between the first Radon transform feature map and the second Radon transform feature map through the deep neural network;

[0018] Step S4: using the similarity score to determine the similarity between the flexible object before and after deformation.

[0019] In one embodiment of the present invention, the Radon transform formula in step S2 is: Where (ρ, θ) represents a point in Radon space, ρ = xcosθ + ysinθ, D represents the entire plane of the image, f(x, y) represents the pixel value of a point (x, y) on the image, δ is the Dirac function and ρ is the distance from the straight line to the origin in the (x,y) plane, and θ is the angle between the perpendicular line from the origin to the straight line and the x-axis.

[0020] In one embodiment of the present invention, the deep neural network in step S3 is an improved SimNet network.

[0021] In one embodiment of the present invention, the improved SimNet network includes a first feature extraction module, a second feature extraction module and a similarity score calculation module, wherein the first feature extraction module and the second feature extraction module have the same structure and share weights;

[0022] The first feature extraction module is used to extract features of the first Radon transform feature map;

[0023] The second feature extraction module is used to extract features of the second Radon transform feature map;

[0024] The similarity score calculation module is used to calculate the similarity score according to the features extracted by the first feature extraction module and the second feature extraction module.

[0025] In one embodiment of the present invention, the first feature extraction module and the second feature extraction module both include a first feature extraction unit, a first maximum pooling layer, a second feature extraction unit, a second maximum pooling layer, a third feature extraction unit, an attention unit and an SPPF unit connected in sequence; the first feature extraction unit, the second feature extraction unit and the third feature extraction unit each include a convolution layer and a ReLU layer.

[0026] In one embodiment of the present invention, the similarity score calculation module includes a first linear fully connected layer and a second linear fully connected layer connected in sequence.

[0027] In one embodiment of the present invention, the loss function of the deep neural network in the model training in step S3 is: Among them, λ represents the weight decay size, ω represents the weight of the deep neural network, and ||||2 represents the L2 norm. represents the i-th training sample output by the network, N represents the total number of training samples, and y i represents the label corresponding to the i-th training sample and y i ∈{-1,1}, when the network outputs y i =-1 means dissimilar, when the network outputs y i =1 means similar.

[0028] In one embodiment of the present invention, the method further includes verifying the performance of the deep neural network on an MPEG-7 dataset.

[0029] The present invention provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for detecting similarity of deformable objects based on Radon transform when executing the computer program.

[0030] The present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned deformable object similarity detection method based on Radon transform are implemented.

[0031] The above technical solution of the present invention has the following advantages compared with the prior art:

[0032] The deformable object similarity detection method based on Radon transform described in the present invention uses Radon transform to describe the object, which is a special case of Trace transform. Radon features can well describe the overall features of flexible objects, rather than just based on contour information;

[0033] The present invention uses a deep neural network (improved SimNet network) to match the Radon feature map of the object. Since the Radon feature generated by flexible deformation undergoes certain changes, the SimNet network combines the deep feature with the similarity learning technology, which is superior to the simple rigid metric feature and can greatly improve the similarity detection performance.

[0034] In order to improve the accuracy of similarity detection of deformable objects, the present invention further adjusts the SimNet network and optimizes the feature extraction part (introducing attention units and SPPF units). Experiments show that the improved SimNet network has higher detection accuracy and better detection effect than the unimproved SimNet network. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0036] Figure 1 is a method flow chart of an embodiment of the present invention;

[0037] Figure 2 : is a graph showing different states of the pen and its Radon transform characteristics in an embodiment of the present invention;

[0038] FIG3( a) is a network structure diagram of a Radon transform combined with an improved SimNet network according to an embodiment of the present invention;

[0039] FIG3( b ) is a schematic diagram of the structure of an SPPF unit according to an embodiment of the present invention;

[0040] Figure 4 14 original images of ribbons of different shapes according to an embodiment of the present invention;

[0041] Figure 5 The present invention Figure 4The Radon feature maps corresponding to the 14 original images of ribbons of different shapes;

[0042] Figure 6 The present invention Figure 4 Middle Ribbon 1 and Ribbon 2, and Figure 4 Middle Ribbon 1 and Figure 2 The original image of stroke 1 and its corresponding Radon feature map;

[0043] Figure 7 The present invention will Figure 4 The similarity result graph obtained by inputting the original ribbon image into the improved SimNet network;

[0044] Figure 8 The present invention will Figure 5 The similarity result graph obtained by inputting the Radon feature map corresponding to the original ribbon image into the improved SimNet network;

[0045] Fig. 9 The present invention Figure 4 Middle Ribbon 1 and Figure 2 Comparison of the original image of stroke 1 and its corresponding Radon feature map;

[0046] Fig.10 It is a comparison of the combination of three kinds of cloth in the embodiment of the present invention and the original image of ribbon 1 and cloth 1 and the comparison of their corresponding Radon characteristic images;

[0047] Fig.11 Some typical original images of the MPEG-7 data set and the corresponding Radon feature images in the embodiment of the present invention;

[0048] Fig.12 It is a part of the original image of the self-built data set in the embodiment of the present invention and its corresponding Radon feature image;

[0049] Fig.13 Schematic diagram of Radon transformation in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0051] Embodiment 1

[0052] Reference Figure 1 As shown, the deformable object similarity detection method and electronic device based on Radon transform of the present invention include:

[0053] Step S1: acquiring a first image of the flexible object before deformation and a second image of the flexible object after deformation;

[0054] Step S2: transforming the first image and the second image respectively by Radon transform to obtain a first Radon transform feature map and a second Radon transform feature map;

[0055] Step S3: inputting the first Radon transform feature map and the second Radon transform feature map into a deep neural network, and calculating a similarity score between the first Radon transform feature map and the second Radon transform feature map through the deep neural network;

[0056] Step S4: using the similarity score to determine the similarity between the flexible object before and after deformation.

[0057] This embodiment uses Radon transform to describe the object, which is a special case of Trace transform. Radon features can well describe the overall features of flexible objects, not just based on contour information. Afterwards, a deep neural network (improved SimNet network) is used to match the Radon feature map of the object. Since the Radon features generated by flexible deformation undergo certain changes, this embodiment uses an improved SimNet network for feature matching. The SimNet network combines deep features with similarity learning technology, which is superior to simple rigid metric features and can greatly improve image similarity detection performance.

[0058] The following is a detailed introduction to this embodiment:

[0059] This embodiment uses Radon transformation to transform the original image, and then uses the improved SimNet network to calculate the similarity of the Radon feature map of the object. The deformation of the object will have a certain impact on the Radon feature map, and the improved SimNet network can overcome such impact to avoid problems such as information loss.

[0060] The essence of Radon transform is to transform the original function into space. For the Radon transform of two-dimensional graphics, it is to map a straight line ρ=xcosθ+ysinθ in the plane space to a point (ρ,θ) in Radon space. The Radon transform of continuous graphics is:

[0061]

[0062] in, D is the entire image plane, f(x,y) is the pixel value of a point (x,y) on the image, the characteristic function δ is the Dirac function, ρ is the distance from the straight line to the origin in the (x,y) plane, and θ is the angle between the perpendicular line from the origin to the straight line and the x-axis. For details, see Fig.13The Radon transform can be used to detect linear trends in an image. The direction in which the object is located with more pixels can be defined as its main direction (i.e. Fig.13 The Radon transform along the main direction usually has a large change. Therefore, the variance of the projection in this direction is locally maximum, and the derivative removes the low-frequency component of the variance, so the technique is robust.

[0063] Figure 2 Examples, rotations, degradation maps and corresponding Radon transform feature maps of the pen are shown, where (a) is the example of the pen and its Radon feature map; (b) is the rotation map of the pen and its Radon feature map; (c) is the degradation map of the pen and its Radon feature map. It is not difficult to find that the Radon features are similar in all cases.

[0064] For the Radon feature map, it is similar to a histogram. Therefore, if a dynamic programming operation (DTW) is performed on the Radon feature map, the similarity of a pair of Radon feature maps is obtained, which is a rigid distance measurement method. However, this method only uses a part of the histogram data and does not make full use of the entire Radon feature map. Therefore, this embodiment proposes a method for learning a similarity function from visual data retrieved from images based on Radon transform. Visual similarity is the degree of similarity between two images. Generally speaking, for a group of images φ, there is a pair of images I and i and I j , then define S ij Score their similarity. ij The higher the value, the better the pair of images I i and I j In order to calculate S ij , put the picture I k Perform Radon transform to obtain the Radon feature space R corresponding to the image k , then x k =f(I k ,W k ), where I k ∈φ, f(·) is the Radon transform representation function, W k is the corresponding parameter during the transformation. Therefore, the similarity function g(·) of a pair of input images can be obtained as:

[0065] S ij =g(f(I i ,W i ),f(I j ,W j ))

[0066] Among them, I i,I j ,∈φ, when S ij >S ik When i ,I j Than I i ,I k The similarity is higher.

[0067] In order to make better use of the Radon feature map instead of the metric distance function, this embodiment uses a deep neural network to learn a similarity function from visual data. The main advantage of this network is that it dilutes the label, so that the network has good scalability and can classify categories that have not been trained. Specifically, this embodiment uses a twin neural network to extract features and uses a weighted contrast loss to reduce the imbalance of samples, that is, not only unchanged pixels, but also changed pixels are fully considered in the objective function. The advantage of weighted contrast loss is that the feature vectors of the changed pixel pairs will be far away from each other, while the feature vectors of the unchanged pixel pairs will be close to each other. When the original image is directly input into the SimNet network for similarity calculation, the SimNet network has a good performance for rigid objects. However, when the same object with a large deformation is input into the network, the result obtained is not ideal, and even approaches 0. Therefore, the input of the network is changed to a pair of vectors x1 and x2 represented by Radon features, and the output is a similarity score. In this way, the similarity network (that is, combining the Radon transform and the SimNet network) replaces the metric distance function used in the standard system and learns the similarity function g(·) from the Radon feature vector. Compared with the traditional method of directly inputting the image into the SimNet network, this embodiment changes the image input into the Radon feature input network, thereby overcoming the influence of the object's own deformation on the similarity calculation and making the result more accurate.

[0068] The network structure of this embodiment is shown in Figure 3(a). To distinguish it from the traditional SimNet network, the network name of this embodiment is defined as "improved SimNet network". The network in Figure 3(a) has two branches (branch network1 and branchnetwork2), which share exactly the same structure and the same set of weights. Each branch takes the image as input after Radon transformation, and then applies a series of convolutional layers and maximum pooling layers for feature extraction. The branch outputs are connected and input into a network composed of linear fully connected layers for similarity prediction.

[0069] The improved SimNet network of this embodiment includes a first feature extraction module (branch network1), a second feature extraction module (branch network2) and a similarity score calculation module (decision network), wherein the first feature extraction module and the second feature extraction module have the same structure and share weights; the first feature extraction module is used to extract features of a first Radon transform feature map; the second feature extraction module is used to extract features of a second Radon transform feature map; the similarity score calculation module is used to calculate a similarity score based on the features extracted by the first feature extraction module and the second feature extraction module.

[0070] Further, referring to Figure 3(a), the first feature extraction module and the second feature extraction module both include a first feature extraction unit, a first maximum pooling layer, a second feature extraction unit, a second maximum pooling layer, a third feature extraction unit, an attention unit (SE Block) and an SPPF unit connected in sequence; the first feature extraction unit, the second feature extraction unit and the third feature extraction unit each include a convolutional layer (Conv) and a ReLU layer.

[0071] It is worth mentioning that in order to further improve the accuracy of image classification, this embodiment improves the SimNet network by adding attention units (i.e., the above-mentioned SEBlock) to the first feature extraction module and the second feature extraction module. The attention unit applies a channel attention mechanism, which allows the network to automatically learn the importance of each channel by giving each feature channel a weight, and then enhances useful features according to this importance and suppresses useless features (feature recalibration strategy).

[0072] Furthermore, this embodiment also introduces an SPPF unit after the attention unit (i.e., SE Block). The structure of the SPPF unit is similar to the SPP structure. It mainly draws on the idea of ​​the spatial pyramid, realizes the fusion of local features and global features, enriches the expression ability of the feature map, and is suitable for the situation where the target size difference in the image to be detected is large, and can effectively improve the detection accuracy. The SPPF unit in this embodiment is an improvement on the SPP structure. By changing the parallel pooling layer (Max Pool) of the SPP structure to a serial operation, the same effect as the SPP structure is achieved, while reducing repeated operations and meeting the real-time requirements of the algorithm. The structure of the SPPF unit is shown in Figure 3 (b), where the CBL unit is mainly used for feature extraction, the Concat layer is mainly used to fuse the number of feature channels, and Max Pool represents maximum pooling.

[0073] Furthermore, the similarity score calculation module (decision network) includes a first linear fully connected layer (fully connected layer) and a second linear fully connected layer (fully connected layer) connected in sequence, and each fully connected layer is followed by a ReLU layer.

[0074] This example sets the training of the similarity network as a supervised regression task. Using the L2 norm regularization term and the hinge-based loss term, the loss function is:

[0075]

[0076] Among them, λ represents the weight decay size, ω represents the weight of the neural network, and ||||2 represents the L2 norm. represents the i-th training sample output by the network, N represents the total number of training samples, and y i represents the label corresponding to the i-th training sample and y i ∈{-1,1}(when y i The output is -1 for dissimilarities and 1 for similarities).

[0077] Since the collected Radon feature data is not enough to train a complete network, the main training object is the Omniglot dataset. The parameter training update method uses ASGD, the input size is (105,105,3), the batch size is 32, the learning rate is always 1.0, the momentum parameter is selected as 0.9, the weight decay size is selected as λ=0.0005, the number of iterations is 100, and the ratio of training set to validation set is 0.9. The weights are initialized randomly.

[0078] The experimental comparison is as follows:

[0079] (1) DTW algorithm

[0080] This embodiment performs Radon transform on objects of different shapes, and compares the differences between the DTW algorithm, the SimNet algorithm, and the Radon-based and improved SimNet network, verifying the feasibility and reliability of the method of this embodiment.

[0081] According to the three basic geometric outline shapes proposed by Rasche, including a perfect arc, an arc with a single inflection point, and an L-shaped arc, the ribbon is placed in different shapes taking into full consideration its own characteristics. Figure 4 There are 14 shapes from Ribbon1 to Ribbon14, and the Radon feature maps corresponding to different shapes of ribbons are obtained. Please refer to Figure 5 , Radon1 to Radon14 correspond to Figure 4 Radon characteristic diagram of 14 different ribbon shapes.

[0082] See also Figure 6 ,by Figure 4 Ribbon 1, Ribbon 2 and Figure 2 Taking pen 1 (i.e., pen example) in the figure as an example, the corresponding Radon feature map is subjected to DTW calculation, that is, the DTW distances of ribbon 1, ribbon 2, and ribbon 1 and pen 1 are calculated respectively, and the results obtained are all 0, and the three are judged to be the same object. This is because when the original image is directly subjected to Radon transformation, the background difference is not eliminated, and the result obtained by DTW calculation of the Radon feature map is greatly disturbed. Therefore, this method has great limitations.

[0083] (2) Radon-based and improved SimNet network

[0084] The original ribbon image ( Figure 4 ) and the corresponding Radon feature map ( Figure 5 ) are input into the network in pairs, and then Figure 7 It can be seen that for cases where the ribbon deformation is not serious, such as simple stretching, the similarity is higher than 0.5; however, for cases where the ribbon deformation is serious, such as bending, the similarity is generally lower than 0.5, or even close to 0. Figure 8 It can be seen that only 2% of the similarities are lower than 0.5. Figure 7 , 8 By comparison, the Radon feature similarity is 94% higher than the original image similarity. Therefore, Radon feature similarity recognition has higher robustness when the same object has various deformations.

[0085] Fig. 9 The similarity between the original image and Radon feature of Ribbon 1 and Pen 1 was calculated respectively. The similarity of the original image is 0.977, while the similarity of the Radon feature is 0.595. The results show that for different objects, when the morphological features are very similar, direct comparison of the original image will lead to misjudgment and low reliability. When Radon transformation is performed on different objects and then compared, the Radon features are quite different, the similarity is less than 0.6, and the reliability is high.

[0086] Fig.10The similarity between the original image and Radon feature of three forms of cloth (very flat cloth 1, slightly wrinkled cloth 2, and fully wrinkled cloth 3) was calculated respectively. For the original image, the similarity of the first group (cloth 1 and cloth 2) was 0.895, but the similarities of the second group (cloth 1 and cloth 3) and the third group (cloth 2 and cloth 3) were 0.001 and 0.000 respectively. For the Radon feature map, the results were 1.000, 0.995, and 0.997 respectively, judging that the three are the same object. Finally, the ribbon 1 and cloth 1 were compared, and the similarity of the original image was 0.498, and the similarity of the Radon feature map was 0.955. The results show that when there is a large degree of deformation for the same object, the result obtained by directly calculating the similarity of the original image is not accurate, while the result obtained by calculating the similarity of the Radon feature is close to or even equal to 1, which is very ideal. At the same time, although the Radon feature similarity of ribbon 1 and cloth 1 is high, it is not as high as the similarity of the cloth itself. In fact, ribbon and cloth are very similar and can even be classified as the same type of objects, so this result is acceptable.

[0087] (3) MPEG-7 dataset and self-built dataset

[0088] This embodiment evaluates the classification performance of the improved SimNet network on the MPEG-7 dataset, which contains 1400 different shapes divided into 70 classes with rigid and non-rigid properties, where each class contains more than 10 different shapes. Fig.11 The typical original shapes of the data set and their corresponding Radon feature maps are shown. The experimental method is similar to the above ribbon experiment, and the average classification accuracy is obtained. As shown in Table 1, the method of this embodiment reaches 90.11%, which is 10.51% higher than the original method. It is proved that this method is a stable and robust feature description method. This is because Radon transform and deep learning can achieve more discriminative descriptions.

[0089] Table 1 Comparison of MPEG-7 and self-built dataset results

[0090]

[0091] Deformable objects also exist in the real world, so this embodiment creates a dataset containing rigid and non-rigid objects for testing. The details of the dataset are shown in Table 2 below. Fig.12is a typical original picture and its corresponding Radon feature map. This embodiment also uses the same experiment to evaluate the two methods. As shown in Table 1, the accuracy of the original method and the method of this embodiment are 75.35%, 85.17% and 88.02% respectively. The results show that the method of this embodiment has strong robustness in practical applications. It also shows that the shapes of the same class with deformable transformation may be partially different, however, the Radon transform is sufficient to capture the features.

[0092] Table 2 The number of images in each category in the self-built dataset

[0093] type The number of original images and their corresponding Radon feature maps Ribbon 14+14 Gloves 11+11 Bag 13+13 Cloth 13+13 Pen 9+9 Tassel 10+10 total 70+70

[0094] Embodiment 2

[0095] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the deformable object similarity detection method based on Radon transform described in Embodiment 1 are implemented.

[0096] Embodiment 3

[0097] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the deformable object similarity detection method based on Radon transform described in the first embodiment are implemented.

[0098] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0103] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A deformable object similarity detection method based on Radon transform, Features: include: Step S1: acquiring a first image of the flexible object before deformation and a second image of the flexible object after deformation; Step S2: transforming the first image and the second image respectively by Radon transform to obtain a first Radon transform feature map and a second Radon transform feature map; Step S3: inputting the first Radon transform feature map and the second Radon transform feature map into a deep neural network, and calculating a similarity score between the first Radon transform feature map and the second Radon transform feature map through the deep neural network; The deep neural network in step S3 is an improved SimNet network; The improved SimNet network includes a first feature extraction module, a second feature extraction module and a similarity score calculation module, wherein the first feature extraction module and the second feature extraction module have the same structure and share weights; The first feature extraction module is used to extract features of the first Radon transform feature map; The second feature extraction module is used to extract features of the second Radon transform feature map; The similarity score calculation module is used to calculate the similarity score based on the features extracted by the first feature extraction module and the second feature extraction module; The first feature extraction module and the second feature extraction module each include a first feature extraction unit, a first maximum pooling layer, a second feature extraction unit, a second maximum pooling layer, a third feature extraction unit, an attention unit and an SPPF unit connected in sequence; the first feature extraction unit, the second feature extraction unit and the third feature extraction unit each include a convolution layer and a ReLU layer; The similarity score calculation module includes a first linear fully connected layer and a second linear fully connected layer connected in sequence; The loss function of the deep neural network in the model training in step S3 is: Among them, λ represents the weight decay size, ω represents the weight of the deep neural network, || || 2 represents the L2 norm, represents the i-th training sample output by the network, N represents the total number of training samples, and y i represents the label corresponding to the i-th training sample and y i ∈{-1,1}, when the network outputs y i =-1 means dissimilar, when the network outputs y i =1 means similar; Step S4: using the similarity score to determine the similarity between the flexible object before and after deformation.

2. According to the method for similarity detection of deformable objects based on Radon transform in claim 1, Features: The Radon transform formula in step S2 is: Where (ρ, θ) represents a point in Radon space, ρ = xcosθ + ysinθ, D represents the entire plane of the image, f(x, y) represents the pixel value of a point (x, y) on the image, δ is the Dirac function and ρ is the distance from the straight line to the origin in the (x, y) plane, and θ is the angle between the perpendicular line from the origin to the straight line and the x-axis.

3. The method for similarity detection of deformable objects based on Radon transform according to claim 1, Features: The method also includes the step of verifying the performance of the deep neural network on the MPEG-7 dataset.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the deformable object similarity detection method based on Radon transform as claimed in any one of claims 1 to 3 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the deformable object similarity detection method based on Radon transform as claimed in any one of claims 1 to 3 are implemented.

Images