Casting riser image transfer learning method based on domain alignment

Abstract

The invention discloses a casting riser image transfer learning method based on domain alignment. The casting riser image transfer learning method comprises the steps of S1, acquiring a two-dimensional matrix grayscale image of a casting riser; converting the image into a one-dimensional grayscale image vector; s2, acquiring an image covariance matrix according to the one-dimensional grayscale image vector, and taking the image covariance matrix as sample data; the sample data comprises a source domain sample set and a target domain sample set; s3, performing domain alignment processing on the sample data to obtain a transformation target domain sample set; s4, training through the source domain sample set and obtaining a Riemannian mean minimum distance classifier; and transforming the target domain sample set to realize identification and classification of casting riser images based on a Riemannian mean minimum distance classifier. The problems that the light environment of a production site is complex, the shooting postures of different industrial cameras cannot be completely consistent, the consistency of casting riser image collection is difficult to guarantee, and then the recognition precision of casting riser images is poor are solved.

Description

Technical Field

[0001] The invention relates to the technical fields of casting cleaning, image processing and transfer learning, and in particular to a casting riser image transfer learning method based on domain alignment. Background Art

[0002] With the development of robot technology, industrial robots have gradually been applied to the cleaning process of casting risers, reducing workers' repetitive work, ensuring processing quality, and improving production efficiency. On this basis, the application of image recognition technology makes it possible for robots to autonomously identify the type of risers and perform processing according to the corresponding procedures.

[0003] However, the lack of casting precision leads to differences in size and shape of the same risers of different castings. At the same time, due to the complex lighting environment at the production site and the inability of different industrial cameras to shoot in completely consistent postures, the consistency of casting riser image acquisition is difficult to ensure, which leads to the problem of poor recognition accuracy of casting riser images. The above problems seriously affect the establishment of riser image datasets and the generalization ability of image recognition algorithms, which in turn hinders the intelligent development of casting cleaning robot equipment. Summary of the invention

[0004] The present invention provides a casting riser image transfer learning method based on domain alignment to overcome the above technical problems.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A casting riser image transfer learning method based on domain alignment specifically comprises the following steps:

[0007] S1: Obtain a two-dimensional matrix grayscale image of the casting riser and convert it into a one-dimensional grayscale image vector;

[0008] S2: Obtain an image covariance matrix according to the one-dimensional grayscale image vector, and use the image covariance matrix as sample data;

[0009] And the sample data includes a source domain sample set and a target domain sample set;

[0010] The source domain sample set is the image covariance matrix dataset of casting risers with known riser category labels; the target domain sample set is the covariance matrix dataset of casting riser images with unknown riser category labels;

[0011] S3: Perform domain alignment on sample data;

[0012] The domain alignment process includes calculating and obtaining the Riemann center point of the sample data based on the Riemann distance metric, and performing sample centering process on the sample data based on the Riemann center point to obtain the centralized data;

[0013] According to the centralized data, the average Riemann distances of the source domain sample set and the target domain sample set from the Riemann center point are obtained to achieve a scaling transformation of the target domain sample set and obtain a transformed target domain sample set;

[0014] S4: Train and obtain a Riemann mean minimum distance classifier through the source domain sample set;

[0015] The Riemann mean minimum distance classifier is used to classify and identify the casting riser images of the sample data in the transformed target domain sample set.

[0016] Furthermore, S2 obtains the image covariance matrix according to the one-dimensional grayscale image vector, which specifically includes the following steps:

[0017] S21: Define a one-dimensional grayscale image vector as X = [x 11 ,x 12 ,…,x 1n ,x 21 ,x 22 ,…,x mn ]; Calculate and obtain the pixel mean of a one-dimensional grayscale image vector;

[0018] S22: Obtain a zero-mean image according to the pixel mean;

[0019] S23: Based on the zero-mean image, calculate and obtain the image covariance matrix.

[0020] Furthermore, the calculation formula of the pixel mean is:

[0021]

[0022] Where: μ represents the pixel mean of the one-dimensional grayscale image vector; x ij Represents the pixel value of the i-th row and j-th column in the image vector; m and n represent the length and width of the two-dimensional matrix grayscale image respectively;

[0023] The formula for obtaining the zero-mean image is:

[0024] X zero =X-μ

[0025] The calculation formula of the image covariance matrix is:

[0026]

[0027] Where: p represents the image covariance matrix; X zerorepresents a zero-mean image; Represents X zero The transpose of .

[0028] Furthermore, the S3 specifically includes the following steps:

[0029] S31: Set the number of samples in the source domain sample set to n S , the i-th sample in the source domain is P S,i ;

[0030] Set the number of samples in the target domain sample set to n T , the jth sample in the target domain is P T,j ;

[0031] Based on the Riemann distance metric, the Riemann center points of the sample data are calculated and obtained respectively;

[0032] S32: performing sample centering processing on the source domain sample set and the target domain sample set respectively according to the obtained Riemann center points of the source domain sample set and the target domain sample set to obtain centralized data;

[0033] S33: according to the centralized data, obtaining the average Riemann distance between the source domain sample set and the target domain sample set and the Riemann center point;

[0034] S34: Obtain the sample relative dispersion α of the source domain sample set and the target domain sample set according to the average value of the Riemann distance;

[0035] S35: Performing scaling transformation on the samples of the target domain sample set using the relative discreteness α to obtain a transformed target domain sample set.

[0036] Furthermore, the S3 specifically includes the following steps:

[0037] The calculation formula of the Riemann center point is:

[0038]

[0039] Where: M r represents the Riemann center of the source domain sample set or the target domain sample set and M r ={M S,r ,M T,r},M S,r With M T,r Represent the source domain sample set and the target domain sample set The Riemann center point; δ(P 1 ,P 2 ) represents the Riemann distance between any two samples in the source domain sample set or the target domain sample set; ||·|| F represents the Fibonacci norm; λ iexpress The ith real eigenvalue of s ,n T};

[0040] The expression for sample centering is:

[0041]

[0042] Where: and Respectively represent P S,i With P T,j The matrix form after centralization;

[0043] The expression of the average value of the Riemann distance is:

[0044]

[0045] Where: d S Represents the source domain sample set to the Riemann center point M S,r The average Riemann distance of T Represents the target domain sample set to the Riemann center point M T,r The average Riemann distance of

[0046] The expression of the sample relative discreteness α is:

[0047]

[0048] The expression for scaling transformation is:

[0049]

[0050] Where: Represents the sample of the target domain sample set Samples after scaling transformation.

[0051] Beneficial effect: The present invention provides a casting riser image transfer learning method based on domain alignment.

[0052] The image covariance matrix is ​​obtained by converting the two-dimensional matrix grayscale image of the casting riser into a one-dimensional grayscale image vector, that is, the original grayscale image is used to construct a one-dimensional grayscale image vector group, and then the vector mean is made zero through zero mean transformation, thereby reducing the influence of light intensity on the image itself, and finally the covariance matrix is ​​constructed using the covariance between pixels to obtain a feature matrix with symmetric positive definite properties, so that the distribution of samples can be more effectively described using the Riemann metric; the image covariance matrix data set of the casting riser with known riser category labels is used as the source domain sample set; the covariance matrix data set of the casting riser image with unknown riser category labels is used as the target domain sample set; and the matching of the source domain and target domain sample features is achieved through domain alignment processing, thereby reducing the data distribution difference between different casting riser images, improving the generalization performance of the image recognition algorithm, and achieving the effect that the model trained with source domain samples can be effectively generalized to the target domain samples, thereby enhancing the robustness of the robot visual recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 It is a flow chart of the casting riser image transfer learning method based on domain alignment of the present invention;

[0055] Figure 2 This is a core flow chart of the casting riser image transfer learning method based on domain alignment in this embodiment;

[0056] Figure 3 Schematic diagram of the Riemann mean minimum distance classifier in this embodiment;

[0057] Figure 4 Schematic diagram of converting a two-dimensional matrix grayscale image into a one-dimensional grayscale image vector in this embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] This embodiment provides a casting riser image transfer learning method based on domain alignment, such as Figure 1 to Figure 2 As shown, the specific steps include:

[0060] S1: Obtain a two-dimensional matrix grayscale image of the casting riser; and convert it into a one-dimensional grayscale image vector; specifically, Figure 4 As shown in the figure, assuming that the size of the grayscale image collected by the industrial camera for the riser at a certain position of the casting is m×n, the image is spliced ​​into a vector group with a length of m×n according to each row; that is, the original form of the grayscale image is a two-dimensional matrix of m×n, that is, m rows and n columns; after splicing according to each row, it becomes a one-dimensional row vector with a size (or length) of m×n, or it can be understood as 1 row and m×n columns;

[0061] S2: Obtaining an image covariance matrix according to the one-dimensional grayscale image vector, and using the image covariance matrix as sample data, specifically comprising the following steps:

[0062] S21: Define a one-dimensional grayscale image vector as X = [x 11 ,x 12 ,…,x 1n ,x 21 ,x 22 ,…,x mn ]; Calculate and obtain the pixel mean of a one-dimensional grayscale image vector;

[0063] The calculation formula of the pixel mean is:

[0064]

[0065] Where: μ represents the pixel mean of the one-dimensional grayscale image vector; x ij Represents the pixel value of the i-th row and j-th column in the image vector; m and n represent the length and width of the two-dimensional matrix grayscale image respectively;

[0066] S22: Obtain a zero-mean image according to the pixel mean; that is, subtract the mean from each pixel value so that the mean of the image is 0. The formula for obtaining the zero-mean image is:

[0067] X zero =X-μ

[0068] S23: Based on the zero-mean image, calculate the covariance between image pixels to obtain an image covariance matrix;

[0069] The calculation formula of the image covariance matrix is:

[0070]

[0071] Where: p represents the image covariance matrix; X zerorepresents a zero-mean image; Represents X zero The transpose of

[0072] And the sample data includes a source domain sample set and a target domain sample set;

[0073] The source domain sample set is the image covariance matrix dataset of casting risers with known riser category labels; the target domain sample set is the covariance matrix dataset of casting riser images with unknown riser category labels; in actual engineering, the naming of riser categories is completed by manually numbering the riser parts (such as the riser at position 1, the riser at position 2... the riser at position n are the first, second...nth types respectively), which is only used to distinguish and has no actual precise meaning, which is different from general classification tasks such as classifying cats and dogs and classifying medical images;

[0074] In this embodiment, the original grayscale image is first used to construct a vector, that is, the two-dimensional matrix description is converted into a one-dimensional grayscale image vector through vector reorganization; then the vector mean is made zero through zero mean transformation, thereby reducing the influence of light intensity on the image itself; finally, the covariance between pixels is used to construct a covariance matrix, and a characteristic matrix with symmetric positive definite properties is obtained, so that the Riemann metric can be used to more effectively describe the distribution of samples in the future;

[0075] S3: Perform domain alignment on sample data;

[0076] The domain alignment process includes calculating and obtaining the Riemann center point of the sample data based on the Riemann distance metric, and performing sample centering processing on the sample data based on the Riemann center point (ie, making the Riemann center point of the sample a unit matrix) to obtain centralized data;

[0077] According to the centralized data, the average Riemann distances of the source domain sample set and the target domain sample set from the Riemann center point are obtained to achieve the scaling transformation of the target domain sample set and obtain the transformed target domain sample set;

[0078] The specific steps include:

[0079] S31: Set the number of samples in the source domain sample set to n S , the i-th sample in the source domain is P S,i ;

[0080] Set the number of samples in the target domain sample set to n T , the jth sample in the target domain is P T,j ;

[0081] And based on the Riemann distance metric, the source domain samples are calculated separately and target domain samples The Riemann center MS,r With M T,r ; Any two samples P in the source domain sample set or the target domain sample set 1 and P 2 The Riemann distance between is defined as:

[0082]

[0083] The Riemann center point of the source domain sample set or the target domain sample set is calculated and obtained according to the Riemann distance; the solution formula of the Riemann center point is:

[0084]

[0085] Where: M r represents the Riemann center of the source domain sample set or the target domain sample set and M r ={M S,r ,M T,r},M S,r With M T,r Represent the source domain sample set and the target domain sample set The Riemann center point; δ(P 1 ,P 2 ) represents the Riemann distance between any two samples in the source domain sample set or the target domain sample set; ||·|| F represents the Fibonacci norm; λ i express The ith real eigenvalue of s ,n T};

[0086] S32: performing sample centering processing on the source domain sample set and the target domain sample set respectively according to the obtained Riemann center points of the source domain sample set and the target domain sample set to obtain centralized data;

[0087] The expression for sample centering is:

[0088]

[0089] Where: and Respectively represent P S,i With P T,j The matrix form after centralization;

[0090] S33: Obtain the average Riemann distance between the source domain sample set and the target domain sample set and the Riemann center point according to the centralized data;

[0091] The expression of the average value of the Riemann distance is:

[0092]

[0093] Where: d S Represents the source domain sample set to the Riemann center point M S,r The average Riemann distance of T Represents the target domain sample set to the Riemann center point M T,r The average Riemann distance of

[0094] In this embodiment, according to the source of the samples, the constructed covariance matrix is ​​divided into a source domain sample set and a target domain sample set, and then the Riemann center points of the source domain samples and the target domain samples are respectively calculated based on the Riemann distance metric, and the samples of the source domain and the target domain are centered using their respective Riemann center points. After the samples are centered, the Riemann center points of the source domain and the target domain samples are both unit matrices, so they have the same geometric center in the Riemann geometric space;

[0095] S34: Obtain the sample relative discreteness α of the source domain sample set and the target domain sample set according to the average value of the Riemann distance; the expression of the sample relative discreteness α is:

[0096]

[0097] S35: scaling the samples of the target domain sample set using the relative discreteness α to obtain a transformed target domain sample set;

[0098] The expression for scaling transformation is:

[0099]

[0100] Where: represents the samples of the target domain sample set after scaling transformation;

[0101] In this embodiment, the average Riemann distances of each sample in the source domain sample set and the target domain sample set to the Riemann center are calculated respectively; then the quotient of the two is used as a standard for measuring the difference in dispersion between the source domain and the target domain samples, and the target domain samples are scaled using the value so that the average distance of the target domain samples from the Riemann center is the same or similar to that of the source domain samples;

[0102] S4: Train and obtain a Riemann mean minimum distance classifier through the source domain sample set;

[0103] The Riemann mean minimum distance classifier is used to classify and identify the casting riser images of the sample data in the transformed target domain sample set.

[0104] The principle of the Riemann mean minimum distance classifier is as follows Figure 3 As shown:

[0105] This schematic diagram takes binary classification as an example, and the Riemann means A and B are the Riemann center points of the two categories of samples respectively. For the covariance matrix samples located on the Riemann manifold, the Riemann distance from the unknown sample to the Riemann center point of each category can more accurately describe the similarity between the unknown sample and the samples of each category, that is, the closer the distance to the center point, the greater the similarity is considered, and vice versa. For sample C, if its Riemann distance from the center point A is less than its Riemann distance from the center point B, it will be judged as the category represented by A, otherwise it will be judged as the category represented by B. In this embodiment, the samples of the source domain and the target domain have the same or similar data distribution, and the domain alignment processing is realized; after the domain alignment processing, the Riemann mean minimum distance classifier trained with the source domain samples can effectively classify the unknown samples in the target domain in the Riemann space, improve the generalization performance of the model, and realize transfer learning.

[0106] In this embodiment, the two-dimensional matrix grayscale image of the casting riser is converted into a one-dimensional grayscale image vector to obtain an image covariance matrix, that is, a one-dimensional grayscale image vector group is constructed using the original grayscale image, and then the vector mean is made zero by zero mean transformation, thereby reducing the influence of light intensity on the image itself, and finally a covariance matrix is ​​constructed using the covariance between pixel points to obtain a feature matrix with a symmetric positive definite property, so that the distribution of samples can be more effectively described using the Riemann metric; the image covariance matrix data set of the casting riser with a known riser category label is used as the source domain sample set; the covariance matrix data set of the casting riser image with an unknown riser category label is used as the target domain sample set; and the matching of the source domain and target domain sample features is achieved through domain alignment processing, thereby reducing the data distribution difference between different casting riser images, improving the generalization performance of the image recognition algorithm, achieving the effect that the model trained with source domain samples can be effectively generalized to the target domain samples, and enhancing the robustness of the robot visual recognition system.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A casting riser image transfer learning method based on domain alignment, characterized in that: The specific steps include: S1: Obtain a two-dimensional matrix grayscale image of the casting riser and convert it into a one-dimensional grayscale image vector; S2: Obtain an image covariance matrix according to the one-dimensional grayscale image vector, and use the image covariance matrix as sample data; And the sample data includes a source domain sample set and a target domain sample set; The source domain sample set is the image covariance matrix dataset of casting risers with known riser category labels; the target domain sample set is the covariance matrix dataset of casting riser images with unknown riser category labels; S3: Perform domain alignment on sample data; The domain alignment process includes calculating and obtaining the Riemann center point of the sample data based on the Riemann distance metric, and performing sample centering process on the sample data based on the Riemann center point to obtain the centralized data; According to the centralized data, the average Riemann distances of the source domain sample set and the target domain sample set from the Riemann center point are obtained to achieve a scaling transformation of the target domain sample set and obtain a transformed target domain sample set; S4: Train and obtain a Riemann mean minimum distance classifier through the source domain sample set; The Riemann mean minimum distance classifier is used to classify and identify the casting riser images of the sample data in the transformed target domain sample set.

2. The method for casting riser image transfer learning based on domain alignment according to claim 1, characterized in that: In S2, the image covariance matrix is ​​obtained according to the one-dimensional grayscale image vector, which specifically includes the following steps: S21: Define a one-dimensional grayscale image vector as X = [x 11 ,x 12 ,…,x 1n ,x 21 ,x 22 ,…,x mn ]; Calculate and obtain the pixel mean of a one-dimensional grayscale image vector; S22: Obtain a zero-mean image according to the pixel mean; S23: Based on the zero-mean image, calculate and obtain the image covariance matrix.

3. The method for casting riser image transfer learning based on domain alignment according to claim 2, characterized in that: The calculation formula of the pixel mean is: Where: μ represents the pixel mean of the one-dimensional grayscale image vector; x ij Represents the pixel value of the i-th row and j-th column in the image vector; m and n represent the length and width of the two-dimensional matrix grayscale image respectively; The formula for obtaining the zero-mean image is: X zero =X-μ The calculation formula of the image covariance matrix is: Where: p represents the image covariance matrix; X zero represents a zero-mean image; Represents X zero The transpose of .

4. The method for casting riser image transfer learning based on domain alignment according to claim 3, characterized in that: The S3 specifically includes the following steps: S31: Set the number of samples in the source domain sample set to n s , the i-th sample in the source domain is P S,i ; Set the number of samples in the target domain sample set to n T , the jth sample in the target domain is P T,j ; Based on the Riemann distance metric, the Riemann center points of the sample data are calculated and obtained respectively; S32: performing sample centering processing on the source domain sample set and the target domain sample set according to the obtained Riemann center points of the source domain sample set and the target domain sample set to obtain centralized data; S33: Obtain the average Riemann distance between the source domain sample set and the target domain sample set and the Riemann center point according to the centralized data; S34: Obtain the sample relative dispersion α of the source domain sample set and the target domain sample set according to the average value of the Riemann distance; S35: Performing scaling transformation on the samples of the target domain sample set using the relative discreteness α to obtain a transformed target domain sample set.

5. The method for casting riser image transfer learning based on domain alignment according to claim 4, characterized in that: The S3 specifically includes the following steps: The calculation formula of the Riemann center point is: Where: M r represents the Riemann center of the source domain sample set or the target domain sample set and M r =, and respectively represent the Riemann center points of the source domain sample set and the target domain sample set; δ(P1,P2) represents the Riemann distance between any two samples in the source domain sample set or the target domain sample set; represents the Fibonacci norm; represents The th real eigenvalue of ; represents the number of samples and =; The expression for sample centering is: Where: and Respectively represent the matrix forms after centralization; The expression of the average value of the Riemann distance is: Where: d S Represents the source domain sample set to the Riemann center point M S,r The average Riemann distance of T Represents the target domain sample set to the Riemann center point M T,r The average Riemann distance of The expression of the sample relative discreteness α is: The expression for scaling transformation is: Where: Represents the sample of the target domain sample set Samples after scaling transformation.

Images