State monitoring method for automatic switch of switch machine

Abstract

The invention discloses a method for monitoring the state of an automatic switch of a switch machine, which relates to the technical field of image processing and comprises the following steps of: shooting an image of the automatic switch by utilizing shooting equipment in the switch machine; a nut area image in the automatic shutter image is positioned, fine-grained classification is carried out on the nut area image to judge the state of the nut, and classification is carried out by comparing the cosine similarity of feature vectors during classification; and sliding the static contact sheet template image in the automatic shutter image to perform template matching, and outputting the length of the successfully matched static contact sheet template image as the contact depth of the static contact sheet. According to the invention, the state of the nut and the contact depth of the static contact sheet are monitored, the stability and safety of the turnout switch machine are ensured, and the stability and accuracy of electrical signals are ensured.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning, image processing and target detection, and more specifically to a method for monitoring the state of an automatic switch. Background Art

[0002] With the development of railway technology, switch machines have emerged. Switch machines are important signal infrastructure equipment used to reliably switch the position of switches, change the direction of switch opening, lock the switch point rail, and reflect the position of switches. It can well ensure driving safety, improve transportation efficiency, and improve the labor intensity of drivers. Automatic openers and closeers are a device of switch machines, which are used to control the opening and closing of switches and signal transmission.

[0003] When the automatic switch is running, it is necessary to monitor the loosening and missing nuts of the automatic switch, the contact depth of the static contact piece, etc. This is because: the nuts of the automatic switch must be kept tight to prevent mechanical failure caused by loose or missing nuts. Loose or missing nuts may affect the normal operation of the automatic switch, and then affect the accuracy of the railway signal system. The contact depth of the moving contact in the static contact piece should be no less than 4mm, which is a key indicator to ensure the stability of the electrical connection. At the same time, when the moving contact is pulled by hand, its swing should not be greater than 3.5mm to ensure that the gap between the moving contact and the static contact seat is no less than 3mm, thereby avoiding poor contact or electrical short circuit problems.

[0004] The current monitoring of the loosening and missing nuts of the automatic switch opening and closing device and the contact depth of the static contact piece has the following difficulties:

[0005] 1. If Figure 2 As shown in the figure, the camera distortion in the image is extremely serious, while the pixels occupied by the target are very small, which belongs to the category of small target detection and is a key and difficult problem in the field of target detection.

[0006] 2. If Figure 3 As shown, different models of switch machine closers have different appearance characteristics and different camera angle distortions, which have a serious impact on the calculation of the contact depth of the static contact piece.

[0007] In the prior art, the patent with publication number CN115100593A discloses an online monitoring method and device for the working condition of a switch machine based on image information processing, the method comprising: receiving image information of the switch machine static contact group collected by a camera; obtaining the access depth of the switch machine static node through a trained image processing model according to the image information; and identifying the working condition of the switch machine according to the access depth of the switch machine static node. Compared with the prior art, the present invention has the advantages of online monitoring, high accuracy, labor saving, and high efficiency. Although it monitors the access depth of the switch machine static node, it does not monitor the loosening and missing of the nut of the automatic switch machine opener and closeer. Summary of the invention

[0008] In order to overcome the defects in the prior art, the present invention discloses a method for monitoring the state of an automatic switch door, so as to monitor the loosening and missing of the nut of the automatic switch door and the contact depth of the static contact piece.

[0009] In order to achieve the above objectives, the technical solution adopted by the present invention is:

[0010] A method for monitoring the state of an automatic switch of a switch machine comprises the following steps:

[0011] 1. Image acquisition

[0012] S1. Use the internal shooting equipment of the switch machine to shoot the image of the automatic switch;

[0013] In the present invention, the photographing device of the automatic switch is installed inside the switch machine, located just above the automatic switch, to photograph and collect the image of the automatic switch.

[0014] 2. Fine-grained classification of nut positioning

[0015] S2, locating the nut region image in the automatic opener and closer image, and performing fine-grained classification on the nut region image to determine the nut state, and performing classification by comparing the cosine similarity of the feature vectors during classification;

[0016] In the above steps, deep learning weights are used to locate all nuts in the automatic opener and closer.

[0017] Preferably, the fine-grained classification of the nut region image includes:

[0018] Use the Resnet-18 classification network for fine-grained classification, and use the Arcface loss function to calculate the loss of the classification network;

[0019] The classification is normalized through the softmax activation function, the scores are converted into probability distribution, and the category with the highest probability is selected for output to obtain a fine-grained classification result.

[0020] In the present invention, the Resnet-18 classification network is combined with the Arcface loss function to perform loss calculation, and finally the score is calculated by the softmax activation function. The principle is to classify by comparing the cosine similarity of the feature vectors. The formula is as follows:

[0021] Preferably, the Arcface loss function is:

[0022]

[0023] Among them, L is the loss value, N is the batch size, C is the number of categories, s is the scaling factor, and m is the introduced angle, which is set to 0.5. is the central angle between the sample and the true category, θ j is the center angle between the sample and other categories. In the implementation process, the angle is calculated by the dot product of the sample feature vector and the category center vector. i is the i-th batch, j is the j-th category, and y " For sample.

[0024] In the present invention, after the loss is obtained, it is normalized by softmax, and the formula is as follows:

[0025] Preferably, the classification normalization by the softmax activation function includes:

[0026]

[0027] Among them, P is the probability, y " is the sample value, x " is the true category value, C is the number of categories, s is the scaling factor, and m is the introduced angle. is the central angle between the sample and the true category, θ j is the central angle between the sample and other categories, and j is the jth category.

[0028] In the present invention, the category with the largest value is finally selected for output to obtain the result.

[0029] In the present invention, for fine-grained classification and nut status, the softmax activation function will regularize the probability values ​​of the three categories of nut looseness, missingness, and breakage into one, and finally take the maximum value of these three probability values ​​for output.

[0030] 3. Calculation of static contact length

[0031] S3. Slide the static contact sheet template image in the automatic opener and closer image to perform template matching, and output the length of the successfully matched static contact sheet template image as the static contact sheet contact depth.

[0032] Preferably, the step of sliding the static contact sheet template image in the automatic opener and closer image for template matching and outputting the length of the successfully matched static contact sheet template image as the static contact sheet contact depth comprises:

[0033] The automatic opener and closer image is processed by average filtering to obtain the target image. Different template images of the static contact piece are slid in the target image, and the similarity and maximum similarity between the template image and the target image are calculated at different angles. When the maximum similarity exceeds the threshold, the template image and the target image are successfully matched, and the length of the successfully matched template image is output as the contact depth of the static contact piece.

[0034] Preferably, the step of performing average filtering on the automatic opener / closer image to obtain the target image comprises:

[0035]

[0036] Among them, I(x,y) is the pixel value of the original image at position (x,y), I ′ (x,y) is the pixel value of the filtered image at position (x,y), n is the set of pixels in the window, N is the total number of pixels in the window, i is the number on the x-axis, and j is the number on the y-axis.

[0037] In the present invention, three types of template images are set for the static contact piece template image, namely, static contact piece missing, static contact piece normal, and static contact piece too long, and subsequent operations are performed according to the template with the highest matching similarity.

[0038] Preferably, the template matching includes:

[0039] Rotate the template image, and the rotation angle range is [0,360°];

[0040] During the rotation process, the normalized cross-correlation method is used to calculate the similarity between the template image and the target image;

[0041] Traverse all possible positions and angles in the target image and find the position and angle corresponding to the maximum similarity;

[0042] Set a threshold to determine whether the maximum similarity exceeds the threshold. If so, the template image and the target image are successfully matched;

[0043] The length of the successfully matched template image is output as the contact depth of the static contact piece.

[0044] Preferably, rotating the template image includes:

[0045] T4(x,y)=T(xcos(θ)-ysin(θ),xsin(θ)+ycos(θ))

[0046] Wherein, T4 is the coordinate after rotation transformation, T is the original coordinate point, θ is the rotation angle, x is the x coordinate, and y is the y coordinate.

[0047] Preferably, the normalized cross-correlation method is used, comprising:

[0048]

[0049] Among them, s is the similarity, x ； is the x coordinate of the target image, y ； is the y coordinate of the target image, θ is the rotation angle, x is the x coordinate of the template image, y is the y coordinate of the template image, I is the offset of the reference point, T θ is the pixel value, μ " is the average value of the target image, μ T is the average value of the template image.

[0050] Preferably, the setting of the threshold value to determine whether the maximum similarity exceeds the threshold value, if so, the template image and the target image are successfully matched, includes:

[0051] Set the threshold to 0.8 to determine the maximum similarity Is it greater than 0.8? If so, the match is successful.

[0052] Beneficial effects of the present invention:

[0053] 1. In the case of fine-grained classification, the present invention uses feature vectors to distinguish categories, which will greatly increase the gap between different categories, making the classification effect excellent, increasing the strength of classification, and solving the problem of being unable to distinguish categories due to small feature differences.

[0054] 2. The camera distortion in the image is extremely serious, and the pixels occupied by the target are very small, which belongs to the category of small target detection and is a key and difficult problem in the field of target detection. In the present invention, the above problems are solved by fine-grained classification and modification of the loss function.

[0055] 3. The appearance characteristics of switch machine switches of different models are different, and the camera angle distortion is different, which has a serious impact on the calculation of the contact depth of the static contact piece. In the present invention, template matching is used to improve generalization to reduce the output value error caused by external features and distortion.

[0056] 4. In the present invention, the state of the nut and the contact depth of the static contact piece are monitored to ensure the stability and safety of the turnout machine and the stability and accuracy of the electrical signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of the state monitoring method of the automatic switch opening and closing device of the present invention;

[0058] Figure 2 The image of the automatic opener and closer of the present invention;

[0059] Figure 3The images are of different types of automatic openers and closeers of the present invention;

[0060] Figure 4 It is the template image of the present invention. DETAILED DESCRIPTION

[0061] The concept, specific structure and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings to fully understand the purpose, characteristics and effects of the present invention.

[0062] Example 1

[0063] A method for monitoring the state of an automatic switch switch, such as Figure 1 As shown, the following steps are included:

[0064] S1. Use the internal shooting equipment of the switch machine to shoot the image of the automatic switch;

[0065] S2, locating the nut region image in the automatic opener and closer image, and performing fine-grained classification on the nut region image to determine the nut state, and performing classification by comparing the cosine similarity of the feature vectors during classification;

[0066] S3. Slide the static contact sheet template image in the automatic opener and closer image to perform template matching, and output the length of the successfully matched static contact sheet template image as the static contact sheet contact depth.

[0067] Example 2

[0068] This embodiment further elaborates on step S1 based on embodiment 1. In step S1, the photographing device of the automatic switch is installed inside the switch machine, located directly above the automatic switch, to photograph and collect the image of the automatic switch, and then the image of the automatic switch is processed by steps S2 and S3 to obtain the nut state and the contact depth of the static contact piece.

[0069] Example 3

[0070] This embodiment further describes step S2 based on embodiment 2.

[0071] In step S2, the Resnet-18 classification network is combined with the Arcface loss function to calculate the loss, and finally the score is calculated by the softmax activation function. The principle is to classify by comparing the cosine similarity of the feature vectors. The formula is as follows:

[0072]

[0073] Among them, L is the loss value, N is the batch size, C is the number of categories, s is the scaling factor, and m is the introduced angle, which is set to 0.5. is the central angle between the sample and the true category, θ j is the center angle between the sample and other categories. In the implementation process, the angle is calculated by the dot product of the sample feature vector and the category center vector. i is the i-th batch, j is the j-th category, and y " For sample.

[0074] After obtaining the loss, normalize it through softmax, the formula is as follows:

[0075]

[0076] Among them, P is the probability, y " is the sample value, x " is the true category value, C is the number of categories, s is the scaling factor, and m is the introduced angle. is the central angle between the sample and the true category, θ j is the central angle between the sample and other categories, and j is the jth category.

[0077] For fine-grained classification and nut status, the softmax activation function will normalize the probability values ​​of the three categories of loose nuts, missing nuts, and broken nuts into one, and finally take the maximum value of these three probability values ​​for output to obtain the result.

[0078] Example 4

[0079] This embodiment further describes step S3 based on embodiment 3.

[0080] The input image is first subjected to average filtering, the formula is as follows:

[0081]

[0082] Among them, I(x,y) is the pixel value of the original image at position (x,y), I ′ (x,y) is the pixel value of the filtered image at position (x,y), n is the set of pixels in the window, N is the total number of pixels in the window, i is the number of the x-axis, and j is the number of the y-axis. The image is smoothed by averaging filtering.

[0083] Then create a static contact sheet target template, the template image is as follows Figure 4 As shown. By sliding the template in the target image, the similarity between the template and the target image area is calculated at different angles. The position and angle with the highest similarity are the matching results. The specific steps are as follows:

[0084] 3.1 Template rotation:

[0085] The template is rotated within an angle range of [0,360°], and the formula is as follows:

[0086] T4(x,y)=T(xcos(θ)-ysin(θ),xsin(θ)+ycos(θ))

[0087] Wherein, T4 is the coordinate after rotation transformation, T is the original coordinate point, θ is the rotation angle, x is the x coordinate, and y is the y coordinate.

[0088] 3.2 Calculate similarity:

[0089] The similarity between the template and the target image area is calculated using the normalized cross-correlation method, and the formula is as follows:

[0090]

[0091] Among them, s is the similarity, x ； is the x coordinate of the target image, y ； is the y coordinate of the target image, θ is the rotation angle, x is the x coordinate of the template image, y is the y coordinate of the template image, I is the offset of the reference point, T θ is the pixel value, μ " is the average value of the target image, μ T is the average value of the template image.

[0092] 3.3 Traverse all positions and angles:

[0093] Traverse all possible positions (x, y) and angles θ in the target image and find the position and angle corresponding to the maximum similarity. The formula is as follows:

[0094]

[0095] 3.4 Determine the matching score:

[0096] Set the threshold to 0.8 to determine the maximum similarity Whether it exceeds 0.8, if so, the match is considered successful.

[0097] 3.5 Calculate the length:

[0098] The length of the successfully matched template is output, which is the contact depth of the static contact piece.

[0099] In this embodiment, images of automatic switch machines of models ZYJ-7 and ZDJ-9 are collected for test optimization to avoid inaccurate output values ​​due to insufficient generalization.

[0100] The above is a specific description of the implementation mode of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention, and these equivalents or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for monitoring the state of an automatic switch switch, characterized in that: The following steps are involved: Use the internal shooting equipment of the switch machine to take the image of the automatic switch; Locate the nut area image in the automatic opener and closer image, and perform fine-grained classification on the nut area image to determine the nut state. During classification, classification is performed by comparing the cosine similarity of the feature vectors. The static contact sheet template image is slid in the automatic opener and closer image for template matching, and the length of the successfully matched static contact sheet template image is output as the static contact sheet contact depth.

2. A method for monitoring the state of an automatic switch of a switch machine according to claim 1, characterized in that: The fine-grained classification of the nut region image includes: Use the Resnet-18 classification network for fine-grained classification, and use the Arcface loss function to calculate the loss of the classification network; The classification is normalized through the softmax activation function, the scores are converted into probability distribution, and the category with the highest probability is selected for output to obtain a fine-grained classification result.

3. A method for monitoring the state of an automatic switch switch as claimed in claim 2, characterized in that: The Arcface loss function is: Among them, L is the loss value, N is the batch size, C is the number of categories, s is the scaling factor, and m is the introduced angle, which is set to 0.

5. is the central angle between the sample and the true category, θ j is the central angle between the sample and other categories. In the implementation process, the angle is calculated by the dot product of the sample feature vector and the category center vector. i is the i-th batch, j is the j-th category, and y " For sample.

4. A method for monitoring the state of an automatic switch switch as claimed in claim 2, characterized in that: The classification normalization by the softmax activation function includes: Among them, P is the probability, y " is the sample value, x " is the true category value, C is the number of categories, s is the scaling factor, and m is the introduced angle. is the central angle between the sample and the true category, θ j is the central angle between the sample and other categories, and j is the jth category.

5. A method for monitoring the state of an automatic switch switch as claimed in claim 1, characterized in that: The step of sliding the static contact sheet template image in the automatic opener and closer image to perform template matching and outputting the length of the successfully matched static contact sheet template image as the static contact sheet contact depth includes: The automatic opener and closer image is processed by average filtering to obtain the target image. Different template images of the static contact piece are slid in the target image, and the similarity and maximum similarity between the template image and the target image are calculated at different angles. When the maximum similarity exceeds the threshold, the template image and the target image are successfully matched, and the length of the successfully matched template image is output as the contact depth of the static contact piece.

6. A method for monitoring the state of an automatic switch switch as claimed in claim 5, characterized in that: The method of performing average filtering on the automatic opener / closer image to obtain the target image comprises: Among them, I(x,y) is the pixel value of the original image at position (x,y), I ′ (x,y) is the pixel value of the filtered image at position (x,y), n is the set of pixels in the window, N is the total number of pixels in the window, i is the number on the x-axis, and j is the number on the y-axis.

7. A method for monitoring the state of an automatic switch switch as claimed in claim 5, characterized in that: The template matching includes: Rotate the template image, and the rotation angle range is [0,360°]; During the rotation process, the normalized cross-correlation method is used to calculate the similarity between the template image and the target image; Traverse all possible positions and angles in the target image and find the position and angle corresponding to the maximum similarity; Set a threshold to determine whether the maximum similarity exceeds the threshold. If so, the template image and the target image are successfully matched; The length of the successfully matched template image is output as the contact depth of the static contact piece.

8. A method for monitoring the state of an automatic switch of a switch machine as claimed in claim 7, characterized in that: The rotating the template image comprises: T4(x,y)=T(xcos(θ)-ysin(θ),xsin(θ)+ycos(θ)) Wherein, T4 is the coordinate after rotation transformation, T is the original coordinate point, θ is the rotation angle, x is the x coordinate, and y is the y coordinate.

9. A method for monitoring the state of an automatic switch switch as claimed in claim 7, characterized in that: The method of using normalized cross-correlation comprises: Among them, s is the similarity, x ； is the x coordinate of the target image, y ； is the y coordinate of the target image, θ is the rotation angle, x is the x coordinate of the template image, y is the y coordinate of the template image, I is the offset of the reference point, T θ is the pixel value, μ " is the average value of the target image, μ T is the average value of the template image.

10. A method for monitoring the state of an automatic switch switch according to claim 7, characterized in that: The setting of the threshold value and judging whether the maximum similarity exceeds the threshold value, if so, the template image and the target image are successfully matched, include: Set the threshold to 0.8 to determine the maximum similarity Is it greater than 0.8? If so, the match is successful.

Images