Intelligent turnout maintenance evaluation method and device based on semantic segmentation, and medium

Abstract

This invention relates to a method, equipment, and medium for the inspection and evaluation of intelligent switch machines based on semantic segmentation. The method includes the following steps: Step S1, the client acquires an image of the switch machine's operating status and sends it to the server as the image to be inspected; Step S2, the server segments the image to be inspected using a trained semantic segmentation model; Step S3, the server calculates the contact depth of the moving and stationary contacts of the automatic switch machine's switch based on the semantic segmentation results, the gap between the moving and stationary contacts, and the distance between the center of the moving contact post and the center line of the stationary contact spring; Step S4, the server sends the segmentation results and contact distances to the client, and the client determines whether the switch machine's operating status is normal based on the contact distances. Compared with existing technologies, this invention has advantages such as strong anti-interference ability, stable segmentation effect, and high accuracy.

Description

Technical Field

[0001] This invention relates to maintenance technology for rail transit equipment, and in particular to a method, equipment, and medium for the maintenance and evaluation of intelligent switch machines based on semantic segmentation. Background Technology

[0002] Urban rail transit maintenance and assessment is a crucial part of ensuring the normal operation of subways. Traditional methods involve manual measurement of switch machine operating conditions, which has the following unavoidable drawbacks: 1. Workers are prone to fatigue from prolonged work, leading to measurement errors. 2. Subjective errors exist due to differences in personnel. 3. Manual measurement is slow, inefficient, and time-consuming. Therefore, there is an urgent need for an intelligent switch machine maintenance and assessment device to simplify the daily workflow of urban rail transit maintenance personnel and provide standardized switch machine inspection and assessment.

[0003] A search of Chinese Patent Publication No. CN113155504A reveals an intelligent testing system for switch machines in rail transit, including a data collection module. This module comprises a sensor module, a conditioning and conversion circuit, and a control module. The data collection module synchronously collects real-time operating status parameters of the switch machine. The sensors collect data on the switch machine's switching torque, operating current, voltage, operating environment, and indicator rod notch. The detected signals are amplified and filtered by the conditioning and conversion circuit, converted into standard pulse signals, and then processed by A / D conversion before being transmitted to a fault prediction and processing module via a communication cable. The control module includes a control center, a power supply module, a communication exchange module, and a turnout control module. However, monitoring the operating status of the switch machine requires installing sensor modules, conditioning and conversion circuits, and a control module on the switch machine itself to collect information, resulting in cumbersome installation and operation. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a method, equipment and medium for the maintenance and evaluation of intelligent switch machines based on semantic segmentation.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] According to a first aspect of the present invention, a method for the maintenance and evaluation of intelligent switch machines based on semantic segmentation is provided, the method comprising the following steps:

[0007] Step S1: The client collects images of the switch machine's working status and sends them to the server as images to be detected;

[0008] Step S2: The server uses the trained semantic segmentation model to segment the image to be detected;

[0009] Step S3: The server calculates the contact depth of the moving and stationary contacts of the automatic switch machine based on the semantic segmentation results, the gap between the moving and stationary contacts, and the distance between the center of the moving contact post and the center line of the stationary contact spring.

[0010] Step S4: The server sends the segmentation results and contact distance to the client. The client determines whether the switch machine is working properly based on the contact distance.

[0011] As a preferred technical solution, step S1 specifically includes:

[0012] Step S 11 The client-side APP captures images of the switch machine's working status;

[0013] Step S 12 The client-side app sends the captured images to the server via the local area network as images to be detected.

[0014] As a preferred technical solution, the client APP has the functions of calling the camera to take pictures, selecting images, displaying the interface, and communicating with the server.

[0015] As a preferred technical solution, step S2 specifically includes:

[0016] Step S 21 Take a set number of images of switch machines to create a training dataset;

[0017] Step S 22 To augment the dataset;

[0018] Step S 23 LabelMe is used to label the dataset and generate corresponding JSON files, which store the category names and edge coordinates of each category.

[0019] Step S 24 The dataset and its corresponding labels are divided into training and validation sets according to a set ratio and stored in a specified folder;

[0020] Step S 25 Build a semantic segmentation network model, set the network parameters, and start training the model;

[0021] Step S 26 After obtaining the trained model and testing its performance on the validation set, the image to be detected is segmented.

[0022] As a preferred technical solution, step S 22 Data augmentation in this context includes adjusting the brightness of the dataset, adding noise, and rotating the data.

[0023] As a preferred technical solution, step S 24 The ratio is set to 4:1.

[0024] As a preferred technical solution, step S3 specifically includes:

[0025] Step S 31 Based on the color information, the required categories, including the switch motor contact cylinder, the stationary contact spring sheet, and the base gap, are extracted from the segmentation results.

[0026] Step S 32 Calculate the positive bounding rectangle of different instance categories to obtain the width, height and center coordinate information of different categories;

[0027] Step S 33 Based on the actual length of the diameter of the moving contact cylinder and the width and height information of the corresponding segmentation mask, the actual length represented by each pixel in the image is calculated.

[0028] Step S 34 Based on the width, height, and center coordinate information of different categories, the contact depth of the moving and stationary contacts of the automatic switch machine opener, the gap between the moving and stationary contacts, and the distance between the center of the moving contact column and the center line of the stationary contact spring are calculated. Then, the actual distance is obtained by multiplying the actual length represented by each pixel.

[0029] As a preferred technical solution, in step S4, the segmentation results, the contact depth of the moving and stationary contacts of the automatic opener / closer, the gap between the moving contact and the stationary contact seat, the distance between the center of the moving contact column and the center line of the stationary contact spring, and the working status of the switch machine are displayed on the APP interface.

[0030] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0031] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described thereon.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] 1) This invention applies image semantic segmentation methods from the field of machine vision to intelligent evaluation of switch machines. The semantic segmentation method based on deep learning has strong anti-interference ability and stable segmentation effect.

[0034] 2) This invention can intelligently calculate the contact depth of the moving and stationary contacts of the switch, the gap between the moving and stationary contacts, and the distance between the center of the moving contact column and the center line of the stationary contact spring based on the semantic segmentation results. This is a pixel-level distance calculation with an accuracy of three decimal places, which is more accurate than traditional manual measurement methods.

[0035] 4) This invention can automatically save the segmentation results for easy viewing later;

[0036] 5) This invention uses a client APP to call the camera to take pictures. Based on the collected image information, the working status of the switch machine can be evaluated. There is no need to install additional hardware on the switch machine equipment, and the operation is simple.

[0037] 6) The client APP of this invention transmits the image to be detected to the server through the local area network to evaluate the status of the switch machine. No additional communication cable needs to be laid, which is convenient to operate and reduces costs. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention;

[0039] Figure 2 Select an image interface for the client app;

[0040] Figure 3 This is the client app's waiting screen for receiving data.

[0041] Figure 4 This is the interface for communication errors in the client-side app.

[0042] Figure 5 This indicates that the client app has successfully received the data and the switch is functioning normally.

[0043] Figure 6 The client app successfully received the data, but the switch machine malfunctioned.

[0044] Figure 7 Label the training set samples;

[0045] Figure 8 This is a diagram of a semantic segmentation network framework.

[0046] Figure 9 To extract the result images of different segmentation categories;

[0047] Figure 10 To calculate the center coordinates and circumscribed rectangles for different segmentation categories. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0049] refer to Figure 1-10 To illustrate the technical solution of the present invention, firstly, refer to... Figure 1 According to the overall solution flowchart, the specific steps are as follows:

[0050] Step S1: The client APP captures images of the switch machine's working status;

[0051] Step S2: The client APP sends the captured image to the server via the local area network as the image to be detected;

[0052] Step S3: The server calls the trained semantic segmentation model to segment the image to be detected;

[0053] Step S4: Based on the color information, extract the required categories from the segmentation results, including the switch motor contact cylinder, the stationary contact spring sheet, and the base gap.

[0054] Step S5: Calculate the positive bounding rectangle of different instance categories to obtain the width, height and center coordinate information of different categories;

[0055] Step S6: Based on the actual length of the diameter of the moving contact cylinder and the width and height information of the corresponding segmentation mask, the actual length represented by each pixel in the image can be calculated.

[0056] Step S7: Based on the width, height and center coordinate information of different categories, calculate the contact depth of the moving and stationary contacts of the automatic switch machine, the gap between the moving and stationary contacts, and the distance between the center of the moving contact column and the center line of the stationary contact spring, and calculate how many pixels each contains. Then multiply by the actual length represented by each pixel to obtain the corresponding actual distance.

[0057] Step S8: The server sends the segmentation result and contact distance to the client APP. The client APP receives and saves the data, and at the same time judges whether the switch machine is working properly based on the contact distance. Finally, the segmentation result, the contact depth of the moving and stationary contacts of the automatic opener, the gap between the moving contact and the stationary contact seat, the distance between the center of the moving contact column and the center line of the stationary contact spring, and the working status of the switch machine are displayed on the APP interface.

[0058] The specific steps of S1 are as follows:

[0059] Step S 11Design an app that includes functions such as camera access for taking pictures, image selection, interface display, and communication with a server.

[0060] The specific steps of S3 are as follows:

[0061] Step S 31 Take a sufficient number of switch machine images to create a training dataset;

[0062] Step S 32 To improve the robustness of the training model against interference, data augmentation is performed on the dataset, including operations such as changing the brightness, adding noise, and rotating the dataset.

[0063] Step S 33 LabelMe is used to label the dataset and generate corresponding JSON files. The JSON files store the category names and edge coordinates of each category.

[0064] Step S 34 The dataset and its corresponding labels are divided into training and validation sets in a 4:1 ratio and stored in a specified folder;

[0065] Step S 35 Build a semantic segmentation network model, set the network parameters, and start training the model;

[0066] Step S 36 After obtaining the trained model and testing its performance on the validation set, the image to be detected is segmented.

[0067] Based on this, refer to Figure 2-10 The software operation and data processing flow of the present invention are described in detail.

[0068] First refer to Figure 2-6 This invention introduces the client-side APP interface and processing flow designed in this invention, including the following steps:

[0069] Step 100: In the IP address field in the upper left corner, enter the server IP address to establish a connection with the server;

[0070] Step 101: Click the "Take Photo" button to capture an image of the switch machine's operating status, which will be displayed in the left image bar as the image to be inspected. Alternatively, click the "Select Image" button to select the image to be inspected from the folder; the status bar in the upper left corner displays the software's running status in real time.

[0071] Step 102: Click the "Upload Image" button to send the image to be tested to the server. At this time, the button is unclickable to prevent accidental operation, and the status bar will show "Receiving". If the waiting time exceeds 20 seconds, it is considered a communication error, and the waiting state will automatically end, with the status bar showing "Error 1, please check the network and entered data". After checking for errors, you can try uploading again.

[0072] Step 103: Receive and save data. The received data includes semantic segmentation results, the contact depth of the moving and stationary contacts of the switch machine, the bottom clearance of the switch machine, and the distance between the center of the moving contact post and the center line of the stationary contact spring. If the contact depth of the moving and stationary contacts of the switch machine is not less than 4mm and the bottom clearance of the switch machine is greater than 3mm, the switch machine is considered to be working normally; otherwise, it is considered abnormal.

[0073] Step 104: Results are displayed. The left side of the APP interface shows the segmentation results, and the right side shows the working status of the switch machine, the contact depth of the moving and stationary contacts of the switch, the gap between the moving and stationary contacts, and the distance between the center of the moving contact column and the center line of the stationary contact spring.

[0074] First refer to Figure 7-8 This paper introduces the training and usage process of the semantic segmentation model of the present invention.

[0075] Includes the following steps:

[0076] Step 105: Training set data augmentation. First, the training images are rotated by 90 degrees, 180 degrees, and 270 degrees respectively. Then, the brightness and saturation of the images are adjusted, and the thresholds are randomly adjusted to 0.3, 0.5, and 0.8. Finally, noise is randomly added with a probability of 0.01, resulting in a dataset of 3000 images. Through data augmentation, the generalization ability of the model is improved.

[0077] Step 106: Labelme is used to annotate the training set images. Dots are used to mark the edge contours of the categories. According to the calculation requirements, three categories are marked: moving contact cylinder, stationary contact spring sheet, and moving and stationary contact base gap.

[0078] Step 107: Build the Deeplabv3-plus semantic segmentation network, setting network parameters including num_classes to 4, input image size to 512*512, batch_size to 16, learning rate to 5e-5, and training epochs to 300. Start training the model. To accelerate network convergence, freeze the backbone network for the first 150 training iterations, at which point the loss converges to 0.1. After that, unfreeze the backbone network, and the final loss converges to 0.06.

[0079] Step 108: The trained model with the suffix "pth" is obtained. The performance is tested using the validation set images. The average segmentation accuracy (AP) on the test set reaches 94.6%.

[0080] Step 109: The server opens a listener and receives the image to be detected sent by the client;

[0081] Step 110: Call the trained semantic segmentation model to segment the image to be detected. The model can predict which category each pixel belongs to and obtain a segmentation mask image. Different categories are represented by different colors.

[0082] First refer to Figure 9-10 This paper introduces the pixel-level distance calculation strategy and method of the present invention.

[0083] Includes the following steps:

[0084] Step 111: Convert the segmented mask image from RGB space to HSV space. Based on the color information of different categories, extract the pixels belonging to the turning motor contact cylinder, stationary contact spring sheet and base gap respectively to facilitate distance calculation.

[0085] Step 112: By calculating the bounding rectangles of different categories, we can obtain the number of pixels contained in the height and width of each category, as well as the center coordinates of each category.

[0086] Step 113: The actual diameter of the moving contact cylinder is 4mm. Based on the height of the circumscribed rectangle of the moving contact, the actual distance of each pixel can be calculated.

[0087] Step 114: Based on the Y-axis distance from the upper boundary of the stationary contact spring sheet to the center of the stationary contact, multiplied by the actual distance of each pixel, the contact depth of the moving and stationary contacts of the automatic opener / closer can be calculated.

[0088] Step 115: Based on the X-axis distance from the center of the stationary contact to the stationary contact spring sheet, multiply by the actual distance of each pixel point to calculate the distance between the center of the moving contact post and the center line of the stationary contact spring sheet.

[0089] Step 116: Based on the height information of the circumscribed rectangle of the base gap, multiply by the actual distance of each pixel to calculate the base gap between the moving contact and the stationary contact.

[0090] Step 117: Determine the working status of the switch machine based on the calculated contact distance. If the contact depth of the switch machine's moving and stationary contacts is less than 0, it is diagnosed as the switch machine's moving and stationary contacts not making contact. If the contact depth of the switch machine's moving and stationary contacts is not less than 4mm, and the gap between the switch machine's base and the contact depth is greater than 3mm, the diagnosis is that the switch machine's contact is normal. Otherwise, the switch machine's contact is abnormal.

[0091] The above is an introduction to the method embodiments. The following embodiments using electronic devices and storage media will further illustrate the solution of the present invention.

[0092] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0093] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0094] The processing unit performs the various methods and processes described above, such as the methods of the present invention. For example, in some embodiments, the methods of the present invention may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the methods of the present invention described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute the methods of the present invention by any other suitable means (e.g., by means of firmware).

[0095] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0096] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0098] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for the maintenance and evaluation of intelligent switch machines based on semantic segmentation, characterized in that, The method includes the following steps: Step S1: The client collects images of the switch machine's working status and sends them to the server as images to be detected; Step S2: The server uses the trained semantic segmentation model to segment the image to be detected; Step S3: The server calculates the contact depth of the moving and stationary contacts of the automatic switch machine based on the semantic segmentation results, the gap between the moving and stationary contacts, and the distance between the center of the moving contact post and the center line of the stationary contact spring. Step S4: The server sends the segmentation results, the contact depth of the moving and stationary contacts of the automatic switch, the gap between the moving contact and the stationary contact seat, and the distance between the center of the moving contact column and the center line of the stationary contact spring to the client. The client determines: if the contact depth of the moving and stationary contacts of the automatic switch is not less than 4mm and the gap between the moving contact and the stationary contact seat is greater than 3mm, then the switch machine is working normally; otherwise, it is abnormal. The specific steps of step S3 are as follows: Step S 31 Based on the color information, the required categories, including the switch motor contact cylinder, the stationary contact spring sheet, and the base gap, are extracted from the segmentation results. Step S 32 Calculate the positive bounding rectangles for different instance categories to obtain the width, height, and center coordinates of the bounding rectangles for different categories; Step S 33 Based on the actual length of the diameter of the moving contact cylinder and the width and height information of the corresponding segmentation mask, the actual length represented by each pixel in the image is calculated. Step S 34 The contact depth of the moving and stationary contacts of the automatic opener is calculated by multiplying the Y-axis distance from the upper boundary of the stationary contact spring sheet to the center of the stationary contact by the actual length of each pixel; the distance between the center of the moving contact post and the center line of the stationary contact spring sheet is calculated by multiplying the X-axis distance from the center of the stationary contact spring sheet to the center of the stationary contact by the actual length of each pixel; and the gap between the moving contact and the stationary contact seat is calculated by multiplying the height of the outer rectangle of the base gap by the actual length of each pixel.

2. The intelligent switch machine maintenance and evaluation method based on semantic segmentation according to claim 1, characterized in that, The specific steps of S1 are as follows: Step S 11 The client-side APP captures images of the switch machine's working status; Step S 12 The client-side app sends the captured images to the server via the local area network as images to be detected.

3. The intelligent switch machine maintenance and evaluation method based on semantic segmentation according to claim 2, characterized in that, The client APP has the functions of calling the camera to take pictures, selecting images, displaying the interface, and communicating with the server.

4. The intelligent switch machine maintenance and evaluation method based on semantic segmentation according to claim 1, characterized in that, The specific steps of S2 are as follows: Step S 21 Take a set number of images of switch machines to create a training dataset; Step S 22 To augment the dataset; Step S 23 LabelMe is used to label the dataset and generate corresponding JSON files, which store the category names and edge coordinates of each category. Step S 24 The dataset and its corresponding labels are divided into training and validation sets according to a set ratio and stored in a specified folder; Step S 25 Build a semantic segmentation network model, set the network parameters, and start training the model; Step S 26 After obtaining the trained model and testing its performance on the validation set, the image to be detected is segmented.

5. The intelligent switch machine maintenance and evaluation method based on semantic segmentation according to claim 4, characterized in that, Step S 22 Data augmentation in this context includes adjusting the brightness of the dataset, adding noise, and rotating the data.

6. The intelligent switch machine maintenance and evaluation method based on semantic segmentation according to claim 4, characterized in that, The step S 24 The ratio is set to 4:

1.

7. The intelligent switch machine maintenance and evaluation method based on semantic segmentation according to claim 1, characterized in that, In step S4, the segmentation results, the contact depth of the moving and stationary contacts of the automatic opener / closer, the gap between the moving contact and the stationary contact seat, the distance between the center of the moving contact column and the center line of the stationary contact spring, and the working status of the switch machine are displayed on the APP interface.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images