Railway kilometer post identification method based on smart phone

Abstract

The invention discloses a railway kilometer post identification method based on a smart phone, and relates to the technical field of image processing and railway kilometer post identification. The method comprises the following steps: acquiring an image containing a railway kilometer post by using a smart phone camera, and preprocessing the acquired image; identifying the digital information of the railway kilometer post through a convolutional neural network; calibrating the relative distance between the smart phone and the railway kilometer post based on the internal reference of the camera and the coordinates of the feature points; the corresponding relation between the railway kilometer post and the geographic position is established by combining latitude and longitude coordinates collected by the GPS module; and finally, the recognition result is presented on the smart phone APP in an image-text form, and storage and export are supported. According to the method, automatic acquisition, identification, positioning and storage of railway kilometer post information are realized, the method has the characteristics of flexible deployment and strong adaptability, smart phones of different models can execute data acquisition, and exported detection results can be checked on other equipment or platforms.

Description

Technical Field

[0001] The invention relates to the field of image processing and railway kilometer markings, and in particular to a railway kilometer marking recognition method based on a smart phone. Background Art

[0002] With the rapid development of railway transportation, railway kilometer marks, as important signs of railway lines, are of great significance for the positioning, maintenance and management of railway lines. The traditional way of identifying railway kilometer marks mainly relies on manual visual inspection and labeling, which is not only labor-intensive, but also prone to errors and low efficiency. With the advancement of information technology, the automatic identification technology of railway kilometer marks based on smartphones has gradually become a trend. By utilizing the powerful computing and communication capabilities of smartphones, through image recognition and positioning technology, the automatic identification, positioning and data storage of railway kilometer marks can be realized, thereby providing efficient and accurate support for railway line management and maintenance.

[0003] However, there are still some problems with the existing technology: on the one hand, the recognition accuracy of railway kilometer markers is greatly affected by factors such as ambient lighting, angle, and stains, and traditional image processing methods are difficult to ensure recognition results in complex environments; on the other hand, the existing technology mostly uses a single data processing method and lacks efficient multi-dimensional data integration and positioning accuracy improvement solutions. Therefore, how to improve the recognition accuracy of railway kilometer markers and ensure the efficient application of equipment under different conditions has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above problems, the present invention proposes a railway kilometer mark recognition method based on a smartphone, which combines image processing, deep learning and positioning technology, collects images through a smartphone camera, recognizes deep learning models, calibrates distances and GPS positioning, and automatically recognizes and locates railway kilometer marks, thereby realizing accurate kilometer mark information collection and storage. This method not only improves the recognition accuracy of railway kilometer marks, but also can work stably in a variety of environments, providing strong support for the digital management and maintenance of railway lines.

[0005] The technical solution adopted by the present invention is: a method for identifying railway kilometer markers based on a smart phone, the method comprising: using a smart phone camera to capture images and perform image preprocessing, identifying digital information in the railway kilometer markers, calibrating the distance between the smart phone and the railway kilometer markers, using the smart phone GPS to collect longitude and latitude coordinates and matching them with the railway kilometer marker information, and storing and displaying the recognition results.

[0006] The specific implementation steps of the method are: a. Collect images along the railway using a smartphone camera to obtain image data containing railway kilometer mark information; and pre-process the collected images; b. Input the preprocessed image into the convolutional neural network and use the neural network model to recognize the digital information in the railway kilometer mark; c. Based on the identified railway kilometer mark information, the camera internal parameter matrix and feature point coordinates are used to calibrate the distance between the smartphone and the railway kilometer mark through the feature matching algorithm and the PnP algorithm; d. Use the GPS module of the smart phone to collect the longitude and latitude coordinates, and match the longitude and latitude coordinates with the identified railway kilometer mark information to form a corresponding relationship; e. Through the smartphone APP, the railway kilometer mark recognition results are displayed in graphic and text form. At the same time, the recognition results and corresponding relationships are stored in the device, and data export is supported.

[0007] The beneficial results of the present invention are as follows: the present invention provides a method for automatic identification, positioning and storage of railway kilometer marks based on a smartphone, which combines deep learning technology and positioning function to effectively solve the problems of low recognition efficiency, poor positioning accuracy and cumbersome data recording in traditional methods. First, the kilometer mark images along the railway are collected by a smartphone, and the kilometer mark information in the image is automatically detected and identified using a pre-trained deep learning model, which can achieve high-precision recognition results under complex backgrounds and various lighting conditions; then, with the help of the GPS positioning function of the smartphone, the longitude and latitude coordinates of the corresponding kilometer mark are obtained in real time to ensure the accuracy of positioning; then, the recognition results and the longitude and latitude data are associated and stored to generate digital data that can be directly used for railway management. Compared with the prior art, the present invention has significant advantages such as simple operation, low equipment cost, high recognition and positioning accuracy, and automatic data storage. Its innovation lies in the realization of the full process intelligence of railway kilometer mark information from collection to storage, which significantly improves the efficiency and reliability of data collection, and provides strong support for railway line management, maintenance and digital system construction. At the same time, the present invention also has good scalability, can adapt to different line environments and application requirements, and lays a solid foundation for the information upgrade of the railway industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of a method for identifying railway kilometer marks according to the present invention; Figure 2 This is a schematic diagram of image preprocessing of the present invention; Figure 3 This is a schematic diagram of the Resnt residual network of the present invention; Figure 4 This is a flow chart of railway kilometer mark identification of the present invention; Figure 5 This is a flow chart of the distance calibration between a smart phone and a railway kilometer mark of the present invention; Figure 6 This is the GPS kilometer mark matching flow chart of the present invention; Figure 7 This is a flow chart of mobile phone APP login in the present invention; Figure 8 This is a flow chart of the mobile phone APP operation page of the present invention. DETAILED DESCRIPTION

[0009] The present invention will be further described below with reference to the accompanying drawings and embodiments: like Figure 1 As shown, a method for identifying railway kilometer marks based on a smartphone includes collecting images with a smartphone camera, image preprocessing, identifying railway kilometer marks, calibrating the distance between the smartphone and the railway kilometer mark, collecting longitude and latitude coordinates with the smartphone GPS to form a corresponding relationship with the railway kilometer mark and presenting and storing it, and the final result is displayed through a smartphone APP, thereby realizing automatic identification, positioning and storage of railway kilometer mark information. The present invention makes the deployment method more flexible, and different mobile phones can be used for collecting railway kilometer mark information, and the detection results can be exported and viewed on different devices and platforms.

[0010] The method specifically comprises the following steps: a. Use smartphone cameras to collect images along the railway and obtain image data containing railway kilometer mark information; then pre-process the collected images, including image scaling, grayscale processing, binarization processing and morphological operations, to improve image quality and optimize subsequent recognition calculations; b. Input the preprocessed image into the convolutional neural network, recognize the digital information on the railway kilometer mark through the automatic feature extraction mechanism (such as texture, edge, shape, etc.), and use the deep learning algorithm to improve the accuracy and robustness of recognition; c. After completing the digital information recognition, the distance between the smartphone and the railway kilometer mark is calibrated by combining the OpenCV feature matching algorithm and the PnP algorithm. The rotation vector and translation vector are calculated through the camera internal parameter matrix and the spatial three-dimensional coordinates of the target feature points to determine the spatial position relationship between the smartphone and the target; d. Based on the calibrated distance data between the smartphone and the railway kilometer marker, as well as the mobile speed information built into the device, the positioning accuracy of the kilometer marker position is further improved. At the same time, the GPS module of the smartphone is used to collect the longitude and latitude coordinates, and the location data is correspondingly related to the identified railway kilometer marker information; e. The recognition results (including railway kilometer mark numbers, calibrated distances, corresponding latitude and longitude coordinates, etc.) are presented in graphic form through a dedicated smartphone application. Users can export recognition data and store it locally on the device or in the cloud in a predefined format for subsequent viewing and analysis.

[0011] This method can improve the recognition accuracy of railway kilometer markers and efficiently associate and store railway kilometer markers with their geographic coordinates, providing reliable data support for railway management and maintenance.

[0012] The present invention uses a smartphone APP as an application carrier, and this APP is developed based on the Android or iOS platform. In terms of development technology selection, the Android platform uses Kotlin as the development language and integrates PyTorch Mobile as the machine learning framework to make full use of the native characteristics of the Android system; the iOS platform uses Swift language development and combines Apple's own CoreML machine learning framework to obtain the best system performance and user experience.

[0013] like Figure 2 As shown, in the present invention, the images along the railway collected by the smartphone camera undergo the following preprocessing process to prepare for the input of the subsequent neural network: a1. Image size adjustment: The collected original images are first resized to a standard format of 600 × 600. This step unifies the image size through scaling operations, facilitates subsequent processing, and reduces the complexity of the algorithm.

[0014] a2. Convert color image to grayscale image: Convert color images to grayscale images. By retaining only the brightness information of the image, the data dimension can be effectively reduced while retaining key target information, thereby simplifying the computational complexity.

[0015] a3. Threshold processing of grayscale image: Threshold segmentation is performed on the grayscale image to divide the pixel values ​​in the image into two categories: foreground and background. The processed image is converted into a black and white binary image, the target area is clearer, and the background noise is effectively removed.

[0016] a4. Expansion operation: A dilation operation is applied to the binary image to enhance the connectivity of the target area, fill small holes caused by noise or acquisition conditions, make the target area more complete, and facilitate subsequent area screening.

[0017] a5. Elimination of small connected domains: All connected domains in the image are analyzed and screened according to their area size. The connected domains with smaller areas are eliminated, and the important target areas with larger areas related to railway kilometer marks are retained to improve the accuracy of the processing results.

[0018] a6. Extraction of Region of Interest (ROI): Regions of interest (ROIs) are extracted from the processed images. These regions contain digital information of railway kilometer markers and are key input data for further analysis.

[0019] a7. ROI area size scaling: The extracted ROI area is scaled to the standard input size of 28 × 28 required by the neural network. This operation ensures that the format of the input data is consistent with the network structure, laying the foundation for subsequent feature extraction and recognition.

[0020] The convolutional neural network in the present invention is combined with the residual network ResNet residual network structure, and its core design includes direct transmission of input information and nonlinear transformation process to improve the training effect of deep network. Specifically, Figure 3 As shown, the input is x, which first enters the weighted operation unit, which applies the weight matrix and bias to the input x for linear transformation to generate an intermediate result. Then, the intermediate result is introduced into the nonlinear characteristics through the activation function, thereby enhancing the expression ability of the network; then, the result output by the activation function passes through another weighted operation unit to generate a new feature result f(x). In order to realize the residual connection, the input x is directly passed to the addition operation node and added to f(x)-x to form a residual output. Through this design, the network can directly use the input information to avoid the problem of information attenuation or gradient disappearance caused by the deepening of the network layer; finally, the output of the residual connection is further processed by the activation function unit to generate the final output of the network. The entire network structure significantly improves the training efficiency and model performance while ensuring the depth of the model through the residual connection. The convolutional neural network combined with the residual network of the present invention can more accurately identify the digital information on the railway kilometer mark, even if it is blurred due to light, angle or stains, it can have a high recognition accuracy.

[0021] The complete convolutional neural network structure adopted by the present invention is as follows: Figure 4 As shown in the figure, the specific process for railway kilometer mark identification is as follows: b1. Input data: The image generated after the preprocessing process is used as the input of the network, which contains the visual information of the railway kilometer landmarks.

[0022] b2. Feature extraction stage: Conv (convolutional layer): The input image first passes through a convolutional layer, using a set of convolution kernels to extract local spatial features. The convolution operation can capture low-level features such as edges and textures to generate feature maps; Norm (normalization layer): The extracted feature map passes through the normalization layer (such as Batch Normalization) to normalize the data, stabilize the network training process, and accelerate convergence; ReLU (activation function): The normalized features are activated by the ReLU function to increase the nonlinear expression ability of the model, enabling the network to learn complex mapping relationships.

[0023] b3. Feature dimensionality reduction stage: MaxPool (Maximum Pooling Layer): The activated feature map passes through the maximum pooling layer to further reduce the spatial resolution of the feature map while retaining the main features. Maximum pooling reduces the amount of data and prevents overfitting by selecting the maximum value of the local area.

[0024] b4. Deep feature extraction stage: ResNet_block (three residual blocks): The pooled feature map passes through three residual blocks in sequence. Each residual block contains multiple layers of weighted operations and activation functions, and passes the input information directly to the output through residual connections to alleviate the gradient vanishing problem. Each residual block extracts higher-level features, such as structural patterns and regional characteristics, to provide richer semantic information for subsequent classification tasks.

[0025] b5. Feature dimensionality reduction and conversion stage: GlobalAvgPool (global average pooling layer): The deep feature map is processed by the global average pooling layer to reduce the dimension, averaging the values ​​of each feature channel to generate a compact global feature representation. This process reduces the number of parameters while retaining global information; FlattenLayer: Global features are flattened into a one-dimensional vector as input to the subsequent fully connected layer to facilitate classification or prediction tasks.

[0026] b6. Output stage: Predicted probability values: The final output of the network is a set of predicted probability values, which represent the probability distribution of each possible number in the input image. Based on the probability values, the model can accurately recognize the numbers on the railway kilometer mark.

[0027] like Figure 5 As shown, the specific process of calibrating the distance between a smartphone and a railway kilometer mark in the present invention is as follows: c1. First, the system obtains the camera's intrinsic parameter matrix from the smartphone camera, which is a matrix describing the camera's internal parameters, including focal length, principal point coordinates, and distortion parameters. These intrinsic parameter data provide precise geometric references for subsequent processing; c2. Next, the image collected from the camera is gray-scaled to convert the color image into a gray-scale image, reducing data complexity and retaining key information of the image; c3. Then, the SIFT (Scale-Invariant Feature Transform) feature point detection algorithm in the OpenCV library is used to identify key feature points from the grayscale image. The SIFT algorithm extracts invariant feature points in the image, making it stable under scaling, rotation and lighting changes; c4. Based on the pixel coordinates of the detected feature points and the camera's intrinsic parameter matrix, the system uses triangulation to calculate the three-dimensional coordinates of the feature points corresponding to the object points. Triangulation is based on geometric principles and combines information from multiple perspectives to deduce the position of the object in three-dimensional space; c5. Then, according to the 3D coordinates and pixel coordinates of the object point derived from the intrinsic parameter matrix, the solvePnP (Perspective-n-Point) function in the OpenCV library is called to solve the rotation vector and translation vector: the rotation vector represents the rotation relationship of the camera coordinate system relative to the object coordinate system; the translation vector represents the displacement relationship of the camera coordinate system relative to the object coordinate system; c6. Finally, the transformation matrix between the camera and the object is calculated by the rotation vector and the translation vector. This matrix can be used to accurately derive the distance and spatial relationship between the object (such as a railway kilometer mark) and the camera, providing data support for subsequent recognition and positioning.

[0028] like Figure 6 As shown in the figure, the specific process of matching the longitude and latitude coordinates collected by the smartphone GPS with the railway kilometer mark is as follows: d1: Get GPS location information The smartphone starts and receives satellite signals through the built-in GPS module, obtains the latitude and longitude coordinate data of the current location in real time, including latitude (Latitude) and longitude (Longitude), and continuously updates the location data at fixed time intervals to ensure the continuity and real-time nature of the trajectory.

[0029] d2: Calculate kilometer mark position information Based on the current position obtained by GPS and the previously calculated distance between the camera and the kilometer marker, the accurate position information of the current railway kilometer marker is calculated through the geometric positioning algorithm, and the position deviation is corrected to ensure the accuracy of the kilometer marker positioning.

[0030] d3: Matching kilometer markers with location information The calculated kilometer marker location information is matched with the kilometer marker digital information obtained by image recognition, and the synchronization of the matching data is verified through a timestamp mechanism to ensure the consistency of the location and kilometer marker number.

[0031] d4: Data binding and storage The matched kilometer mark digital information is associated and bound with the currently calculated longitude and latitude coordinates to generate complete corresponding relationship data and store it in the database. Finally, the matching results can be displayed in real time through the smartphone APP to provide users with accurate kilometer mark location information.

[0032] The login process of the smartphone APP is as follows Figure 7 As shown, it mainly includes the following steps: First, the user enters the account and password as the basic credentials for identity verification on the login interface. After the input is completed, the system will initiate a verification request through the network and send the user's account and password to the server for identity verification; When the server receives the verification request, the system will compare the input account and password with the user data stored in the background. If the verification is successful, the server returns a response that the verification is passed, and the user is immediately granted access rights, successfully logs in, and enters the main interface of the smartphone APP; If the verification fails, it means that the user entered an incorrect account or password. The system will return an error message and prompt the user with a specific question on the login interface, such as "The account or password is incorrect, please re-enter." At this time, the user needs to re-enter the correct account and password and initiate the verification request again; The login process ensures that the access rights of the APP are limited to legitimate users through account and password verification, thus enhancing the security of the system. At the same time, the error message provided by the login interface can effectively guide users to complete the login operation, thereby improving the user experience.

[0033] The operation process of the main interface of the smartphone APP is as follows Figure 8 As shown, the specific steps are as follows: e1. First, after entering the main interface, the user needs to select the input source to specify the source of the image data to be detected. The input source includes multiple options: image file, image folder, camera real-time acquisition, or video file. The user can select the appropriate input source type according to the needs.

[0034] e2. Next, the user needs to select the model file for railway kilometer mark detection. This model file is trained based on a convolutional neural network and is used to recognize digital information on railway kilometer marks. Selecting the correct model file is a key step to ensure detection accuracy.

[0035] e3. After completing the selection of the input source and model file, the user clicks the "Start" button on the main interface. At this time, the system will load the specified model file and detect the data in the input source. During the detection process, the system will automatically perform image preprocessing, feature extraction, recognition and other operations to achieve accurate detection of railway kilometer markers.

[0036] e4. Finally, after the test is completed, the system will present the test results on the APP interface for users to view in real time. In addition, users can also choose to export the test results as files for further analysis or backup. This process provides users with a convenient operating experience and efficient testing capabilities.

[0037] The model file used to identify railway kilometer marks in the present invention is generated by the following steps: first, 8,000 railway kilometer mark images are collected under different lighting conditions and weather environments to ensure data diversity; then, the collected images are accurately annotated using a manual annotation method to generate a high-quality training data set. Next, the annotated image data set is imported into the deep learning model training environment, which runs on the Ubuntu 22.04.3 LTS operating system and is configured with CUDA 12.3 version and a Tesla T4 GPU with 16G video memory to ensure sufficient computing power and compatibility support. During the model training process, the iterative optimization method is used to tune the parameters, and the optimal model weights are obtained through multiple rounds of training, and finally a model file dedicated to railway kilometer mark detection is generated. The recognition performance of this model under complex lighting and weather conditions has been fully verified, and it can meet the actual needs of automatic recognition of railway kilometer marks.

Claims

1. A railway kilometer mark recognition method based on a smart phone, characterized in that: The method includes: using a smartphone camera to capture images and perform image preprocessing, identifying digital information in railway kilometer markers, calibrating the distance between the smartphone and the railway kilometer markers, using the smartphone GPS to capture longitude and latitude coordinates corresponding to the railway kilometer marker information, and presenting and storing the recognition results.

2. The method for identifying railway kilometer marks based on a smart phone according to claim 1, characterized in that: The specific implementation steps of the method are: a. Collect images along the railway using a smartphone camera to obtain image data containing railway kilometer mark information; and pre-process the collected images; b. Input the preprocessed image into the convolutional neural network and use the neural network model to recognize the digital information in the railway kilometer mark; c. Based on the identified railway kilometer mark information, the camera internal parameter matrix and feature point coordinates are used to calibrate the distance between the smartphone and the railway kilometer mark through the feature matching algorithm and the PnP algorithm; d. Use the GPS module of the smart phone to collect the longitude and latitude coordinates, and match the longitude and latitude coordinates with the identified railway kilometer mark information to form a corresponding relationship; e. Through the smartphone APP, the railway kilometer mark recognition results are displayed in graphic and text form. At the same time, the recognition results and corresponding relationships are stored in the device, and data export is supported.

3. The method for identifying railway kilometer marks based on a smart phone according to claim 2, characterized in that: The specific steps of the image preprocessing include: a1. Scale the collected images to the specified size to unify the image format; a2. Grayscale the scaled image to reduce data complexity; a3. Convert the grayscale image into a binary image through threshold segmentation to highlight the target area; a4. Apply dilation operation to enhance the connectivity of the target area; a5. Eliminate the connected domains with smaller areas to retain the main target areas related to the railway kilometer mark; a6. Extract the area of ​​interest containing the railway kilometer mark; a7. Scale the region of interest to the standard image input size accepted by the neural network.

4. The method for identifying railway kilometer marks based on a smart phone according to claim 2, characterized in that: The method comprises the following steps: using a neural network model to identify digital information in railway kilometer marks: b1. Input the image generated by image preprocessing into the convolutional neural network; b2. In the feature extraction stage, the image passes through the following layers in sequence: convolution layer, which extracts local features of the image; normalization layer, which stabilizes the training and accelerates network convergence; activation function, which enhances the nonlinear expression ability of the network; b3. In the feature dimension reduction stage, the maximum pooling layer is used to reduce the spatial resolution of the feature map and retain the main features; b4. In the deep feature extraction stage, deep features are extracted through multiple residual blocks in sequence, where the residual blocks alleviate the gradient vanishing problem and enhance network performance through residual connections; b5. In the feature conversion stage, the global average pooling layer is used to reduce the dimensionality of the deep feature map and flatten it into a one-dimensional vector; b6. Finally, the predicted probability value is calculated through the fully connected layer to generate the recognition result of the railway kilometer mark digital information in the input image.

5. The method for identifying railway kilometer marks based on a smart phone according to claim 2, characterized in that: The method includes the following steps of calibrating the distance between the smart phone and the railway kilometer mark: c1. Obtain the camera's intrinsic parameter matrix from the smartphone camera, where the intrinsic parameter matrix includes focal length, principal point coordinates, and distortion parameters; c2. grayscale the images along the railway collected by the smartphone camera to obtain a grayscale image; c3, using SIFT feature point detection algorithm to extract feature points from the grayscale image, wherein the feature points are invariant under scaling, rotation and illumination changes; c4. Based on the pixel coordinates of the extracted feature points and the internal parameter matrix, the three-dimensional coordinates of the object points corresponding to the feature points are calculated using a triangulation method; c5. According to the 3D coordinates of the object point and the pixel coordinates of the feature point, the solvePnP function is called to solve the rotation vector and translation vector, where the rotation vector is used to represent the rotation relationship of the camera coordinate system relative to the object coordinate system; the translation vector is used to represent the displacement relationship of the camera coordinate system relative to the object coordinate system; c6. Calculate a transformation matrix based on the rotation vector and the translation vector, and use the transformation matrix to derive the distance and spatial position relationship between the railway kilometer mark and the smart phone.

6. The method for identifying railway kilometer marks based on a smart phone according to claim 2, characterized in that: The step of matching the latitude and longitude coordinates with the identified railway kilometer mark information in the method includes: d1. Obtain the latitude and longitude coordinate information of the current location through the built-in GPS module of the smartphone; d2. Calculate the specific location information of the railway kilometer marker based on the acquired GPS location information and the previously calculated distance between the smartphone camera and the kilometer marker; d3, matching the calculated railway kilometer mark position information with the railway kilometer mark digital information obtained by the image recognition module; d4. Associate and bind the matched railway kilometer mark digital information with the longitude and latitude coordinate data calculated in step d1 to generate corresponding relationship data between the railway kilometer mark and the longitude and latitude.

7. The method for identifying railway kilometer marks based on a smart phone according to claim 2, characterized in that: The smartphone APP operation process includes: e1. The user selects an input source through the main interface and specifies the source of the image data to be detected. The input source includes a picture file, a picture folder, a real-time camera acquisition, or a video file; e2. The user selects a model file for railway kilometer mark detection. The model file is a file trained based on a convolutional neural network and is used to identify digital information on railway kilometer marks. e3. After completing the selection of the input source and model file, the user clicks the "Start" button, the system loads the model file, and performs detection processing on the data in the input source, including image preprocessing, feature extraction, and recognition of railway kilometer mark digital information; e4. After the test is completed, the system presents the test results in the APP interface, and the user can choose to export the test results as a file for analysis or saving.

Images