Rail surface condition determination method, apparatus, device, and computer-readable storage medium
By acquiring target parameters from track images and utilizing a prediction model, the problem of low accuracy in track surface condition recognition was solved, enabling accurate recognition under complex conditions and improving the accuracy and safety of track surface condition detection.
Patent Information
- Application Number
- PCT/CN2025/093490
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2026-11-12
AI Technical Summary
In existing technologies, track surface condition recognition is affected by lighting, viewing angle, occlusion, and sharpness, resulting in low recognition accuracy and the inability to detect slippery conditions in a timely manner, which affects train safety and operational efficiency.
By acquiring target parameters such as acquisition angle, sharpness, occlusion area, and acquisition brightness of track images, a prediction model is used to identify the track surface state. The prediction model is trained on track surface images under different angles, sharpness, occlusion area, and brightness, and can accurately identify the track surface state under low sharpness, large occlusion area, and incorrect angle.
It improves the accuracy of rail surface condition determination, enabling accurate identification of rail surface condition under complex conditions, reducing safety hazards, and ensuring safe train operation.
Smart Images

Figure CN2025093490_12112026_PF_FP_ABST
Abstract
Description
Methods, apparatus, equipment and computer-readable storage media for determining rail surface condition Technical Field
[0001] This application belongs to the field of rail transit technology, and particularly relates to a method, apparatus, equipment, and computer-readable storage medium for determining track surface condition. Background Technology
[0002] Slippery rail surfaces are a significant factor affecting the safety and operational efficiency of rail transit. In rainy or snowy weather, water or ice easily accumulates on the track surface, directly reducing the adhesion between the train wheels and rails. This can lead to serious safety hazards such as train slippage and increased braking distance. Failure to detect and identify these slippery conditions in a timely manner can result in train derailments, frequent accidents, and severely impact passenger safety and the normal operation of freight transport.
[0003] In the exemplary technology, the state of the track surface is determined by image recognition. However, in application, image recognition is affected by lighting, viewing angle, occlusion, and sharpness, which can lead to the track surface state not being accurately identified, i.e., the accuracy of determining the track surface state is low. Summary of the Invention
[0004] This application provides a method, apparatus, device, and computer-readable storage medium for determining the track surface state, which addresses the problem of determining processing strategies.
[0005] Firstly, this application provides a method for determining the state of a rail surface, the method comprising:
[0006] Obtain the current first track plane image;
[0007] Determine the target parameters of the first track surface image, wherein the target parameters include at least one of the following: acquisition angle, sharpness, track surface occlusion area, and acquisition brightness of the first track surface image;
[0008] The first track surface image and the target parameters are input into the prediction model to predict the state of the track surface in the future time period. The prediction model is trained based on each second track surface image, which includes track surface images with different acquisition angles, different resolutions, different occlusion areas, and track surface images acquired under different brightness levels.
[0009] In some possible implementations of the first aspect, the target parameters include the sharpness of the first track surface image, and determining the target parameters of the first track surface image includes:
[0010] Convert the first track surface image into a grayscale image;
[0011] Based on the pixel values of each pixel in the first track surface image, determine the grayscale value of each pixel in the grayscale image;
[0012] Based on the gray values of the pixels in the grayscale image, determine the variance of the second derivative image corresponding to the grayscale image;
[0013] The sharpness of the first track surface image is determined based on the variance.
[0014] In some possible implementations of the first aspect, before inputting the first track plane image into the prediction model, the method further includes:
[0015] Acquire each second track surface image and the acquisition time point corresponding to each second track surface image. Each second track surface image includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different track surface occlusion areas, and track surface images acquired under different brightness levels.
[0016] The label of the second track surface image is determined based on the known state of the track surface in each second track surface image;
[0017] Based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point, construct the first training sample corresponding to the second track surface image;
[0018] The preset model is trained based on each of the first training samples to obtain the prediction model.
[0019] In some possible implementations of the first aspect, acquiring the images of each of the second orbital planes includes:
[0020] Obtain the training orbital plane image;
[0021] By changing different parameters of the training track image, multiple second track images are obtained. The parameter changes of the training track image include one of the following: changing the image acquisition angle, changing the sharpness, changing the track occlusion area, and changing the acquisition brightness.
[0022] In some possible implementations of the first aspect, the preset model includes a deep learning network and a deep convolutional neural network, wherein the deep learning network is equipped with a self-attention mechanism, and the feature layers in the deep convolutional neural network are equipped with a residual attention mechanism module.
[0023] In some possible implementations of the first aspect, training the preset model based on each of the first training samples includes:
[0024] The preset model is trained based on each of the first training samples to obtain an intermediate model;
[0025] Obtain historical images of each third orbital plane of the orbit, and construct a second training sample corresponding to each third orbital plane image based on each third orbital plane image;
[0026] The intermediate model is adjusted based on each of the second training samples to obtain the prediction model corresponding to the orbit.
[0027] In some possible implementations of the first aspect, acquiring the history of each third orbital plane image of the orbit includes:
[0028] Determine the monitoring time period for the track;
[0029] Historical track surface images acquired by the image acquisition device during the monitoring period are used as the respective third track surface images.
[0030] In some possible implementations of the first aspect, acquiring the history of each third orbital plane image of the orbit includes:
[0031] Determine the location of the orbit and the corresponding climate information.
[0032] Historical orbital images acquired by the image acquisition device under the climate conditions corresponding to the climate information are used as the third orbital images.
[0033] In some possible implementations of the first aspect, constructing the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point includes:
[0034] Based on the various reference parameters of the second track surface image, the configuration and learning weight of the corresponding second track surface image are determined. Each of the reference parameters includes the acquisition angle, sharpness, occlusion area of the track surface in the second track surface image, and acquisition brightness of the second track surface image.
[0035] Based on the second track surface image, the corresponding label of the second track surface image, the acquisition time point, and the learning weight, a first training sample corresponding to the second track surface image is constructed.
[0036] Secondly, this application provides a track surface condition determination device, comprising:
[0037] The first acquisition module is used to acquire the current first track surface image of the track;
[0038] The first determining module is used to determine the target parameters of the first track surface image, wherein the target parameters include at least one of the acquisition angle, sharpness, track surface occlusion area, and acquisition brightness of the first track surface image.
[0039] The input module is used to input the first track surface image and the target parameters into the prediction model to predict the state of the track surface in the future time period. The prediction model is trained based on each second track surface image, which includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different occlusion areas, and track surface images acquired under different brightness levels.
[0040] In some possible implementations of the second aspect, the first determining module includes:
[0041] A conversion unit is used to convert the first track surface image into a grayscale image;
[0042] The first determining unit is configured to determine the grayscale value of each pixel in the grayscale image based on the pixel value of each pixel in the first track surface image.
[0043] The second determining unit is used to determine the variance of the second derivative image corresponding to the grayscale image based on the grayscale values of the pixels in the grayscale image.
[0044] The third determining unit is used to determine the sharpness of the first track surface image based on the variance.
[0045] Among the possible implementations of the second aspect are:
[0046] The second acquisition module is used to acquire each second track surface image and the acquisition time point corresponding to each second track surface image. Each second track surface image includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different track surface occlusion areas, and track surface images acquired under different brightness levels.
[0047] The second determining module is used to determine the label of the second track surface image based on the known state of the track surface in each second track surface image;
[0048] The construction module is used to construct the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point;
[0049] The training module is used to train the preset model based on each of the first training samples to obtain the prediction model.
[0050] In some possible implementations of the second aspect, the second acquisition module includes:
[0051] The first acquisition unit is used to acquire training orbital plane images;
[0052] The modification unit is used to modify different parameters of the training track image to obtain multiple second track images. The parameter modification of the training track image includes one of the following: image acquisition angle modification, sharpness modification, track occlusion area modification, and acquisition brightness modification.
[0053] In some possible implementations of the second aspect, the preset model is a deep learning model for object detection, and the deep learning model includes a residual attention mechanism module.
[0054] In some possible implementations of the second aspect, the training module includes:
[0055] The training unit is used to train the preset model based on each of the first training samples to obtain an intermediate model;
[0056] The second acquisition unit is used to acquire each third orbital plane image of the history of the orbit, and to construct a second training sample corresponding to each third orbital plane image based on each third orbital plane image;
[0057] An adjustment unit is used to adjust the intermediate model according to each of the second training samples to obtain the prediction model corresponding to the orbit.
[0058] In some possible implementations of the second aspect, the second acquisition unit includes:
[0059] The first determining subunit is used to determine the monitoring time period of the track;
[0060] The first acquisition subunit is used to acquire historical track surface images collected by the image acquisition device during the monitoring period, as each of the third track surface images.
[0061] In some possible implementations of the second aspect, the second acquisition unit includes:
[0062] The second determining subunit is used to determine the location of the orbit and the corresponding climate information.
[0063] The second acquisition subunit is used to acquire historical orbital maps collected by the image acquisition device under the climate conditions corresponding to the climate information, as each of the third orbital images.
[0064] In some possible implementations of the second aspect, the building module includes:
[0065] The configuration unit is used to configure the learning weight of the second track surface image according to various reference parameters of the second track surface image. Each of the reference parameters includes the acquisition angle, sharpness, occlusion area of the track surface in the second track surface image, and acquisition brightness of the second track surface image.
[0066] The construction unit is used to construct the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, the acquisition time point, and the learning weight.
[0067] Thirdly, this application provides a track surface state determination device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the track surface state determination method described in the first aspect or any implementable method of the first aspect.
[0068] Fourthly, this application provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the orbital state determination method described in the first aspect or any implementable embodiment of the first aspect.
[0069] The track surface state determination method, apparatus, device, and computer-readable storage medium of this application acquire target parameters such as the acquisition angle, clarity, track surface occlusion area, and acquisition brightness of track images. By inputting the current track surface image and target parameters into a prediction model, the state of the track surface in a future time period can be obtained. The prediction model is trained using track surface images acquired at different acquisition angles, with different clarity, different occlusion areas, and under different brightness levels. Even if the current track surface image has low clarity, a large occlusion area, an incorrect acquisition angle, or dim lighting, the prediction model can still accurately identify the state of the track surface, thus improving the accuracy of track surface state determination. Attached Figure Description
[0070] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0071] Figure 1 is a schematic flowchart of one of the methods for determining the track surface state provided in the embodiments of this application;
[0072] Figure 2 is a second schematic flowchart of a method for determining the track surface state provided in an embodiment of this application;
[0073] Figure 3 is a schematic diagram of the structure of a prediction model provided in an embodiment of this application;
[0074] Figure 4 is a third flowchart illustrating a method for determining the track surface state provided in an embodiment of this application;
[0075] Figure 5 is a fourth flowchart illustrating a method for determining the track surface state provided in an embodiment of this application;
[0076] Figure 6 is a fifth flowchart illustrating a method for determining the track surface state provided in an embodiment of this application;
[0077] Figure 7 is a structural schematic diagram of a track surface state determination device provided in an embodiment of this application;
[0078] Figure 8 is a structural schematic diagram of a track surface condition determination device provided in an embodiment of this application. Detailed Implementation
[0079] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.
[0080] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0081] Slippery rail surfaces are a significant factor affecting the safety and operational efficiency of rail transit. In rainy or snowy weather, water or ice easily accumulates on the track surface, directly reducing the adhesion between the train wheels and rails. This can lead to serious safety hazards such as train slippage and increased braking distance. Failure to detect and identify these slippery conditions in a timely manner can result in train derailments, frequent accidents, and severely impact passenger safety and the normal operation of freight transport.
[0082] In the exemplary technology, the state of the track surface is determined by image recognition. However, in application, image recognition is affected by lighting, viewing angle, occlusion, and sharpness, which can lead to the track surface state not being accurately identified, i.e., the accuracy of determining the track surface state is low.
[0083] To address the problems mentioned in the background art, this application provides a method, apparatus, device, and computer-readable storage medium for determining the track surface state.
[0084] By acquiring target parameters such as the acquisition angle, clarity, track surface occlusion area, and acquisition brightness of the track image, and inputting the current track surface image and target parameters into the prediction model, the state of the track surface in the future time period can be obtained. The prediction model is trained using track surface images acquired at different acquisition angles, with different clarity, different occlusion areas, and under different brightness levels. Even if the current track surface image has low clarity, a large occlusion area, an incorrect acquisition angle, or dim lighting, the prediction model can still accurately identify the state of the track surface, thus improving the accuracy of track surface state determination.
[0085] The method for determining the track surface state provided in the embodiments of this application will be described below. Figure 1 shows one of the schematic flowcharts of the method for determining the track surface state provided in an embodiment of this application. As shown in Figure 1, the method for determining the track surface state includes:
[0086] Step S101: Obtain the current first track surface image.
[0087] In this embodiment, the executing entity is the rail surface state determination device. For ease of description, the term "device" will be used to refer to the rail surface state determination device below. The device can be any terminal device with rail surface state determination function.
[0088] Tracks refer to the routes paved with steel or other materials for trains, trams, and other vehicles to travel on. The track surface refers to the surface that contacts the vehicle, and its condition includes normal, rained, snowed, rusted, and covered in fallen leaves. The system needs to constantly monitor the condition of the track surface to ensure the safe operation of the vehicles.
[0089] After acquiring an image of the track surface, the image acquisition device sends the image to the device. This image is defined as the first track surface image, which refers to an image containing the track surface. Specifically, the image acquisition device can use a real-time streaming media transmission protocol to transmit the acquired video stream data to a data server next to the track. The data server parses the video stream in real time to generate a high-definition image and transmits the high-definition image to the device at a predetermined period, so that the device uses the high-definition image as the current first track surface image.
[0090] Step S102: Determine the target parameters of the first track surface image. The target parameters include at least one of the following: acquisition angle, sharpness, track surface occlusion area, and acquisition brightness of the first track surface image.
[0091] After acquiring the first track surface image, the device determines the target parameters of the first track surface image. The target parameters include at least one of the following: the acquisition angle of the first track surface image, the sharpness, the track surface occlusion area, and the acquisition brightness.
[0092] In one example, the target parameters include the acquisition angle. The image acquisition component can use the vertical direction of the track surface as a reference to determine the angle between its own acquisition direction and the vertical direction of the track surface. This angle is the acquisition angle of the first track surface image. In other words, the image acquisition component sends the first track surface image and the acquisition angle to the device.
[0093] In another example, the target parameters include the sharpness of the first track surface image. The sharpness can be transmitted by the image acquisition device, that is, the sharpness is an inherent parameter of the image acquisition device, and the image acquisition device sends the sharpness to the device.
[0094] In another example, the target parameters include the degree of track occlusion in the first track image. A track image typically contains M pixels displaying the track. The device determines the number of pixels displaying the track in the first track image, and the ratio of this number to M determines the degree of track occlusion in the first track image.
[0095] In another example, the target parameter includes the acquisition brightness, which refers to the ambient brightness when the image acquisition unit acquires the track surface image. When the image acquisition unit acquires the first track surface image, it obtains the ambient brightness and sends the ambient brightness and the first track surface image to the device, which then uses the ambient brightness as the acquisition brightness.
[0096] Step S103: Input the first track surface image and target parameters into the prediction model to predict the state of the track surface in the future time period. The prediction model is trained based on each second track surface image, which includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different occlusion areas, and track surface images acquired under different brightness levels.
[0097] The device is equipped with a prediction model that can predict the state of the track surface in the future. The prediction model is trained using various second track surface images, which include track surface images acquired from different angles, with different resolutions, with different occlusion areas, and with different brightness levels.
[0098] Since each second track surface image contains track surface images from different acquisition angles, the trained prediction model can identify the state of the track surface in the track surface image acquired at any acquisition angle. Even if the acquisition angle of the image acquisition device is incorrect and the track surface portion of the acquired track occupies a small area of the track surface image, the prediction model can still identify the state of the track surface in the first track surface image.
[0099] Since the second track surface image contains track surface images of different resolutions, the trained prediction model can identify the state of the track surface in track surface images of any resolution. Even if the image resolution of the first track surface image is low, the prediction model can still identify the state of the track surface in the first track surface image.
[0100] Since the second track surface image contains track surface images with different occlusion areas, the trained prediction model can identify the state of the track surface with any occlusion area. Even if the occlusion area of the track surface in the first track surface image is large, the prediction model can still identify the state of the track surface.
[0101] Since the second track surface image contains track surface images acquired under different brightness levels, the trained prediction model can identify the state of the track surface in track surface images acquired under any brightness level. Even in the first track surface image acquired under high or low brightness levels, the prediction model can identify the state of the track surface in the first track surface image.
[0102] By acquiring target parameters, the prediction model can determine the degree of missing information in the first track surface image. The more missing information, the more feature layers the prediction model uses, thus enabling it to accurately identify the state of the track surface in the first track surface image based on multiple feature layers.
[0103] After obtaining the status of the rail surface in the future time period, if the status indicator shows that the rail surface is in a state of rain, snow, fallen leaves, or rust, the device will output an alarm message to prompt the track maintenance personnel to deal with the rail surface in time so that the rail surface status is changed back to normal.
[0104] In this embodiment, target parameters such as the acquisition angle, clarity, track surface occlusion area, and acquisition brightness of the track image are acquired. By inputting the current track surface image and the target parameters into the prediction model, the state of the track surface in the future time period can be obtained. The prediction model is trained using track surface images acquired at different acquisition angles, with different clarity, with different occlusion areas, and with different brightness levels. Even if the current track surface image has low clarity, a large occlusion area, an incorrect acquisition angle, or dim lighting, the prediction model can still accurately identify the state of the track surface, thus improving the accuracy of track surface state determination.
[0105] In one embodiment, the target parameter includes the sharpness of the first track surface image, which is determined by the richness of detail features in the first track surface image.
[0106] For example, the device first converts the first track surface image into a grayscale image. The conversion method uses a weighted average of the human visual system's sensitivity to different colors, that is, the grayscale value of each pixel in the grayscale image = 0.299 * red brightness value + 0.587 * green brightness value + 0.144 * blue brightness value.
[0107] The device determines the grayscale value of each pixel in the grayscale image based on the pixel values of each pixel in the first track surface image. In other words, the grayscale value of each pixel in the grayscale image is calculated based on the above formula.
[0108] The device determines the variance of the second derivative image corresponding to the grayscale image based on the grayscale value of the pixel in the grayscale image, and thus determines the sharpness of the first track surface image through the variance.
[0109] Variance is a statistic that measures the degree of dispersion of data. It is calculated by measuring the variance of the second derivative of a grayscale image to quantify the transversal of grayscale variations. The formula for calculating variance is: σ 2 =∑[(x_i-μ) 2 ] / N
[0110] Where, σ 2 is the variance, x_i is the gray value of each pixel in the image, μ is the average gray value of the image, and N is the total number of pixels in the image.
[0111] Since the binary derivative image of a grayscale image can highlight high-frequency information in the track surface image, such as edges and speckles, and variance can quantify the intensity of this high-frequency information, a larger variance indicates more high-frequency information such as edges and speckles in the track surface image, resulting in higher image clarity; conversely, a smaller variance indicates fewer details in the track surface image, resulting in lower image clarity.
[0112] In this embodiment, the clarity of the first track surface image is accurately determined by converting the first track surface image into a grayscale image and then using the second derivative image of the grayscale image.
[0113] Referring to Figure 2, which shows a second schematic flowchart of a method for determining the track surface state according to an embodiment of this application, based on the embodiment shown in Figure 1, before step S102, the method further includes:
[0114] Step S201: Obtain each second track surface image and the acquisition time point corresponding to each second track surface image. Each second track surface image includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different track surface occlusion areas, and track surface images acquired under different brightness levels.
[0115] In this embodiment, the device acquires various second track surface images, including track surface images from different acquisition angles, track surface images with different resolutions, track surface images with different occlusion areas, and track surface images acquired under different brightness levels.
[0116] Since it is necessary to train a model to predict the state of the track surface in the future time period, the model is made to learn the relationship between the state of the track surface and time. That is, after the device obtains the second track surface image, it obtains the acquisition time point of the second track surface image, thereby obtaining each second track surface image in the time series.
[0117] Step S202: Determine the label of the second track surface image based on the known state of the track surface in each second track surface image.
[0118] The state of the track surface in the second track surface image is known, and this known state can be manually labeled and uploaded. Based on the known state of the track surface in each second track surface image, the device determines a label for each image. For example, if the track surface state in the second track surface image is normal, the label value is 0; if the track surface state in the second track surface image is abnormal, the label value is set to 1. Alternatively, labels can be obtained by labeling the second track surface images using a labeling tool.
[0119] Step S203: Construct the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point.
[0120] After determining the label of each second track surface image, a training sample corresponding to the second track surface image can be constructed based on the second track surface image, the label of the second track surface image, and the acquisition time point. This training sample is defined as the first training sample.
[0121] Step S204: Train the preset model based on each first training sample to obtain the prediction model.
[0122] After obtaining each first training sample, the preset model is trained using each first training sample. When the network parameters of the preset model no longer converge, the training of the model is complete, and the prediction model can be obtained.
[0123] The pre-defined model includes deep learning networks and deep convolutional neural networks. The deep learning network includes a self-attention mechanism, which can learn the complex and non-linear dependencies between time steps and sequences, thus enabling the trained prediction model to accurately predict the orbital state over a future period of time.
[0124] Deep convolutional neural networks (DCNNs) are networks used for object detection. DCNNs serve as the backbone network for multi-scale feature map extraction and also include a neck network responsible for multi-scale feature map fusion. A DCNN consists of an input network, a backbone network, a neck network, and an output network. The input network handles image preprocessing and can employ data augmentation to improve model performance. The backbone network has strong feature extraction capabilities and high computational efficiency. The neck network uses an FPN (Feature Pyramid Networks) + PAN (Path Aggregation Network) structure to fuse information from different feature map levels, enhancing feature diversity and robustness. The output network outputs the object detection results, including classification and regression branches. It can use computational loss functions to mitigate class imbalance and non-maximum suppression to handle overlapping bounding boxes.
[0125] Furthermore, a residual attention mechanism module is incorporated into the deep convolutional neural network. This module is designed for detecting anomalies on the track surface. By combining residual networks and attention mechanisms, the residual attention mechanism effectively improves the performance of multi-label classification tasks. Specifically, firstly, it addresses the vanishing gradient problem that traditional deep networks often encounter during training by using residual blocks. Residual blocks allow for identity mappings, making the network easier to train and optimize. Secondly, the initial convolutional layers and residual blocks are used to extract multi-level feature representations from the input image. These features capture the complex texture information of the track surface, and by adding attention mechanisms to feature channels and locations, the network can focus more on important features and regions on the track surface, thereby improving classification performance. Thirdly, multi-label classification is ultimately achieved by analyzing the probability distribution of each category output by the multi-label classification output.
[0126] Referring to Figure 3, which shows a deep learning model for object detection with a residual attention mechanism module, this deep learning model is the preset model. Specifically, the backbone network includes a cross-stage layer, a convolutional module, and a residual self-attention mechanism module, while the neck network includes a connection layer and a residual self-attention mechanism module.
[0127] Furthermore, deep learning networks can be integrated with deep convolutional neural networks (DCNNs). For example, the feature layer containing the self-attention mechanism in a deep learning network can be replaced with the backbone in a DCNN to enhance global information capture capabilities, thereby improving the accuracy of DCNNs in detecting small objects and complex scenes. Additionally, deep learning networks and DCNNs can be connected in parallel, fusing features extracted by the feature layers in the DCNN with the output of the deep learning network; or, deep learning networks and DCNNs can process images in parallel, and then fuse the two parallel results. This combination can improve both the accuracy and efficiency of object detection. By configuring deep learning networks and DCNNs in these ways, the attention mechanism in the deep learning network can enhance the detection capabilities of the DCNN, focusing on key information and ignoring redundant information, thus improving the accuracy of the prediction model in predicting the state of the orbital plane. The cross-scale capability of deep learning networks combined with the multi-scale detection of DCNNs can comprehensively cover targets of different sizes, improving the prediction coverage and accuracy of the prediction model.
[0128] In this embodiment, the device obtains multiple first training samples through each second track plane image, and then trains a high-performance prediction model through each first training sample.
[0129] In one embodiment, the parameters of the training track surface image can be changed by the device, thereby increasing the diversity of the track surface image and training a prediction model that can identify more accurately.
[0130] For example, the device acquires multiple training track surface images. The device modifies different parameters of each training track surface image to obtain multiple second track surface images of the training track surface images. The parameter modifications to the training track surface images include one or more of the following: image acquisition angle modification, sharpness modification, track surface occlusion area modification, and acquisition brightness modification.
[0131] When it comes to changing the image acquisition angle, a series of different angles θ are set based on the direction perpendicular to the center of the track surface. j j = 1, 2, 3, 4, ..., m; m is the number of different acquisition angles. For example, when θ j When θ = 0, the image acquired is a view of the rail surface from directly above. This image clearly shows the overall outline of the rail surface, its surface flatness, and the distribution of the rail fasteners, and can be used for model learning of the standard morphological features of the rail surface. When θ = 0, the image acquired is a view of the rail surface from directly above. j At an angle of 30°, the acquired images can show part of the transition area between the side and front of the rail surface, allowing the model to learn features such as the wear condition of the rail surface side and the connection details between the sleeper and the rail surface. Images from different acquisition angles. The track surface condition is presented from multiple perspectives. Specifically, the track surface in the training track surface image can be deflected to set track surface images under different image acquisition angles.
[0132] When it comes to changes in sharpness, details in the track surface image can be blurred. The more details are blurred, the lower the sharpness of the track surface image.
[0133] When changes are involved in the area obstructed by the rail surface, the obstruction ratio is O. p Where p = 1, 2, 3, 4, ..., s; s is the number of different occlusion ratios. By adding occluders of different sizes and shapes to the training orbital image, images with different occlusion areas are generated. The model learns to accurately determine the orbital surface status even when some information is missing.
[0134] When it involves changing the brightness of the sampled image, set the brightness parameter to b. r r = 1, 2, 3, 4, ..., t; t is the number of different brightness levels. By adjusting the brightness of the training orbital image, images at different brightness levels are generated. This allows the model to adapt to the features of track surface images under different lighting conditions.
[0135] In this embodiment, the parameters of the training track surface image are changed to increase the diversity of the track surface image, thereby training a prediction model that can identify more accurately.
[0136] Referring to Figure 4, which shows a third schematic flowchart of a method for determining the track surface state according to an embodiment of this application, based on the embodiment shown in Figure 2, step S204 includes:
[0137] Step S401: Train the preset model based on each first training sample to obtain an intermediate model.
[0138] In this embodiment, the device first trains a preset model using each first training sample to obtain a pre-trained model, which is defined as an intermediate model. After obtaining the intermediate model, the device fine-tunes the intermediate model so that the fine-tuned intermediate model can more accurately predict the state of the track surface.
[0139] Step S402: Obtain the historical images of each third orbital plane, and construct the second training sample corresponding to each third orbital plane image based on each third orbital plane image.
[0140] The device acquires historical images of each track surface for the track that needs to be monitored, and each track surface image is defined as a third track surface image. Based on each third track surface image, the device constructs a second training sample corresponding to each third track surface image. The construction method of the second training sample is the same as that of the first training sample, and will not be repeated here.
[0141] Step S403: Adjust the intermediate model according to each second training sample to obtain the prediction model corresponding to the track.
[0142] After obtaining each second training sample, the device trains the intermediate model to achieve fine-tuning of the intermediate model. The intermediate model after training is the prediction model.
[0143] In this embodiment, the device first trains the preset model using each first training sample to obtain an intermediate model, and then adjusts the intermediate model using the second training samples corresponding to each third track surface image in the history of the track to obtain a prediction model suitable for monitoring the track surface state, thereby improving the prediction accuracy of the prediction model for the track surface state.
[0144] Referring to Figure 5, which illustrates a fourth flowchart of a method for determining the track surface state according to an embodiment of this application, based on the embodiment shown in Figure 2 or Figure 4, step S402 includes:
[0145] Step S501: Determine the monitoring time period for the track.
[0146] Step S502: Obtain historical track surface images collected by the image acquisition device during the monitoring period, as each third track surface image.
[0147] In this embodiment, the device needs to train a prediction model adapted to the track. Since the historical track surface images collected are limited, directly using these historical images to train the model would result in low prediction accuracy. Therefore, the device first trains a pre-set model based on each first training sample to obtain a pre-trained intermediate model, and then trains the intermediate model using a second training sample constructed from the historical track surface images to obtain the prediction model.
[0148] The condition of the track surface is prone to anomalies during fixed time periods throughout the day. For example, between 5:00 AM and 7:00 AM, there may be dew or frost on the track surface. Therefore, this period needs to be monitored closely, making it a key monitoring timeframe. The monitoring timeframe refers to the period during which the track surface is likely to exhibit anomalies. " Likely to exhibit anomalies" means that the probability of the track surface exhibiting an abnormal condition within this timeframe exceeds a preset threshold. This probability is obtained statistically. For example, if the track surface exhibits an abnormal condition 7 times between 5:00 AM and 7:00 AM over ten days, meaning the track surface is abnormal every single morning between 5:00 AM and 7:00 AM within those 7 days, then the probability of anomaly is 7 / 10 = 70%.
[0149] To address this, the device acquires historical track surface images collected by the image acquisition equipment during the detection period, using these images as third track surface images. This enables the trained prediction model to accurately predict the state of the track surface during the detection period.
[0150] In this embodiment, the device determines the monitoring time period of the track and acquires historical track surface images collected by the image acquisition device during the detection time period as each third track surface image, so that the trained prediction model can accurately predict the state of the track surface during the detection time period.
[0151] Referring to Figure 6, which illustrates a fifth flowchart of a method for determining the track surface state according to an embodiment of this application, based on the embodiment shown in Figure 2 or Figure 4, step S402 includes:
[0152] Step S601: Determine the location of the orbit and the corresponding climate information.
[0153] Step S602: Obtain historical orbital images acquired by the image acquisition device under the climate conditions corresponding to the climate information, as each third orbital image.
[0154] In this embodiment, the device needs to train a prediction model adapted to the track. Since the historical track surface images collected are limited, directly using these historical images to train the model would result in low prediction accuracy. Therefore, the device first trains a pre-set model based on each first training sample to obtain a pre-trained intermediate model, and then trains the intermediate model using a second training sample constructed from the historical track surface images to obtain the prediction model.
[0155] The condition of the track surface is related to the climate of the track's location. For example, if the climate at the track's location is rainy, the track surface will mostly be covered in rainwater and rust. The device then needs to acquire historical track surface images during the rainy season as the third track surface image. In other words, the device acquires historical track surface images collected by image acquisition equipment under the corresponding climatic conditions. Furthermore, when the abnormal condition of the track surface is unrelated to climate, the device can also acquire a third track surface image based on location. For example, if the track is located at a high altitude, where trees are planted, there will be fallen leaves on the track surface, thus requiring historical track surface images as the third track surface image.
[0156] To address this, the device acquires historical track surface images collected by the image acquisition equipment during the detection period, using these images as third track surface images. This enables the trained prediction model to accurately predict the state of the track surface during the detection period.
[0157] In this embodiment, the device determines the monitoring time period of the track and acquires historical track surface images collected by the image acquisition device during the detection time period as each third track surface image, so that the trained prediction model can accurately predict the state of the track surface during the detection time period.
[0158] In one embodiment, during the training of the preset model, a dynamic weight adjustment mechanism is used, which enables the preset model to automatically adjust the weights of different labels based on the difficulty of the classification task, so as to balance the classification performance of different categories.
[0159] In one example, the device configures learning weights for the second track surface image based on various reference parameters, including the acquisition angle, sharpness, occlusion area of the track surface in the second track surface image, and acquisition brightness of the second track surface image. For example, the device determines a score corresponding to each target parameter, and each target parameter is assigned a weight. A total score is obtained by weighting the scores and weights. The larger the total score, the more difficult it is to identify the state of the track surface in the second track surface image; therefore, a larger learning weight is set. Specifically, the device determines the numerical range in which the total score falls, and uses the weight corresponding to the numerical range as the learning weight; the larger the numerical range, the larger the learning weight.
[0160] When the target parameter is sharpness, the smaller the sharpness, the higher the score corresponding to sharpness; when the target parameter is occlusion area, the larger the occlusion area, the higher the score corresponding to occlusion area; when the target parameter is acquisition brightness, if the acquisition brightness exceeds the set brightness range, it will be difficult to recognize the track surface image, and the score corresponding to acquisition brightness is the first value; if the acquisition brightness is within the set brightness range, the score corresponding to acquisition brightness is the second value, which is less than the first value; the acquisition angle refers to the angle between the acquisition direction of the image acquisition device and the plane where the track surface is located. The larger the absolute value of the difference between the angle and 90°, the higher the score corresponding to the acquisition angle.
[0161] It should be noted that after obtaining each first training sample, the device uses 70% of the first training samples as the training set and the remaining first training samples as the validation set. After training the preset model using each first training sample in the training set, the first training samples in the validation set are used to validate the trained preset model. If the validation is successful, the trained preset model is used as the prediction model.
[0162] Figure 7 is a schematic diagram of a track surface state determination device provided in an embodiment of this application. As shown in Figure 7, the track surface state determination device 700 may include:
[0163] The first acquisition module 710 is used to acquire the current first track surface image of the track;
[0164] The first determining module 720 is used to determine the target parameters of the first track surface image. The target parameters include at least one of the following: the acquisition angle, the sharpness, the track surface occlusion area, and the acquisition brightness of the first track surface image.
[0165] The input module 720 is used to input the first track surface image and target parameters into the prediction model to predict the state of the track surface in the future time period. The prediction model is trained based on each second track surface image, which includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different occlusion areas, and track surface images acquired under different brightness levels.
[0166] In one embodiment, the first determining module includes:
[0167] A conversion unit is used to convert the first track surface image into a grayscale image;
[0168] The first determining unit is used to determine the grayscale value of each pixel in the grayscale image based on the pixel value of each pixel in the first track surface image.
[0169] The second determining unit is used to determine the variance of the second derivative image corresponding to the grayscale image based on the grayscale values of the pixels in the grayscale image.
[0170] The third determining unit is used to determine the sharpness of the first track surface image based on the variance.
[0171] In one embodiment, it further includes:
[0172] The second acquisition module is used to acquire each second track surface image and the acquisition time point corresponding to each second track surface image. Each second track surface image includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different track surface occlusion areas, and track surface images acquired under different brightness levels.
[0173] The second determining module is used to determine the label of the second track surface image based on the known state of the track surface in each second track surface image;
[0174] The construction module is used to construct the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point;
[0175] The training module is used to train the preset model based on each first training sample to obtain the prediction model.
[0176] In one embodiment, the second acquisition module includes:
[0177] The first acquisition unit is used to acquire training orbital plane images;
[0178] The modification unit is used to change different parameters of the training track image to obtain multiple second track images. The parameter changes of the training track image include one of the following: image acquisition angle change, sharpness change, track occlusion area change, and acquisition brightness change.
[0179] In one embodiment, the preset model includes a deep learning network and a deep convolutional neural network. The deep learning network is equipped with a self-attention mechanism, and the feature layers in the deep convolutional neural network are equipped with a residual attention mechanism module.
[0180] In one embodiment, the training module includes:
[0181] The training unit is used to train the preset model based on each first training sample to obtain an intermediate model;
[0182] The second acquisition unit is used to acquire each third orbital plane image of the orbit's history, and to construct a second training sample corresponding to each third orbital plane image based on each third orbital plane image.
[0183] The adjustment unit is used to adjust the intermediate model based on each second training sample to obtain the prediction model corresponding to the orbit.
[0184] In one embodiment, the second acquisition unit includes:
[0185] The first determining sub-unit is used to determine the monitoring time period of the track;
[0186] The first acquisition subunit is used to acquire historical track surface images collected by the image acquisition device during the monitoring period, as images of each third track surface.
[0187] In one embodiment, the second acquisition unit includes:
[0188] The second determining subunit is used to determine the location of the orbit and the corresponding climate information.
[0189] The second acquisition subunit is used to acquire historical orbital images collected by the image acquisition device under the climate conditions corresponding to the climate information, as each third orbital image.
[0190] In one embodiment, the construction module includes:
[0191] The configuration unit is used to configure the learning weights of the second track surface image according to various reference parameters of the second track surface image. The various reference parameters include the acquisition angle, sharpness, occlusion area of the track surface in the second track surface image, and acquisition brightness of the second track surface image.
[0192] The construction unit is used to construct the first training sample corresponding to the second track plane image based on the second track plane image, the label corresponding to the second track plane image, the acquisition time point, and the learning weight.
[0193] It is understood that the track surface state determination device 700 in this application embodiment can correspond to the execution subject of the track surface state determination method described in this application embodiment. For specific details of the operation and / or function of each module / unit of the track surface state determination device 700, please refer to the description of the corresponding part in the track surface state determination method provided in the above application embodiment. For the sake of brevity, it will not be repeated here.
[0194] Figure 8 shows a schematic diagram of a track surface state determination device according to an embodiment of this application. As shown in Figure 8, the device may include a processor 801 and a memory 802 storing computer program instructions.
[0195] Specifically, the processor 801 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0196] Memory 802 may include mass storage for information or instructions. For example, and not limitingly, memory 802 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk drive, magneto-optical disk drive, magnetic tape drive, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 802 may include removable or non-removable (or fixed) media, or memory 802 may be a non-volatile solid-state memory. Memory 802 may be internal or external to the device depending on the track surface condition.
[0197] In one instance, memory 802 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.
[0198] The processor 801 reads and executes the computer program instructions stored in the memory 802 to implement the method described in the embodiments of this application and achieve the corresponding technical effects achieved by the execution of the method in the embodiments of this application. For the sake of brevity, it will not be described in detail here.
[0199] In one example, the track surface status determination device may also include a communication interface 803 and a bus 810. As shown in Figure 8, the processor 801, memory 802, and communication interface 803 are connected via the bus 810 and communicate with each other.
[0200] The communication interface 803 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.
[0201] Bus 810 includes hardware, software, or both, that couples components of an online information flow metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 810 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.
[0202] The track surface condition determination device can execute the track surface condition determination method in the embodiments of this application, thereby achieving the corresponding technical effects of the track surface condition determination method described in the embodiments of this application.
[0203] Furthermore, in conjunction with the track surface state determination method in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the track surface state determination methods in the above embodiments. Examples of readable storage media can be non-transitory machine-readable media, such as electronic circuits, semiconductor memory devices, read-only memory (ROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, etc.
[0204] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0205] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0206] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0207] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.
[0208] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.
Claims
1. A method for determining the state of a rail surface, comprising: Obtain the current first track plane image; Determine the target parameters of the first track surface image, wherein the target parameters include at least one of the following: acquisition angle, sharpness, track surface occlusion area, and acquisition brightness of the first track surface image; The first track surface image and the target parameters are input into the prediction model to predict the state of the track surface in the future time period. The prediction model is trained based on each second track surface image, which includes track surface images with different acquisition angles, different resolutions, different occlusion areas, and track surface images acquired under different brightness levels.
2. The method for determining the track surface state according to claim 1, wherein the target parameter includes the clarity of the first track surface image, and determining the target parameter of the first track surface image includes: Convert the first track surface image into a grayscale image; Based on the pixel values of each pixel in the first track surface image, determine the grayscale value of each pixel in the grayscale image; Based on the gray values of the pixels in the grayscale image, determine the variance of the second derivative image corresponding to the grayscale image; The sharpness of the first track surface image is determined based on the variance.
3. The method according to claim 1, further comprising, before inputting the first track surface image into the prediction model: Acquire each second track surface image and the acquisition time point corresponding to each second track surface image. Each second track surface image includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different track surface occlusion areas, and track surface images acquired under different brightness levels. The label of the second track surface image is determined based on the known state of the track surface in each second track surface image; Based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point, construct the first training sample corresponding to the second track surface image; The preset model is trained based on each of the first training samples to obtain the prediction model.
4. The method according to claim 3, wherein acquiring each second track plane image comprises: Obtain the training orbital plane image; By changing different parameters of the training track image, multiple second track images are obtained. The parameter changes of the training track image include one of the following: changing the image acquisition angle, changing the sharpness, changing the track occlusion area, and changing the acquisition brightness.
5. The method according to claim 3, wherein the preset model includes a deep learning network and a deep convolutional neural network, the deep learning network is provided with a self-attention mechanism, and the feature layer in the deep convolutional neural network is provided with a residual attention mechanism module.
6. The method according to claim 3, wherein training the preset model based on each of the first training samples comprises: The preset model is trained based on each of the first training samples to obtain an intermediate model; Obtain historical images of each third orbital plane of the orbit, and construct a second training sample corresponding to each third orbital plane image based on each third orbital plane image; The intermediate model is adjusted based on each of the second training samples to obtain the prediction model corresponding to the orbit.
7. The method according to claim 6, wherein acquiring the historical images of each third orbital plane of the orbit comprises: Determine the monitoring time period for the track; Historical track surface images acquired by the image acquisition device during the monitoring period are used as the respective third track surface images.
8. The method according to claim 6, wherein acquiring the historical images of each third orbital plane of the orbit comprises: Determine the location of the orbit and the corresponding climate information. Historical orbital images acquired by the image acquisition device under the climate conditions corresponding to the climate information are used as the third orbital images.
9. The method according to any one of claims 3-8, wherein constructing the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point comprises: Based on the various reference parameters of the second track surface image, the configuration and learning weight of the corresponding second track surface image are determined. Each of the reference parameters includes the acquisition angle, sharpness, occlusion area of the track surface in the second track surface image, and acquisition brightness of the second track surface image. Based on the second track surface image, the corresponding label of the second track surface image, the acquisition time point, and the learning weight, a first training sample corresponding to the second track surface image is constructed.
10. A track surface condition determination device, comprising: The first acquisition module is used to acquire the current first track surface image of the track; The first determining module is used to determine the target parameters of the first track surface image, wherein the target parameters include at least one of the acquisition angle, sharpness, track surface occlusion area, and acquisition brightness of the first track surface image. The input module is used to input the first track surface image and the target parameters into the prediction model to predict the state of the track surface in the future time period. The prediction model is trained based on each second track surface image, which includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different occlusion areas, and track surface images acquired under different brightness levels.
11. The apparatus according to claim 10, wherein the first determining module comprises: A conversion unit is used to convert the first track surface image into a grayscale image; The first determining unit is configured to determine the grayscale value of each pixel in the grayscale image based on the pixel value of each pixel in the first track surface image. The second determining unit is used to determine the variance of the second derivative image corresponding to the grayscale image based on the grayscale values of the pixels in the grayscale image. The third determining unit is used to determine the sharpness of the first track surface image based on the variance.
12. The apparatus of claim 10, further comprising: The second acquisition module is used to acquire each second track surface image and the acquisition time point corresponding to each second track surface image. Each second track surface image includes track surface images with different acquisition angles, track surface images with different resolutions, track surface images with different track surface occlusion areas, and track surface images acquired under different brightness levels. The second determining module is used to determine the label of the second track surface image based on the known state of the track surface in each second track surface image; The construction module is used to construct the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, and the acquisition time point; The training module is used to train the preset model based on each of the first training samples to obtain the prediction model.
13. The apparatus according to claim 12, wherein the second acquisition module comprises: The first acquisition unit is used to acquire training orbital plane images; The modification unit is used to modify different parameters of the training track image to obtain multiple second track images. The parameter modification of the training track image includes one of the following: image acquisition angle modification, sharpness modification, track occlusion area modification, and acquisition brightness modification.
14. The apparatus according to claim 12, wherein the preset model is a deep learning model for object detection, and the deep learning model includes a residual attention mechanism module.
15. The apparatus according to claim 12, wherein the training module comprises: The training unit is used to train the preset model based on each of the first training samples to obtain an intermediate model; The second acquisition unit is used to acquire each third orbital plane image of the history of the orbit, and to construct a second training sample corresponding to each third orbital plane image based on each third orbital plane image; An adjustment unit is used to adjust the intermediate model according to each of the second training samples to obtain the prediction model corresponding to the orbit.
16. The apparatus according to claim 15, wherein the second acquiring unit comprises: The first determining subunit is used to determine the monitoring time period of the track; The first acquisition subunit is used to acquire historical track surface images collected by the image acquisition device during the monitoring period, as each of the third track surface images.
17. The apparatus of claim 15, wherein the second acquiring unit comprises: The second determining subunit is used to determine the location of the orbit and the corresponding climate information. The second acquisition subunit is used to acquire historical orbital maps collected by the image acquisition device under the climate conditions corresponding to the climate information, as each of the third orbital images.
18. The apparatus according to any one of claims 12-17, wherein the building module comprises: The configuration unit is used to configure the learning weight of the second track surface image according to various reference parameters of the second track surface image. Each of the reference parameters includes the acquisition angle, sharpness, occlusion area of the track surface in the second track surface image, and acquisition brightness of the second track surface image. The construction unit is used to construct the first training sample corresponding to the second track surface image based on the second track surface image, the label corresponding to the second track surface image, the acquisition time point, and the learning weight.
19. A track surface condition determination device, comprising: Processor, and memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the orbital surface state determination method as described in any one of claims 1-9.
20. A computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the orbital state determination method as described in any one of claims 1-9.
