Locomotive sanding state real-time monitoring method and system based on multi-vision volume measurement

Abstract

The invention discloses a locomotive sanding state real-time monitoring method and system based on multi-vision volume measurement. The method comprises the following steps: acquiring a sand surface image sequence in a sand box in a locomotive sanding process in real time; carrying out image enhancement and defogging processing on the collected original sand grain surface image to obtain a high-quality sand grain surface image; sand grain surface self-adaptive three-dimensional point cloud reconstruction is carried out on the obtained high-quality sand grain surface image; on the basis of the sand grain surface three-dimensional point cloud reconstruction result, the sand spraying consumption volume is calculated; and classifying the faults of the sanding device and optimizing the operation strategy of a driver according to the sanding consumption volume. The method can realize real-time high-precision non-contact measurement of the locomotive sanding consumption volume, sanding device fault classification and locomotive driver operation strategy optimization under special operation conditions, has the characteristics of accuracy, high efficiency, strong generalization ability and the like, and is expected to be developed into a long-term monitoring method of the locomotive sanding state.

Description

Technical Field

[0001] This invention relates to the field of rail transit technology, specifically to a method and system for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement. Background Technology

[0002] With the development of high-speed and heavy-haul railway transportation, the stability of wheel-rail adhesion performance has become a core factor restricting the traction and braking performance of trains. As a key component for improving wheel-rail adhesion, the sand spreading device can significantly enhance the adhesion between the wheel and the rail through a reasonable sand content, effectively preventing the risk of locomotive slippage during traction or skidding during braking, thus ensuring the safety and stability of train operation. However, with the continuous increase in locomotive carrying capacity and speed, the frequency of use and load intensity of the sand spreading device have also increased significantly, leading to a surge in failure rates such as sand pipe blockage and sand spreader wear. Furthermore, the traction and braking requirements of locomotives vary dynamically under different line conditions, carrying capacity, and weather conditions. The previous method of relying on human experience to judge the appropriate sand amount under these conditions is difficult to accurately adapt to actual needs, severely restricting the effective performance of locomotive traction and braking. Therefore, it is necessary to conduct real-time monitoring and quantitative analysis of locomotive sand spreading amount to effectively guide the daily maintenance of the locomotive sand spreading device and optimize driver operation strategies based on different operating conditions, effectively ensuring the long-term healthy and safe operation of the locomotive.

[0003] Currently, the quantitative assessment of sand spreading and the fault detection of equipment rely on trackside static detection methods, such as weighing, machine vision-based two-dimensional image comparison, and optical detection. However, these methods have significant limitations: the detection process must be carried out in static environments such as maintenance yards, which is cumbersome and inefficient. Not only do they require sand recovery and environmental cleaning, but they also cannot operate effectively in adverse weather conditions such as rain, snow, and sandstorms, resulting in a significant decrease in monitoring accuracy or even failure. More importantly, these methods are essentially static or post-event detections, unable to monitor the actual sand spreading volume of the sand spreading device in real time during locomotive operation. Therefore, they cannot accurately control the sand spreading amount based on dynamically changing track conditions, load, weather, and wheel-rail adhesion. This can easily lead to excessive sand spreading, wasting resources, or insufficient sand spreading, causing adhesion failure and other safety problems. Therefore, developing a real-time monitoring method for locomotive sand spreading status based on multi-vision volume measurement will effectively overcome the limitations of existing methods and achieve long-term stable monitoring of sand spreading content and equipment status. Summary of the Invention

[0004] This invention provides a method and system for real-time monitoring of locomotive sand spreading status based on multi-vision volume measurement, in order to solve the above-mentioned technical problems existing in existing trackside static or post-event detection methods.

[0005] According to a first aspect, one embodiment provides a method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement, the method comprising:

[0006] Real-time acquisition of image sequences of sand particles on the surface inside the sand box during the sand spreading process of the locomotive;

[0007] Image enhancement and dehazing were performed on the original sand grain surface images to obtain high-quality sand grain surface images;

[0008] Adaptive 3D point cloud reconstruction of the sand grain surface is performed on the obtained high-quality sand grain surface image;

[0009] Based on the three-dimensional point cloud reconstruction results of the sand grain surface, the volume of sand consumption is calculated.

[0010] Based on the volume of sand consumption, classify sand spreading device malfunctions and optimize driver operation strategies.

[0011] Furthermore, real-time acquisition of image sequences of sand particles on the surface of the sand box during the sand spreading process of the locomotive specifically includes:

[0012] High-resolution industrial cameras at different angles are deployed on top of the sand spreading box of the locomotive to capture image sequences of the surface of sand particles inside the sand spreading box during the sand spreading process, and multiple LED lights provide supplementary light sources for the cameras.

[0013] Furthermore, image enhancement and dehazing processing are performed on the acquired sand grain surface images to obtain high-quality sand grain surface images, specifically including:

[0014] Image enhancement processing includes adding Gaussian noise, adjusting brightness, and random exposure processing.

[0015] Furthermore, image enhancement and dehazing processing are performed on the acquired sand grain surface images to obtain high-quality sand grain surface images, specifically including:

[0016] A deep learning model is used to perform dehazing on the sand grain surface image after image enhancement.

[0017] Furthermore, adaptive 3D point cloud reconstruction of the sand grain surface is performed on the obtained high-quality sand grain surface image, specifically including:

[0018] Based on a deep learning model, the correspondence between multiple pixels across different viewpoints is established autonomously to achieve self-supervised, pose-free, multi-view accurate camera pose capture.

[0019] Then, based on accurate multi-view stereo matching, a sparse point cloud is generated, and the point cloud noise is reduced by filtering. At the same time, downsampling is used to reduce redundancy or upsampling is used to increase the integrity of local details and improve the reconstruction accuracy.

[0020] Finally, the Alpha-shape algorithm is used to extract the outer contour of the point cloud and combined with Poisson reconstruction to generate an implicit surface, thus achieving high-precision reconstruction of the sand grain surface.

[0021] Furthermore, based on the three-dimensional point cloud reconstruction results of the sand grain surface, the volume of sand consumption is calculated, specifically including:

[0022] Using the state of the sand box before sand application as the reference plane, the volume difference between the sand grain surface reconstructed from the point cloud and the reference plane is calculated in real time and used as the volume of sand consumption. The formula is as follows:

[0023] ;

[0024] In the formula: This refers to the volume of sand consumed during sand spreading. Before spreading sand The volume of space between the surface of the sand grains and the highest sand loading position at any given time; After sprinkling sand The volume of space between the surface of the sand grains and the highest sand loading position at any given time; Before spreading sand The surface coordinates of the sand grain at that moment; After sprinkling sand The coordinates of the sand grain surface at that moment.

[0025] Furthermore, based on the volume of sand consumption, fault classification of the sand spreading device and optimization of driver operation strategies are carried out, specifically including:

[0026] A parallel residual network based on depthwise separable convolution is used for fault classification of the sand spreading device, as detailed below:

[0027] First, multi-branch deep residual operations are performed in parallel to extract different feature information at different depths in each branch. The single-branch deep residual network is represented as follows:

[0028] ;

[0029] In the formula, It is the real-time monitoring of sand consumption volume. It is the output after passing through several separable depthwise convolutional layers, batch normalization, and activation layers. Represents residual connections. It is the feature extraction output of a single depth residual block;

[0030] Subsequently, feature fusion is performed on the information from each branch. The Softmax function is used to obtain the weights of each branch and assign them to each branch to obtain the fused features. :

[0031] ;

[0032] ;

[0033] In the formula: This is the feature extraction output for the m-th depth residual branch; The attention weights for each branch, and m represents the number of branches in the deep residual structure.

[0034] Finally, based on the feature fusion results, a single-layer fully connected network and the Softmax function are used to classify the faults of the sand spreading device.

[0035] Furthermore, based on the volume of sand consumption, fault classification of the sand spreading device and optimization of driver operation strategies are carried out, specifically including:

[0036] Fault classification is determined based on the percentage of actual sand consumption V relative to the set standard consumption V0, as follows:

[0037] When V=V0, the sand spreading device is working properly and the sand spreading degree is excellent;

[0038] When 75% V0 < V ≤ 100% V0, the sand spreading device has a minor malfunction, but the sand spreading degree is good.

[0039] When 50% V0 < V ≤ 75% V0, the sand spreading device is moderately faulty, and the sand spreading degree is poor.

[0040] When V≤50%V0, the sand spreading device is seriously malfunctioning and cannot operate normally.

[0041] Furthermore, based on the volume of sand consumption, fault classification of the sand spreading device and optimization of driver operation strategies are carried out, specifically including:

[0042] Based on the real-time sand-spreading consumption volume and locomotive operating condition information, the sand-spreading effect under the corresponding operating conditions is evaluated, and sand-spreading operation optimization suggestions or early warning information are generated. The sand-spreading consumption data, operating condition diagnosis results and operation suggestions under the corresponding operating conditions are output to the on-board display unit or maintenance system in real time. Among them, the locomotive operating condition information includes track conditions, traction / braking commands, locomotive speed, sandblasting pipe pressure and weather conditions.

[0043] According to a second aspect, one embodiment provides a real-time monitoring system for locomotive sand-spreading status based on multi-vision volume measurement, the system comprising:

[0044] The image acquisition module is used to acquire real-time image sequences of the surface of sand particles inside the sand box during the sand spreading process of the locomotive;

[0045] The image preprocessing module is used to enhance and dehaze the acquired raw sand grain surface image to obtain a high-quality sand grain surface image.

[0046] The 3D point cloud reconstruction module is used to perform adaptive 3D point cloud reconstruction of the sand grain surface from the obtained high-quality sand grain surface image.

[0047] The sand-spreading consumption volume calculation module is used to calculate the sand-spreading consumption volume based on the three-dimensional point cloud reconstruction results of the sand grain surface.

[0048] The fault classification and handling module is used to classify faults of the sand spreading device and optimize driver operation strategies based on the volume of sand consumption.

[0049] According to a third aspect, one embodiment provides an electronic device, the device comprising: a processor and a memory;

[0050] The memory is used to store one or more program instructions;

[0051] The processor is configured to run one or more program instructions to perform the steps of a real-time monitoring method for locomotive sand spreading status based on multi-vision volume measurement as described in any of the preceding claims.

[0052] According to a fourth aspect, one embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in any of the preceding claims.

[0053] This invention provides a method and system for real-time monitoring of locomotive sand spreading status based on multi-vision volume measurement, which has the following beneficial effects:

[0054] 1. This invention uses multi-view images from the top of the sand spreading box as monitoring signals. Through image enhancement processing and a deep learning-based defogging method, it can adaptively achieve high-fidelity sand particle distribution images, making monitoring data acquisition more convenient and reliable, without the need for additional equipment for data acquisition.

[0055] 2. This invention employs a deep learning-based multi-view adaptive 3D point cloud reconstruction method. It does not require obtaining actual camera parameters and position / pose information, and can accurately establish the correspondence between pixels from different viewpoints from any number of view photographs. It then integrates filtering, point cloud density enhancement, and the Alpha-shape algorithm to generate a high-fidelity 3D point cloud of sand flow, achieving accurate calculation of the sand consumption volume. This method, through adaptive acquisition of point cloud reconstruction information, significantly reduces the complexity of manually calibrating camera parameters and avoids problems such as decreased reconstruction accuracy due to human measurement errors, effectively improving the accuracy and adaptability of sand volume calculation.

[0056] 3. This invention, through quantitative analysis of sand-spreading consumption volume and fault classification of sand-spreading equipment based on deep learning methods, identifies and promptly warns of sand-spreading anomalies. This effectively avoids the problem of reduced locomotive adhesion caused by insufficient sand spreading due to equipment failure, and also avoids the problems of reduced production capacity and wasted labor resources caused by excessive maintenance. This will effectively guide the daily maintenance operation of sand-spreading equipment and reduce maintenance costs. Furthermore, this invention can effectively guide locomotive drivers' operating strategies under special working conditions based on real-time monitoring of sand-spreading consumption volume and operating condition information, ensuring the long-term safe and stable operation of the locomotive. Attached Figure Description

[0057] Figure 1 A flowchart illustrating a method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement, as provided in an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of the distribution of multi-view cameras and the installation position of LED lights inside the locomotive sand box in a real-time monitoring method for locomotive sand spreading status based on multi-view volume measurement provided in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the structure of a real-time monitoring model for locomotive sand spreading status in a method for real-time monitoring of locomotive sand spreading status based on multi-vision volume measurement, provided in an embodiment of the present invention.

[0060] Figure 4 This is a schematic diagram illustrating the calculation of locomotive sand-spreading volume consumption in a real-time monitoring method for locomotive sand-spreading status based on multi-vision volume measurement, provided in an embodiment of the present invention.

[0061] Figure 5 This is a schematic diagram of sand particle surface reconstruction in a real-time monitoring method for locomotive sand spreading status based on multi-vision volume measurement, provided in an embodiment of the present invention.

[0062] Figure 6 This is a flowchart illustrating the integration of the locomotive sand spreading system and the real-time sand spreading status monitoring program in a locomotive sand spreading status real-time monitoring method based on multi-vision volume measurement, provided as an embodiment of the present invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0064] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0065] The first embodiment of this invention provides a real-time monitoring method for locomotive sand spreading status based on multi-vision volume measurement. The following is in conjunction with... Figure 1 and Figure 3 Please provide a detailed explanation.

[0066] like Figure 1 As shown, in step S100, a sequence of images of the surface of sand particles in the sand box during the sand spreading process of the locomotive is acquired in real time.

[0067] In this embodiment, the image sequence of sand particles on the surface inside the sand box during the locomotive sand spreading process is obtained by deploying high-resolution industrial cameras at different angles on the top of the locomotive sand spreading box, such as... Figure 2 As shown. The camera utilizes a 6-megapixel global shutter CMOS sensor (model: Hikvision MV-CA060-10GC), supporting a dynamic frame rate of 17fps and IP30 protection. Its operating temperature ranges from 0℃ to 50℃, making it suitable for the high-temperature heating requirements of the heating rods in the sandbox, as well as the vibration environment of locomotives. A dust cover is added to protect the lens. The light source system employs NVC adjustable spectrum LED strips, continuously and evenly arranged below the camera. PWM pulse modulation enables dynamic color temperature adjustment to eliminate sand reflection interference and compensate for low-light environments. Through multi-directional camera deployment and supplementary light sources, the three-dimensional morphology of the sand flow can be fully captured, ensuring clear image texture.

[0068] like Figure 1As shown, in step S200, the acquired sand grain surface image is enhanced and dehazed to obtain a high-quality sand grain surface image.

[0069] In this embodiment, the image enhancement process includes adding Gaussian noise, adjusting brightness, and random exposure processing. The probability density function of the Gaussian noise is defined as: (1)

[0070] In the formula: Let be the probability density function of Gaussian noise; Let be a random variable, representing the value of the noise; The standard deviation controls the width of the Gaussian distribution; The mean;

[0071] In this embodiment, the defogging process preferably employs the UniDet-D sand and fog removal model based on deep learning, such as... Figure 3 As shown, this model, through end-to-end feature learning, specifically eliminates scattering noise from suspended particles in sand flow imaging, significantly improving the visibility of sand grain surface texture. This processing allows the image to fully preserve the spatial structural features of sand grain distribution, providing a high-fidelity input source for subsequent 3D point cloud reconstruction and enhancing the model's monitoring robustness.

[0072] It should be noted that the Gaussian noise injection, brightness adjustment, and random exposure processing in the above embodiments are merely exemplary technical means. Any enhancement method that can improve image quality and meet the technical objectives is applicable to this invention. Furthermore, the UniDet-D model is a preferred solution rather than a limiting choice; any dehazing algorithm that can effectively suppress medium scattering effects and enhance the discernibility of target features is also applicable to this invention.

[0073] like Figure 1 As shown, in step S300, the obtained high-quality sand grain surface image is subjected to adaptive three-dimensional point cloud reconstruction of the sand grain surface.

[0074] In this embodiment, a deep learning model and three-dimensional geometric calculation methods are used to achieve adaptive three-dimensional point cloud reconstruction of the dehazed sand particle distribution image. The adaptive three-dimensional point cloud reconstruction process is completed as follows:

[0075] First, the deep learning model adopts the "camera pose adaptive capture" model. The camera pose adaptive capture model does not need to explicitly rely on the traditional camera physical calibration parameters and position pose. It can autonomously establish the correspondence between multiple pixels across the viewpoint and fully realize the "self-supervised pose-free" multi-view accurate camera pose capture. The specific optimization process is as follows: (1) Use the powerful vision basic model Vision Transformer (ViT) to extract features from each pose-free image, match and aggregate the multi-view features, and construct a hard three-dimensional latent voxel; (2) Introduce Plück coordinates to encode the position and orientation of the camera to more comprehensively represent the spatial position information, and use the self-attention mechanism to adaptively refine the correspondence in the latent voxel. At the same time, based on the shared features, use a lightweight MLP network to regress the camera pose of each input view relative to a normal space (with the first input view as a reference).

[0076] Then, based on accurate multi-view stereo matching, a sparse point cloud is generated. Statistical filtering, radius filtering and other filtering methods are used to reduce point cloud noise. At the same time, downsampling is used to reduce redundancy or upsampling (K-nearest neighbor difference) is used to increase the integrity of local details and improve reconstruction accuracy.

[0077] Finally, the Alpha-shape algorithm is used to extract the outer contour of the point cloud and combined with Poisson reconstruction to generate an implicit surface, thereby achieving high-precision reconstruction of the sand grain surface.

[0078] It should be noted that the adaptive 3D point cloud reconstruction in the above embodiments is only an exemplary technical means, and any method that can achieve adaptive capture of camera parameters and attitude and 3D point cloud reconstruction is applicable to the present invention.

[0079] like Figure 1 As shown, in step S400, the volume of sand consumed is calculated based on the three-dimensional point cloud reconstruction results of the sand grain surface.

[0080] In this embodiment, the state of the sand box before sand application is used as the reference plane. The volume difference between the reconstructed sand surface point cloud and the reference plane is calculated in real time and used as the volume of sand application consumption. The volume calculation diagram is shown below. Figure 4 As shown, the formula is as follows: (2)

[0081] In the formula: This refers to the volume of sand consumed during this sand spreading operation. This represents the volume of space between the surface of the sand grains and the highest sand loading position before sand spreading (at time t1); This represents the volume of space between the surface of the sand grains and the highest sand loading position after sand is spread (at time t2). The coordinates of the sand grain surface before sand application (at time t1); Let be the coordinates of the sand grain surface after sand application (at time t2). The sand grain surface coordinates in the formula refer to the set of coordinates of n points on the surface of the sand grains in the sand box at time t1 or t2, such as... Figure 5 As shown, by collecting the coordinate values ​​of these n points, the upper surface of the remaining sand in the sand box at that moment is reconstructed, thereby completing the calculation of the remaining sand volume at that moment.

[0082] like Figure 1 As shown, in step S500, the sand spreading device malfunctions are classified and the driver's operation strategy is optimized based on the sand spreading consumption volume.

[0083] In this embodiment, the fault classification of the sand-spreading device is implemented using a parallel residual network based on depthwise separable convolution (DWConv). The DWConv network effectively extracts key sensitive features, reduces computational resource consumption, and simultaneously utilizes the deep residual structure to address the gradient explosion and model performance degradation problems that easily occur in deep training. The DWConv parallel residual convolution network is completed through the following sub-steps:

[0084] First, multi-branch deep residual operations are performed in parallel to extract different feature information at different depths from each branch. A single-branch deep residual network can be represented as: (3)

[0085] In the formula, It is the real-time monitoring of sand consumption volume. It is the output after passing through several separable depthwise convolutional layers (DWConv), batch normalization, and ReLU activation layers. Y represents the residual connection (element-wise addition), and Y is the feature extraction output of a single depth residual block.

[0086] Subsequently, feature fusion is performed on the information from each branch. The Softmax function is used to obtain the weights of each branch, which are then assigned to each branch to obtain the fused features. : (4) (5)

[0087] In the formula: This is the feature extraction output for the m-th depth residual branch; The attention weights for each branch, and m represents the number of branches in the deep residual structure.

[0088] Finally, based on the feature fusion results, a single-layer fully connected network and the Softmax function are used to classify the faults of the sand spreading device.

[0089] It should be noted that the above-mentioned multi-branch parallel deep residual network preferably has a parallel structure of three branches, and the depthwise separable convolution kernel size of each branch is 3×1. Although the convolution kernel size of each branch is the same, the convolution kernel value will dynamically change during network training and inference, extracting different fault feature information. In addition, the number of branches in the parallel structure and the size of the convolution kernel of each branch will affect the time and accuracy of fault classification. Besides the preferred scheme in this embodiment, multi-branch parallel deep residual networks with other parameters can also be used according to the classification time and accuracy requirements, and are equally applicable to this invention.

[0090] In this embodiment, the classification rule for sand spreading device malfunctions is based on the percentage of actual sand spreading consumption (V) relative to the standard specification consumption (V0), as follows:

[0091] According to TB / T 3254-2019 "Sand Spreading Device for Locomotives and Rolling Stock", the standard consumption (V0) of the sand spreading device for locomotives should be 0.3 L / min to 0.7 L / min.

[0092] When V equals V0, the sand spreading device is functioning normally and the sand spreading degree is excellent.

[0093] When 75% V0 < V ≤ 100% V0, the sand spreading device has a minor malfunction, but the sand spreading degree is good.

[0094] When 50% V0 < V ≤ 75% V0, the sand spreading device is moderately faulty, and the sand spreading degree is poor.

[0095] When V≤50%V0, the sand spreading device is seriously malfunctioning and cannot operate normally.

[0096] In addition, by repeatedly monitoring the volume of sand consumption and the fault information of the locomotive sand spreading equipment, a non-linear judgment relationship between the volume of sand consumption and equipment faults is established. Subsequently, the cause of equipment faults can be directly determined based on the volume of sand consumption, and corresponding solutions can be taken in a timely manner.

[0097] In this embodiment, the sand-spreading effect under the operating condition can be evaluated in real time based on the sand-spreading consumption volume and locomotive operating condition information, and sand-spreading operation optimization suggestions or early warning information can be generated, such as... Figure 6 As shown. The joint integration process of the locomotive sand spreading system and the real-time sand spreading status monitoring program is as follows:

[0098] First, the locomotive driver operates the sand-spreading command device according to external conditions, issues the initial sand-spreading command and starts the real-time sand-spreading status monitoring program, and controls the pressurization of the sand-spreading pipeline through the sand-spreading control module and the monitoring program to achieve initial sand-spreading and sand-spreading status monitoring.

[0099] Then, the sand spreading status monitoring program calculates the volume of sand consumed in the current sand spreading operation, and determines whether the sand spreading equipment is faulty based on the volume of sand consumed. If the equipment is faulty, it is diagnosed and repaired; if the equipment is normal, it is jointly controlled with the locomotive operating condition information collected by other external sensors of the locomotive, and proposes operation optimization suggestions or early warning information under the operating condition, and feeds back to the locomotive driver.

[0100] Subsequently, the driver determines whether the locomotive is in a safe operating condition based on the feedback results. If it is in a safe condition, the driver operates the sand-spreading command device, issues a sand-spreading stop command, and stops the real-time sand-spreading status monitoring program. If it is in an unsafe operating condition, the driver continues to operate the sand-spreading command device, issues a sand-spreading command, and starts the real-time sand-spreading status monitoring program to complete the rail sand-spreading and sand-spreading status monitoring. Through the above continuous feedback, the locomotive's safe operation is ultimately achieved, and multiple sand-spreading status monitoring operations are completed.

[0101] Finally, the volume data of sand consumption for each application, the results of working condition diagnosis, and the operation suggestions are output to the vehicle display unit or maintenance system in real time for data backup and subsequent adjustment and optimization.

[0102] It should be noted that the preferred locomotive operating condition information includes track conditions, traction / braking commands, locomotive speed, sandblasting pipe pressure, and weather conditions. Other richer operating condition information, including but not limited to this preferred scheme, is also applicable to this invention.

[0103] In summary, this invention enables real-time, high-precision, non-contact measurement of the volume of sand consumed by locomotives, classification of sand-spreading device faults, and optimization of locomotive driver operating strategies under special operating conditions. Compared with existing technologies, this invention represents a significant advancement.

[0104] Corresponding to the above-disclosed method for real-time monitoring of locomotive sand-spreading status based on multi-view volume measurement, this invention also discloses a system for real-time monitoring of locomotive sand-spreading status based on multi-view volume measurement, which specifically includes:

[0105] The image acquisition module is used to acquire real-time image sequences of the surface of sand particles inside the sand box during the sand spreading process of the locomotive;

[0106] The image preprocessing module is used to enhance and dehaze the acquired raw sand grain surface image to obtain a high-quality sand grain surface image.

[0107] The 3D point cloud reconstruction module is used to perform adaptive 3D point cloud reconstruction of the sand grain surface from the obtained high-quality sand grain surface image.

[0108] The sand-spreading consumption volume calculation module is used to calculate the sand-spreading consumption volume based on the three-dimensional point cloud reconstruction results of the sand grain surface.

[0109] The fault classification and handling module is used to classify faults of the sand spreading device and optimize driver operation strategies based on the volume of sand consumption.

[0110] It should be noted that for a detailed description of the locomotive sand-spreading status real-time monitoring system based on multi-vision volume measurement provided in the embodiments of the present invention, please refer to the relevant description of the locomotive sand-spreading status real-time monitoring method based on multi-vision volume measurement provided in the embodiments of the present invention, which will not be repeated here.

[0111] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0112] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement, characterized in that, The method includes: Real-time acquisition of image sequences of sand particles on the surface inside the sand box during the sand spreading process of the locomotive; Image enhancement and dehazing were performed on the original sand grain surface images to obtain high-quality sand grain surface images; Adaptive 3D point cloud reconstruction of the sand grain surface is performed on the obtained high-quality sand grain surface image; Based on the three-dimensional point cloud reconstruction results of the sand grain surface, the volume of sand consumption is calculated. Based on the volume of sand consumption, classify sand spreading device malfunctions and optimize driver operation strategies.

2. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Real-time acquisition of image sequences of sand particles on the surface inside the sand box during the locomotive sand spreading process, specifically including: High-resolution industrial cameras at different angles are deployed on top of the sand spreading box of the locomotive to capture image sequences of the surface of sand particles inside the sand spreading box during the sand spreading process, and multiple LED lights provide supplementary light sources for the cameras.

3. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Image enhancement and dehazing processes were performed on the acquired sand grain surface images to obtain high-quality sand grain surface images, specifically including: Image enhancement processing includes adding Gaussian noise, adjusting brightness, and random exposure processing.

4. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Image enhancement and dehazing processes were performed on the acquired sand grain surface images to obtain high-quality sand grain surface images, specifically including: A deep learning model is used to perform dehazing on the sand grain surface image after image enhancement.

5. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, The obtained high-quality sand grain surface image is subjected to adaptive 3D point cloud reconstruction of the sand grain surface, specifically including: Based on a deep learning model, the correspondence between multiple pixels across different viewpoints is established autonomously to achieve self-supervised, pose-free, multi-view accurate camera pose capture. Then, based on accurate multi-view stereo matching, a sparse point cloud is generated, and the point cloud noise is reduced by filtering. At the same time, downsampling is used to reduce redundancy or upsampling is used to increase the integrity of local details and improve the reconstruction accuracy. Finally, the Alpha-shape algorithm is used to extract the outer contour of the point cloud and combined with Poisson reconstruction to generate an implicit surface, thus achieving high-precision reconstruction of the sand grain surface.

6. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Based on the 3D point cloud reconstruction results of the sand grain surface, the volume of sand consumption is calculated, specifically including: Using the state of the sand box before sand application as the reference plane, the volume difference between the sand grain surface reconstructed from the point cloud and the reference plane is calculated in real time and used as the volume of sand consumption. The formula is as follows: ； In the formula: This refers to the volume of sand consumed during sand spreading. Before spreading sand The volume of space between the surface of the sand grains and the highest sand loading position at any given time; After sprinkling sand The volume of space between the surface of the sand grains and the highest sand loading position at any given time; Before spreading sand The surface coordinates of the sand grain at that moment; After sprinkling sand The coordinates of the sand grain surface at that moment.

7. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Based on the volume of sand consumption, classify sand spreading device malfunctions and optimize driver operation strategies, specifically including: A parallel residual network based on depthwise separable convolution is used for fault classification of the sand spreading device, as detailed below: First, multi-branch deep residual operations are performed in parallel to extract different feature information at different depths in each branch. The single-branch deep residual network is represented as follows: ； In the formula, It is the real-time monitoring of sand consumption volume. It is the output after passing through several separable depthwise convolutional layers, batch normalization, and activation layers. Represents residual connectivity. It is the feature extraction output of a single depth residual block; Subsequently, feature fusion is performed on the information from each branch. The Softmax function is used to obtain the weights of each branch and assign them to each branch to obtain the fused features. : ； ； In the formula: This is the feature extraction output for the m-th depth residual branch; The attention weights for each branch, and m represents the number of branches in the deep residual structure. Finally, based on the feature fusion results, a single-layer fully connected network and the Softmax function are used to classify the faults of the sand spreading device.

8. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Based on the volume of sand consumption, classify sand spreading device malfunctions and optimize driver operation strategies, specifically including: Fault classification is determined based on the percentage of actual sand consumption V relative to the set standard consumption V0, as follows: When V=V0, the sand spreading device is working properly and the sand spreading degree is excellent; When 75% V0 < V ≤ 100% V0, the sand spreading device has a minor malfunction, but the sand spreading degree is good. When 50% V0 < V ≤ 75% V0, the sand spreading device is moderately faulty, and the sand spreading degree is poor. When V≤50%V0, the sand spreading device is seriously malfunctioning and cannot operate normally.

9. The method for real-time monitoring of locomotive sand-spreading status based on multi-vision volume measurement as described in claim 1, characterized in that, Based on the volume of sand consumption, classify sand spreading device malfunctions and optimize driver operation strategies, specifically including: Based on the real-time sand-spreading consumption volume and locomotive operating condition information, the sand-spreading effect under the corresponding operating conditions is evaluated, and sand-spreading operation optimization suggestions or early warning information are generated. The sand-spreading consumption data, operating condition diagnosis results and operation suggestions under the corresponding operating conditions are output to the on-board display unit or maintenance system in real time. Among them, the locomotive operating condition information includes track conditions, traction / braking commands, locomotive speed, sandblasting pipe pressure and weather conditions.

10. A real-time monitoring system for locomotive sand-spreading status based on multi-vision volume measurement, characterized in that, The system includes: The image acquisition module is used to acquire real-time image sequences of the surface of sand particles inside the sand box during the sand spreading process of the locomotive; The image preprocessing module is used to enhance and dehaze the acquired raw sand grain surface image to obtain a high-quality sand grain surface image. The 3D point cloud reconstruction module is used to perform adaptive 3D point cloud reconstruction of the sand grain surface from the obtained high-quality sand grain surface image. The sand-spreading consumption volume calculation module is used to calculate the sand-spreading consumption volume based on the three-dimensional point cloud reconstruction results of the sand grain surface. The fault classification and handling module is used to classify faults of the sand spreading device and optimize driver operation strategies based on the volume of sand consumption.

Images