Heavy-load train coupler force measurement method and system based on machine vision

Abstract

The embodiment of the present invention provides a method and system for measuring the coupler force of a heavy-load train based on machine vision, which belongs to the field of railway train operation monitoring. The method includes: collecting an image of the coupler area of ​​a heavy-load train, and performing decontamination and enhancement processing on the coupler area image in combination with working condition information to obtain an enhanced coupler area image; performing coupler feature target recognition on the enhanced coupler area image based on an improved DEIM‑DFine network model trained with exclusive data enhancement, and obtaining the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; based on the pixel coordinates of the coupler feature target and the corresponding working condition information, using the CNN‑ResMLP network model to perform quantitative identification of the coupler force to obtain a coupler force identification result. The solution of the present invention improves the accuracy and physical rationality of coupler force identification, thereby effectively supporting the stable measurement and monitoring needs of the coupler force of heavy-load trains.

Description

Technical Field

[0001] The present invention relates to the field of railway train operation detection, and in particular to a heavy-load train coupler force measurement method based on machine vision and a heavy-load train coupler force measurement system based on machine vision. Background Art

[0002] With the continuous advancement of railway transportation equipment and technology, heavy-haul trains have become an important means of transporting bulk materials such as minerals and energy. Due to their long train lengths and heavy loads, heavy-haul trains subject their coupler systems to significant longitudinal traction and impact forces during operation. These forces also vary in complexity, which can lead to safety hazards such as coupler failure, vehicle uncoupling, and cargo shifting. Therefore, accurately determining the force status of couplers during operation is crucial for ensuring the safe operation of heavy-haul trains and improving transportation efficiency.

[0003] Currently, coupler force detection relies on data collection using strain sensors embedded within the coupler or mechanical sensors installed between the train bodies. However, these detection methods often suffer from the following issues: Firstly, the coupler structure is complex, installation space is limited, and the long-term operating environment is harsh. Sensors are prone to wear and drift, resulting in high maintenance costs. Secondly, due to the limitations of sensor installation location, traditional detection methods often fail to fully reflect the complex stress state and dynamic changes of the coupler, making them difficult to adapt to the high-precision, long-term force monitoring requirements of the complex operating conditions of heavy-load trains.

[0004] At the same time, some current force detection methods based on video monitoring or image recognition rely on manual feature extraction or fixed algorithms for analysis. Faced with the harsh operating conditions frequently encountered during train operation, such as oil pollution, vibration, strong light, rain and snow, image quality degrades significantly, and target features are easily obscured or distorted, resulting in large fluctuations in detection accuracy and frequent false positives and omissions. Existing methods struggle to stably and accurately capture coupler force trends, especially under the challenging operating conditions of heavily loaded trains. This leads to problems such as an inability to reflect the coupler's operating status in real time and an inability to support safety warnings and scheduling decisions.

[0005] In summary, how to break through the environmental adaptability of coupler force monitoring technology under complex working conditions, reduce dependence on high-maintenance cost sensors, and improve the accuracy and reliability of coupler force monitoring are technical problems that urgently need to be solved in the current field of heavy-load train operation safety control. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a method and system for measuring the coupler force of a heavy-load train based on machine vision, so as to at least solve the problems of the existing coupler force detection technology under the complex working conditions of heavy-load trains, such as poor environmental adaptability, low monitoring stability, and dependence on sensors with high maintenance costs.

[0007] In order to achieve the above-mentioned objectives, the first aspect of the present invention provides a method for measuring the coupler force of a heavy-load train based on machine vision, the method comprising: collecting an image of the coupler area of ​​a heavy-load train, and performing decontamination and enhancement processing on the coupler area image in combination with working condition information to obtain an enhanced coupler area image; performing coupler feature target recognition on the enhanced coupler area image based on an improved DEIM-DFine network model trained with exclusive data enhancement, and obtaining the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; based on the pixel coordinates of the coupler feature target and the corresponding working condition information, using the CNN-ResMLP network model to perform quantitative identification of the coupler force to obtain a coupler force recognition result.

[0008] Optionally, the coupler area image is decontaminated and enhanced in combination with the working condition information to obtain an enhanced coupler area image, including: detecting the current running speed value of the coupler and the ambient light intensity value to determine the current working condition type based on the current running speed value of the coupler and the ambient light intensity value; selecting an image decontamination processing flow and an image enhancement processing flow according to the determined working condition type; executing the image decontamination processing flow to perform stain area detection and image restoration processing on the coupler area image; executing the image enhancement processing flow to perform any one or more of contrast enhancement processing, image sharpening processing, dehazing processing and noise reduction processing in the order of matching the determined working condition type.

[0009] Optionally, the enhanced coupler area image is taken as input, and coupler feature target recognition is performed on the enhanced coupler area image based on the improved DEIM-DFine network model trained with exclusive data enhancement; wherein, the improved DEIM-Dfine network model trained with exclusive data enhancement introduces a CARAFE upsampling module in the feature fusion part, wherein the CARAFE upsampling module includes a kernel prediction module and a feature reconstruction module, the kernel prediction module generates a position-related upsampling kernel based on the coupler area image, and the feature reconstruction module receives the upsampling kernel and performs content-aware upsampling processing on the coupler area image; based on the feature map after upsampling processing, coupler feature target recognition is performed in the improved DEIM-DFine network model trained with exclusive data enhancement to obtain preliminary pixel coordinates of the coupler feature target.

[0010] Optionally, before performing coupler feature target recognition on the enhanced coupler area image based on the improved DEIM-DFine network model that has been trained with exclusive data enhancement, the method also includes: executing training of the improved DEIM-DFine network model that has been trained with exclusive data enhancement, including: performing image complex working condition simulation enhancement processing on the enhanced historical coupler area image to generate processed image data; wherein the image complex working condition simulation enhancement processing includes any one or more of oil occlusion processing, dynamic blur processing, low illumination degradation processing, rain and snow cover processing, and vibration jitter simulation processing; adding the processed image data to the coupler feature target recognition training data set, and executing improved DEIM-DFine network model training based on the coupler feature target recognition training data set to obtain an improved DEIM-DFine network model that has been trained with exclusive data enhancement.

[0011] Optionally, after obtaining the preliminary pixel coordinates of the coupler feature target, the method further includes performing dynamic time series filtering on the preliminary pixel coordinates of the obtained coupler feature target; wherein the rules of dynamic time series filtering include: receiving continuous frame coupler area images, extracting preliminary pixel coordinates of the coupler feature target in each frame; calculating pixel coordinate changes between continuous frames, and performing weighted averaging processing, change amplitude filtering processing and outlier removal processing in sequence based on the calculation results of the pixel coordinate changes to generate pixel coordinates of the coupler feature target after dynamic time series filtering.

[0012] Optionally, based on the pixel coordinates of the coupler feature target and the corresponding working condition information, a CNN-ResMLP network model is used to perform quantitative identification of the coupler force to obtain a coupler force identification result, including: inputting the pixel coordinates of the coupler feature target and the corresponding working condition information as input data into the CNN-ResMLP network model for quantitative identification of the coupler force; wherein the CNN-ResMLP network model includes at least one convolution layer whose convolution kernel size is greater than a first convolution kernel size threshold, and at least one convolution layer whose convolution kernel size is less than a second convolution kernel size threshold, And a multi-layer perceptron module including a residual connection structure; wherein, the first convolution kernel size threshold is greater than the second convolution kernel size threshold; the convolution layer whose convolution kernel size is greater than the first convolution kernel size threshold is used to receive the pixel coordinates and coupler working condition information of the coupler feature target after dynamic temporal filtering processing, and perform global feature extraction; the convolution layer whose convolution kernel size is less than the second convolution kernel size threshold is used to perform local feature extraction; the multi-layer perceptron module performs feature fusion and nonlinear mapping based on the extracted global features and local features, and outputs the coupler force recognition result.

[0013] Optionally, the method further includes performing model training of a CNN-ResMLP network model.

[0014] Optionally, after obtaining the coupler force identification result, the method further includes: comparing the coupler force identification result with historical identification trend data, current coupler operating condition information and a theoretical reasonable range of coupler force calculated based on a coupler physical constraint model; when the comparison result deviates from a preset theoretical reasonable range, performing error filtering processing; wherein, the error filtering processing includes: performing error compensation processing on the coupler force identification result according to the comparison result, and generating a corrected coupler force identification result.

[0015] The second aspect of the present invention provides a heavy-load train coupler force measurement system based on machine vision, the system comprising: an acquisition unit for acquiring images of the coupler area of ​​a heavy-load train, and performing decontamination and enhancement processing on the coupler area image in combination with working condition information to obtain an enhanced coupler area image; a positioning unit for performing coupler feature target recognition on the enhanced coupler area image based on an improved DEIM-DFine network model trained with exclusive data enhancement, and obtaining the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; an identification unit for performing quantitative identification of the coupler force based on the pixel coordinates of the coupler feature target and the corresponding working condition information using a CNN-ResMLP network model to obtain a coupler force recognition result.

[0016] Through the above technical scheme, the scheme of the present invention can improve the image quality of the coupler feature target under complex working conditions by collecting the coupler area image and decontaminating and enhancing the coupler area image in combination with the working condition information; the improved DEIM-DFine network model that has been trained with exclusive data enhancement is used to identify the coupler feature target in the enhanced coupler area image, and the pixel coordinates of the coupler feature target are obtained in combination with dynamic time series filtering, which is helpful to stably extract the coupler feature information in a complex dynamic environment; based on the pixel coordinates and working condition information of the coupler feature target, the CNN-ResMLP network is used to quantitatively identify the coupler force, which can improve the accuracy and physical rationality of the coupler force identification, thereby effectively supporting the stable measurement and monitoring needs of the coupler force of heavy-load trains.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0019] Figure 1 This is a flowchart of a method for measuring the coupler force of a heavy-load train based on machine vision provided by one embodiment of the present invention;

[0020] Figure 2 1 is a schematic diagram of the installation position of a camera provided in one embodiment of the present invention;

[0021] Figure 3 It is a schematic structural diagram of an improved DEIM-DFine network model provided by one embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of a CNN-ResMLP network model provided by one embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of coupler force identification results provided by one embodiment of the present invention;

[0024] Figure 6 This is a system structure diagram of a heavy-load train coupler force measurement system based on machine vision provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0026] Figure 1 This is a flowchart of a method for measuring the coupler force of a heavy-load train based on machine vision provided by one embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a method for measuring the coupler force of a heavy-load train based on machine vision, the method comprising:

[0027] Step S10: collecting a coupler area image of a heavy-load train, and performing decontamination and enhancement processing on the coupler area image in combination with working condition information to obtain an enhanced coupler area image.

[0028] Specifically, the coupler area image is decontaminated and enhanced in combination with the working condition information to obtain an enhanced coupler area image, including: detecting the current running speed value of the coupler and the ambient light intensity value to determine the current working condition type based on the current running speed value of the coupler and the ambient light intensity value; selecting an image decontamination processing flow and an image enhancement processing flow according to the determined working condition type; executing the image decontamination processing flow to perform stain area detection and image restoration processing on the coupler area image; executing the image enhancement processing flow to perform any one or more of contrast enhancement processing, image sharpening processing, dehazing processing and noise reduction processing in the order of matching the determined working condition type.

[0029] In practical applications, the acquisition of images of the coupler area of ​​heavy-load trains is limited by the complex and ever-changing train operating environment. The coupler area is often affected by factors such as oil, mud, rain and snow cover, strong light, low light, and vibration, which greatly reduce image clarity and target recognizability. Therefore, the collected coupler area images usually need to undergo certain decontamination and enhancement processing to ensure that the subsequent image recognition algorithm can obtain clear, accurate, and usable coupler feature target information. In this embodiment, to improve the targetedness and adaptability of image processing, auxiliary judgment of coupler operating condition information is introduced during the decontamination and enhancement process to dynamically adjust the image processing process and parameters to better adapt to different operating conditions.

[0030] Specifically, when performing decontamination and enhancement processing on the coupler area image, it is first necessary to detect the current operating speed value of the coupler and the light intensity value in the acquisition environment. The speed value can be obtained through the train control system or the speed measuring device, while the light intensity value can be obtained by the ambient light sensor integrated in the camera device or the image automatic exposure feedback information. Based on the obtained speed value and light intensity value, the current operating environment can be divided into multiple combined working condition types such as low speed and low light, high speed and low light, low speed and strong light, and high speed and strong light according to the preset working condition type classification rules. Each working condition type corresponds to a preset image processing strategy table, so that the corresponding image decontamination processing process and image enhancement processing process are dynamically matched according to the current working condition type.

[0031] In the specific image decontamination processing, stain area detection is usually performed first. The stain or occlusion area in the coupler area image is identified by methods based on color distribution, texture change, morphological features, etc., and image restoration processing is performed on the identified stain area. For example, an algorithm based on image texture reconstruction or an image restoration method based on context completion is used for restoration to reduce the impact of stain occlusion on the coupler target area.

[0032] After the decontamination process is completed, the image enhancement process phase begins. In this phase, one or more of the following processes, including contrast enhancement, image sharpening, defogging, and noise reduction, is performed on the coupler area image in the order of processing that matches the current working condition type. Specifically, contrast enhancement processing can use histogram equalization or adaptive contrast enhancement algorithm to improve the clarity of the coupler target boundary; image sharpening processing can use methods based on high-pass filtering or gradient enhancement to enhance the edge information of the target area; defogging processing can use transmittance estimation and inversion based on the atmospheric scattering model to reduce the image graying phenomenon caused by haze; noise reduction processing can use methods based on non-local mean, bilateral filtering, or deep learning denoising networks to reduce image noise interference. The activation, execution order, and parameter settings of the above-mentioned processing procedures are dynamically adjusted according to the working condition type to ensure that the coupler area image after image processing has sufficient clarity, stability, and feature fidelity to meet the input requirements of the subsequent coupler feature target recognition link.

[0033] Step S20: performing coupler feature target recognition on the enhanced coupler area image based on the improved DEIM-DFine network model trained with exclusive data enhancement, and acquiring the pixel coordinates of the coupler feature target in combination with dynamic temporal filtering.

[0034] Specifically, the enhanced coupler area image is taken as input, and coupler feature target recognition is performed on the enhanced coupler area image based on the improved DEIM-DFine network model trained with exclusive data enhancement; wherein, the improved DEIM-Dfine network model trained with exclusive data enhancement introduces a CARAFE upsampling module in the feature fusion part, wherein the CARAFE upsampling module includes a kernel prediction module and a feature reconstruction module, the kernel prediction module generates a position-related upsampling kernel based on the coupler area image, the feature reconstruction module receives the upsampling kernel and performs content-aware upsampling processing on the coupler area image; based on the feature map after upsampling processing, coupler feature target recognition is performed in the improved DEIM-DFine network model trained with exclusive data enhancement to obtain the preliminary pixel coordinates of the coupler feature target.

[0035] Furthermore, the rules for obtaining the pixel coordinates of the coupler feature target in combination with dynamic time series filtering include: receiving continuous frame coupler area images, extracting the preliminary pixel coordinates of the coupler feature target in each frame; calculating the pixel coordinate changes between continuous frames, and performing weighted averaging processing, change amplitude filtering processing and abnormal point elimination processing in sequence based on the calculation results of the pixel coordinate changes to generate the pixel coordinates of the coupler feature target after dynamic time series filtering processing.

[0036] In the implementation of the present invention, after the decontamination and enhancement processing of the coupler area image is completed, in order to further accurately extract the coupler feature target information from the clear and stable image, the enhanced coupler area image can be used to identify the coupler feature target based on the improved DEIM-DFine network model that has been trained with exclusive data enhancement. This network model has been adapted based on DEIM-DFine and has been trained and optimized by introducing a specially constructed coupler scene data set to enhance the target detection capability under complex working condition images. Specifically, the enhanced coupler area image is used as input to the improved DEIM-DFine network model, which applies the CARAFE upsampling module in the feature fusion process, thereby improving the problems of information loss and insufficient spatial resolution in the traditional upsampling process.

[0037] The CARAFE upsampling module integrates two submodules: a kernel prediction module and a feature reconstruction module. First, the kernel prediction module uses lightweight convolution and feature encoding to generate a unique upsampling kernel for each pixel position based on the input enhanced coupler region image. This upsampling kernel is dynamically generated and has content-aware properties corresponding to the pixel location. This avoids the edge blurring and detail loss problems associated with traditional fixed interpolation methods. The feature reconstruction module then receives the upsampling kernel output by the kernel prediction module and uses this kernel to perform content-aware upsampling on the low-resolution feature map of the input image to generate a more context-aware high-resolution feature map. This ensures that the coupler feature region in the image retains structural integrity and texture clarity after upsampling. Based on this high-resolution feature map, coupler feature target recognition is performed in the object detection branch and keypoint detection branch of the improved DEIM-DFine network model, obtaining preliminary pixel coordinates of the coupler feature target, ensuring an accurate foundation for subsequent processing.

[0038] Furthermore, to mitigate the jitter and fluctuation of target pixel coordinates in coupler images during heavy-haul train operation caused by factors such as vibration, jitter, and illumination variations, and to prevent unstable or even distorted recognition results between consecutive frames, a dynamic time-series filtering strategy can be used to further process the pixel coordinates of coupler feature targets. The specific rules include: first, receiving multiple consecutive frames of coupler region images and extracting preliminary pixel coordinates of the coupler feature targets in each frame to generate pixel coordinate time series data. Next, the pixel coordinate changes between consecutive frames are calculated, including parameters such as the pixel position change amplitude, position change speed, and change trend. Based on the calculated pixel coordinate change results, weighted averaging, change amplitude filtering, and outlier removal are sequentially performed. The weighted averaging process smoothes the pixel coordinates of consecutive frames using a certain weight to reduce jumps caused by vibration. The change amplitude filtering process sets a change threshold to filter out anomalous data points with change amplitudes exceeding a reasonable range. The outlier removal process eliminates anomalous coordinates with sudden changes based on the difference between the historical trajectory and the current frame data, further improving the continuity and stability of the pixel coordinates. After the above-mentioned dynamic time series filtering processing, the pixel coordinates of the coupler feature target that are smoothed and time-series continuous can be obtained, which significantly improves the stability and reliability of subsequent quantitative identification of coupler force.

[0039] This coupler feature target recognition and dynamic time series filtering processing process, by combining image space feature extraction, time series dynamic stabilization and other technical paths, effectively improves the availability, stability and robustness of the pixel coordinates of the coupler feature target under complex working conditions, making the entire coupler force recognition chain have stronger anti-interference ability and recognition accuracy.

[0040] Preferably, before performing coupler feature target recognition on the enhanced coupler area image based on the improved DEIM-DFine network model that has been trained with exclusive data enhancement, the method also includes: executing training of the improved DEIM-DFine network model that has been trained with exclusive data enhancement, including: performing image complex working condition simulation enhancement processing on the enhanced historical coupler area image to generate processed image data; wherein the image complex working condition simulation enhancement processing includes any one or more of oil occlusion processing, dynamic blur processing, low illumination degradation processing, rain and snow cover processing and vibration jitter simulation processing; adding the processed image data to the coupler feature target recognition training data set, and executing improved DEIM-DFine network model training based on the coupler feature target recognition training data set to obtain the improved DEIM-DFine network model that has been trained with exclusive data enhancement.

[0041] In this embodiment of the present invention, to ensure that the improved DEIM-DFine network model maintains high recognition accuracy and environmental adaptability when faced with complex operating condition images of heavy-load trains, it is preferred that dedicated data augmentation training of the network model be performed before formally performing coupler feature target recognition on the enhanced coupler area images. This training process primarily involves constructing a training dataset with rich operating condition characteristics and performing targeted training and optimization on the improved DEIM-DFine network model based on this dataset to enhance its robust recognition capability for coupler feature targets.

[0042] Specifically, historically collected images of the coupler area are first enhanced by simulating complex working conditions. This process aims to simulate the various interference factors commonly found in the coupler area during actual operation. By artificially constructing data with image interference conditions such as oil occlusion, motion blur, low-light degradation, rain and snow cover, and vibration, the training dataset is expanded to include a wider range of working conditions, enabling the network model to learn richer scene features, thereby enhancing generalization and interference resistance. Specifically, oil stain occlusion processing can simulate the presence of oil stains, mud and other materials on the coupler surface by randomly superimposing irregular stain textures or opaque occlusion areas; dynamic blur processing can simulate the image smearing or blurring caused by high-speed train running through image blur processing based on motion convolution kernel; low-light degradation processing uses the method of reducing brightness and enhancing noise to simulate low-light scenes at night, in tunnels or in bad weather; rain and snow cover processing constructs the image of the coupler area in rainy and snowy weather by superimposing visual elements such as snowflakes, raindrops, and ice debris, and combining a certain degree of image blur and light distortion; vibration jitter simulation processing can simulate the problem of reduced image stability during severe vibration of the train by introducing micro-displacement disturbances, image jitter, offset and other means.

[0043] The above-mentioned image complex working condition simulation enhancement processing can select one or more processing methods for combined application to generate processed image data with complex working condition characteristics. Subsequently, these processed image data are added to the coupler feature target recognition training data set, so that the entire training data set covers a more comprehensive coupler working condition scenario. Based on the training data set, the training process of the improved DEIM-DFine network model is executed, which specifically includes feature extraction, target detection, loss function calculation, gradient backpropagation, parameter optimization and other steps until the network model converges, and an improved DEIM-DFine network model that has been trained with exclusive data enhancement is obtained. The trained model can still have a strong coupler feature target recognition capability when facing multiple complex interferences in the coupler area image, and can accurately locate the pixel coordinates of the coupler target, laying a reliable foundation for the subsequent force recognition process.

[0044] Step S30: Based on the pixel coordinates of the coupler feature target and the corresponding working condition information, the CNN-ResMLP network model is used to perform quantitative identification of the coupler force to obtain a coupler force identification result.

[0045] Specifically, the pixel coordinates of the coupler feature target and the corresponding working condition information are used as input data and input into the CNN-ResMLP network model for quantitative identification of coupler force; wherein, the CNN-ResMLP network model includes at least one convolution layer with a convolution kernel size greater than a first convolution kernel size threshold, at least one convolution layer with a convolution kernel size less than a second convolution kernel size threshold, and a multi-layer perceptron module including a residual connection structure; the first convolution kernel size threshold is greater than the second convolution kernel size threshold; the convolution layer with a convolution kernel size greater than the first convolution kernel size threshold is used to receive the pixel coordinates of the coupler feature target and the coupler working condition information after dynamic temporal filtering processing, and perform global feature extraction; the convolution layer with a convolution kernel size less than the second convolution kernel size threshold is used to perform local feature extraction; the multi-layer perceptron module is based on the global features and the local features, and performs feature fusion and nonlinear mapping to output the coupler force identification result.

[0046] In this embodiment of the present invention, after obtaining the pixel coordinates of the coupler's characteristic targets after dynamic time-series filtering and combining them with the corresponding coupler operating condition information, the CNN-ResMLP network model can be used to quantitatively identify the coupler force. The input data here primarily consists of two components: the pixel coordinates of the coupler's characteristic targets after dynamic time-series filtering, and the operating condition information corresponding to the coupler's state at that moment, including but not limited to vehicle speed, acceleration, traction status, braking status, etc. These input data are combined and fed as features into the CNN-ResMLP network model for quantitative coupler force identification.

[0047] The CNN-ResMLP network model's structure has been specifically designed and optimized, primarily consisting of: at least one convolutional layer with a kernel size greater than a first kernel size threshold, at least one convolutional layer with a kernel size less than a second kernel size threshold, and a multilayer perceptron (MLP) module with residual connections. The first kernel size threshold is greater than the second kernel size threshold. This means that the combination of large and small kernel convolutional layers achieves a complementary combination of global and local feature extraction in coupler force recognition scenarios.

[0048] Specifically, convolutional layers with kernel sizes larger than the first kernel size threshold primarily process the pixel coordinates and operating condition information of the coupler's feature targets, extracting global features from this data. These features primarily reflect the spatial distribution of the coupler's overall force state, the reasonable range of coupler force under the mechanical constraint model, and the force variations under different operating conditions. This convolutional layer covers the entire input feature map through a larger receptive field, enhancing understanding of the coupler's global force pattern.

[0049] Meanwhile, convolutional layers with kernel sizes smaller than the second kernel size threshold focus on extracting local features, including detailed changes in the pixel coordinates of key coupler points in the image, sudden changes in local forces, and small-scale differences in force details caused by changes in working conditions. These convolutional layers use a smaller receptive field to extract fine-grained features from the input, ensuring that key local information is not missed during coupler force identification.

[0050] The output features of the above two convolutional layers will be extracted and retained in parallel, and merged before entering the multi-layer perceptron module. The multi-layer perceptron module is a standard MLP structure, but a residual connection structure is introduced on this basis. The main purpose of this design is to alleviate the gradient vanishing problem that may occur in deep networks while maintaining the ability to extract deep features, while improving the model's ability to retain feature information and avoid information loss during deep transmission. In this module, global features and local features are fused through feature splicing or weighted fusion, and combined with nonlinear mapping functions to achieve a comprehensive understanding of the input data and high-dimensional feature space projection, and finally output the coupler force recognition results.

[0051] This process fully leverages the spatial information carried by the pixel coordinates of the coupler's characteristic targets, as well as the environmental constraints provided by the operating conditions. By combining this with supervised optimization using the coupler's physical constraint model, the CNN-ResMLP network model accurately and stably performs quantitative identification of coupler forces. Overall, this process effectively addresses the challenges of traditional methods in coupler force identification under complex operating conditions, including poor robustness and high accuracy fluctuations. It also possesses significant engineering application value and potential for widespread adoption.

[0052] Preferably, the method further comprises performing model training of the CNN-ResMLP network model.

[0053] In one possible implementation, before performing quantitative coupler force identification, the method further includes training the CNN-ResMLP network model. As the core model for coupler force identification, the training process of the CNN-ResMLP network model directly determines the accuracy, robustness, and adaptability of subsequent identification results to complex operating conditions. Therefore, prior to model deployment, establishing a rational training process to fully exploit the structural and operating condition characteristics of the training data is crucial for improving overall identification performance.

[0054] Specifically, the model training phase uses data samples containing the pixel coordinates of the coupler feature targets and the corresponding working condition information as input, and the labeled values ​​of the coupler force as the training target, and optimizes the network parameters through supervised learning. During the training process, the different structural layers of the CNN-ResMLP network undertake different tasks: the convolutional layer is used to extract the spatial structural features of the input data, including the distribution changes of the coupler image coordinates in different frames, the pixel offset trend under mechanical action, etc.; the MLP module is responsible for nonlinear mapping and high-dimensional feature space modeling, so as to fully capture the deep correlation in the input data. The training process adopts the standard forward propagation, loss function calculation, backpropagation and parameter update mechanism, and continues to iterate until the model converges on the validation set.

[0055] In terms of training data preparation, we cover a variety of typical coupler operating conditions, such as image samples at different speeds, lighting conditions, and different traction or braking states, to ensure the model has good generalization capabilities. Regularization techniques such as batch normalization and dropout can be introduced during training to prevent model overfitting. For optimizer selection, mainstream optimizers such as Adam and SGD can be used, and hyperparameters such as learning rate and weight decay can be appropriately set to ensure that the model achieves high recognition accuracy while maintaining training speed.

[0056] Through the aforementioned model training process, the CNN-ResMLP network learns the deep mapping relationship between coupler pixel spatial variations and coupler forces, thereby improving its ability to predict coupler forces under unknown operating conditions. The trained model can be directly used to quantitatively identify coupler forces based on actual operating data, demonstrating high practicality and engineering deployment value.

[0057] In a possible implementation, the method for measuring the coupler force of a heavy-load train based on machine vision proposed in the present invention is implemented based on the following process.

[0058] S1: Collect images of the coupler area of ​​heavy-load trains, annotate and train the coupler force-related features of the coupler area images, and obtain a visual detection model for coupler feature targets.

[0059] In a specific embodiment, Figure 2The image of the coupler area of ​​a heavy-load train is obtained by a camera installed on the car body directly above the coupler. For the acquired coupler area image, the coupler feature target is annotated and the visual detection model is trained. After the training is completed, for each frame of newly input coupler area image, the visual detection model will identify and locate the coupler feature target and return the pixel coordinates of the coupler feature. It should be noted that the image obtained directly above is more complete and facilitates the identification of coupler feature targets. However, the camera can also be installed in other positions where the complete coupler can be captured.

[0060] S2: Perform coupler feature target recognition on the pre-processed coupler area image to obtain the pixel coordinates of the coupler.

[0061] In one specific embodiment, a deep learning model is used to identify coupler features from preprocessed images of the coupler region. It should be noted that identifying target features from images is a prior art technique, and the deep learning model used in this embodiment is merely a preferred method for identifying coupler features from images. Other prior art methods for identifying coupler features from images are also applicable to the present invention.

[0062] In a specific embodiment, the deep learning model adopts an improved DEIM-DFine network model, such as Figure 3 The improved DEIM-DFine network model is based on the DEIM-DFine network model. In its feature fusion part, the CARAFE upsampling module is used to replace the original neighbor value upsampling, thereby improving the model's edge and fine-grained perception capabilities.

[0063] In the above embodiment, the CARAFE upsampling module is introduced on the basis of the DEIM-DFine network model. The CARAFE upsampling module is an existing technology, which realizes efficient upsampling of feature maps in a content-aware manner. This module is mainly composed of two key components, namely the kernel prediction module and the feature reconstruction module. The kernel prediction module is used to generate position-related upsampling kernels, and the feature reconstruction module uses the prediction kernel to perform content-aware upsampling operations. Compared with traditional fixed upsampling methods such as bilinear interpolation, CARAFE can adaptively predict the optimal upsampling kernel based on the content of the input features, thereby better preserving image details and semantic information. Among them, the kernel prediction module can be mainly divided into two steps: feature kernelization and reconstruction and normalization, which can be expressed by the following formula:

[0064] ;

[0065] in, represents the final upsampling kernel after normalization, Represents the original feature map of the input.

[0066] The feature reorganization module mainly reorganizes the upsampled feature map based on the obtained upsampled kernel, which can be expressed by the following formula:

[0067] ;

[0068] in, Indicates location The corresponding local neighborhood, represents the sampling points in the neighborhood, and Represent the input and output feature maps respectively.

[0069] It should be noted that the above-mentioned improved DEIM-DFine network model is only a preferred deep learning model of the present invention. Other target detection deep learning models that can accurately identify coupler features and other deep learning models can also be applied to the present invention.

[0070] S3: Quantitatively identify coupler forces using a deep learning model based on the pixel coordinates of coupler features.

[0071] In a specific embodiment, it is necessary to first obtain the measured coupler force and the pixel coordinates of the corresponding coupler feature. The pixel coordinates of the coupler feature are obtained using the aforementioned visual inspection model. The measured coupler force and the corresponding coupler feature pixel coordinates are used as the training set to train the deep learning model. After the model training is complete, for each new set of coupler feature pixel coordinates, a corresponding coupler force will be quantitatively identified and output.

[0072] In a specific embodiment, a deep learning model is used to quantitatively identify the coupler force of a heavy-load train. In this embodiment, a CNN-ResMLP network model is used to quantitatively identify the coupler force. The network structure of the CNN-ResMLP network model is as follows: Figure 4 As shown, in the network design, the pixel horizontal and vertical coordinates of two coupler feature targets are input. These are first passed through a large-kernel convolutional layer module to extract global information, followed by a small-kernel convolutional module to exploit local information associations between the data, achieving multi-scale feature extraction. Finally, a residual-based MLP module fuses deep information while preserving shallow-level information, outputting the final coupler force. Compared to other methods, the CNN-ResMLP network model can simultaneously consider global associations and local details, while also offering high computational efficiency, making it more suitable for quantitative coupler force identification.

[0073] In a specific embodiment, using a heavy-haul train as an example, the machine vision-based coupler force measurement method described in the present invention was used to calculate the identified coupler force value, and the actual coupler force value was obtained through strain gauge measurements on the heavy-haul train. In this embodiment, a camera was mounted on the vehicle body directly above the coupler. A modified DEIM-Dfine network model was used to identify coupler features from preprocessed images of the coupler area, obtaining the coupler's pixel coordinates. Based on these coupler pixel coordinates, the CNN-ResMLP network model was used to identify the coupler force of the heavy-haul train.

[0074] In this embodiment, the identification result of the coupler force is as follows: Figure 5 As shown. Figure 5 It can be seen that the identification value calculated by the present invention is very consistent with the true value measured by the strain gauge. The present invention can accurately and quantitatively identify the coupler force of a heavy-load train.

[0075] Figure 6 This is a system structure diagram of a heavy-load train coupler force measurement system based on machine vision provided by one embodiment of the present invention. Figure 6 As shown, an embodiment of the present invention provides a heavy-load train coupler force measurement system based on machine vision, and the system includes: an acquisition unit, used to acquire a coupler area image of a heavy-load train, and perform decontamination and enhancement processing on the coupler area image in combination with working condition information to obtain an enhanced coupler area image; a positioning unit, used to perform coupler feature target recognition on the enhanced coupler area image based on an improved DEIM-DFine network model trained with exclusive data enhancement, and obtain the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; an identification unit, used to perform quantitative identification of the coupler force based on the pixel coordinates of the coupler feature target and the corresponding working condition information using a CNN-ResMLP network model to obtain a coupler force recognition result.

[0076] Those skilled in the art will appreciate that all or part of the steps in the methods described in the aforementioned embodiments can be performed by instructing the relevant hardware through a program. The program, stored in a storage medium, includes instructions for causing a microcontroller, chip, or processor to execute all or part of the steps in the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0077] The above describes in detail the optional embodiments of the present invention in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, a variety of simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the scope of protection of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will no longer describe the various possible combinations separately.

[0078] In addition, the various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A method for measuring the coupler force of a heavy-load train based on machine vision, characterized in that: The method comprises: Collect the image of the coupler area of ​​the heavy-load train, and perform decontamination and enhancement processing on the image of the coupler area in combination with the working condition information to obtain the enhanced image of the coupler area; Based on the improved DEIM-DFine network model trained with exclusive data enhancement, coupler feature target recognition is performed on the enhanced coupler area image, and pixel coordinates of the coupler feature target are obtained in combination with dynamic time series filtering; Based on the pixel coordinates of the coupler feature target and the corresponding working condition information, the CNN-ResMLP network model is used to perform quantitative identification of the coupler force to obtain the coupler force identification result.

2. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 1, characterized in that: The coupler area image is decontaminated and enhanced in combination with the working condition information to obtain an enhanced coupler area image, including: Detecting a current running speed value of the coupler and an ambient light intensity value, so as to determine a current working condition type based on the current running speed value of the coupler and the ambient light intensity value; Select the image decontamination processing process and image enhancement processing process according to the determined working condition type; Executing the image decontamination process to perform stain area detection and image restoration processing on the coupler area image; The image enhancement processing flow is executed to sequentially perform any one or more of contrast enhancement processing, image sharpening processing, defogging processing and noise reduction processing in the order of matching the determined working condition types.

3. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 1, characterized in that: The enhanced coupler area image is subjected to coupler feature target recognition based on the improved DEIM-DFine network model trained with exclusive data enhancement, including: The enhanced coupler area image is used as input, and the coupler feature target recognition is performed on the enhanced coupler area image based on the improved DEIM-DFine network model trained with exclusive data enhancement; wherein, The improved DEIM-Dfine network model, which has undergone exclusive data augmentation training, introduces a CARAFE upsampling module in the feature fusion part, wherein the CARAFE upsampling module includes a kernel prediction module and a feature reconstruction module. The kernel prediction module generates a position-dependent upsampling kernel based on the coupler area image, and the feature reconstruction module receives the upsampling kernel and performs content-aware upsampling processing on the coupler area image. Based on the upsampled feature map, coupler feature target recognition is performed in the improved DEIM-DFine network model trained with exclusive data enhancement to obtain the preliminary pixel coordinates of the coupler feature target.

4. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 3, characterized in that: Before performing coupler feature target recognition on the enhanced coupler region image based on the improved DEIM-DFine network model trained with dedicated data augmentation, the method further includes: executing training of the improved DEIM-DFine network model trained with dedicated data augmentation, including: The enhanced coupler area image is subjected to image complex working condition simulation enhancement processing to generate processed image data; wherein, The image complex working condition simulation enhancement processing includes any one or more of oil occlusion processing, dynamic blur processing, low illumination degradation processing, rain and snow cover processing, and vibration jitter simulation processing; The processed image data is added to the coupler feature target recognition training dataset, and the improved DEIM-DFine network model training is performed based on the coupler feature target recognition training dataset to obtain the improved DEIM-DFine network model that has undergone exclusive data augmentation training.

5. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 3, characterized in that: The rules for obtaining the pixel coordinates of the coupler feature target in combination with dynamic temporal filtering include: Receive continuous frame coupler area images and extract preliminary pixel coordinates of the coupler feature target in each frame; Calculate the pixel coordinate changes between consecutive frames, and based on the calculation results of the pixel coordinate changes, perform weighted averaging processing, change amplitude filtering processing and abnormal point elimination processing in sequence to generate the pixel coordinates of the coupler feature target after dynamic time series filtering processing.

6. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 1, characterized in that: Based on the pixel coordinates of the coupler feature target and the corresponding working condition information, the CNN-ResMLP network model is used to quantitatively identify the coupler force, and the coupler force identification results are obtained, including: The pixel coordinates of the coupler feature target and the corresponding working condition information are used as input data and input into the CNN-ResMLP network model for quantitative identification of coupler force; wherein, The CNN-ResMLP network model includes at least one convolutional layer whose convolution kernel size is larger than a first convolution kernel size threshold, at least one convolutional layer whose convolution kernel size is smaller than a second convolution kernel size threshold, and a multilayer perceptron module including a residual connection structure; The first convolution kernel size threshold is greater than the second convolution kernel size threshold; the convolution layer with a convolution kernel size greater than the first convolution kernel size threshold is used to receive the pixel coordinates of the coupler feature target and the coupler working condition information after dynamic temporal filtering, and perform global feature extraction; the convolution layer with a convolution kernel size less than the second convolution kernel size threshold is used to perform local feature extraction; The multi-layer perceptron module performs feature fusion and nonlinear mapping based on the extracted global features and local features, and outputs the coupler force recognition result.

7. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 1, characterized in that: The method also includes: performing model training of the CNN-ResMLP network model.

8. The method for measuring the coupler force of a heavy-load train based on machine vision according to claim 1, characterized in that: After obtaining the coupler force identification result, the method further includes: Compare the coupler force identification results with historical identification trend data, current coupler operating condition information, and the theoretical reasonable range of coupler force calculated based on the coupler physical constraint model; When the comparison result deviates from a preset theoretical reasonable range, error filtering processing is performed; wherein, the error filtering processing includes: performing error compensation processing on the coupler force identification result according to the comparison result, and generating a corrected coupler force identification result.

9. A heavy-load train coupler force measurement system based on machine vision, characterized in that: The system comprises: An acquisition unit is used to acquire an image of the coupler area of ​​a heavy-load train, and perform decontamination and enhancement processing on the image of the coupler area in combination with working condition information to obtain an enhanced image of the coupler area; A positioning unit is used to identify the coupler feature target on the enhanced coupler area image based on the improved DEIM-DFine network model trained with exclusive data enhancement, and obtain the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; The identification unit is used to perform quantitative identification of the coupler force based on the pixel coordinates of the coupler feature target and the corresponding working condition information using the CNN-ResMLP network model to obtain a coupler force identification result.

Images