Machine vision-based heavy haul train coupler force measurement method and system

Abstract

The embodiment of the invention provides a heavy haul train coupler force measuring method and system based on machine vision, and belongs to the field of railway train operation monitoring. The method comprises the following steps: collecting a coupler area image of the heavy haul train, and carrying out decontamination and enhancement processing on the coupler area image in combination with working condition information to obtain an enhanced coupler area image; on the basis of an improved DEIM-DFine network model subjected to exclusive data enhancement training, carrying out coupler feature target identification on the enhanced coupler region image, and obtaining pixel coordinates of a coupler feature target in combination with dynamic time sequence filtering; and based on the pixel coordinates of the coupler feature target and the corresponding working condition information, performing coupler force quantitative identification by using a CNN-ResMLP network model to obtain a coupler force identification result. According to the scheme, the accuracy and physical rationality of coupler force recognition are improved, and therefore the requirement for stable measurement and monitoring of the coupler force of the heavy-load train is effectively met.

Description

Technical Field

[0001] The present invention relates to the field of railway train operation detection. Specifically, it relates to a method for measuring the coupler force of a heavy-haul train based on machine vision and a system for measuring the coupler force of a heavy-haul train based on machine vision. Background Art

[0002] With the continuous progress of railway transportation equipment and technology, heavy-haul trains have become an important means for transporting bulk materials such as minerals and energy. Due to their long formation length and large load capacity, the longitudinal traction force and impact force borne by the coupler system during operation are relatively large, and the force changes are complex. In severe cases, it is easy to cause safety hazards such as coupler failure, vehicle uncoupling, and cargo offset. Therefore, accurately obtaining the force state of the coupler during operation is an important basis for ensuring the safe operation of heavy-haul trains and improving transportation efficiency.

[0003] Currently, the detection of coupler force mostly relies on strain sensors buried inside the coupler or mechanical sensing devices installed between car bodies for data collection. However, such detection means generally have the following problems: on the one hand, the coupler structure is complex, the installation space is limited, the long-term operating environment is harsh, the sensors are prone to wear and drift, and the maintenance cost is high; on the other hand, limited by the installation position of the sensors, traditional detection means often have difficulty in comprehensively reflecting the complex force state and dynamic change process of the coupler, and it is difficult to meet the requirements of high-precision and long-term force monitoring under the complex working conditions of heavy-haul trains.

[0004] At the same time, current force detection methods based on video monitoring or image recognition mostly rely on manual feature extraction or fixed algorithms for analysis. In the face of harsh working conditions such as oil stains, vibrations, strong lights, rain and snow that frequently occur during train operation, the image quality drops significantly, the target features are easily blocked or distorted, resulting in large fluctuations in detection accuracy and frequent false alarms and missed detections. Especially under the special working conditions of heavy-haul trains, the existing methods are difficult to stably and accurately obtain the change trend of coupler force, and there are problems such as being unable to reflect the operating state of the coupler in real time and being unable to support safety warning and dispatching decisions.

[0005] In summary, how to break through the environmental adaptability of coupler force monitoring technology under complex working conditions, reduce the dependence on high-maintenance-cost sensors, and improve the accuracy and reliability of coupler force monitoring is a technical problem that urgently needs to be solved in the current field of heavy-haul train operation safety control. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a method and system for measuring the coupler force of a heavy-haul train based on machine vision, so as to at least solve the problems of poor environmental adaptability, low monitoring stability, and dependence on high-maintenance-cost sensors existing in the existing coupler force detection technology under the complex working conditions of heavy-haul trains.

[0007] To achieve the above object, a first aspect of the present invention provides a method for measuring the coupler force of a heavy-haul train based on machine vision. The method includes: collecting images of the coupler area of the heavy-haul train, and performing decontamination and enhancement processing on the coupler area images in combination with the working condition information to obtain enhanced coupler area images; performing coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation, and obtaining the pixel coordinates of the coupler feature targets by combining dynamic time series filtering; performing quantitative recognition of the coupler force using a CNN-ResMLP network model based on the pixel coordinates of the coupler feature targets and the corresponding working condition information to obtain a coupler force recognition result.

[0008] Optionally, performing decontamination and enhancement processing on the coupler area images in combination with the working condition information to obtain enhanced coupler area images includes: detecting the current operating speed value and the ambient light intensity value of the coupler to determine the current working condition type based on the current operating speed value and the ambient light intensity value of the coupler; selecting an image decontamination processing flow and an image enhancement processing flow according to the determined working condition type; executing the image decontamination processing flow to perform stain area detection and image restoration processing on the coupler area images; executing the image enhancement processing flow to sequentially perform any one or more of contrast enhancement processing, image sharpening processing, defogging processing, and noise reduction processing in the order matching the determined working condition type.

[0009] Optionally, taking the enhanced coupler area images as input, performing coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation; wherein, the improved DEIM-Dfine network model trained with exclusive data augmentation introduces a CARAFE upsampling module in the feature fusion part, and the CARAFE upsampling module includes a kernel prediction module and a feature recombination module. The kernel prediction module generates a position-related upsampling kernel based on the coupler area images, and the feature recombination module receives the upsampling kernel and performs content-aware upsampling processing on the coupler area images; performing coupler feature target recognition in the improved DEIM-DFine network model trained with exclusive data augmentation on the feature map after upsampling processing to obtain the preliminary pixel coordinates of the coupler feature targets.

[0010] Optionally, before identifying the coupler feature target from the enhanced coupler region image based on the improved DEIM-DFine network model trained with exclusive data augmentation, the method further includes: performing training on the improved DEIM-DFine network model trained with exclusive data augmentation, including: performing image complex working condition simulation enhancement processing on the enhanced historical coupler region images to generate processed image data; wherein, the image complex working condition simulation enhancement processing includes any one or more of oil stain occlusion processing, motion blur processing, low illuminance degradation processing, rain and snow coverage processing, and vibration and jitter simulation processing; adding the processed image data to the coupler feature target recognition training dataset, and performing training on the improved DEIM-DFine network model based on the coupler feature target recognition training dataset to obtain the improved DEIM-DFine network model trained with exclusive data augmentation.

[0011] Optionally, after obtaining the preliminary pixel coordinates of the coupler feature target, the method further includes performing dynamic time-series filtering processing on the obtained preliminary pixel coordinates of the coupler feature target; wherein, the rules of the dynamic time-series filtering processing include: receiving consecutive frames of coupler region images, and extracting the preliminary pixel coordinates of the coupler feature target for each frame; calculating the pixel coordinate changes between consecutive frames, and sequentially performing weighted average processing, change amplitude filtering processing, and outlier removal processing based on the calculation results of the pixel coordinate changes to generate the pixel coordinates of the coupler feature target after dynamic time-series filtering processing.

[0012] Optionally, based on the pixel coordinates of the coupler feature target and the corresponding working condition information, using the CNN-ResMLP network model to perform quantitative identification of the coupler force to obtain the coupler force identification result, including: using the pixel coordinates of the coupler feature target and the corresponding working condition information as input data, and inputting them into the CNN-ResMLP network model for quantitative identification of the coupler force; wherein, the CNN-ResMLP network model includes at least one convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold, at least one convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold, and a multi-layer perceptron module including a residual connection structure; wherein, the first convolutional kernel size threshold is greater than the second convolutional kernel size threshold; the convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold is used to receive the pixel coordinates of the coupler feature target after dynamic time-series filtering processing and the coupler working condition information, and perform global feature extraction; the convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold is used to perform local feature extraction; the multi-layer perceptron module performs feature fusion and non-linear mapping based on the extracted global features and local features, and outputs the coupler force identification result.

[0013] Optionally, the method further includes performing model training on the CNN-ResMLP network model. Optionally, after obtaining the coupler force recognition result, the method further includes: comparing the coupler force recognition result with historical recognition trend data, current coupler working condition information, and the theoretically reasonable range of coupler force calculated based on the coupler physical constraint model; when the comparison result deviates from the preset theoretically reasonable range, performing error filtering processing; wherein, the error filtering processing includes: performing error compensation processing on the coupler force recognition result according to the comparison result, and generating a corrected coupler force recognition result.

[0014] The second aspect of the present invention provides a machine vision-based heavy-haul train coupler force measurement system, the system includes: an acquisition unit, configured to acquire images of the coupler area of the heavy-haul train, and perform decontamination and enhancement processing on the coupler area images in combination with the working condition information to obtain enhanced coupler area images; a positioning unit, configured to perform coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation, and obtain the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; an identification unit, configured to perform quantitative identification of the coupler force based on the pixel coordinates of the coupler feature target and the corresponding working condition information by using a CNN-ResMLP network model to obtain a coupler force recognition result.

[0015] Through the above technical solutions, the solution of the present invention can improve the image quality of coupler feature targets under complex working conditions by acquiring coupler area images and performing decontamination and enhancement processing on the coupler area images in combination with the working condition information; using an improved DEIM-DFine network model trained with exclusive data augmentation to perform coupler feature target recognition on the enhanced coupler area images, and obtaining the pixel coordinates of the coupler feature target in combination with dynamic time series filtering helps to stably extract coupler feature information in a complex dynamic environment; based on the pixel coordinates of the coupler feature target and the working condition information, using a CNN-ResMLP network to perform quantitative identification of the coupler force can improve the accuracy and physical rationality of coupler force recognition, thereby effectively supporting the stable measurement and monitoring requirements of the coupler force of heavy-haul trains.

[0016] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific embodiments section. Brief Description of the Drawings

[0017] The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification, and are used together with the following specific embodiments to explain the embodiments of the present invention, but do not constitute a limitation to the embodiments of the present invention. In the drawings: Figure 1 is a flowchart of the steps of a machine vision-based heavy-haul train coupler force measurement method provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the installation position of a camera provided by an embodiment of the present invention; Figure 3 It is a schematic structural diagram of an improved DEIM-DFine network model provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of a CNN-ResMLP network model provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of the coupler force identification result provided by an embodiment of the present invention; Figure 6 It is a system structure diagram of a coupler force measurement system for heavy-haul trains based on machine vision provided by an embodiment of the present invention. Specific Embodiments

[0018] The following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.

[0019] Figure 1 It is a step flow chart of a coupler force measurement method for heavy-haul trains based on machine vision provided by an embodiment of the present invention. As Figure 1 shown, an embodiment of the present invention provides a coupler force measurement method for heavy-haul trains based on machine vision, and the method includes: Step S10: Collect an image of the coupler area of the heavy-haul train, and perform decontamination and enhancement processing on the coupler area image in combination with the working condition information to obtain an enhanced coupler area image.

[0020] Specifically, performing decontamination and enhancement processing on the coupler area image in combination with the working condition information to obtain an enhanced coupler area image includes: detecting the current running speed value and the ambient light intensity value of the coupler to determine the current working condition type based on the current running speed value and the ambient light intensity value of the coupler; selecting an image decontamination processing flow and an image enhancement processing flow according to the determined working condition type; executing the image decontamination processing flow to perform stain area detection and image restoration processing on the coupler area image; executing the image enhancement processing flow to sequentially perform any one or more of contrast enhancement processing, image sharpening processing, defogging processing, and noise reduction processing in the order matching the determined working condition type.

[0021] In practical applications, for the acquisition of images of the coupler area of heavy-haul trains, due to the complex and changeable train operation environment, the coupler area is often interfered by factors such as oil stains, mud, rain and snow coverage, strong light, low light, vibration, etc. These factors greatly reduce the image clarity and target recognizability. Therefore, the acquired images of the coupler area usually need to undergo certain decontamination and enhancement processing to ensure that subsequent image recognition algorithms can obtain clear, accurate and usable coupler feature target information. In this embodiment, in order to improve the pertinence and adaptability of image processing, the auxiliary judgment of the coupler working condition information is introduced during the decontamination and enhancement processing to dynamically adjust the image processing process and parameters to better adapt to different working condition environments.

[0022] Specifically, when performing decontamination and enhancement processing on the coupler area image, it is first necessary to detect the current operating speed value of the coupler and the light intensity value in the acquisition environment. The speed value can be obtained through the train control system or the speed measurement device, while the light intensity value can be obtained by the ambient light sensor integrated in the camera device or the image automatic exposure feedback information. Based on the obtained speed value and light intensity value, the current operating environment can be divided into multiple combined working condition types such as low speed and low light, high speed and low light, low speed and strong light, high speed and strong light, etc. according to the preset working condition type division rules. Each working condition type corresponds to a preset image processing strategy table, so as to dynamically match the corresponding image decontamination processing process and image enhancement processing process according to the current working condition type.

[0023] In the specific image decontamination processing, usually the stain area detection is preferentially performed. By methods based on color distribution, texture change, morphological features, etc., the stain or occlusion area in the coupler area image is identified, and image restoration processing is performed on the identified stain area. For example, an algorithm based on image texture reconstruction or an image restoration method based on context completion is used for restoration to reduce the impact of stain occlusion on the coupler target area.

[0024] After the decontamination process is completed, the image enhancement process stage is entered. In this stage, according to the processing sequence matched by the current working condition type, one or more of contrast enhancement processing, image sharpening processing, dehazing processing, and noise reduction processing are performed on the coupler area image. Specifically, for contrast enhancement processing, histogram equalization or adaptive contrast enhancement algorithms can be used to improve the clarity of the coupler target boundary; for image sharpening processing, methods based on high-pass filtering or gradient enhancement can be used to strengthen the edge information of the target area; for dehazing processing, transmittance estimation and inversion can be performed based on the atmospheric scattering model to reduce the image graying phenomenon caused by haze; for noise reduction processing, methods based on non-local means, bilateral filtering, or deep learning denoising networks can be used to reduce image noise interference. The enabling, execution order, and parameter setting of the above processing flow are dynamically adjusted according to the working condition type to ensure that the coupler area image after image processing has sufficient clarity, stability, and feature fidelity to meet the input requirements of the subsequent coupler feature target recognition link.

[0025] Step S20: Based on the improved DEIM-DFine network model trained with exclusive data augmentation, perform coupler feature target recognition on the enhanced coupler area image, and obtain the pixel coordinates of the coupler feature target by combining dynamic temporal filtering.

[0026] Specifically, take the enhanced coupler area image as the input, and perform coupler feature target recognition on the enhanced coupler area image based on the improved DEIM-DFine network model trained with exclusive data augmentation; among them, the improved DEIM-Dfine network model trained with exclusive data augmentation introduces a CARAFE upsampling module in the feature fusion part, and the CARAFE upsampling module includes a kernel prediction module and a feature recombination module. The kernel prediction module generates a position-related upsampling kernel based on the coupler area image, and the feature recombination module receives the upsampling kernel and performs content-aware upsampling processing on the coupler area image; perform coupler feature target recognition on the feature map after upsampling processing in the improved DEIM-DFine network model trained with exclusive data augmentation to obtain the preliminary pixel coordinates of the coupler feature target.

[0027] Furthermore, the rules for obtaining the pixel coordinates of the coupler feature target by combining dynamic temporal filtering include: receiving consecutive frame coupler area images, extracting the preliminary pixel coordinates of the coupler feature target in each frame; calculating the pixel coordinate changes between consecutive frames, and sequentially performing weighted average processing, change amplitude filtering processing, and outlier removal processing based on the calculation results of the pixel coordinate changes to generate the pixel coordinates of the coupler feature target after dynamic temporal filtering processing.

[0028] In the implementation of the present invention, after the decontamination and enhancement processing of the coupler area image is completed, in order to further accurately extract the coupler feature target information from the clear and stable image, the improved DEIM-DFine network model trained by exclusive data augmentation can be used to identify the coupler feature target in the enhanced coupler area image. This network model is adaptively improved on the basis of DEIM-DFine and is trained and optimized by introducing a specially constructed coupler scenario dataset to improve the target detection ability under complex working condition images. Specifically, the above-enhanced coupler area image is used as the input and input into the improved DEIM-DFine network model. This network model applies the CARAFE upsampling module in the feature fusion process, thereby improving the problems of information loss and insufficient spatial resolution in the traditional upsampling process.

[0029] Among them, the CARAFE upsampling module internally integrates two sub-modules: a kernel prediction module and a feature recombination module. First, the kernel prediction module will generate an exclusive upsampling kernel for each pixel position based on the input enhanced coupler area image by using lightweight convolution and feature encoding methods. The upsampling kernel is dynamically generated and has content-aware characteristics corresponding to the pixel position. In this way, problems such as edge blurring and detail loss existing in the traditional fixed interpolation method can be avoided. Then, the feature recombination module receives the upsampling kernel output by the kernel prediction module and performs content-aware upsampling operations on the low-resolution feature map of the input image to generate a high-resolution feature map with stronger context awareness ability, ensuring that the coupler feature area in the image can still maintain a complete structure and clear texture after upsampling. Based on this high-resolution feature map, the coupler feature target recognition is continued in the target detection branch and the key point detection branch of the improved DEIM-DFine network model, thereby obtaining the preliminary pixel coordinates of the coupler feature target and ensuring the accurate basis for subsequent processing.

[0030] Furthermore, in order to overcome the jitter and fluctuation of the target pixel coordinates of the coupler image during the operation of the heavy-haul train due to factors such as vibration, jitter, and light change, and to avoid unstable or even offset and distorted recognition results between consecutive frames, a dynamic time-series filtering strategy can be combined to further process the pixel coordinates of the coupler feature target. The specific rules are as follows. First, receive consecutive multi-frame coupler region images, and respectively extract the preliminary pixel coordinates of the coupler feature target in each frame image to form pixel coordinate time-series data. Then, calculate the change in pixel coordinates between consecutive frames. The calculation process includes parameters such as the change amplitude of the pixel position, the position change speed, and the change trend. Based on the calculation results of the pixel coordinate change, weighted average processing, change amplitude filtering processing, and outlier removal processing are sequentially performed. Among them, the weighted average processing smooths the pixel coordinates of consecutive frames according to a certain weight to reduce the jump phenomenon caused by vibration; the change amplitude filtering processing sets a change threshold to filter out abnormal data points whose change amplitude exceeds a reasonable range; the outlier removal processing eliminates the abnormal coordinates of instantaneous mutations based on the difference between the historical trajectory and the current frame data to further improve the continuity and stability of the pixel coordinates. After the above dynamic time-series filtering processing, the pixel coordinates of the coupler feature target that are filtered, smoothed, and time-series continuous can be obtained, significantly improving the stability and reliability of subsequent coupler force quantitative recognition.

[0031] The coupler feature target recognition and dynamic time-series filtering processing flow effectively improves the usability, stability, and robustness of the pixel coordinates of the coupler feature target under complex working conditions by combining technical paths such as image spatial feature extraction and time-series dynamic stability, making the entire coupler force recognition chain have stronger anti-interference ability and recognition accuracy.

[0032] Preferably, before performing coupler feature target recognition on the enhanced coupler region image based on the improved DEIM-DFine network model trained with exclusive data augmentation, the method further includes: performing training on the improved DEIM-DFine network model trained with exclusive data augmentation, including: performing image complex working condition simulation enhancement processing on the enhanced historical coupler region images to generate processed image data; wherein, the image complex working condition simulation enhancement processing includes any one or more of oil stain occlusion processing, dynamic blur processing, low illuminance degradation processing, rain and snow coverage processing, and vibration and jitter simulation processing; adding the processed image data to the coupler feature target recognition training data set, and performing training on the improved DEIM-DFine network model based on the coupler feature target recognition training data set to obtain the improved DEIM-DFine network model trained with exclusive data augmentation.

[0033] In the embodiment of the present invention, in order to ensure that the improved DEIM-DFine network model can maintain a high recognition accuracy and environmental adaptability when facing the complex working condition images of heavy-loaded trains, preferably, before formally performing coupler feature target recognition on the enhanced coupler area image, exclusive data enhancement training of the network model is performed. The training process mainly includes constructing a training data set with rich working condition characteristics, and performing targeted training optimization on the improved DEIM-DFine network model based on the data set to enhance its robust recognition ability of coupler feature targets.

[0034] Specifically, based on the historically collected images of the coupler area, the complex working condition simulation enhancement processing can be performed first. The purpose of this processing process is to simulate the various interference factors commonly seen in the coupler area during actual operation, and to expand the working condition coverage of the training data set by artificially constructing data with image interference conditions such as oil occlusion, dynamic blur, low illumination degradation, rain and snow coverage, and vibration jitter, so that the network model can learn richer scene features, thereby having stronger generalization and anti-interference capabilities. Specifically, oil stain occlusion processing can simulate the presence of oil stains, mud and other materials on the surface of the coupler by randomly superimposing irregular stain textures or opaque occlusion areas; dynamic blur processing can simulate the image smear or blurring caused by high-speed train running through image blur processing based on motion convolution kernel; low-light degradation processing uses the method of reducing brightness and enhancing noise to simulate low-light scenes at night, in tunnels or in bad weather; rain and snow cover processing constructs the image of the coupler area in rainy and snowy weather by superimposing visual elements such as snowflakes, raindrops, ice debris, and combining a certain degree of image blur and light distortion; vibration jitter simulation processing can simulate the problem of reduced image stability during severe vibration of the train by introducing micro-displacement disturbances, image jitter, offset and other means.

[0035] The above-mentioned image complex working condition simulation enhancement processing can select one or more processing methods for combined application to generate processed image data with complex working condition characteristics. Subsequently, these processed image data are added to the coupler feature target recognition training data set, so that the entire training data set covers a more comprehensive coupler working condition scene. Based on the training data set, the training process of the improved DEIM-DFine network model is executed, specifically including feature extraction, target detection, loss function calculation, gradient backpropagation, parameter optimization and other steps, until the network model converges, and the improved DEIM-DFine network model that has been trained with exclusive data enhancement is obtained. The trained model can still have a strong coupler feature target recognition capability when facing a variety of complex interferences in the coupler area image, and can accurately locate the pixel coordinates of the coupler target, laying a reliable foundation for the subsequent force recognition process.

[0036] Step S30: Based on the pixel coordinates of the coupler feature target and the corresponding working condition information, use the CNN-ResMLP network model to quantitatively identify the coupler force and obtain the coupler force identification result.

[0037] Specifically, take the pixel coordinates of the coupler feature target and the corresponding working condition information as input data and input them into the CNN-ResMLP network model for quantitative identification of the coupler force. Among them, the CNN-ResMLP network model includes at least one convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold, at least one convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold, and a multi-layer perceptron module containing a residual connection structure. The first convolutional kernel size threshold is greater than the second convolutional kernel size threshold. The convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold is used to receive the pixel coordinates of the coupler feature target and the coupler working condition information after dynamic time series filtering and perform global feature extraction. The convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold is used to perform local feature extraction. The multi-layer perceptron module is based on the global feature and the local feature, and performs feature fusion and non-linear mapping to output the coupler force identification result.

[0038] In the embodiment of the present invention, after obtaining the pixel coordinates of the coupler feature target after dynamic time series filtering and combining the corresponding coupler working condition information, the coupler force can be quantitatively identified based on the CNN-ResMLP network model. The input data here mainly includes two parts. One is the pixel coordinates of the coupler feature target after dynamic time series filtering, and the other is the working condition information corresponding to the coupler state at this moment, specifically including but not limited to vehicle speed, acceleration, traction state, braking state, etc. Combine the above input data as features and input them into the CNN-ResMLP network model for quantitative identification of the coupler force.

[0039] The structure of the CNN-ResMLP network model is designed and optimized specifically, mainly consisting of the following parts: at least one convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold, at least one convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold, and a multi-layer perceptron (MLP) module containing a residual connection structure. Here, the first convolutional kernel size threshold is greater than the second convolutional kernel size threshold. That is to say, the combined use of the convolutional layer with a large-size convolutional kernel and the convolutional layer with a small-size convolutional kernel is to achieve the complementary combination of global feature extraction and local feature extraction in the coupler force identification scenario.

[0040] Specifically, the convolutional layer with a convolutional kernel size larger than the first convolutional kernel size threshold mainly functions to process the pixel coordinates of the coupler feature target and the working condition information, and extract the global feature information in this data. This part of the features mainly reflects the spatial distribution trend of the overall force state of the coupler, the reasonable force range of the coupler under the mechanical constraint model, and the force change law under different working conditions. This part of the convolutional layer covers the entire input feature map through a larger receptive field to enhance the understanding of the global force mode of the coupler.

[0041] At the same time, the convolutional layer with a convolutional kernel size smaller than the second convolutional kernel size threshold mainly focuses on the extraction of local features, which includes the detailed changes of the pixel coordinates of the key points of the coupler in the image, the sudden change of local force, and the detailed force differences caused by working condition changes in a small range. This part of the convolutional layer performs fine-grained feature extraction on the input features through a smaller receptive field to ensure that local key information is not missed during the coupler force recognition process.

[0042] The output features of the above two convolutional layers are extracted and retained in parallel and merged before entering the multi-layer perceptron module. The multi-layer perceptron module is a standard MLP structure, but on this basis, a residual connection structure is introduced. The main purpose of this design is to alleviate the possible vanishing gradient problem in the deep network while maintaining the deep feature extraction ability, and at the same time improve the model's ability to retain feature information and avoid information loss during the deep transmission process. In this module, the global features and local features are fused through methods such as feature concatenation or weighted fusion, and combined with the non-linear mapping function to achieve comprehensive understanding of the input data and high-dimensional feature space projection, and finally output the coupler force recognition result.

[0043] Through the above processing, the spatial information carried by the pixel coordinates of the coupler feature target and the environmental constraint information provided by the working condition information can be fully utilized, and supervised optimization is carried out by combining the coupler physical constraint model, so that the CNN-ResMLP network model can accurately and stably complete the quantitative recognition of the coupler force. Overall, this process can effectively solve the problems of poor robustness and large accuracy fluctuations of traditional methods in recognizing coupler forces under complex working conditions, and at the same time has good engineering application value and promotion potential.

[0044] Preferably, the method further includes performing model training on the CNN-ResMLP network model.

[0045] In a possible implementation, before performing the quantitative identification of the coupler force, the method further includes: training the CNN-ResMLP network model. The CNN-ResMLP network model, as the core model for carrying out the coupler force identification task, its training process directly determines the accuracy, robustness of the subsequent identification results, as well as the adaptability to complex working conditions. Therefore, before model deployment, by constructing a reasonable training process and fully excavating the structural features and working condition information features in the training data, it is an important guarantee for improving the overall identification performance.

[0046] Specifically, in the model training stage, data samples including the pixel coordinates of the coupler feature target and the corresponding working condition information are used as inputs, and the labeled value of the coupler force is used as the training target. The network parameters are optimized through the method of supervised learning. During the training process, different structural layers of the CNN-ResMLP network undertake different tasks: the convolutional layer is used to extract the spatial structure features of the input data, including the distribution changes of the coupler image coordinates in different frames, the pixel offset trend under mechanical action, etc.; the MLP module is responsible for non-linear mapping and high-dimensional feature space modeling to fully capture the deep correlations in the input data. The training process adopts the standard forward propagation, loss function calculation, backpropagation and parameter update mechanism, and continuously iterates until the model converges on the validation set.

[0047] In terms of training data preparation, cover a variety of typical coupler working conditions, such as image samples under different speeds, different illuminations, different traction or braking states, to ensure that the model has good generalization ability. Regularization techniques such as batch normalization and Dropout can be introduced during training to prevent the model from overfitting. In terms of optimizer selection, mainstream optimizers such as Adam and SGD can be used, and hyperparameters such as the learning rate and weight decay can be reasonably set to ensure that the model achieves a high recognition accuracy while ensuring the training speed.

[0048] Through the above model training process, the CNN-ResMLP network can learn the deep mapping relationship between the coupler pixel spatial changes and the coupler force, thereby improving its prediction ability for the coupler force under unknown working conditions. The trained model can be directly used for the quantitative identification of the coupler force in actual operation data, and has high practicality and engineering deployment value.

[0049] In a possible implementation, the method for measuring the coupler force of a heavy-haul train based on machine vision proposed by the solution of the present invention is implemented based on the following process.

[0050] S1: Collect images of the coupler area of the heavy-haul train, and perform label and training on the strong correlation features of the coupler force for the images of the coupler area to obtain a visual detection model for the coupler feature target.

[0051] In a specific embodiment, such as Figure 2, the image of the coupler area of the heavy-haul train is obtained by a camera installed on the car body directly above the coupler. For the obtained coupler area image, the coupler feature targets are labeled and a visual detection model is trained. After the training is completed, for each newly input coupler area image, the visual detection model will identify and locate the coupler feature targets and return the pixel coordinates of the coupler features. It should be noted that the picture obtained directly above is more complete and convenient for identifying the coupler feature targets. However, the camera can also be installed at other positions where a complete coupler can be photographed.

[0052] S2: Perform coupler feature target recognition on the preprocessed coupler area image to obtain the pixel coordinates of the coupler.

[0053] In a specific embodiment, a deep learning model is used to perform coupler feature target recognition on the preprocessed coupler area image. It should be noted that target feature recognition of images is a prior art. The deep learning model in this embodiment is only the preferred recognition method of the present invention, and other methods in the prior art that can recognize coupler feature targets from images can also be applied to the present invention. In a specific embodiment, the deep learning model uses an improved DEIM-DFine network model, as Figure 3 shown. On the basis of the DEIM-DFine network model, the improved DEIM-DFine network model uses the CARAFE upsampling module instead of the original nearest neighbor upsampling in its feature fusion part, improving the model's perception ability in terms of edges and fine-grainedness.

[0054] In the above embodiment, the CARAFE upsampling module is introduced on the basis of the DEIM-DFine network model. The CARAFE upsampling module is a prior art, which realizes efficient upsampling of the feature map through a content-aware method. This module is mainly composed of two key components, namely the kernel prediction module and the feature recombination module. The kernel prediction module is used to generate position-related upsampling kernels, and the feature recombination module uses the predicted kernels to perform content-aware upsampling operations. Compared with traditional fixed upsampling methods such as bilinear interpolation, CARAFE can adaptively predict the optimal upsampling kernel according to the content of the input features, so as to better retain image details and semantic information. Among them, the kernel prediction module can be mainly divided into two steps: feature kernelization recombination and normalization, which can be expressed by the following formula: ; where, represents the final upsampling kernel after normalization, represents the original input feature map.

[0055] The feature recombination module mainly performs feature recombination on the upsampled feature map based on the obtained upsampling kernel, which can be expressed by the following formula: ; Among them, represents the position corresponding local neighborhood, represents the sampling points within the neighborhood, and respectively represent the feature maps of the input and output.

[0056] It should be noted that the above improved DEIM-DFine network model is only a preferred deep learning model of the present invention, and other object detection deep learning models that can accurately identify the coupler features and other deep learning models can also be applicable to the present invention.

[0057] S3: According to the pixel coordinates of the coupler features, use the deep learning model to quantitatively identify the coupler force.

[0058] In a specific embodiment, it is necessary to first obtain the measured coupler force and the corresponding pixel coordinates of the coupler features. The pixel coordinates of the coupler features are obtained by the above visual detection model. The measured coupler force and the corresponding coupler feature pixel coordinates are used as the training set to train the deep learning model. After the model training is completed, for a group of newly input pixel coordinates of the coupler features, a corresponding coupler force will be quantitatively identified and output.

[0059] In a specific embodiment, a deep learning model is used to quantitatively identify the coupler force of a heavy-haul train. In this embodiment, the CNN-ResMLP network model is used to quantitatively identify the coupler force. The network structure of the CNN-ResMLP network model is as Figure 4 shown. In the design of the network, the input is the pixel horizontal and vertical coordinates of two coupler feature targets. First, it passes through a large-kernel convolutional layer module for extracting global information, and then through a small-kernel convolutional module for mining the local information association between data, realizing multi-scale feature extraction. Finally, it passes through an MLP module with a residual design, which fuses deep information while retaining shallow information, and outputs the final coupler force. The CNN-ResMLP network model can consider both global association and local details compared with other methods, and has a relatively high calculation efficiency, making it more suitable for coupler force quantitative identification.

[0060] In a specific embodiment, taking a certain heavy-haul train as an example, the recognition value of the coupler force of the heavy-haul train is calculated by using the machine vision-based heavy-haul train coupler force measurement method described in the present invention, and the true value of the coupler force is obtained by measuring with strain gauges on the heavy-haul train. In this embodiment, the camera is installed on the car body directly above the coupler, and the improved DEIM-Dfine network model is used to identify the coupler feature target in the preprocessed coupler area image to obtain the pixel coordinates of the coupler. According to the pixel coordinates of the coupler, the coupler force of the heavy-haul train is identified by the CNN-ResMLP network model.

[0061] In this embodiment, the recognition result of the coupler force is as Figure 5 shown. From Figure 5 it can be seen that the recognition value calculated by the present invention is in good agreement with the true value measured by the strain gauge. The present invention can accurately and quantitatively identify the coupler force of the heavy-haul train.

[0062] Figure 6 FIG. is the system structure diagram of a machine vision-based heavy-haul train coupler force measurement system provided by an embodiment of the present invention. As Figure 6 shown, the embodiment of the present invention provides a machine vision-based heavy-haul train coupler force measurement system, and the system includes: a collection unit, configured to collect an image of the coupler area of the heavy-haul train, and perform decontamination and enhancement processing on the coupler area image in combination with the working condition information to obtain an enhanced coupler area image; a positioning unit, configured to perform coupler feature target recognition on the enhanced coupler area image based on an improved DEIM-DFine network model trained by exclusive data enhancement, and obtain the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; an identification unit, configured to perform quantitative identification of the coupler force by using the CNN-ResMLP network model based on the pixel coordinates of the coupler feature target and the corresponding working condition information to obtain a coupler force recognition result. Those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program is stored in a storage medium, including several instructions for causing a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disc that can store program codes.

[0063] The optional embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. In addition, it should be noted that various specific technical features described in the above specific embodiments can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the embodiments of the present invention will not separately describe various possible combination methods.

[0064] In addition, any combination can be made between various different embodiments of the present invention as long as it does not violate the idea of the embodiments of the present invention, and it should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A method for measuring the coupler force of a heavy-haul train based on machine vision, characterized in that The method includes: Collecting images of the coupler area of a heavy-haul train, and performing decontamination and enhancement processing on the coupler area images in combination with the working condition information to obtain enhanced coupler area images; Performing coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation, and obtaining the pixel coordinates of the coupler feature target in combination with dynamic time series filtering; Based on the pixel coordinates of the coupler feature target and the corresponding working condition information, using a CNN-ResMLP network model to perform quantitative recognition of the coupler force and obtaining the coupler force recognition result.

2. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 1, wherein Performing decontamination and enhancement processing on the coupler area images in combination with the working condition information to obtain enhanced coupler area images, including: Detecting the current operating speed value and the environmental light intensity value of the coupler to determine the current working condition type based on the current operating speed value and the environmental light intensity value of the coupler; Selecting an image decontamination processing flow and an image enhancement processing flow according to the determined working condition type; Executing the image decontamination processing flow to perform stain area detection and image restoration processing on the coupler area images; Executing the image enhancement processing flow to sequentially perform any one or more of contrast enhancement processing, image sharpening processing, defogging processing, and noise reduction processing in the order matched to the determined working condition type.

3. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 1, wherein Performing coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation, including: Taking the enhanced coupler area images as input and performing coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation; wherein, The improved DEIM-Dfine network model trained with exclusive data augmentation introduces a CARAFE upsampling module in the feature fusion part, wherein the CARAFE upsampling module includes a kernel prediction module and a feature recombination module, the kernel prediction module generates a position-related upsampling kernel based on the coupler area images, and the feature recombination module receives the upsampling kernel and performs content-aware upsampling processing on the coupler area images; Performing coupler feature target recognition in the improved DEIM-DFine network model trained with exclusive data augmentation based on the upsampled feature map to obtain the preliminary pixel coordinates of the coupler feature target.

4. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 3, wherein, Before performing coupler feature target recognition on the enhanced coupler area images based on an improved DEIM-DFine network model trained with exclusive data augmentation, the method further includes: performing training on the improved DEIM-DFine network model trained with exclusive data augmentation, including: Performing image complex working condition simulation enhancement processing on the enhanced coupler area images to generate processed image data; wherein, The image complex working condition simulation enhancement processing includes any one or more of oil stain occlusion processing, dynamic blur processing, low illuminance degradation processing, rain and snow coverage processing, and vibration and jitter simulation processing. Add the processed image data to the coupler feature target recognition training dataset, and perform improved DEIM-DFine network model training based on the coupler feature target recognition training dataset to obtain an improved DEIM-DFine network model trained with exclusive data augmentation.

5. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 3, wherein, The rules for combining dynamic time series filtering to obtain the pixel coordinates of the coupler feature target include: Receive consecutive frame coupler region images, and extract the preliminary pixel coordinates of the coupler feature target for each frame; Calculate the pixel coordinate changes between consecutive frames, and sequentially perform weighted average processing, change amplitude filtering processing, and outlier rejection processing based on the calculation results of the pixel coordinate changes to generate the pixel coordinates of the coupler feature target after dynamic time series filtering.

6. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 1, wherein, Based on the pixel coordinates of the coupler feature target and the corresponding working condition information, use the CNN-ResMLP network model to perform quantitative coupler force recognition, and obtain the coupler force recognition result, including: Use the pixel coordinates of the coupler feature target and the corresponding working condition information as input data, and input them into the CNN-ResMLP network model for quantitative coupler force recognition; where The CNN-ResMLP network model includes at least one convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold, at least one convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold, and a multi-layer perceptron module containing a residual connection structure; Among them, the first convolutional kernel size threshold is greater than the second convolutional kernel size threshold; the convolutional layer with a convolutional kernel size greater than the first convolutional kernel size threshold is used to receive the pixel coordinates of the coupler feature target after dynamic time series filtering and the coupler working condition information, and perform global feature extraction; the convolutional layer with a convolutional kernel size less than the second convolutional kernel size threshold is used to perform local feature extraction; The multi-layer perceptron module performs feature fusion and non-linear mapping based on the extracted global features and local features, and outputs the coupler force recognition result.

7. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 1, wherein The method further includes: performing model training on the CNN-ResMLP network model.

8. The method for measuring the coupler force of a heavy-haul train based on machine vision according to claim 1, characterized in that, After obtaining the coupler force recognition result, the method further includes: Compare the coupler force recognition result with the historical recognition trend data, the current coupler working condition information, and the theoretically reasonable range of coupler force calculated based on the coupler physical constraint model; When the comparison result deviates from the preset theoretically reasonable range, perform error filtering processing; where the error filtering processing includes: performing error compensation processing on the coupler force recognition result according to the comparison result, and generating a corrected coupler force recognition result.

9. A coupler force measurement system for heavy-haul trains based on machine vision, characterized in that, The system includes: An acquisition unit, configured to acquire coupler region images of a heavy-haul train, and perform decontamination and enhancement processing on the coupler region images in combination with working condition information to obtain enhanced coupler region images; A positioning unit, configured to perform coupler feature target recognition on the enhanced coupler region images based on an improved DEIM-DFine network model trained with exclusive data augmentation, and combine dynamic time series filtering to obtain the pixel coordinates of the coupler feature target; An identification unit, configured to perform quantitative identification of the coupler force by using a CNN-ResMLP network model based on the pixel coordinates of the coupler feature target and the corresponding working condition information, so as to obtain a coupler force identification result.

Images