Wheel set damage detection system and method based on machine vision

Abstract

The present invention discloses a wheel set damage detection system and method based on machine vision, which relates to the technical field of sensor detection. The present invention analyzes the wheel set image, calculates the quality of the wheel set image, and obtains the visual detection quality threshold by using the image quality in all records; matches the environmental features with the wheel set image features to form a relationship pair; calculates the real-time wheel set image quality, and judges whether the real-time visual detection is qualified by using the visual detection quality threshold; uses the relationship pair to judge the environmental features that cause the failure of the real-time visual detection; collects the environmental data when different sensors detect wheel set damage failures in history to obtain the environmental factors affecting the detection quality of different sensors; selects an auxiliary detection sensor by using the types of environmental data that cause the failure of the real-time visual detection and the environmental factors of different sensors, and uses the auxiliary detection sensor and the visual sensor to detect the wheel set damage simultaneously.

Description

Technical Field

[0001] The present invention relates to the technical field of sensor detection, and specifically to a wheel set damage detection system and method based on machine vision. Background Technique

[0002] With the rapid development of railway transportation, the running speed and load of trains are continuously increasing. As a key component of trains, the quality of the wheel set is directly related to the safe operation of trains. During long-term operation, the wheel set will be damaged due to various factors, such as the sliding friction between the wheel and the rail during emergency braking, wheel brake shoe problems, improper operation of train drivers, and the impact of the wheel on the track. These may cause damages such as wheel peeling, abrasion, pit wear, and wheel polygon. These damages not only affect the running safety of trains, but also shorten the service life of wheels and increase maintenance costs. Therefore, efficient and accurate detection methods are needed to ensure the reliability of the wheel set. With the continuous improvement of computer performance and the continuous improvement of image processing algorithms, the wheel set damage detection technology based on machine vision has been further developed. A variety of image preprocessing methods have emerged, such as grayscale conversion, filtering, image enhancement, etc., to improve the image quality and highlight the damage features. At the same time, feature extraction and classification algorithms are also becoming increasingly rich. Algorithms such as edge detection, region segmentation, and template matching are widely used in the positioning and recognition of wheel set damages. The application of these technologies has greatly improved the accuracy and reliability of wheel set damage detection, and can detect more types and finer damages.

[0003] However, during the operation of the wheel set, the environment changes diversely and rapidly. Many environmental data will have a great impact on the visual detection results, resulting in unqualified image quality and unable to successfully perform damage detection. And currently, adjusting the environmental data according to the specific environmental impact is not applicable to all scenarios and the adjustment effect is not high. Therefore, it is crucial to reduce the impact of environmental data on visual detection in various scenarios and improve the detection quality. Summary of the Invention

[0004] The purpose of the present invention is to provide a wheel set damage detection system and method based on machine vision to solve the problems raised in the prior art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A wheel set damage detection method based on machine vision, the method includes the following steps:

[0007] S100. Collect the detection failure records when the historical vision sensor detects the wheel set damage, extract the wheel set images generated by the vision sensor in the records, analyze the wheel set images, calculate the quality of the wheel set images, and obtain the visual detection quality threshold using the image quality in all records;

[0008] Further, the specific steps for obtaining the visual detection quality threshold using the image quality in all records are as follows:

[0009] S101. Collect the records of detection failures when the historical visual sensor detects wheel set damage, extract the wheel set images generated by the visual sensor detection in the records, collect each pixel point in the wheel set image as P(x, y), calculate the sum of the square of the horizontal gradient and the square of the vertical gradient of each pixel point, and then sum the squares of all pixel points to obtain the sharpness of each wheel set image;

[0010] S102. Extract the pixel values of each pixel point in the wheel set image, calculate the average pixel value of all pixel points, calculate the square of the difference between the pixel value of each pixel point and the average pixel value, and sum and average the squared differences of all pixel points to obtain the contrast of each wheel set image;

[0011] S103. Convert the wheel set image into a digital signal, extract the average power of the signal and the average power of the noise in each wheel set image, and calculate the signal-to-noise ratio SNR of each wheel set image using the average power of the signal and the noise;

[0012] S104. Using the principal component analysis method, convert the three quality characteristics of sharpness, contrast, and signal-to-noise ratio SNR into a feature matrix, construct the covariance matrix of the feature matrix, calculate the eigenvalues and eigenvectors of the covariance matrix, calculate the proportion of the eigenvalue of each quality characteristic to the total eigenvalue as the weight of each quality characteristic. After obtaining the weights corresponding to the three quality characteristics, perform weighted summation of the three quality characteristics in each wheel set image to obtain the quality of each wheel set image;

[0013] S105. After obtaining the quality of all detected failed wheel set images in history, calculate the average value and standard deviation of the wheel set image quality, and use the average value minus the standard deviation to obtain the visual detection quality threshold Imy.

[0014] By collecting historical detection failure records and analyzing the quality of wheel set images, the visual detection quality threshold is obtained, which provides a quantitative standard for judging the quality of real-time wheel set images, helps to accurately identify detection failure situations caused by image quality problems, and avoids misjudgment and missed judgment.

[0015] The quality threshold can also be an important indicator for evaluating the performance of the visual detection system. By comparing the relationship between the image quality collected at different times or by different devices and the threshold, the performance change of the system can be understood, potential degradation or faults of the system can be detected in time, and a basis for the maintenance and upgrade of the system can be provided.

[0016] S200. In the historical visual detection failure records, collect the environmental data and wheel set images in the records, extract different types of environmental features and wheel set image features that cause visual detection failure, and match the environmental features with the wheel set image features to form relationship pairs.

[0017] Further, the specific steps for matching the environmental data with the wheel set image features to form relationship pairs are as follows:

[0018] S201. Extract the environmental data and wheel set images from the records of failed detections when the historical visual sensor detects wheel set damage, extract the environmental data when the visual sensor successfully detects wheel set damage, calculate the difference between all environmental data at the time of successful and failed detections, and for each failed detection record, extract the type of environmental data corresponding to the maximum difference as the environmental feature that causes the visual sensor to fail to detect wheel set damage. Let all the environmental features that cause the visual sensor to fail to detect wheel set damage be {H1, H2, H3,..., H n}, where H1, H2, H3,..., H n represent the 1st, 2nd, 3rd,..., nth environmental features that cause the visual sensor to fail to detect wheel set damage, and n is a positive integer.

[0019] S202. For the environmental features that cause the visual sensor to fail to detect wheel set damage in each record, use the convolutional neural network CNN to analyze all the wheel set images in the records and extract the wheel set image features; construct a relationship pair for each failed detection record as H i →F i ={T1, T2, T3,..., T m}, where H i represents the i-th environmental feature that causes the visual sensor to fail to detect wheel set damage, T1, T2, T3,..., T m represent the 1st, 2nd, 3rd,..., mth wheel set image features, and m is a positive integer. F i represents the set of wheel set image features in the i-th relationship pair; construct relationship pairs for all environmental features in the same way.

[0020] Extracting environmental features and wheel set image features and forming relationship pairs can deeply analyze the reasons for detection failure, clarify the influence of different environmental factors on wheel set image features, and thus more accurately judge whether the detection result is reliable in real-time detection.

[0021] S300. When using the visual sensor to detect wheel set damage in real-time, view the generated real-time wheel set image, calculate the quality of the real-time wheel set image, and use the visual detection quality threshold to judge whether the real-time visual detection is qualified.

[0022] Further, the specific steps for using the visual detection quality threshold to determine whether real-time visual detection is qualified are as follows:

[0023] S301. When using a vision sensor to detect wheel set damage in real time, view the generated real-time wheel set image, calculate the three quality features of the sharpness Sharpness_s, contrast Contrast_s, and signal-to-noise ratio SNR_s of the real-time wheel set image respectively, and calculate the real-time wheel set image quality Ims according to the corresponding weights;

[0024] S302. Use the visual detection quality threshold to judge the quality of the real-time wheel set image. When Ims≥Imy, it is judged that the real-time visual detection of wheel set damage is qualified; when Ims<Imy, it is judged that the real-time visual detection of ferry damage is unqualified.

[0025] Using the visual detection quality threshold to judge whether real-time visual detection is qualified can timely detect the situation where the image quality does not meet the requirements and ensure the accuracy of the detection results.

[0026] S400. After judging that the real-time visual detection is unqualified, collect the wheel set image features during the real-time visual detection, and use the relationship pair to judge the environmental features that cause the real-time visual detection to fail;

[0027] Further, the specific steps for using the relationship pair to judge the environmental features that cause the real-time visual detection to fail are as follows:

[0028] S401. After judging that the real-time visual detection is unqualified, collect the wheel set image features during the real-time visual detection, and calculate the similarity between the real-time wheel set image features and the wheel set image features in all relationship pairs. The formula is:

[0029] ;

[0030] In the formula, Sim(Fs, F i ) represents the similarity between the real-time wheel set image feature set and the wheel set image feature set in the i-th relationship pair, fs j represents the j-th type of wheel set image feature in the real-time wheel set image feature set, and f j represents the j-th type of wheel set image feature in the wheel set image feature set of the i-th relationship pair; m represents the total number of wheel set image features in the wheel set image feature set of the i-th relationship pair; calculate the similarity between the real-time wheel set image feature set and the wheel set image feature sets of n relationship pairs in the same way;

[0031] S402. Judge all the calculated similarities, select the wheel set image feature set in the relationship pair with the highest similarity, and extract the environmental features corresponding to the wheel set image feature set in the relationship pair as the environmental features that cause the real-time visual detection to fail.

[0032] By analyzing the characteristics of the wheel set images and matching them with the previously established relationship between the environmental characteristics and the wheel set image characteristics, it is possible to accurately identify which specific environmental factors have caused the current detection failure. For example, whether it is due to excessive or insufficient light, or factors such as dust occlusion that affect the image quality, thereby leading to detection failure. This helps to quickly locate the root cause of the problem.

[0033] S500. Based on the data that changes when the wheel set is damaged, obtain the corresponding types of sensors used to detect wheel set damage. Collect the environmental data when different sensors have failed to detect wheel set damage in the past, and obtain the environmental factors that affect the detection quality of different sensors.

[0034] Furthermore, the specific steps to obtain the environmental factors that affect the detection quality of different sensors are as follows:

[0035] S501. Based on the data that changes when the wheel set is damaged, obtain the corresponding types of sensors used to detect wheel set damage. Collect the environmental data when different sensors have failed to detect wheel set damage in the past, extract the failure rates of different sensors in detecting wheel set damage historically. The failure rate represents the ratio of the number of detection failures to the total number of detections; calculate the correlation coefficient between the detection failures of different sensors and each type of environmental data. The formula is:

[0036] ;

[0037] In the formula, ρ(S u , E v ) represents the correlation coefficient between the u-th type of sensor and the v-th type of environmental data, cov(S u , E v ) represents the covariance between the u-th type of sensor and the v-th type of environmental data, σS u represents the average failure rate of the u-th type of sensor, σE v represents the average value of the v-th type of environmental data, S u represents the failure rate of the u-th type of sensor, and E v represents the v-th type of environmental data;

[0038] S502. Calculate the correlation coefficients between each sensor and different environmental data, and select the one with the largest correlation coefficient as the environmental factor that affects the detection quality of the corresponding sensor; in the same way, obtain the environmental factors that affect the detection quality of each type of sensor.

[0039] By analyzing the data that changes when the wheel set is damaged, it is possible to clarify which type of sensor can most effectively detect these changes, thereby specifically selecting the appropriate type of sensor. This helps to avoid using inappropriate sensors, improve the detection efficiency and accuracy, and reduce resource waste.

[0040] After understanding the environmental factors that affect the detection quality of different sensors, these environmental factors can be monitored and controlled during actual detection, or sensors more suitable for the current environment can be selected according to changes in environmental conditions for detection. This can reduce the interference of environmental factors on the detection results of sensors, improve the reliability and stability of detection, and reduce the occurrence of false alarms and missed detections.

[0041] S600. Select an auxiliary detection sensor based on the types of environmental data that cause real-time vision detection to fail and the environmental factors of different sensors, and use the auxiliary detection sensor and the vision sensor to detect wheel set damage simultaneously.

[0042] Further, the specific steps for using the auxiliary detection sensor and the vision sensor to detect wheel set damage simultaneously are as follows:

[0043] S601. Screen the environmental factors of different sensors according to the environmental characteristics that cause real-time vision detection to fail, and regard the sensors corresponding to the environmental factors different from the environmental characteristics that cause real-time vision detection to fail as auxiliary sensors during real-time detection;

[0044] S602. Use the auxiliary sensor to perform a secondary detection on the wheel set damage to obtain the detection results of all auxiliary sensors. Quantify the detection results of the real-time vision sensor and all auxiliary sensors, and assign a weight β to all quantified detection results. The weight is based on the ratio of the number of successful detections to the total number of successful detections in the history of each type of sensor; then perform weighted fusion on all quantified detection results. The formula is:

[0045] ;

[0046] In the formula, J final represents the detection result after fusion, β d represents the weight of the d-th auxiliary sensor, J d represents the detection result of the d-th auxiliary sensor, β s represents the weight of the vision sensor, J s represents the detection result of the vision sensor, and C represents the total number of auxiliary sensors; use the detection result after fusion to determine whether there is damage to the wheel set.

[0047] When the vision sensor fails to detect due to specific environmental factors, the auxiliary detection sensor can provide additional detection information based on its adaptability to this environmental factor or different detection principles, making up for the deficiencies of the vision sensor, thereby improving the overall detection accuracy and reducing the situation of missed detections and false detections.

[0048] By combining multiple sensors for detection, the system has stronger adaptability to different environmental conditions. Even in a complex and changeable environment, it can ensure that some sensors can work properly, making the detection system more stable and reliable, and reducing the risk of system failures caused by environmental factors.

[0049] The wheel set damage detection system based on machine vision includes a data collection module, a visual detection quality threshold calculation module, a relationship pair construction module, a real-time visual detection judgment module, a sensor analysis module, and an auxiliary detection module.

[0050] The data collection module is used to collect records of the failure of visual sensors to detect wheel set damage in history.

[0051] The visual detection quality threshold calculation module is used to extract the wheel set images generated by the visual sensors in the records, analyze the wheel set images, calculate the quality of the wheel set images, and obtain the visual detection quality threshold using the image quality in all records.

[0052] The relationship pair construction module is used to collect the environmental data and wheel set images in the records, extract different types of environmental features and wheel set image features that cause visual detection failures, and match the environmental features with the wheel set image features to form relationship pairs.

[0053] The real-time visual detection judgment module is used to judge whether the quality of the real-time visual sensor's detection of wheel set damage is qualified, and when it is unqualified, it judges the environmental features that cause the unqualified.

[0054] The sensor analysis module is used to obtain the types of sensors corresponding to the detection of wheel set damage based on the data that changes when the wheel set is damaged, and calculate the environmental factors that affect the detection quality of different sensors.

[0055] The auxiliary detection module is used to select auxiliary detection sensors using the types of environmental data that cause real-time visual detection failures and the environmental factors of different sensors, and use the auxiliary detection sensors and visual sensors to detect wheel set damage simultaneously.

[0056] The real-time visual detection judgment module includes a detection quality judgment unit and an environmental feature analysis unit.

[0057] The detection quality judgment unit is used to judge whether the real-time visual detection is qualified using the visual detection quality threshold.

[0058] The environmental feature analysis unit is used to collect the wheel set image features during real-time visual detection after judging that the real-time visual detection is unqualified, and use the relationship pairs to judge the environmental features that cause the real-time visual detection to fail.

[0059] The auxiliary detection module includes an auxiliary sensor determination unit and a result fusion unit.

[0060] The auxiliary sensor determination unit is configured to use all sensors corresponding to environmental factors different from the environmental features that cause real-time vision detection failure as auxiliary sensors during real-time detection;

[0061] The result fusion unit is configured to quantify the results detected by all sensors, assign weight values, and perform weighted fusion on the detection results of all sensors to obtain the final detection result.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0063] 1. The present invention extracts environmental features and wheel set image features and forms a relationship pair, which can deeply analyze the reasons for detection failure, clarify the influence of different environmental factors on wheel set image features, and thus more accurately judge whether the detection result is reliable during real-time detection.

[0064] 2. When the vision sensor fails to detect due to specific environmental factors, the auxiliary detection sensor can provide additional detection information by virtue of its adaptability to the environmental factors or different detection principles, make up for the deficiencies of the vision sensor, thereby improving the overall detection accuracy and reducing the situations of missed detection and false detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a module distribution diagram of the wheel set damage detection system based on machine vision of the present invention;

[0066] Figure 2 It is a step schematic diagram of the wheel set damage detection method based on machine vision of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0068] Embodiment: As Figure 1 - Figure 2 shown, the present invention provides a technical solution,

[0069] A wheel set damage detection method based on machine vision, the method includes the following steps:

[0070] S100. Collect the detection failure records when the historical vision sensor detects wheel set damage, extract the wheel set images generated by the vision sensor in the records, analyze the wheel set images, calculate the quality of the wheel set images, and obtain the vision detection quality threshold by using the image quality in all records;

[0071] The specific steps to obtain the visual detection quality threshold using the image quality in all records are as follows:

[0072] S101. Collect the records of detection failures when the historical visual sensor detects wheel set damage, extract the wheel set images generated by the visual sensor in the records, collect each pixel point in the wheel set image as P(x, y), calculate the sum of the square of the horizontal gradient and the square of the vertical gradient of each pixel point, and then sum the squares of all pixel points to obtain the sharpness of each wheel set image; the formula is:

[0073] ;

[0074] In the formula, represents the horizontal gradient of the pixel point P(x, y), represents the vertical gradient of the pixel point P(x, y), and G represents the total number of pixel points;

[0075] S102. Extract the pixel values of each pixel point in the wheel set image, calculate the average pixel value of all pixel points, calculate the square of the difference between the pixel value of each pixel point and the average pixel value, and sum and average the squared differences of all pixel points to obtain the contrast of each wheel set image; the formula is:

[0076] ;

[0077] In the formula, R represents the average value of the pixel values of all pixel points, and Ps(x, y) g represents the pixel value of the pixel point P(x, y);

[0078] S103. Convert the wheel set image into a digital signal, extract the average power of the signal and the average power of the noise in each wheel set image, and calculate the signal-to-noise ratio SNR of each wheel set image using the average power of the signal and the average power of the noise;

[0079] S104. Using the principal component analysis method, convert the three quality characteristics of sharpness, contrast, and signal-to-noise ratio SNR into a feature matrix, construct the covariance matrix of the feature matrix, calculate the eigenvalues and eigenvectors of the covariance matrix, calculate the proportion of the eigenvalue of each quality characteristic in the total eigenvalues as the weight of each quality characteristic. After obtaining the weights corresponding to the three quality characteristics, perform weighted summation of the three quality characteristics in each wheel set image to obtain the quality of each wheel set image; the formula is:

[0080] ;

[0081] In the formula, Q represents the quality of the wheel set image, ω1 represents the weight of sharpness, ω2 represents the weight of contrast, and ω3 represents the weight of signal-to-noise ratio;

[0082] S105. After obtaining the quality of all wheel set images that failed in detection in history, calculate the average value and standard deviation of the wheel set image quality, and use the average value minus the standard deviation to obtain the visual detection quality threshold Imy.

[0083] By collecting historical detection failure records and analyzing the quality of wheel set images, the visual detection quality threshold is obtained, which provides a quantitative standard for judging the quality of real-time wheel set images, helps to accurately identify detection failure cases caused by image quality problems, and avoids misjudgment and missed judgment.

[0084] The quality threshold can also be an important indicator for evaluating the performance of the visual detection system. By comparing the relationship between the image quality collected at different times or by different devices and the threshold, the performance change of the system can be understood, potential degradation or faults of the system can be detected in time, and a basis for system maintenance and upgrade can be provided.

[0085] S200. In the historical visual detection failure records, collect the environmental data and wheel set images in the records, extract different types of environmental features and wheel set image features that cause visual detection failure, and match the environmental features and wheel set image features to form relationship pairs;

[0086] The specific steps for matching the environmental data and wheel set image features to form relationship pairs are as follows:

[0087] S201. Extract the environmental data and wheel set images from the records of historical visual sensor detection of wheel set damage failure, extract the environmental data when the visual sensor detects wheel set damage successfully, calculate the difference between all environmental data at the time of detection success and failure, and extract the type of environmental data corresponding to the maximum difference for each detection failure record as the environmental feature that causes the visual sensor to detect wheel set damage failure. Suppose all the environmental features that cause the visual sensor to detect wheel set damage failure are {H1, H2, H3,..., H n}, H1, H2, H3,..., H n represent the 1st, 2nd, 3rd,..., nth environmental features that cause the visual sensor to detect wheel set damage failure, and n is a positive integer;

[0088] S202. For the environmental features that cause the visual sensor to detect wheel set damage failure in each record, use the convolutional neural network CNN to analyze all the wheel set images in the records and extract the wheel set image features; construct a relationship pair for each failure record as H i →F i ={T1, T2, T3,..., T m}, where Hi represents the i-th environmental feature that causes the vision sensor to fail to detect wheel set damage, T1, T2, T3, ..., T m represents the 1st, 2nd, 3rd, ..., m-th wheel set image features, where m is a positive integer, F i represents the set of wheel set image features in the i-th relationship pair; construct relationship pairs for all environmental features in the same way.

[0089] Extracting environmental features and wheel set image features and forming relationship pairs can deeply analyze the reasons for detection failures, clarify the influence of different environmental factors on wheel set image features, and thus more accurately judge whether the detection results are reliable in real-time detection.

[0090] S300. When using a vision sensor to detect wheel set damage in real-time, view the generated real-time wheel set image, calculate the quality of the real-time wheel set image, and use the vision detection quality threshold to judge whether the real-time vision detection is qualified;

[0091] The specific steps for using the vision detection quality threshold to judge whether the real-time vision detection is qualified are as follows:

[0092] S301. When using a vision sensor to detect wheel set damage in real-time, view the generated real-time wheel set image, calculate three quality features of the real-time wheel set image, namely sharpness Sharpness_s, contrast Contrast_s, and signal-to-noise ratio SNR_s, and calculate the real-time wheel set image quality Ims according to the corresponding weights;

[0093] S302. Use the vision detection quality threshold to judge the quality of the real-time wheel set image. When Ims ≥ Imy, judge that the real-time vision detection of wheel set damage is qualified; when Ims < Imy, judge that the real-time vision detection of ferry damage is unqualified.

[0094] Using the vision detection quality threshold to judge whether the real-time vision detection is qualified can timely detect situations where the image quality does not meet the requirements and ensure the accuracy of the detection results.

[0095] S400. After judging that the real-time vision detection is unqualified, collect the wheel set image features during the real-time vision detection, and use the relationship pair to judge the environmental features that cause the real-time vision detection to fail;

[0096] The specific steps for using the relationship pair to judge the environmental features that cause the real-time vision detection to fail are as follows:

[0097] S401. After judging that the real-time vision detection is unqualified, collect the wheel set image features during the real-time vision detection, and calculate the similarity between the real-time wheel set image features and the wheel set image features in all relationship pairs. The formula is:

[0098] ;

[0099] In the formula, Sim(Fs, F i ) represents the similarity between the real-time wheel set image feature set and the wheel set image feature set in the i-th relationship pair. fs j represents the j-th type of wheel set image feature in the real-time wheel set image feature set, and f j represents the j-th type of wheel set image feature in the wheel set image feature set of the i-th relationship pair; m represents the total number of wheel set image features in the wheel set image feature set of the i-th relationship pair; the similarities between the real-time wheel set image feature set and the wheel set image feature sets of n relationship pairs are calculated in the same way;

[0100] S402. Judge all the calculated similarities, select the wheel set image feature set in the relationship pair with the highest similarity, and extract the environmental features corresponding to the wheel set image feature set in the relationship pair as the environmental features causing the failure of real-time visual detection.

[0101] By analyzing the wheel set image features and matching them with the previously established relationship pairs of environmental features and wheel set image features, it is possible to accurately find out which specific environmental factors cause the current detection failure. For example, whether it is too strong or too weak light, or factors such as dust occlusion that affect the image quality, thereby causing the detection failure, which helps to quickly locate the root cause of the problem.

[0102] S500. Obtain the types of sensors used to detect wheel set damage based on the data that changes when the wheel set is damaged, collect the environmental data when different sensors fail to detect wheel set damage historically, and obtain the environmental factors affecting the detection quality of different sensors;

[0103] The specific steps to obtain the environmental factors affecting the detection quality of different sensors are as follows:

[0104] S501. Obtain the types of sensors used to detect wheel set damage based on the data that changes when the wheel set is damaged, collect the environmental data when different sensors fail to detect wheel set damage historically, extract the failure rates of different sensors' historical detections of wheel set damage, and the failure rate represents the ratio of the number of detection failures to the total number of detections; calculate the correlation coefficients between the detection failures of different sensors and each type of environmental data. The formula is:

[0105] ;

[0106] In the formula, ρ(S u , E v ) represents the correlation coefficient between the u-th type of sensor and the v-th type of environmental data, cov(S u , E v ) represents the covariance between the u-th type of sensor and the v-th type of environmental data, σS u represents the average value of the failure rates of the u-th type of sensor, and σEv represents the average value of the v-th type of environmental data, S u represents the failure rate of the u-th type of sensor, E v represents the v-th type of environmental data;

[0107] S502. Calculate the correlation coefficients between each sensor and different environmental data, and select the one with the largest correlation coefficient as the environmental factor affecting the detection quality of the corresponding sensor; similarly, calculate the environmental factors affecting the detection quality of each sensor.

[0108] By analyzing the data that changes when the wheel set is damaged, it is possible to clarify which type of sensor can most effectively detect these changes, and thus select the appropriate type of sensor accordingly. This helps to avoid using unsuitable sensors, improve the detection efficiency and accuracy, and reduce resource waste.

[0109] After understanding the environmental factors affecting the detection quality of different sensors, these environmental factors can be monitored and controlled during actual detection, or sensors more suitable for the current environment can be selected according to the changes in environmental conditions. This can reduce the interference of environmental factors on the detection results of sensors, improve the reliability and stability of detection, and reduce the occurrence of false alarms and missed detections.

[0110] S600. Select auxiliary detection sensors using the types of environmental data that cause real-time visual detection failure and the environmental factors of different sensors, and use the auxiliary detection sensors and visual sensors to detect the wheel set damage simultaneously.

[0111] The specific steps for using the auxiliary detection sensors and visual sensors to detect the wheel set damage simultaneously are as follows:

[0112] S601. Screen the environmental factors of different sensors according to the environmental characteristics that cause real-time visual detection failure, and regard the sensors corresponding to the environmental factors different from the environmental characteristics that cause real-time visual detection failure as auxiliary sensors during real-time detection;

[0113] S602. Use the auxiliary sensors to conduct secondary detection on the wheel set damage to obtain the detection results of all auxiliary sensors, quantify the detection results of the real-time visual sensor and all auxiliary sensors, and assign weights β to all quantified detection results. The weights are based on the ratio of the number of successful detections to the total number of successful detections in the history of each sensor; then, perform weighted fusion on all quantified detection results. The formula is:

[0114] ;

[0115] In the formula, J final represents the fused detection result, β d represents the weight of the d-th type of auxiliary sensor, Jd represents the detection result of the d-th auxiliary sensor, β s represents the weight of the vision sensor, J s represents the detection result of the vision sensor, and C represents the total number of auxiliary sensors; the fused detection result is used to determine whether there is damage to the wheel set.

[0116] When the vision sensor fails to detect due to specific environmental factors, the auxiliary detection sensor can provide additional detection information by virtue of its adaptability to the environmental factors or different detection principles, making up for the deficiencies of the vision sensor, thereby improving the overall detection accuracy and reducing the cases of missed detection and false detection.

[0117] By combining multiple sensors for detection, the system has stronger adaptability to different environmental conditions. Even in a complex and changeable environment, it can ensure that some sensors can work properly, making the detection system more stable and reliable, and reducing the risk of system failure caused by environmental factors.

[0118] A wheel set damage detection system based on machine vision, the wheel set damage detection system includes a data collection module, a vision detection quality threshold calculation module, a relationship pair construction module, a real-time vision detection judgment module, a sensor analysis module, and an auxiliary detection module;

[0119] The data collection module is used to collect the records of the vision sensor's failure to detect wheel set damage in history;

[0120] The vision detection quality threshold calculation module is used to extract the wheel set images generated by the vision sensor in the records, analyze the wheel set images, calculate the quality of the wheel set images, and obtain the vision detection quality threshold using the image quality in all records;

[0121] The relationship pair construction module is used to collect the environmental data and wheel set images in the records, extract different types of environmental features and wheel set image features that cause vision detection failure, and match the environmental features and wheel set image features to form relationship pairs;

[0122] The real-time vision detection judgment module is used to judge whether the quality of the real-time vision sensor's detection of wheel set damage is qualified, and when it is unqualified, judge the environmental features that cause the unqualified;

[0123] The sensor analysis module is used to obtain the corresponding types of sensors for detecting wheel set damage based on the data that changes when the wheel set is damaged, and calculate the environmental factors that affect the detection quality of different sensors;

[0124] The auxiliary detection module is used to select auxiliary detection sensors using the types of environmental data that cause real-time vision detection failure and the environmental factors of different sensors, and use the auxiliary detection sensors and the vision sensor to detect wheel set damage simultaneously.

[0125] The real-time visual detection and judgment module includes a detection quality judgment unit and an environmental feature analysis unit;

[0126] The detection quality judgment unit is used to judge whether the real-time visual detection is qualified by using the visual detection quality threshold;

[0127] The environmental feature analysis unit is used to collect the wheel set image features during the real-time visual detection and use the relationship pair to judge the environmental features that cause the failure of the real-time visual detection after judging that the real-time visual detection is unqualified.

[0128] The auxiliary detection module includes an auxiliary sensor determination unit and a result fusion unit;

[0129] The auxiliary sensor determination unit is used to regard all sensors corresponding to environmental factors different from the environmental features that cause the failure of the real-time visual detection as auxiliary sensors during the real-time detection;

[0130] The result fusion unit is used to quantify the results detected by all sensors, assign weights, and perform weighted fusion on the results detected by all sensors to obtain the final detection result.

[0131] Embodiment: Now analyze a system for detecting wheel set damage using a visual sensor. Calculate the clarity, contrast, and signal-to-noise ratio of the historical wheel set images using formulas. Assume that the weights of each type of data obtained by the principal component analysis method are 0.5, 0.3, and 0.2 respectively; standardize the three types of data to values within the same range, and calculate that the quality of one of the historical wheel set images is 28; calculate the quality threshold as 35 based on the quality of the historical wheel set images;

[0132] Construct a relationship pair based on the historical environmental data. For example: Relationship pair 1 is light → {clarity 3000, brightness 520, exposure 4500}. Use the visual sensor to detect the wheel set damage in real time, calculate the real-time detection quality as 30, and judge that the detection quality is unqualified;

[0133] Extract the wheel set image features as clarity 2900, contrast 90, and exposure 4445; calculate that the highest similarity is relationship pair 1, and judge that the environmental feature that causes the failure of the real-time visual detection is light;

[0134] The sensors used to detect wheel set damage in history also include a laser sensor and an ultrasonic sensor; calculate and analyze that the environmental factors of the two sensors are light and sound waves respectively;

[0135] Select the ultrasonic sensor as the auxiliary sensor during the real-time detection. Assume that the result detected by the visual sensor is quantified to 3, the weight is 0.3, the result detected by the acoustic wave sensor is quantified to 9, and the weight is 0.7. Calculate that the final result is 7.2.

[0136] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, in any regard, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Accordingly, all changes that fall within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims concerned.

Claims

1. A wheelset damage detection method based on machine vision, characterized in that: The method comprises the following steps: S100, collecting historical records of failed detections when visual sensors detect wheelset damage, extracting wheelset images generated by the visual sensors in the records, analyzing the wheelset images, calculating the wheelset image quality, and obtaining a visual detection quality threshold using the image quality in all records; The specific steps to obtain the visual detection quality threshold using the image quality in all records are: S101, collecting the records of failed detections when the visual sensor detects wheelset damage, extracting the wheelset images generated by the visual sensor detection in the records, collecting each pixel point in the wheelset image as P (x, y), calculating the square of the horizontal coordinate gradient and the sum of the square of the vertical coordinate gradient of each pixel point, and then summing the square sums of all the pixels to obtain the sharpness of each wheelset image; S102, extracting the pixel value of each pixel in the wheel pair image, calculating the average pixel value of all the pixels, calculating the square of the difference between the pixel value of each pixel and the average pixel value, and summing and averaging the squares of the difference of all the pixels to obtain the contrast of each wheel pair image; S103, converting the wheelset image into a digital signal, extracting the average power of the signal and the average power of the noise in each wheelset image, and calculating the signal-to-noise ratio (SNR) of each wheelset image using the average power of the signal and the average power of the noise; S104, using the principal component analysis method, converting the three quality features of sharpness, contrast and signal-to-noise ratio (SNR) into a feature matrix, constructing a covariance matrix of the feature matrix, calculating the eigenvalues ​​and eigenvectors of the covariance matrix, calculating the ratio of the eigenvalue of each quality feature to the total eigenvalues ​​as the weight of each quality feature, and after obtaining the weights corresponding to the three quality features, performing weighted summation on the three quality features in each wheel pair image to obtain the quality of each wheel pair image; S105, after obtaining the image qualities of all wheel pairs that have failed detection in history, calculate the average and standard deviation of the wheel pair image qualities, and obtain the visual detection quality threshold Imy by subtracting the standard deviation from the average; S200, in the historical visual inspection failure records, collecting environmental data and wheelset images in the records, extracting different types of environmental features and wheelset image features that cause visual inspection failures, and matching the environmental features with the wheelset image features to form a relationship pair; S300, when using a visual sensor to detect wheelset damage in real time, checking the generated real-time wheelset image, calculating the quality of the real-time wheelset image, and using a visual inspection quality threshold to determine whether the real-time visual inspection is qualified; S400, after determining that the real-time visual inspection is unqualified, collecting wheel set image features during the real-time visual inspection, and using the relationship to determine the environmental features that caused the real-time visual inspection failure; S500, based on the data that changes when a wheelset is damaged, obtain the type of sensor corresponding to the type of wheelset damage detection, collect the environmental data when different sensors fail to detect wheelset damage in the past, and obtain the environmental factors that affect the detection quality of different sensors; The specific steps to obtain the environmental factors that affect the detection quality of different sensors are: S501. Based on the data that changes when a wheelset is damaged, the type of sensor corresponding to the wheelset damage detection is obtained, and the environmental data when different sensors fail to detect wheelset damage in the past are collected. The failure rate of different sensors in the past detecting wheelset damage is extracted. The failure rate represents the ratio of the number of detection failures to the total number of detections. The correlation coefficient between the detection failure of different sensors and each type of environmental data is calculated. The formula is: ； In the formula, ρ(S u , E v ) represents the correlation coefficient between the u-th sensor and the v-th environmental data, cov(S u , E v ) represents the covariance of the u-th sensor and the v-th environment data, σS u represents the average failure rate of the u-th sensor, σE v represents the average value of the vth environmental data, S u represents the failure rate of the u-th sensor, E v Indicates the vth type of environmental data; S502, for each sensor, calculate the correlation coefficient with different environmental data, select the largest correlation coefficient as the environmental factor affecting the detection quality of the corresponding sensor; and calculate the environmental factor affecting the detection quality of each sensor in the same way; S600: Select an auxiliary detection sensor by using the type of environmental data that causes the real-time visual detection failure and the environmental factors of different sensors, and use the auxiliary detection sensor and the visual sensor to simultaneously detect the wheelset damage.

2. The wheelset damage detection method based on machine vision according to claim 1, characterized in that: The specific steps of matching the environmental data with the wheel set image features to form a relationship pair in S200 are: S201, extracting environmental data and wheelset images from the historical records of failed visual sensor detection of wheelset damage, extracting environmental data when the visual sensor successfully detects wheelset damage, calculating the difference between all environmental data when the detection is successful and when it fails, and extracting the type of environmental data corresponding to the maximum difference for each detection failure record as the environmental feature that causes the visual sensor to fail to detect wheelset damage, assuming that all the environmental features that cause the visual sensor to fail to detect wheelset damage are {H1, H2, H3, ..., H n }, H1, H2, H3, ..., H n represents the first, second, third, ..., nth environmental features that cause the visual sensor to fail in detecting wheelset damage, where n is a positive integer; S202, for each record of environmental features that cause the visual sensor to fail to detect wheelset damage, use the convolutional neural network CNN to analyze the wheelset images in all records and extract the wheelset image features; for each failure record, construct a relationship pair H i →F i ={T1, T2, T3, ..., T m }, where H i represents the i-th environmental feature that causes the visual sensor to fail in detecting wheelset damage, T1, T2, T3, ..., T m represents the image features of the 1st, 2nd, 3rd, ..., mth wheelset, where m is a positive integer, and F i Represents the set of wheel pair image features in the i-th relationship pair; similarly, the relationship pairs of all environmental features are constructed.

3. The wheelset damage detection method based on machine vision according to claim 2 is characterized in that: The specific steps of using the visual inspection quality threshold in S300 to determine whether the real-time visual inspection is qualified are: S301, when using a visual sensor to detect wheelset damage in real time, check the generated real-time wheelset image, calculate the three quality features of the real-time wheelset image, namely, sharpness_s, contrast Contrast_s, and signal-to-noise ratio SNR_s, and calculate the real-time wheelset image quality Ims according to the corresponding weights; S302, using the visual detection quality threshold to judge the real-time wheelset image quality, when Ims≥Imy, it is judged that the real-time visual detection of wheelset damage is qualified; when Ims<Imy, it is judged that the real-time visual detection of ferry damage is unqualified.

4. The method for detecting wheelset damage based on machine vision according to claim 3 is characterized in that: The specific steps of using the relationship pair to determine the environmental features that cause the real-time visual detection failure in S400 are: S401, after determining that the real-time visual inspection is unqualified, collect the wheel set image features during the real-time visual inspection, and calculate the similarity between the real-time wheel set image features and the wheel set image features in all related pairs, the formula is: ； In the formula, Sim (Fs, F i ) represents the similarity between the real-time wheelset image feature set and the wheelset image feature set in the i-th relationship pair, fs j represents the jth wheelset image feature in the real-time wheelset image feature set, f j represents the jth wheelset image feature in the i-th wheelset image feature set; m represents the total number of wheelset image features in the i-th wheelset image feature set; similarly, the similarity between the real-time wheelset image feature set and the n-th wheelset image feature set is calculated; S402, judging all calculated similarities, selecting the wheel pair image feature set in the relationship pair with the highest similarity, and extracting the environmental features corresponding to the wheel pair image feature set in the relationship pair as the environmental features that cause the real-time visual detection failure.

5. The method for detecting wheelset damage based on machine vision according to claim 4, characterized in that: The specific steps of using the auxiliary detection sensor and the visual sensor to simultaneously detect the damage to the wheelset in S600 are: S601, selecting environmental factors of different sensors according to environmental features that cause real-time visual detection failure, and using sensors corresponding to environmental factors different from environmental features that cause real-time visual detection failure as auxiliary sensors for real-time detection; S602, after performing secondary detection of the wheelset damage using the auxiliary sensors, the detection results of all the auxiliary sensors are obtained, the detection results of the real-time visual sensor and all the auxiliary sensors are quantified, and a weight β is assigned to all the quantified detection results, and the weight is based on the ratio of the number of successful detections in the history of each sensor to the total number of successful detections; then all the quantified detection results are weightedly fused, and the formula is: ； In the formula, J final represents the detection result after fusion, β d represents the weight of the d-th auxiliary sensor, J d represents the detection result of the d-th auxiliary sensor, β s represents the weight of the visual sensor, J s represents the detection result of the visual sensor, and C represents the total number of auxiliary sensors; The fused detection results are used to determine whether the wheelset is damaged.

6. A wheelset damage detection system based on machine vision using the wheelset damage detection method based on machine vision according to any one of claims 1 to 5, characterized in that: The wheelset damage detection system includes a data collection module, a visual inspection quality threshold calculation module, a relationship pair construction module, a real-time visual inspection judgment module, a sensor analysis module and an auxiliary detection module; The data collection module is used to collect historical records of failures of visual sensors to detect wheelset damage; The visual inspection quality threshold calculation module is used to extract the wheelset image generated by the visual sensor in the record, analyze the wheelset image, calculate the wheelset image quality, and obtain the visual inspection quality threshold using the image quality in all records; The relationship pair building module is used to collect environmental data and wheelset images in the record, extract different types of environmental features and wheelset image features that cause visual detection failure, and match the environmental features with the wheelset image features to form a relationship pair; The real-time visual detection judgment module is used to judge whether the wheel set damage quality detected by the real-time visual sensor is qualified, and if it is unqualified, to judge the environmental characteristics that lead to the unqualified; The sensor analysis module is used to obtain the type of sensor corresponding to the wheel set damage detection based on the data that changes when the wheel set is damaged, and calculate the environmental factors that affect the detection quality of different sensors; The auxiliary detection module is used to select auxiliary detection sensors using the types of environmental data that cause real-time visual detection failure and environmental factors of different sensors, and to detect wheelset damage simultaneously using the auxiliary detection sensors and visual sensors.

7. The wheelset damage detection system based on machine vision according to claim 6 is characterized in that: The real-time visual detection and judgment module includes a detection quality judgment unit and an environmental feature analysis unit; The detection quality judgment unit is used to judge whether the real-time visual detection is qualified by using the visual detection quality threshold; The environmental feature analysis unit is used to collect wheel set image features during real-time visual inspection after determining that the real-time visual inspection is unqualified, and use relationship pairs to determine the environmental features that cause the failure of the real-time visual inspection.

8. The wheelset damage detection system based on machine vision according to claim 6 is characterized in that: The auxiliary detection module includes an auxiliary sensor determination unit and a result fusion unit; The auxiliary sensor determination unit is used to use sensors corresponding to environmental factors different from the environmental features that cause the real-time visual detection failure as auxiliary sensors during real-time detection; The result fusion unit is used to quantify the detection results of all sensors, assign weights, and perform weighted fusion of all sensor detection results to obtain a final detection result.

Images