Wheel set damage detection system and method based on machine vision

Abstract

The invention discloses a wheel set damage detection system and method based on machine vision, and relates to the technical field of sensor detection.The wheel set damage detection method comprises the steps that wheel set images are analyzed, the wheel set image quality is calculated, and a visual detection quality threshold value is obtained through the image quality in all records; matching the environment features with the wheel set image features to form a relation pair; the real-time wheel set image quality is calculated, and whether real-time visual detection is qualified or not is judged through a visual detection quality threshold value; judging environment characteristics causing failure of real-time visual detection by utilizing the relation pair; collecting environmental data when different sensors detect wheel set damage failure in history, and obtaining environmental factors influencing the detection quality of different sensors; an auxiliary detection sensor is selected by utilizing environmental data types causing real-time visual detection failure and environmental factors of different sensors, and the auxiliary detection sensor and a visual sensor are used for detecting wheel set damage at the same time.

Description

Technical Field

[0001] The present invention relates to the field of sensor detection technology, and in particular to a wheelset damage detection system and method based on machine vision. Background Art

[0002] With the rapid development of railway transportation, the speed and load of trains are constantly increasing. As a key component of the train, the quality of wheelsets is directly related to the safe operation of the train. Wheelsets may be damaged due to various factors during long-term operation, such as sliding friction between wheels and rails during emergency braking, wheel brake shoe problems, improper operation of train drivers, and impact of wheels on tracks, which may cause wheels to peel, scratch, pit wear, wheel polygon and other damage. These damages not only affect the safety of train operation, but also shorten the service life of wheels and increase maintenance costs. Therefore, efficient and accurate detection methods are needed to ensure the reliability of wheelsets. With the continuous improvement of computer performance and image processing algorithms, wheelset damage detection technology based on machine vision has been further developed. A variety of image preprocessing methods have emerged, such as graying, filtering, image enhancement, etc., to improve image quality and highlight damage features. At the same time, feature extraction and classification algorithms are becoming increasingly abundant. For example, algorithms such as edge detection, region segmentation, and template matching are widely used in the positioning and identification of wheelset damage. The application of these technologies has greatly improved the accuracy and reliability of wheelset damage detection, and can detect more types and more subtle damage.

[0003] However, the environment changes rapidly and diversely during wheelset work. Many environmental data will have a great impact on the visual inspection results, resulting in unqualified image quality and failure to successfully perform damage detection. In addition, adjusting environmental data according to specific environmental influences is not applicable to all scenarios and the adjustment effect is not high. Therefore, it is crucial to reduce the impact of environmental data on visual inspection and increase inspection quality in a variety of scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a wheelset damage detection system and method based on machine vision to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions: A wheelset damage detection method based on machine vision, the method comprising the following steps: S100, collecting historical records of failed detections when visual sensors detect wheelset damage, extracting wheelset images generated by the visual sensors in the records, analyzing the wheelset images, calculating the wheelset image quality, and obtaining a visual detection quality threshold using the image quality in all records; Furthermore, the specific steps of obtaining the visual detection quality threshold using the image quality in all records are as follows: S101, collecting the records of failed detections when the visual sensor detects wheelset damage, extracting the wheelset images generated by the visual sensor detection in the records, collecting each pixel point in the wheelset image as P (x, y), calculating the square of the horizontal coordinate gradient and the sum of the square of the vertical coordinate gradient of each pixel point, and then summing the square sums of all the pixels to obtain the sharpness of each wheelset image; S102, extracting the pixel value of each pixel in the wheel pair image, calculating the average pixel value of all the pixels, calculating the square of the difference between the pixel value of each pixel and the average pixel value, and summing and averaging the squares of the difference of all the pixels to obtain the contrast of each wheel pair image; S103, converting the wheelset image into a digital signal, extracting the average power of the signal and the average power of the noise in each wheelset image, and calculating the signal-to-noise ratio (SNR) of each wheelset image using the average power of the signal and the average power of the noise; S104, using the principal component analysis method, converting the three quality features of sharpness, contrast and signal-to-noise ratio (SNR) into a feature matrix, constructing a covariance matrix of the feature matrix, calculating the eigenvalues ​​and eigenvectors of the covariance matrix, calculating the ratio of the eigenvalue of each quality feature to the total eigenvalues ​​as the weight of each quality feature, and after obtaining the weights corresponding to the three quality features, performing weighted summation on the three quality features in each wheel pair image to obtain the quality of each wheel pair image; S105, after obtaining the image qualities of all wheel pairs that have failed detection in history, calculate the average and standard deviation of the wheel pair image qualities, and obtain the visual detection quality threshold Imy by subtracting the standard deviation from the average.

[0006] By collecting historical detection failure records and analyzing the wheelset image quality, the visual detection quality threshold is obtained, which provides a quantitative standard for judging the real-time wheelset image quality. This helps to accurately identify detection failures caused by image quality problems and avoid misjudgments and missed judgments.

[0007] The quality threshold can also be used as an important indicator to evaluate the performance of the visual inspection system. By comparing the relationship between the image quality and the threshold collected at different times or with different devices, we can understand the performance changes of the system, timely discover possible degradation or failure of the system, and provide a basis for system maintenance and upgrades.

[0008] S200, in the historical visual inspection failure records, collecting environmental data and wheelset images in the records, extracting different types of environmental features and wheelset image features that cause visual inspection failures, and matching the environmental features with the wheelset image features to form a relationship pair; Furthermore, the specific steps of matching the environmental data with the wheel set image features to form a relationship pair are as follows: S201, extracting environmental data and wheelset images from the historical records of failed detections of wheelset damage by visual sensors, extracting environmental data when the visual sensors successfully detect wheelset damage, calculating the difference between all environmental data when the detections are successful and when they fail, extracting the type of environmental data corresponding to the maximum difference for each detection failure record as the environmental feature that causes the failure of the visual sensor to detect wheelset damage, assuming that all the environmental features that cause the failure of the visual sensor to detect wheelset damage are {H 1 , H 2 , H 3 , ..., H n}, H 1 , H 2 , H 3 , ..., H n represents the first, second, third, ..., nth environmental features that cause the visual sensor to fail in detecting wheelset damage, where n is a positive integer; S202, for each record of environmental features that cause the visual sensor to fail to detect wheelset damage, use the convolutional neural network CNN to analyze the wheelset images in all records and extract the wheelset image features; for each failure record, construct a relationship pair H i →F i ={T 1 , T 2 , T 3 , ..., T m}, where H i represents the i-th environmental feature that causes the visual sensor to fail in detecting wheelset damage, T 1 , T 2 , T 3 , ..., T m represents the image features of the 1st, 2nd, 3rd, ..., mth wheelset, where m is a positive integer, and F i Represents the set of wheel pair image features in the i-th relationship pair; similarly, the relationship pairs of all environmental features are constructed.

[0009] By extracting environmental features and wheelset image features and forming relationship pairs, we can deeply analyze the reasons for detection failure and clarify the impact of different environmental factors on wheelset image features, so as to more accurately judge whether the detection results are reliable in real-time detection.

[0010] S300, when using a visual sensor to detect wheelset damage in real time, checking the generated real-time wheelset image, calculating the quality of the real-time wheelset image, and using a visual inspection quality threshold to determine whether the real-time visual inspection is qualified; Furthermore, the specific steps of using the visual inspection quality threshold to determine whether the real-time visual inspection is qualified are as follows: S301, when using a visual sensor to detect wheelset damage in real time, check the generated real-time wheelset image, calculate the three quality features of the real-time wheelset image, namely, sharpness_s, contrast Contrast_s, and signal-to-noise ratio SNR_s, and calculate the real-time wheelset image quality Ims according to the corresponding weights; S302, using the visual detection quality threshold to judge the real-time wheelset image quality, when Ims≥Imy, it is judged that the real-time visual detection of wheelset damage is qualified; when Ims<Imy, it is judged that the real-time visual detection of ferry damage is unqualified.

[0011] Using the visual inspection quality threshold to judge whether the real-time visual inspection is qualified can timely discover the situation where the image quality does not meet the requirements and ensure the accuracy of the inspection results.

[0012] S400, after determining that the real-time visual inspection is unqualified, collecting wheel set image features during the real-time visual inspection, and using the relationship to determine the environmental features that caused the real-time visual inspection failure; Furthermore, the specific steps of using the relationship to determine the environmental features that cause the failure of real-time visual detection are as follows: S401, after determining that the real-time visual inspection is unqualified, collect the wheel set image features during the real-time visual inspection, and calculate the similarity between the real-time wheel set image features and the wheel set image features in all related pairs, the formula is: ; In the formula, Sim (Fs, F i ) represents the similarity between the real-time wheelset image feature set and the wheelset image feature set in the i-th relationship pair, fs j represents the jth wheelset image feature in the real-time wheelset image feature set, f j represents the jth wheelset image feature in the i-th wheelset image feature set; m represents the total number of wheelset image features in the i-th wheelset image feature set; similarly, the similarity between the real-time wheelset image feature set and the n-th wheelset image feature set is calculated; S402, judging all calculated similarities, selecting the wheel pair image feature set in the relationship pair with the highest similarity, and extracting the environmental features corresponding to the wheel pair image feature set in the relationship pair as the environmental features that cause the real-time visual detection failure.

[0013] By analyzing the wheel set image features and matching them with the previously established relationship between environmental features and wheel set image features, we can accurately identify the specific environmental factors that caused the current detection failure. For example, factors such as excessive or weak light or dust obstruction affect the image quality, leading to detection failure. This helps to quickly locate the root cause of the problem.

[0014] S500. Obtain the types of sensors corresponding to the wheel set damage based on the data that changes when the wheel set is damaged. Collect the environmental data when different sensors failed to detect the wheel set damage in the past, and obtain the environmental factors that affect the detection quality of different sensors. Further, the specific steps to obtain the environmental factors that affect the detection quality of different sensors are as follows: S501. Obtain the types of sensors corresponding to the wheel set damage based on the data that changes when the wheel set is damaged. Collect the environmental data when different sensors failed to detect the wheel set damage in the past, extract the failure rates of different sensors in the historical detection of wheel set damage, where the failure rate represents the ratio of the number of detection failures to the total number of detections. Calculate the correlation coefficient between the detection failure of different sensors and each type of environmental data. The formula is: ; In the formula, ρ(S u , E v ) represents the correlation coefficient between the u-th type of sensor and the v-th type of environmental data, cov(S u , E v ) represents the covariance between the u-th type of sensor and the v-th type of environmental data, σS u represents the average failure rate of the u-th type of sensor, σE v represents the average value of the v-th type of environmental data, S u represents the failure rate of the u-th type of sensor, and E v represents the v-th type of environmental data. S502. Calculate the correlation coefficients between each sensor and different environmental data, and select the one with the largest correlation coefficient as the environmental factor that affects the detection quality of the corresponding sensor; in the same way, obtain the environmental factors that affect the detection quality of each sensor.

[0015] By analyzing the data that changes when the wheel set is damaged, it is possible to clarify which type of sensor can most effectively detect these changes, so as to select the appropriate type of sensor specifically. This helps to avoid using unsuitable sensors, improve the detection efficiency and accuracy, and reduce resource waste.

[0016] After understanding the environmental factors that affect the detection quality of different sensors, it is possible to monitor and control these environmental factors during actual detection, or select a more suitable sensor for the current environment according to the changes in environmental conditions for detection. This can reduce the interference of environmental factors on the detection results of sensors, improve the reliability and stability of detection, and reduce the occurrence of false alarms and missed detections.

[0017] S600. Select an auxiliary detection sensor using the types of environmental data that cause real-time visual detection failure and the environmental factors of different sensors, and use the auxiliary detection sensor and the visual sensor to detect the wheel set damage simultaneously.

[0018] Furthermore, the specific steps of using the auxiliary detection sensor and the visual sensor to simultaneously detect the damage of the wheelset are as follows: S601, selecting environmental factors of different sensors according to environmental features that cause real-time visual detection failure, and using sensors corresponding to environmental factors different from environmental features that cause real-time visual detection failure as auxiliary sensors for real-time detection; S602, after performing secondary detection of the wheelset damage using the auxiliary sensors, the detection results of all the auxiliary sensors are obtained, the detection results of the real-time visual sensor and all the auxiliary sensors are quantified, and a weight β is assigned to all the quantified detection results, and the weight is based on the ratio of the number of successful detections in the history of each sensor to the total number of successful detections; then all the quantified detection results are weightedly fused, and the formula is: ; In the formula, J final represents the detection result after fusion, β d represents the weight of the d-th auxiliary sensor, J d represents the detection result of the d-th auxiliary sensor, β s represents the weight of the visual sensor, J s Represents the detection result of the visual sensor, and C represents the total number of auxiliary sensors. The fused detection result is used to determine whether the wheelset is damaged.

[0019] When the visual sensor fails to detect due to specific environmental factors, the auxiliary detection sensor can provide additional detection information by virtue of its adaptability to the environmental factors or different detection principles, to make up for the shortcomings of the visual sensor, thereby improving the overall detection accuracy and reducing missed detections and false detections.

[0020] By combining multiple sensors for detection, the system has a stronger ability to adapt to different environmental conditions. Even in a complex and changing environment, it can ensure that there are sensors that can work normally, making the detection system more stable and reliable, and reducing the risk of system failures caused by environmental factors.

[0021] A wheelset damage detection system based on machine vision, which includes a data collection module, a visual inspection quality threshold calculation module, a relationship pair construction module, a real-time visual inspection judgment module, a sensor analysis module and an auxiliary inspection module; The data collection module is used to collect historical records of failures of visual sensors to detect wheelset damage; The visual inspection quality threshold calculation module is used to extract the wheelset image generated by the visual sensor in the record, analyze the wheelset image, calculate the wheelset image quality, and obtain the visual inspection quality threshold using the image quality in all records; The relationship pair building module is used to collect environmental data and wheelset images in the record, extract different types of environmental features and wheelset image features that cause visual detection failure, and match the environmental features with the wheelset image features to form a relationship pair; The real-time visual detection judgment module is used to judge whether the wheel set damage quality detected by the real-time visual sensor is qualified, and if it is unqualified, to judge the environmental characteristics that lead to the unqualified; The sensor analysis module is used to obtain the type of sensor corresponding to the wheel set damage detection based on the data that changes when the wheel set is damaged, and calculate the environmental factors that affect the detection quality of different sensors; The auxiliary detection module is used to select auxiliary detection sensors using the types of environmental data that cause real-time visual detection failure and environmental factors of different sensors, and to detect wheelset damage simultaneously using the auxiliary detection sensors and visual sensors.

[0022] The real-time visual detection and judgment module includes a detection quality judgment unit and an environmental feature analysis unit; The detection quality judgment unit is used to judge whether the real-time visual detection is qualified by using the visual detection quality threshold; The environmental feature analysis unit is used to collect wheel set image features during real-time visual inspection after determining that the real-time visual inspection is unqualified, and use relationship pairs to determine the environmental features that cause the failure of the real-time visual inspection.

[0023] The auxiliary detection module includes an auxiliary sensor determination unit and a result fusion unit; The auxiliary sensor determination unit is used to use sensors corresponding to environmental factors different from the environmental features that cause the real-time visual detection failure as auxiliary sensors during real-time detection; The result fusion unit is used to quantify the detection results of all sensors, assign weights, and perform weighted fusion of all sensor detection results to obtain a final detection result.

[0024] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention extracts environmental features and wheelset image features and forms a relationship pair, which can deeply analyze the reasons for detection failure and clarify the impact of different environmental factors on wheelset image features, so as to more accurately judge whether the detection results are reliable in real-time detection.

[0025] 2. In the present invention, when the visual sensor fails to detect due to specific environmental factors, the auxiliary detection sensor can provide additional detection information by virtue of its adaptability to the environmental factors or different detection principles, thereby making up for the shortcomings of the visual sensor, thereby improving the overall detection accuracy and reducing missed detections and false detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a module distribution diagram of the wheelset damage detection system based on machine vision of the present invention; Figure 2 The figure is a schematic diagram of the steps of the wheelset damage detection method based on machine vision of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution. A wheelset damage detection method based on machine vision, the method comprising the following steps: S100, collecting historical records of failed detections when visual sensors detect wheelset damage, extracting wheelset images generated by the visual sensors in the records, analyzing the wheelset images, calculating the wheelset image quality, and obtaining a visual detection quality threshold using the image quality in all records; The specific steps to obtain the visual detection quality threshold using the image quality in all records are: S101, collect the records of failed detection when the visual sensor detects wheelset damage, extract the wheelset image generated by the visual sensor detection in the records, collect each pixel point in the wheelset image as P (x, y), calculate the square of the horizontal coordinate gradient and the sum of the square of the vertical coordinate gradient of each pixel point, and then sum the square sums of all pixel points to obtain the sharpness of each wheelset image; the formula is: ; In the formula, Represents the horizontal coordinate gradient of the pixel point P (x, y), represents the gradient of pixel point P (x, y) in the ordinate, and G represents the total number of pixels; S102, extract the pixel value of each pixel in the wheel pair image, calculate the average pixel value of all pixels, calculate the square of the difference between the pixel value of each pixel and the average pixel value, and sum and average the square of the difference of all pixels to obtain the contrast of each wheel pair image; the formula is: ; In the formula, R represents the average value of all pixel values, Ps (x, y) g Represents the pixel value of pixel point P (x, y); S103, converting the wheelset image into a digital signal, extracting the average power of the signal and the average power of the noise in each wheelset image, and calculating the signal-to-noise ratio (SNR) of each wheelset image using the average power of the signal and the average power of the noise; S104. Using the principal component analysis method, the three quality features of sharpness, contrast and signal-to-noise ratio (SNR) are converted into a feature matrix, a covariance matrix of the feature matrix is ​​constructed, the eigenvalues ​​and eigenvectors of the covariance matrix are calculated, and the proportion of the eigenvalue of each quality feature to the total eigenvalue is calculated as the weight of each quality feature. After obtaining the weights corresponding to the three quality features, the three quality features are weighted and summed in each wheel pair image to obtain the quality of each wheel pair image; the formula is: ; In the formula, Q represents the wheelset image quality, ω 1 Represents the weight of clarity, ω 2 represents the contrast weight, ω 3 represents the weight of the signal-to-noise ratio; S105, after obtaining the image qualities of all wheel pairs that have failed detection in history, calculate the average and standard deviation of the wheel pair image qualities, and obtain the visual detection quality threshold Imy by subtracting the standard deviation from the average.

[0029] By collecting historical detection failure records and analyzing the wheelset image quality, the visual detection quality threshold is obtained, which provides a quantitative standard for judging the real-time wheelset image quality. This helps to accurately identify detection failures caused by image quality problems and avoid misjudgments and missed judgments.

[0030] The quality threshold can also be used as an important indicator to evaluate the performance of the visual inspection system. By comparing the relationship between the image quality and the threshold collected at different times or with different devices, we can understand the performance changes of the system, timely discover possible degradation or failure of the system, and provide a basis for system maintenance and upgrades.

[0031] S200, in the historical visual inspection failure records, collecting environmental data and wheelset images in the records, extracting different types of environmental features and wheelset image features that cause visual inspection failures, and matching the environmental features with the wheelset image features to form a relationship pair; The specific steps of matching the environmental data with the wheel set image features to form a relationship pair are: S201, extracting environmental data and wheelset images from the historical records of failed detections of wheelset damage by visual sensors, extracting environmental data when the visual sensors successfully detect wheelset damage, calculating the difference between all environmental data when the detections are successful and when they fail, extracting the type of environmental data corresponding to the maximum difference for each detection failure record as the environmental feature that causes the failure of the visual sensor to detect wheelset damage, assuming that all the environmental features that cause the failure of the visual sensor to detect wheelset damage are {H 1 , H 2 , H 3 , ..., H n}, H 1 , H 2 , H 3 , ..., H n represents the first, second, third, ..., nth environmental features that cause the visual sensor to fail in detecting wheelset damage, where n is a positive integer; S202, for each record of environmental features that cause the visual sensor to fail to detect wheelset damage, use the convolutional neural network CNN to analyze the wheelset images in all records and extract the wheelset image features; for each failure record, construct a relationship pair H i →F i ={T 1 , T 2 , T 3 , ..., T m}, where H i represents the i-th environmental feature that causes the visual sensor to fail in detecting wheelset damage, T 1 , T 2 , T 3 , ..., T m represents the image features of the 1st, 2nd, 3rd, ..., mth wheelset, where m is a positive integer, and F i Represents the set of wheel pair image features in the i-th relationship pair; similarly, the relationship pairs of all environmental features are constructed.

[0032] By extracting environmental features and wheelset image features and forming relationship pairs, we can deeply analyze the reasons for detection failure and clarify the impact of different environmental factors on wheelset image features, so as to more accurately judge whether the detection results are reliable in real-time detection.

[0033] S300, when using a visual sensor to detect wheelset damage in real time, checking the generated real-time wheelset image, calculating the quality of the real-time wheelset image, and using a visual inspection quality threshold to determine whether the real-time visual inspection is qualified; The specific steps of using visual inspection quality threshold to determine whether real-time visual inspection is qualified are as follows: S301, when using a visual sensor to detect wheelset damage in real time, check the generated real-time wheelset image, calculate the three quality features of the real-time wheelset image, namely, sharpness_s, contrast Contrast_s, and signal-to-noise ratio SNR_s, and calculate the real-time wheelset image quality Ims according to the corresponding weights; S302, using the visual detection quality threshold to judge the real-time wheelset image quality, when Ims≥Imy, it is judged that the real-time visual detection of wheelset damage is qualified; when Ims<Imy, it is judged that the real-time visual detection of ferry damage is unqualified.

[0034] Using the visual inspection quality threshold to judge whether the real-time visual inspection is qualified can timely discover the situation where the image quality does not meet the requirements and ensure the accuracy of the inspection results.

[0035] S400, after determining that the real-time visual inspection is unqualified, collecting wheel set image features during the real-time visual inspection, and using the relationship to determine the environmental features that caused the real-time visual inspection failure; The specific steps of using relationship pairs to determine the environmental features that cause the failure of real-time visual detection are: S401, after determining that the real-time visual inspection is unqualified, collect the wheel set image features during the real-time visual inspection, and calculate the similarity between the real-time wheel set image features and the wheel set image features in all related pairs, the formula is: ; In the formula, Sim (Fs, F i ) represents the similarity between the real-time wheelset image feature set and the wheelset image feature set in the i-th relationship pair, fs j represents the jth wheelset image feature in the real-time wheelset image feature set, f j represents the jth wheelset image feature in the i-th wheelset image feature set; m represents the total number of wheelset image features in the i-th wheelset image feature set; similarly, the similarity between the real-time wheelset image feature set and the n-th wheelset image feature set is calculated; S402, judging all calculated similarities, selecting the wheel pair image feature set in the relationship pair with the highest similarity, and extracting the environmental features corresponding to the wheel pair image feature set in the relationship pair as the environmental features that cause the real-time visual detection failure.

[0036] By analyzing the wheel set image features and matching them with the previously established relationship between environmental features and wheel set image features, we can accurately identify the specific environmental factors that caused the current detection failure. For example, factors such as excessive or weak light or dust obstruction affect the image quality, leading to detection failure. This helps to quickly locate the root cause of the problem.

[0037] S500, based on the data that changes when a wheelset is damaged, obtain the type of sensor corresponding to the type of wheelset damage detection, collect the environmental data when different sensors fail to detect wheelset damage in the past, and obtain the environmental factors that affect the detection quality of different sensors; The specific steps to obtain the environmental factors that affect the detection quality of different sensors are: S501. Based on the data that changes when a wheelset is damaged, the type of sensor corresponding to the wheelset damage detection is obtained, and the environmental data when different sensors fail to detect wheelset damage in the past are collected. The failure rate of different sensors in the past detecting wheelset damage is extracted. The failure rate represents the ratio of the number of detection failures to the total number of detections. The correlation coefficient between the detection failure of different sensors and each type of environmental data is calculated. The formula is: ; In the formula, ρ(S u , E v ) represents the correlation coefficient between the u-th sensor and the v-th environmental data, cov(S u , E v ) represents the covariance of the u-th sensor and the v-th environment data, σS u represents the average failure rate of the u-th sensor, σE v represents the average value of the vth environmental data, S u represents the failure rate of the u-th sensor, E v Indicates the vth type of environmental data; S502, calculate the correlation coefficient between each sensor and different environmental data, and select the one with the largest correlation coefficient as the environmental factor affecting the detection quality of the corresponding sensor; and calculate the environmental factor affecting the detection quality of each sensor in the same way.

[0038] By analyzing the data of changes in wheelset damage, it is possible to determine which type of sensor can most effectively detect these changes, thereby selecting the appropriate sensor type in a targeted manner. This helps avoid the use of unsuitable sensors, improves detection efficiency and accuracy, and reduces resource waste.

[0039] After understanding the environmental factors that affect the detection quality of different sensors, these environmental factors can be monitored and controlled in actual detection, or sensors that are more suitable for the current environment can be selected for detection according to changes in environmental conditions. This can reduce the interference of environmental factors on sensor detection results, improve the reliability and stability of detection, and reduce the occurrence of false alarms and missed alarms.

[0040] S600: Select an auxiliary detection sensor by using the type of environmental data that causes the real-time visual detection failure and the environmental factors of different sensors, and use the auxiliary detection sensor and the visual sensor to simultaneously detect the wheelset damage.

[0041] The specific steps of using auxiliary detection sensors and visual sensors to detect wheelset damage simultaneously are as follows: S601, selecting environmental factors of different sensors according to environmental features that cause real-time visual detection failure, and using sensors corresponding to environmental factors different from environmental features that cause real-time visual detection failure as auxiliary sensors for real-time detection; S602, after performing secondary detection of the wheelset damage using the auxiliary sensors, the detection results of all the auxiliary sensors are obtained, the detection results of the real-time visual sensor and all the auxiliary sensors are quantified, and a weight β is assigned to all the quantified detection results, and the weight is based on the ratio of the number of successful detections in the history of each sensor to the total number of successful detections; then all the quantified detection results are weightedly fused, and the formula is: ; In the formula, J final represents the detection result after fusion, β d represents the weight of the d-th auxiliary sensor, J d represents the detection result of the d-th auxiliary sensor, β s represents the weight of the visual sensor, J s Represents the detection result of the visual sensor, and C represents the total number of auxiliary sensors. The fused detection result is used to determine whether the wheelset is damaged.

[0042] When the visual sensor fails to detect due to specific environmental factors, the auxiliary detection sensor can provide additional detection information by virtue of its adaptability to the environmental factors or different detection principles, to make up for the shortcomings of the visual sensor, thereby improving the overall detection accuracy and reducing missed detections and false detections.

[0043] By combining multiple sensors for detection, the system has a stronger ability to adapt to different environmental conditions. Even in a complex and changing environment, it can ensure that there are sensors that can work normally, making the detection system more stable and reliable, and reducing the risk of system failures caused by environmental factors.

[0044] A wheelset damage detection system based on machine vision, which includes a data collection module, a visual inspection quality threshold calculation module, a relationship pair construction module, a real-time visual inspection judgment module, a sensor analysis module and an auxiliary inspection module; The data collection module is used to collect historical records of failures of visual sensors to detect wheelset damage; The visual inspection quality threshold calculation module is used to extract the wheelset image generated by the visual sensor in the record, analyze the wheelset image, calculate the wheelset image quality, and obtain the visual inspection quality threshold using the image quality in all records; The relationship pair building module is used to collect environmental data and wheelset images in the record, extract different types of environmental features and wheelset image features that cause visual detection failure, and match the environmental features with the wheelset image features to form a relationship pair; The real-time visual detection judgment module is used to judge whether the wheel set damage quality detected by the real-time visual sensor is qualified, and if it is unqualified, to judge the environmental characteristics that lead to the unqualified; The sensor analysis module is used to obtain the type of sensor corresponding to the wheel set damage detection based on the data that changes when the wheel set is damaged, and calculate the environmental factors that affect the detection quality of different sensors; The auxiliary detection module is used to select auxiliary detection sensors using the types of environmental data that cause real-time visual detection failure and environmental factors of different sensors, and to detect wheelset damage simultaneously using the auxiliary detection sensors and visual sensors.

[0045] The real-time visual detection and judgment module includes a detection quality judgment unit and an environmental feature analysis unit; The detection quality judgment unit is used to judge whether the real-time visual detection is qualified by using the visual detection quality threshold; The environmental feature analysis unit is used to collect wheel set image features during real-time visual inspection after determining that the real-time visual inspection is unqualified, and use relationship pairs to determine the environmental features that cause the failure of the real-time visual inspection.

[0046] The auxiliary detection module includes an auxiliary sensor determination unit and a result fusion unit; The auxiliary sensor determination unit is used to use sensors corresponding to environmental factors different from the environmental features that cause the real-time visual detection failure as auxiliary sensors during real-time detection; The result fusion unit is used to quantify the detection results of all sensors, assign weights, and perform weighted fusion of all sensor detection results to obtain a final detection result.

[0047] Embodiment: A system using a visual sensor to detect wheelset damage is analyzed, and the clarity, contrast and signal-to-noise ratio of historical wheelset images are calculated using a formula. The weights of each type of data obtained by principal component analysis are assumed to be 0.5, 0.3 and 0.2 respectively; the three types of data are standardized to values ​​in the same range, and the quality of one of the historical wheelset images is calculated to be 28; the quality threshold is calculated to be 35 based on the quality of the historical wheelset images; Relationship pairs are constructed based on historical environmental data. For example, relationship pair 1 is illumination → {clarity 3000, brightness 520, exposure 4500}. The visual sensor is used to detect the damage of the wheelset in real time. The real-time detection quality is calculated to be 30, and the detection quality is judged to be unqualified. The extracted wheel pair image features are clarity 2900, contrast 90, and exposure 4445. The highest similarity is obtained through calculation, and it is determined that the environmental feature that causes the failure of real-time visual detection is illumination. The sensors used to detect wheelset damage in the collection history include laser sensors and ultrasonic sensors. The calculation and analysis show that the environmental factors of the two sensors are light and sound waves. In real-time detection, the ultrasonic sensor is selected as the auxiliary sensor. The detection result of the visual sensor is quantified to 3, the weight is 0.3, and the detection result of the acoustic sensor is quantified to 9, the weight is 0.7. The final result is calculated to be 7.2.

[0048] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A wheelset damage detection method based on machine vision, characterized in that: The method comprises the following steps: S100, collecting historical records of failed detections when visual sensors detect wheelset damage, extracting wheelset images generated by the visual sensors in the records, analyzing the wheelset images, calculating the wheelset image quality, and obtaining a visual detection quality threshold using the image quality in all records; S200, in the historical visual inspection failure records, collecting environmental data and wheelset images in the records, extracting different types of environmental features and wheelset image features that cause visual inspection failures, and matching the environmental features with the wheelset image features to form a relationship pair; S300, when using a visual sensor to detect wheelset damage in real time, checking the generated real-time wheelset image, calculating the quality of the real-time wheelset image, and using a visual inspection quality threshold to determine whether the real-time visual inspection is qualified; S400, after determining that the real-time visual inspection is unqualified, collecting wheel set image features during the real-time visual inspection, and using the relationship to determine the environmental features that caused the real-time visual inspection failure; S500, based on the data that changes when a wheelset is damaged, obtain the type of sensor corresponding to the type of wheelset damage detection, collect the environmental data when different sensors fail to detect wheelset damage in the past, and obtain the environmental factors that affect the detection quality of different sensors; S600: Select an auxiliary detection sensor by using the type of environmental data that causes the real-time visual detection failure and the environmental factors of different sensors, and use the auxiliary detection sensor and the visual sensor to simultaneously detect the wheelset damage.

2. The wheelset damage detection method based on machine vision according to claim 1, characterized in that: The specific steps of obtaining the visual detection quality threshold by using the image quality in all records in S100 are: S101, collecting the records of failed detections when the visual sensor detects wheelset damage, extracting the wheelset images generated by the visual sensor detection in the records, collecting each pixel point in the wheelset image as P (x, y), calculating the square of the horizontal coordinate gradient and the sum of the square of the vertical coordinate gradient of each pixel point, and then summing the square sums of all the pixels to obtain the sharpness of each wheelset image; S102, extracting the pixel value of each pixel in the wheel pair image, calculating the average pixel value of all the pixels, calculating the square of the difference between the pixel value of each pixel and the average pixel value, and summing and averaging the squares of the difference of all the pixels to obtain the contrast of each wheel pair image; S103, converting the wheelset image into a digital signal, extracting the average power of the signal and the average power of the noise in each wheelset image, and calculating the signal-to-noise ratio (SNR) of each wheelset image using the average power of the signal and the average power of the noise; S104, using the principal component analysis method, converting the three quality features of sharpness, contrast and signal-to-noise ratio (SNR) into a feature matrix, constructing a covariance matrix of the feature matrix, calculating the eigenvalues ​​and eigenvectors of the covariance matrix, calculating the ratio of the eigenvalue of each quality feature to the total eigenvalues ​​as the weight of each quality feature, and after obtaining the weights corresponding to the three quality features, performing weighted summation on the three quality features in each wheel pair image to obtain the quality of each wheel pair image; S105, after obtaining the image qualities of all wheel pairs that have failed detection in history, calculate the average and standard deviation of the wheel pair image qualities, and obtain the visual detection quality threshold Imy by subtracting the standard deviation from the average.

3. The wheelset damage detection method based on machine vision according to claim 1, characterized in that: The specific steps of matching the environmental data with the wheel set image features to form a relationship pair in S200 are: S201, extracting environmental data and wheelset images from the historical records of failed visual sensor detection of wheelset damage, extracting environmental data when the visual sensor successfully detects wheelset damage, calculating the difference between all environmental data when the detection is successful and when it fails, and extracting the type of environmental data corresponding to the maximum difference for each detection failure record as the environmental feature that causes the visual sensor to fail to detect wheelset damage, assuming that all the environmental features that cause the visual sensor to fail to detect wheelset damage are {H1, H2, H3, ..., H n }, H1, H2, H3, ..., H n represents the first, second, third, ..., nth environmental features that cause the visual sensor to fail in detecting wheelset damage, where n is a positive integer; S202, for each record of environmental features that cause the visual sensor to fail to detect wheelset damage, use the convolutional neural network CNN to analyze the wheelset images in all records and extract the wheelset image features; for each failure record, construct a relationship pair H i →F i ={T1, T2, T3, ..., T m }, where H i represents the i-th environmental feature that causes the visual sensor to fail in detecting wheelset damage, T1, T2, T3, ..., T m represents the image features of the 1st, 2nd, 3rd, ..., mth wheelset, where m is a positive integer, and F i Represents the set of wheel pair image features in the i-th relationship pair; similarly, the relationship pairs of all environmental features are constructed.

4. The wheelset damage detection method based on machine vision according to claim 2 is characterized in that: The specific steps of using the visual inspection quality threshold in S300 to determine whether the real-time visual inspection is qualified are: S301, when using a visual sensor to detect wheelset damage in real time, check the generated real-time wheelset image, calculate the three quality features of the real-time wheelset image, namely, sharpness_s, contrast Contrast_s, and signal-to-noise ratio SNR_s, and calculate the real-time wheelset image quality Ims according to the corresponding weights; S302, using the visual detection quality threshold to judge the real-time wheelset image quality, when Ims≥Imy, it is judged that the real-time visual detection of wheelset damage is qualified; when Ims<Imy, it is judged that the real-time visual detection of ferry damage is unqualified.

5. The method for detecting wheelset damage based on machine vision according to claim 3 is characterized in that: The specific steps of using the relationship pair to determine the environmental features that cause the real-time visual detection failure in S400 are: S401, after determining that the real-time visual inspection is unqualified, collect the wheel set image features during the real-time visual inspection, and calculate the similarity between the real-time wheel set image features and the wheel set image features in all related pairs, the formula is: ； In the formula, Sim (Fs, F i ) represents the similarity between the real-time wheelset image feature set and the wheelset image feature set in the i-th relationship pair, fs j represents the jth wheelset image feature in the real-time wheelset image feature set, f j represents the jth wheelset image feature in the i-th wheelset image feature set; m represents the total number of wheelset image features in the i-th wheelset image feature set; similarly, the similarity between the real-time wheelset image feature set and the n-th wheelset image feature set is calculated; S402, judging all calculated similarities, selecting the wheel pair image feature set in the relationship pair with the highest similarity, and extracting the environmental features corresponding to the wheel pair image feature set in the relationship pair as the environmental features that cause the real-time visual detection failure.

6. The wheelset damage detection method based on machine vision according to claim 1, characterized in that: The specific steps of obtaining the environmental factors that affect the detection quality of different sensors in S500 are: S501. Based on the data that changes when a wheelset is damaged, the type of sensor corresponding to the wheelset damage detection is obtained, and the environmental data when different sensors fail to detect wheelset damage in the past are collected. The failure rate of different sensors in the past detecting wheelset damage is extracted. The failure rate represents the ratio of the number of detection failures to the total number of detections. The correlation coefficient between the detection failure of different sensors and each type of environmental data is calculated. The formula is: ； In the formula, ρ(S u , E v ) represents the correlation coefficient between the u-th sensor and the v-th environmental data, cov(S u , E v ) represents the covariance of the u-th sensor and the v-th environment data, σS u represents the average failure rate of the u-th sensor, σE v represents the average value of the vth environmental data, S u represents the failure rate of the u-th sensor, E v Indicates the vth type of environmental data; S502, calculate the correlation coefficient between each sensor and different environmental data, and select the one with the largest correlation coefficient as the environmental factor affecting the detection quality of the corresponding sensor; and calculate the environmental factor affecting the detection quality of each sensor in the same way.

7. The method for detecting wheelset damage based on machine vision according to claim 6, characterized in that: The specific steps of using the auxiliary detection sensor and the visual sensor to simultaneously detect the damage to the wheelset in S600 are: S601, selecting environmental factors of different sensors according to environmental features that cause real-time visual detection failure, and using sensors corresponding to environmental factors different from environmental features that cause real-time visual detection failure as auxiliary sensors for real-time detection; S602, after performing secondary detection of the wheelset damage using the auxiliary sensors, the detection results of all the auxiliary sensors are obtained, the detection results of the real-time visual sensor and all the auxiliary sensors are quantified, and a weight β is assigned to all the quantified detection results, and the weight is based on the ratio of the number of successful detections in the history of each sensor to the total number of successful detections; then all the quantified detection results are weightedly fused, and the formula is: ； In the formula, J final represents the detection result after fusion, β d represents the weight of the d-th auxiliary sensor, J d represents the detection result of the d-th auxiliary sensor, β s represents the weight of the visual sensor, J s represents the detection result of the visual sensor, and C represents the total number of auxiliary sensors; The fused detection results are used to determine whether the wheelset is damaged.

8. The wheel set damage detection system based on machine vision is characterized by: The wheelset damage detection system includes a data collection module, a visual inspection quality threshold calculation module, a relationship pair construction module, a real-time visual inspection judgment module, a sensor analysis module and an auxiliary detection module; The data collection module is used to collect historical records of failures of visual sensors to detect wheelset damage; The visual inspection quality threshold calculation module is used to extract the wheelset image generated by the visual sensor in the record, analyze the wheelset image, calculate the wheelset image quality, and obtain the visual inspection quality threshold using the image quality in all records; The relationship pair building module is used to collect environmental data and wheelset images in the record, extract different types of environmental features and wheelset image features that cause visual detection failure, and match the environmental features with the wheelset image features to form a relationship pair; The real-time visual detection judgment module is used to judge whether the wheel set damage quality detected by the real-time visual sensor is qualified, and if it is unqualified, to judge the environmental characteristics that lead to the unqualified; The sensor analysis module is used to obtain the type of sensor corresponding to the wheel set damage detection based on the data that changes when the wheel set is damaged, and calculate the environmental factors that affect the detection quality of different sensors; The auxiliary detection module is used to select auxiliary detection sensors using the types of environmental data that cause real-time visual detection failure and environmental factors of different sensors, and to detect wheelset damage simultaneously using the auxiliary detection sensors and visual sensors.

9. The wheelset damage detection system based on machine vision according to claim 8, characterized in that: The real-time visual detection and judgment module includes a detection quality judgment unit and an environmental feature analysis unit; The detection quality judgment unit is used to judge whether the real-time visual detection is qualified by using the visual detection quality threshold; The environmental feature analysis unit is used to collect wheel set image features during real-time visual inspection after determining that the real-time visual inspection is unqualified, and use relationship pairs to determine the environmental features that cause the failure of the real-time visual inspection.

10. The wheelset damage detection system based on machine vision according to claim 8, characterized in that: The auxiliary detection module includes an auxiliary sensor determination unit and a result fusion unit; The auxiliary sensor determination unit is used to use sensors corresponding to environmental factors different from the environmental features that cause the real-time visual detection failure as auxiliary sensors during real-time detection; The result fusion unit is used to quantify the detection results of all sensors, assign weights, and perform weighted fusion of all sensor detection results to obtain a final detection result.

Images