A method and apparatus for analyzing the integrity of a vehicle underbody component
By using a pre-trained semantic segmentation model and image comparison technology, the problems of insufficient generalization ability and insufficient dataset in the integrity analysis of locomotive undercarriage components are solved, and efficient and accurate integrity assessment of undercarriage components is achieved.
Patent Information
- Application Number
- CN202310594651.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2043-05-24
AI Technical Summary
Existing technologies lack generalization ability in the integrity analysis of locomotive undercarriage components, are difficult to collect negative samples, and have high dataset requirements, which leads to a decrease in the accuracy of detection models or their failure, especially when there is a variety of undercarriage component categories, making effective detection difficult.
A pre-trained semantic segmentation model is used to compare previous and latest vehicle undercarriage images. Through semantic segmentation, location registration, and image comparison, and by using binary and grayscale image comparison methods, the integrity of vehicle undercarriage components is evaluated, which makes up for the lack of dataset and improves the generalization ability of the detection model.
It enables efficient assessment of the health status of various components under the locomotive undercarriage in the absence of a large amount of training data, expands the detection range, and improves the accuracy and robustness of detection.
Smart Images
Figure CN116664505B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of visual inspection technology, specifically relating to a method and apparatus for analyzing the integrity of vehicle underbody components. Background Technology
[0002] The integrity analysis of the undercarriage components of existing locomotives mainly relies on manual inspection. In some special cases, such as the detection of missing undercarriage bolts, some automated solutions have been proposed, but these cases only account for a very small part compared to the number of categories of undercarriage components.
[0003] In automated integrity analysis solutions, target detection technology is typically used to analyze images of the vehicle's undercarriage. Once the computer identifies a part in the area to be detected, it also assesses the part's integrity. The results then display the target part's location and status. This method can solve single-category component detection problems. While it accurately determines the location and status of individual components, it lacks generalization ability. Furthermore, for most undercarriage components, the lack of negative samples (or the inability to collect negative samples at all) can significantly reduce the accuracy of the detection model or even cause it to fail.
[0004] Meanwhile, current mainstream detection technologies integrate object detection and object judgment, training a deep learning network to directly distinguish the location and state (intact or damaged) of components. However, detection models designed for a limited number of components cannot be directly generalized to the detection of all components. Their generalization ability will be severely tested. Moreover, most models lack generalization ability; they essentially only classify the content of the training dataset. For components beyond the scope of the dataset, the lack of model generalization ability significantly reduces their detection capability.
[0005] Furthermore, due to the widespread use of supervised learning in current technologies, the requirements for datasets are extremely high. However, in practical applications, collecting datasets can be very difficult. Some components are designed for very long lifespans and exhibit strong stability, making the collection of negative samples time-consuming and labor-intensive. Moreover, a small number of negative samples do not have a significant impact on the network. In such cases, the network cannot effectively distinguish between positive and negative samples during training, while collecting a large number of negative samples is simply impossible.
[0006] In summary, the aforementioned problems have hindered the application of locomotive undercarriage integrity analysis. Summary of the Invention
[0007] Based on the above technical problems, this application proposes a method and apparatus for analyzing the integrity of vehicle underbody components.
[0008] Firstly, this application proposes a method for analyzing the integrity of vehicle underbody components, comprising the following steps:
[0009] The historical undercarriage image of the locomotive collected previously is input into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image with the undercarriage component whose integrity is to be detected as the region of interest.
[0010] The locomotive undercarriage image collected in this instance is input into a pre-trained semantic segmentation model to obtain the latest semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest.
[0011] The previous semantic segmentation image is registered with the latest semantic segmentation image to obtain the latest registered image;
[0012] The regions of interest in the previous semantic segmentation image and the latest registration image are compared to obtain the comparison result, which is then determined as the integrity analysis result of the vehicle underbody components.
[0013] The pre-training process of the semantic segmentation model includes the following steps:
[0014] Collect historical images of the locomotive's undercarriage;
[0015] The historical undercarriage images of the locomotive are randomly cropped, and the historical undercarriage images of the locomotive and the randomly cropped images are combined into a training set.
[0016] The training set is input into the semantic segmentation network for training to obtain the semantic segmentation model.
[0017] The semantic segmentation network is a deepLabV3+ training network.
[0018] The step of performing positional registration between the previous semantic segmentation image and the latest semantic segmentation image to obtain the latest registered image includes the following steps:
[0019] Extract the region of interest from the previous semantic segmentation image as the template image;
[0020] The size of the template image is used as the window size. In the latest semantic segmentation image, the window is slidable according to the window size. In each sliding window process, the position difference between each pixel in the window and the corresponding pixel in the template image is compared according to the position arrangement of the pixels.
[0021] After the window traverses all the positions in the latest semantic segmentation images, the pixel of the corresponding template image with the smallest position difference is taken as the best registration point.
[0022] Using the optimal registration point as a reference, the latest semantic segmentation image is registered with the template image to obtain the latest registered image.
[0023] The step of comparing the regions of interest in the previous semantic segmentation image and the latest registration image to obtain the comparison result includes the following steps:
[0024] The previous semantic segmentation image and the latest registration image are compared as binary images to obtain the binary image comparison result.
[0025] If the binary image comparison result shows that the undercarriage is incomplete, then the final comparison result is that the undercarriage is incomplete.
[0026] If the binary image comparison result shows that the vehicle bottom part is complete, then the previous semantic segmentation image and the latest registration image are compared in grayscale to obtain the grayscale comparison result, and the grayscale comparison result is used as the final comparison result.
[0027] The step of performing binary image comparison to obtain the binary image comparison result includes the following steps:
[0028] Based on the region of interest, the previous semantic segmentation image and the latest registration image are binarized to obtain the previous binary image and the latest binary image, respectively.
[0029] According to the order of pixel positions, the pixel values of the corresponding pixels in the previous binary image and the latest binary image are compared and the absolute values are taken to obtain a difference image of binary pixel values.
[0030] Calculate the rate of change of the difference map of binarized pixel values;
[0031] When the rate of change is greater than or equal to the first threshold, the binary image comparison result is: the undercarriage component to be detected in the undercarriage image collected this time is incomplete.
[0032] The rate of change of the difference map of binarized pixel values is calculated using the following formula:
[0033]
[0034] Where N1 is the number of pixels whose pixel values have changed in both the previous and latest binary images, N all is the total number of pixels in the difference map, and Var is the rate of change of the binary difference map.
[0035] The step of comparing the previous semantic segmentation image with the latest registration image in grayscale to obtain the grayscale comparison result includes the following steps:
[0036] The previous semantic segmentation image and the latest registration image are converted to grayscale to obtain the previous grayscale image and the latest grayscale image, respectively.
[0037] According to the order of pixel positions, the grayscale difference is calculated between the corresponding pixels of the previous grayscale image and the corresponding pixels of the latest grayscale image, and the absolute value is taken to obtain the grayscale difference image.
[0038] Calculate the average gray-level difference of all pixels in the gray-level difference image;
[0039] When the average grayscale difference is greater than or equal to the second threshold, the grayscale comparison result is: the undercarriage component to be detected in the undercarriage image of the locomotive collected this time is incomplete.
[0040] Secondly, this application proposes a vehicle underbody component integrity analysis device, comprising:
[0041] The first semantic segmentation module is used to input the previously acquired historical undercarriage image of the locomotive into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image with the undercarriage component whose integrity is to be detected as the region of interest.
[0042] The second semantic segmentation module is used to input the locomotive undercarriage image collected in the current session into the pre-trained semantic segmentation model to obtain the latest semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest.
[0043] The image registration module is used to perform positional registration between the previous semantic segmentation image and the latest semantic segmentation image to obtain the latest registered image;
[0044] The image comparison module is used to compare the regions of interest in the previous semantic segmentation image and the latest registration image to obtain a comparison result, and the comparison result is determined as the integrity analysis result of the vehicle underbody components.
[0045] Thirdly, this application proposes an electronic device, including: one or more processors, and a memory, the memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the vehicle underbody component integrity analysis method.
[0046] Fourthly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the vehicle underbody component integrity analysis method.
[0047] Beneficial effects:
[0048] This application proposes a method and apparatus for analyzing the integrity of undercarriage components. Addressing the issue of limited but large-sized images of the acquired components, a training method for the dataset is established. A method comparing undercarriage images from the previous locomotive with those from the most recent locomotive is employed, along with a specific quantification method for the comparison process, leading to conclusions on the integrity of the undercarriage components. This solves the problem of limited negative samples in existing technologies, expands the range of components to be inspected, and enables the assessment of the health status of various undercarriage components. Attached Figure Description
[0049] Figure 1 This is a flowchart of a method for analyzing the integrity of a vehicle underbody component according to an embodiment of this application;
[0050] Figure 2 This is a schematic diagram of the original training set images in an embodiment of this application;
[0051] Figure 3 This is for the pre-training of the semantic segmentation model in the embodiments of this application;
[0052] Figure 4 This is a flowchart illustrating the position registration process in an embodiment of this application.
[0053] Figure 5 This is a flowchart illustrating the comparison of regions of interest in the previous semantic segmentation image and the latest registration image in an embodiment of this application.
[0054] Figure 6 This is a flowchart illustrating the binary image comparison process in an embodiment of this application.
[0055] Figure 7 This is a flowchart illustrating the grayscale image comparison process in an embodiment of this application.
[0056] Figure 8 shows the historical undercarriage map and simulation results of the locomotive collected in the previous embodiment of this application. Figure 8(a) shows the historical undercarriage map of the locomotive collected in the previous embodiment, and Figure 8(b) shows the simulation results.
[0057] Figure 9 This is a schematic diagram of a vehicle underbody component integrity analysis device according to an embodiment of this application. Detailed Implementation
[0058] The present disclosure will be further described below with reference to the embodiments shown in the accompanying drawings.
[0059] The integrity analysis of the undercarriage components of existing locomotives mainly relies on manual inspection. In some special cases, such as the detection of missing undercarriage bolts, some automated solutions have been proposed, but these cases only account for a very small part compared to the number of categories of undercarriage components.
[0060] In automated integrity analysis solutions, target detection technology is typically used to analyze images of the vehicle's undercarriage. Once the computer identifies a part in the area to be detected, it also assesses the part's integrity. The results then display the target part's location and status. This method can solve single-category component detection problems. While it accurately determines the location and status of individual components, it lacks generalization ability. Furthermore, for most undercarriage components, the lack of negative samples (or the inability to collect negative samples at all) can significantly reduce the accuracy of the detection model or even cause it to fail.
[0061] Meanwhile, current mainstream detection technologies integrate object detection and object judgment, training a deep learning network to directly distinguish the location and state (intact or damaged) of components. However, detection models designed for a limited number of components cannot be directly generalized to the detection of all components. Their generalization ability will be severely tested. Moreover, most models lack generalization ability; they essentially only classify the content of the training dataset. For components beyond the scope of the dataset, the lack of model generalization ability significantly reduces their detection capability.
[0062] Furthermore, due to the widespread use of supervised learning in current technologies, the requirements for datasets are extremely high. However, in practical applications, collecting datasets can be very difficult. Some components are designed for very long lifespans and exhibit strong stability, making the collection of negative samples time-consuming and labor-intensive. Moreover, a small number of negative samples do not have a significant impact on the network. In such cases, the network cannot effectively distinguish between positive and negative samples during training, while collecting a large number of negative samples is simply impossible.
[0063] This application proposes a method and apparatus for analyzing the integrity of vehicle undercarriage components. Combining the advantages of semantic segmentation technology and the stability of image comparison technology, it analyzes the integrity of vehicle undercarriage components. In particular, when a training dataset is lacking, a method of breaking down a single vehicle undercarriage image into smaller parts is used to improve the utilization of the acquired images, compensate for the scarcity of the original dataset, and ultimately obtain relatively high-quality analysis results.
[0064] Example 1:
[0065] This embodiment proposes a method for analyzing the integrity of vehicle underbody components, such as... Figure 1 As shown, it includes the following steps:
[0066] Step S1: Input the previously collected historical undercarriage image into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest;
[0067] Historical vehicle undercarriage images are used to train the semantic segmentation model. Multiple acquisitions are generally required, and the more historical undercarriage images available, the more accurate the pre-trained segmentation model. However, in practical applications, the number of historical undercarriage images is usually insufficient. Therefore, this application, on the one hand, increases the training set samples based on the available historical undercarriage images; on the other hand, it inputs previously acquired historical undercarriage images of locomotives into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image, where the undercarriage component to be detected as the region of interest. The latest semantic segmentation image is then compared with the previous semantic segmentation image. If the difference in their regions of interest is significant, it indicates that the undercarriage component to be detected is incomplete. This compensates for the scarcity of the original dataset and ultimately yields higher-quality analysis results.
[0068] The pre-training process of the semantic segmentation model includes, for example: Figure 3 As shown, the steps are as follows:
[0069] Step S101: Acquire historical images of the locomotive's undercarriage;
[0070] Collect historical images of the locomotive's undercarriage as many times as possible, and at least once.
[0071] Step S102: Randomly crop the historical undercarriage image of the locomotive, and combine the historical undercarriage image of the locomotive and the randomly cropped image into a training set;
[0072] Step S103: Input the training set into the semantic segmentation network for training to obtain a semantic segmentation model. The semantic segmentation network is a deepLabV3+ training network.
[0073] Generally speaking, the original training set is relatively sparse, such as Figure 2 As shown, to compensate for the lack of dataset and considering the large image size of individual historical vehicle undercarriage images, the collected historical vehicle undercarriage images are randomly cropped to reduce the image size and the amount of information in a single historical vehicle undercarriage image. To increase the model's generalization ability, a random cropping method is adopted to obtain multiple randomly cropped images. This ensures the availability of data while improving the network's robustness. Furthermore, the input to the deepLabV3+ training network is adapted to align with the information density range that the deepLabV3+ training network can accept, in order to achieve the best performance of the trained semantic segmentation model.
[0074] The deepLabV3+ training network employs an encoder-decoder structure, where the encoder provides semantics and the decoder gradually recovers boundary information. Compared to the deepLabV3 training network, the deepLabV3+ training network uses it as its own encoder and adopts a simple and effective decoder method, thus improving the performance of the original model.
[0075] Step S2: Input the locomotive undercarriage image collected in this session into the pre-trained semantic segmentation model to obtain the latest semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest;
[0076] Step S3: Perform positional registration between the previous semantic segmentation image and the latest semantic segmentation image to obtain the latest registered image, such as... Figure 4 As shown, it includes the following steps:
[0077] Step S3.1: Extract the region of interest from the previous semantic segmentation image as the template image;
[0078] Step S3.2: Using the size of the template image as the window size, slide the window in the latest semantic segmentation image according to the window size. In each sliding window process, compare the position difference between each pixel in the window and the corresponding pixel in the template image according to the position arrangement order of the pixels.
[0079] Step S3.3: After the window has traversed all the positions in the latest semantic segmentation images, the pixel of the corresponding template image with the smallest position difference is taken as the best registration point.
[0080] Step S3.4: Using the best registration point as a reference, perform positional registration between the latest semantic segmentation image and the template image to obtain the latest registered image.
[0081] Geometric errors may arise from either previously acquired historical images of the locomotive's undercarriage or currently acquired images after semantic segmentation, significantly impacting subsequent comparison steps. To eliminate these errors, registration of the two regions of interest is necessary. Furthermore, this registration process should be fast and efficient.
[0082] In this embodiment, a template image matching method is used to treat one image as a template and find the matching position in the second image, thereby completing the registration of the two images.
[0083] In this embodiment, the region of interest from the previous semantic segmentation image is extracted as a template image. Using a template image matching method, a window of the template image size is used to slide up on the latest semantic segmentation image. The positional difference between each pixel in the window and the corresponding pixel in the template image is compared according to the pixel positional order. After the window has traversed all possible positions, the pixel with the smallest difference among all positions is selected as the best matching point. Based on the best registration point, the latest semantic segmentation image and the template image are registered to obtain the latest registered image.
[0084] Step S4: Compare the regions of interest in the previous semantic segmentation image and the latest registration image to obtain the comparison result, and determine the comparison result as the integrity analysis result of the vehicle underbody components.
[0085] The region of interest in the previous semantic segmentation image and the latest registration image are compared to obtain a comparison result, such as... Figure 5 As shown, it includes the following steps:
[0086] Step S4.1: Compare the previous semantic segmentation image and the latest registration image using binary images to obtain the binary image comparison result;
[0087] Step S4.2: If the binary image comparison result shows that the undercarriage component is incomplete, then the final comparison result is that the undercarriage component is incomplete.
[0088] Step S4.3: If the binary image comparison result shows that the vehicle bottom part is complete, then the previous semantic segmentation image and the latest registration image are compared in grayscale to obtain the grayscale comparison result, and the grayscale comparison result is used as the final comparison result.
[0089] The binary image comparison is performed to obtain the binary image comparison result, such as... Figure 6 As shown, it includes the following steps:
[0090] Step S4.1.1: Based on the region of interest, binarize the previous semantic segmentation image and the latest registration image to obtain the previous binary image and the latest binary image, respectively.
[0091] It is understandable that, regardless of the previous semantic segmentation image or the latest registration image, the binarization process based on the region of interest is as follows: if a pixel in the image belongs to the region of interest, then the pixel value of that pixel is set to 1; otherwise, if a pixel in the image does not belong to the region of interest, then the pixel value of that pixel is set to 0. Each pixel in the previous semantic segmentation image or the latest registration image undergoes the above binarization process to obtain the previous binary image and the latest binary image.
[0092] Step S4.1.2: According to the order of pixel position, perform pixel value difference operation on the corresponding pixels of the previous binary image and the latest binary image and take the absolute value to obtain a difference image of binary pixel values;
[0093] The above process is detailed as follows: According to the arrangement order of each pixel in the previous binary image and the new binary image, for example: the pixel value of the first pixel in the previous binary image is 0, and the pixel value of the first pixel in the new binary image is 1, then the difference of the binary pixel values is 0-1=-1, and then its absolute value is taken as 1. Each pixel is processed in the above way to obtain the difference image of the binary pixel values.
[0094] Step S4.1.3: Calculate the rate of change of the difference map of binarized pixel values;
[0095] Step S4.1.4: When the rate of change is greater than or equal to the first threshold, the binary image comparison result is: the undercarriage component to be detected in the undercarriage image collected this time is incomplete.
[0096] The rate of change of the difference map of binarized pixel values is calculated using the following formula:
[0097]
[0098] Where N1 represents the number of pixels whose pixel values have changed in both the previous and latest binary images. For example, if the pixel value at the first position in the previous binary image is 0, and the pixel value at the first position in the latest binary image is 1, this indicates a change in pixel values in both images. Similarly, if the pixel value at the first position in the previous binary image is 1, and the pixel value at the first position in the latest binary image is 0, this also indicates a change in pixel values in both images. If the pixel value at the first position in both the previous and latest binary images is either both 1 or both 0, this does not indicate a change in pixel values in either image, and the value is N. all is the total number of pixels in the difference map, and Var is the rate of change of the binary difference map.
[0099] The previous semantic segmentation image and the latest registration image are compared in grayscale to obtain a grayscale comparison result, such as... Figure 7 As shown, it includes the following steps:
[0100] Step S4.3.1: Perform grayscale processing on the previous semantic segmentation image and the latest registration image respectively to obtain the previous grayscale image and the latest grayscale image respectively;
[0101] It is understandable that grayscale processing involves converting the pixel value of each pixel in the previous semantic segmentation image and the latest registration image into a grayscale value.
[0102] Step S4.3.2: According to the order of pixel positions, perform grayscale value difference calculation between the pixels of the previous grayscale image and the corresponding pixels of the latest grayscale image, and take the absolute value to obtain the grayscale difference image;
[0103] The above process is detailed as follows: According to the arrangement order of each pixel in the previous grayscale image and the new grayscale image, for example: the grayscale value of the first pixel in the previous grayscale image is 30, and the pixel value of the first pixel in the latest grayscale image is 56, then the grayscale value difference is calculated and the absolute value is |30-56|=26. Each pixel is processed in the above way to obtain the grayscale difference image.
[0104] Step S4.3.3: Calculate the average grayscale difference of all pixels in the grayscale difference image;
[0105] Step S4.3.4: When the average grayscale difference is greater than or equal to the second threshold, the grayscale comparison result is: the undercarriage component to be detected in the undercarriage image of the locomotive collected this time is incomplete.
[0106] If the binary image comparison result shows that the undercarriage components are complete, it does not necessarily mean that the undercarriage components are truly complete. The binary image may miss some information. A grayscale comparison should also be performed on the previous semantic segmentation image and the latest registration image. If the grayscale comparison result also shows that the undercarriage components are complete, then it means that the undercarriage components to be detected in the currently acquired locomotive undercarriage image are complete. The simulation result of this embodiment is shown in Figure 8(b), and the historical undercarriage image of the locomotive acquired previously is shown in Figure 8(a). As can be seen from the figure, the semantic component of the main beam has been well separated from the entire undercarriage image.
[0107] This embodiment proposes a method for analyzing the integrity of undercarriage components. It simultaneously inputs historical undercarriage images from previous acquisitions and currently acquired undercarriage images into a pre-trained semantic segmentation model, obtaining previous and latest semantic segmentation images respectively. The previous and latest semantic segmentation images are then registered. The regions of interest in the previous and latest registered images are compared, and the comparison result is determined as the undercarriage component integrity analysis result. This method compensates for the scarcity of the original dataset and obtains relatively accurate undercarriage component integrity analysis results through binary image comparison and grayscale image comparison. The method is simple and effective, improving the utilization of acquired images.
[0108] Example 2:
[0109] This embodiment proposes a device for analyzing the integrity of vehicle underbody components, such as... Figure 9 As shown, it includes: a first semantic segmentation module, a second semantic segmentation module, an image registration module, and an image comparison module;
[0110] The first semantic segmentation module and the second semantic segmentation module are respectively connected to the image registration module, and the image registration module is connected to the image comparison module;
[0111] The first semantic segmentation module is used to input the previously acquired historical undercarriage image of the locomotive into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image with the undercarriage component whose integrity is to be detected as the region of interest.
[0112] The second semantic segmentation module is used to input the locomotive undercarriage image collected in the current session into the pre-trained semantic segmentation model to obtain the latest semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest.
[0113] The image registration module is used to perform positional registration between the previous semantic segmentation image and the latest semantic segmentation image to obtain the latest registered image;
[0114] The image comparison module is used to compare the regions of interest in the previous semantic segmentation image and the latest registration image to obtain a comparison result, and the comparison result is determined as the integrity analysis result of the vehicle underbody components.
[0115] The image comparison module includes: a binary image comparison unit and a grayscale image comparison unit;
[0116] The binary image comparison unit is used to compare the previous semantic segmentation image and the latest registration image in binary image to obtain a binary image comparison result; if the binary image comparison result is that the undercarriage part is incomplete, then the final comparison result is that the undercarriage part is incomplete.
[0117] The grayscale comparison unit is used to compare the previous semantic segmentation image and the latest registration image in grayscale if the binary image comparison result shows that the vehicle bottom part is complete, and to obtain the grayscale comparison result, and use the grayscale comparison result as the final comparison result.
[0118] The binary image comparison unit includes: a binarization processing subunit, a difference image subunit, a rate of change calculation subunit, and a binary image comparison result subunit;
[0119] The binarization processing subunit is used to perform binarization processing on the previous semantic segmentation image and the latest registration image according to the region of interest, so as to obtain the previous binary image and the latest binary image respectively.
[0120] The difference map subunit is used to perform pixel value difference operation on the corresponding pixels of the previous binary map and the latest binary map according to the position order of the pixels, and take the absolute value to obtain a difference map of binary pixel values.
[0121] The rate of change calculation subunit is used to calculate the rate of change of the difference map of the binarized pixel values;
[0122] The binary image comparison result subunit is used to obtain the following binary image comparison result when the rate of change is greater than or equal to the first threshold: the undercarriage component to be detected in the undercarriage image of the locomotive collected this time is incomplete.
[0123] The grayscale image comparison unit includes: a grayscale image processing subunit, a grayscale difference image subunit, a grayscale difference average calculation subunit, and a grayscale image comparison result subunit;
[0124] The grayscale image processing subunit is used to perform grayscale processing on the previous semantic segmentation image and the latest registration image respectively, to obtain the previous grayscale image and the latest grayscale image respectively.
[0125] The grayscale difference image subunit is used to perform grayscale value difference calculation between the pixels of the previous grayscale image and the corresponding pixels of the latest grayscale image according to the positional order of the pixels, and take the absolute value to obtain the grayscale difference image.
[0126] The grayscale difference average calculation subunit is used to calculate the average grayscale difference of all pixels in the grayscale difference map.
[0127] The grayscale comparison result subunit is used to obtain the following grayscale comparison result when the average grayscale difference is greater than or equal to the second threshold: the undercarriage component to be detected in the undercarriage image of the locomotive collected this time is incomplete.
[0128] This embodiment proposes a device for analyzing the integrity of vehicle undercarriage components. It obtains previous and latest semantic segmentation images through a first and a second semantic segmentation module, respectively. An image registration module performs positional registration between the previous and latest semantic segmentation images. Finally, an image comparison module compares the regions of interest in the previous and latest registered images, and the comparison result is determined as the integrity analysis result of the vehicle undercarriage components. This device compensates for the scarcity of the original dataset, obtaining relatively accurate integrity analysis results of vehicle undercarriage components through binary image comparison and grayscale image comparison. The method is simple and effective, improving the utilization of acquired images.
[0129] Example 3:
[0130] This embodiment proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the vehicle underbody component integrity analysis method described above.
[0131] Example 4:
[0132] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory stores instructions, and when the instructions are executed by the one or more processors, the one or more processors perform the vehicle underbody component integrity analysis method.
[0133] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the vehicle undercarriage component integrity analysis method as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and communication components.
[0134] The processor is used to execute all or part of the steps in the vehicle undercarriage component integrity analysis method as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.
[0135] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the vehicle undercarriage component integrity analysis method described in the above embodiments.
[0136] Example 5:
[0137] This embodiment proposes a computer-readable storage medium storing executable instructions, which, when executed, cause a processor to perform the vehicle underbody component integrity analysis method.
[0138] In the various embodiments of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0139] Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the vehicle undercarriage component integrity analysis method described in the various embodiments of this application.
[0140] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes, on which computer programs are stored. When the computer program is executed by the processor, it can implement the various steps of the above-mentioned vehicle undercarriage component integrity analysis method.
[0141] The various embodiments in this disclosure are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0142] The scope of protection of this disclosure is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its scope and spirit. If such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, then the intent of this disclosure also includes such modifications and variations.
Claims
1. A method for analyzing the integrity of vehicle underbody components, characterized in that, Includes the following steps: The historical undercarriage image of the locomotive collected previously is input into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image with the undercarriage component whose integrity is to be detected as the region of interest. The locomotive undercarriage image collected in this instance is input into a pre-trained semantic segmentation model to obtain the latest semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest. The previous semantic segmentation image is registered with the latest semantic segmentation image to obtain the latest registered image; The regions of interest in the previous semantic segmentation image and the latest registration image are compared to obtain the comparison result, which is then determined as the integrity analysis result of the vehicle underbody components. The step of comparing the regions of interest in the previous semantic segmentation image and the latest registration image to obtain the comparison result includes the following steps: The previous semantic segmentation image and the latest registration image are compared as binary images to obtain the binary image comparison result. If the binary image comparison result shows that the undercarriage is incomplete, then the final comparison result is that the undercarriage is incomplete. If the binary image comparison result shows that the vehicle undercarriage is intact, then the previous semantic segmentation image and the latest registration image are compared in grayscale to obtain the grayscale comparison result, and the grayscale comparison result is used as the final comparison result.
2. The method for analyzing the integrity of vehicle underbody components as described in claim 1, characterized in that, The pre-training process of the semantic segmentation model includes the following steps: Collect historical images of the locomotive's undercarriage; The historical undercarriage images of the locomotive are randomly cropped, and the historical undercarriage images of the locomotive and the randomly cropped images are combined into a training set. The training set is input into the semantic segmentation network for training to obtain the semantic segmentation model.
3. The method for analyzing the integrity of vehicle underbody components as described in claim 2, characterized in that, The semantic segmentation network is a deepLabV3+ training network.
4. The method for analyzing the integrity of vehicle underbody components as described in claim 1, characterized in that, The step of performing positional registration between the previous semantic segmentation image and the latest semantic segmentation image to obtain the latest registered image includes the following steps: Extract the region of interest from the previous semantic segmentation image as the template image; The size of the template image is used as the window size. In the latest semantic segmentation image, the window is slidable according to the window size. In each sliding window process, the position difference between each pixel in the window and the corresponding pixel in the template image is compared according to the position arrangement of the pixels. After the window traverses all the positions in the latest semantic segmentation images, the pixel of the corresponding template image with the smallest position difference is taken as the best registration point. Using the optimal registration point as a reference, the latest semantic segmentation image is registered with the template image to obtain the latest registered image.
5. The method for analyzing the integrity of vehicle underbody components as described in claim 1, characterized in that, The step of performing binary image comparison to obtain the binary image comparison result includes the following steps: Based on the region of interest, the previous semantic segmentation image and the latest registration image are binarized to obtain the previous binary image and the latest binary image, respectively. According to the order of pixel positions, the pixel values of the corresponding pixels in the previous binary image and the latest binary image are compared and the absolute values are taken to obtain a difference image of binary pixel values. Calculate the rate of change of the difference map of binarized pixel values; When the rate of change is greater than or equal to the first threshold, the binary image comparison result is: the undercarriage component to be detected in the undercarriage image collected this time is incomplete.
6. The method for analyzing the integrity of vehicle underbody components as described in claim 5, characterized in that, The rate of change of the difference map of binarized pixel values is calculated using the following formula: in, This represents the number of pixels whose pixel values have changed between the previous and latest binary images. This represents the total number of pixels in the difference image. This represents the rate of change in the binary difference plot.
7. The method for analyzing the integrity of vehicle underbody components as described in claim 1, characterized in that, The step of comparing the previous semantic segmentation image with the latest registration image in grayscale to obtain the grayscale comparison result includes the following steps: The previous semantic segmentation image and the latest registration image are converted to grayscale to obtain the previous grayscale image and the latest grayscale image, respectively. According to the order of pixel positions, the grayscale difference is calculated between the corresponding pixels of the previous grayscale image and the corresponding pixels of the latest grayscale image, and the absolute value is taken to obtain the grayscale difference image. Calculate the average gray-level difference of all pixels in the gray-level difference image; When the average grayscale difference is greater than or equal to the second threshold, the grayscale comparison result is: the undercarriage component to be detected in the undercarriage image of the locomotive collected this time is incomplete.
8. A device for analyzing the integrity of vehicle underbody components, characterized in that, include: The first semantic segmentation module is used to input the previously acquired historical undercarriage image of the locomotive into the pre-trained semantic segmentation model to obtain the previous semantic segmentation image with the undercarriage component whose integrity is to be detected as the region of interest. The second semantic segmentation module is used to input the locomotive undercarriage image collected in the current session into the pre-trained semantic segmentation model to obtain the latest semantic segmentation image with the undercarriage component whose integrity to be detected as the region of interest. The image registration module is used to perform positional registration between the previous semantic segmentation image and the latest semantic segmentation image to obtain the latest registered image; The image comparison module is used to compare the regions of interest in the previous semantic segmentation image and the latest registration image to obtain the comparison result, and to determine the comparison result as the integrity analysis result of the vehicle underbody components. The step of comparing the regions of interest in the previous semantic segmentation image and the latest registration image to obtain the comparison result includes the following steps: The previous semantic segmentation image and the latest registration image are compared as binary images to obtain the binary image comparison result. If the binary image comparison result shows that the undercarriage is incomplete, then the final comparison result is that the undercarriage is incomplete. If the binary image comparison result shows that the vehicle undercarriage is intact, then the previous semantic segmentation image and the latest registration image are compared in grayscale to obtain the grayscale comparison result, and the grayscale comparison result is used as the final comparison result.
9. An electronic device, characterized in that, include: One or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the vehicle underbody component integrity analysis method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed, cause the processor to perform the vehicle underbody component integrity analysis method according to any one of claims 1-7.
Citation Information
Patent Citations
Railway wagon bottom floor damage fault detection method
CN111652227A
Method and system for detecting deficiency of fastening bolt of equipment under train
CN115546223A