A railway wagon image analysis method and device, electronic equipment and storage medium

Abstract

This invention relates to the field of image analysis and discloses a method for analyzing railway freight car images. The method includes: acquiring an image of the freight car to be analyzed; reconstructing the freight car image using a trained reconstruction neural network to obtain a reconstructed image; calculating the reconstruction error between the reconstructed image and the freight car image; determining whether there are foreign objects in the freight car image based on the reconstruction error; identifying the foreign object region in the freight car image when foreign objects are present; querying the components to be detected in the freight car and locating the component region in the freight car image; performing fault detection on the component region to determine the fault type and key region of the faulty component; calculating the anomaly score of the key region; determining the abnormal region of the faulty component based on the anomaly score; and fusing information from the foreign object region, fault type, and abnormal region to obtain the analysis result of the freight car image. This invention can improve the real-time performance of railway freight car image analysis.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making, and in particular to a method, apparatus, electronic device, and storage medium for analyzing railway freight car images. Background Technology

[0002] With the continuous expansion of the railway network, the operating mileage and freight volume are also constantly increasing, which places higher demands on the safety of freight car operation. Currently, my country uses the Dynamic Image Detection System (TFDS) for freight car operation faults to analyze and judge the faults of railway freight car components. However, with the increase in railway locomotives and rolling stock and the improvement of vehicle operation rate, the contradiction between the surge in maintenance workload and the increasingly shorter time available for maintenance operations is becoming increasingly prominent. Therefore, there is an urgent need for an intelligent image analysis method for railway freight cars.

[0003] Existing research on railway freight car image analysis mainly adopts methods based on traditional image processing techniques. These methods have high requirements for image quality and require manual design and extraction of fault features. Therefore, they can only identify specific faults, lack versatility, and have slow detection speed and low accuracy, failing to meet the real-time requirements of practical applications. Summary of the Invention

[0004] This invention provides a method and system for analyzing railway freight car images, the main purpose of which is to improve the detection accuracy and real-time performance of railway freight car image analysis.

[0005] To achieve the above objectives, the present invention provides a method for analyzing railway freight car images, comprising:

[0006] The process involves acquiring images of the freight cars to be analyzed, reconstructing the freight car images using a trained reconstruction neural network to obtain reconstructed images, calculating the reconstruction error between the reconstructed images and the freight car images, and determining whether there are any foreign objects in the freight car images based on the reconstruction error.

[0007] When a foreign object is present in the truck image, the foreign object region in the truck image is identified;

[0008] The system queries the components to be inspected in the railway freight car to be analyzed, locates the component area of ​​the component to be inspected in the freight car image, performs fault detection on the component area, and determines whether there are faulty components in the component to be inspected.

[0009] When a faulty component is found in the component to be inspected, the fault type of the faulty component is determined. Based on the fault type, the key area of ​​the faulty component is extracted, the anomaly score of the key area is calculated, and the anomaly area of ​​the faulty component is determined based on the anomaly score.

[0010] Information is fused from the foreign object area, the fault type, and the abnormal area to obtain the analysis results of the truck image.

[0011] Optionally, before reconstructing the truck image using the trained reconstruction neural network to obtain the reconstructed image, the method further includes training the reconstruction neural network, including:

[0012] Initialize the generator and discriminator parameters in the pre-built reconstructed neural network, randomly select truck image samples from the pre-built truck image database, and randomly select noise samples from the pre-built reconstructed neural network from a preset noise distribution.

[0013] Based on the truck image samples and the noise samples, the generator parameters are updated using the generator function in the pre-constructed reconstruction neural network to obtain the updated generator parameters;

[0014] The generator function includes:

[0015]

[0016] in, This indicates updating the generator parameters, z. i Let represent the i-th generator parameter, G represent the truck image sample, D represent the noise sample, i represent the index of the generator parameter, and n represent the number of generator parameters.

[0017] Based on the truck image samples and the noise samples, the discriminator parameters are updated using the discriminator function in the pre-constructed reconstructed neural network to obtain the updated discriminator parameters;

[0018] The discriminator function includes:

[0019]

[0020] Where V represents the updated discriminator parameters, x i Let z represent the parameters of the i-th discriminator, D represent the noise sample, and z represent the noise sample. i Let represent the i-th generator parameter, G represent the truck image sample, i represent the index of the discriminator parameter, and n represent the number of discriminator parameters.

[0021] Iterate through the discriminator update parameters and the generator update parameters. When the pre-built reconstructed neural network reaches Nash equilibrium, obtain the equilibrium generation parameters and the equilibrium discriminator parameters. Based on the equilibrium generation parameters and the equilibrium discriminator parameters, generate the trained reconstructed neural network.

[0022] Optionally, identifying the foreign object region in the truck image includes:

[0023] Obtain the difference image corresponding to the reconstruction error of the truck image, and perform denoising processing on the difference image to obtain a denoised image;

[0024] The denoised image is segmented to obtain a segmented image, and the segmented image is further divided into segments to obtain segmented image blocks;

[0025] Calculate the structural similarity between adjacent segmented image blocks. When the structural similarity is greater than a preset structural similarity threshold, perform similarity fusion processing on the segmented image blocks to obtain fused image blocks.

[0026] Based on the fused image blocks, the foreign object region in the truck image is determined.

[0027] Optionally, calculating the structural similarity between adjacent segmented image blocks includes:

[0028] The structural similarity between adjacent segmented image blocks is calculated using the following formula:

[0029]

[0030] Where SSIM(P,Q) represents the structural similarity between adjacent segmented image blocks, P,Q represent adjacent segmented image blocks, and μ P μ represents the average gray level of the segmented image block P. Q σ represents the average gray level of the segmented image block Q. P σ represents the standard deviation of the gray levels of the segmented image block P. Q C1 represents the grayscale standard deviation of the segmented image block Q, C2 represents the luminance constant, and C3 represents the contrast constant.

[0031] Optionally, the step of performing fault detection on the component area to determine whether there is a faulty component in the component to be detected includes:

[0032] Identify the component names in the component area, extract the regional features of the component area, and concatenate the regional features according to the component names to obtain the concatenated features;

[0033] Extract the stitched normal features from the pre-constructed normal component image database, and calculate the similarity between the stitched features and the stitched normal features;

[0034] When the similarity is greater than a preset similarity threshold, it is determined that the component to be detected does not have a faulty component.

[0035] When the similarity is not greater than a preset similarity threshold, it is determined that the component to be detected is a faulty component.

[0036] Optionally, when a faulty component is found among the components to be tested, determining the fault type of the faulty component includes:

[0037] Obtain the component region of the component to be inspected, and identify the component type corresponding to the component to be inspected;

[0038] Based on the type of component, a trained fault detection network is used to detect the component area and obtain a fault score for the component area.

[0039] Based on the fault score, the fault type of the faulty component is determined.

[0040] Optionally, determining the abnormal region of the faulty component based on the abnormality score includes:

[0041] Configure the mapping relationship between the anomaly scores and the pre-built color data table, and construct a heatmap of the anomaly scores based on the mapping relationship;

[0042] When the color intensity of the heat map is greater than a preset color intensity threshold, the corresponding area of ​​the heat map is identified and the corresponding area is regarded as the abnormal area of ​​the faulty component.

[0043] To address the above problems, the present invention also provides a railway freight car image analysis device, the device comprising:

[0044] The foreign object detection module is used to acquire images of the freight cars to be analyzed, reconstruct the freight car images using a trained reconstruction neural network to obtain a reconstructed image, calculate the reconstruction error between the reconstructed image and the freight car image, and determine whether there are any foreign objects in the freight car image based on the reconstruction error.

[0045] Foreign object recognition module, used to identify foreign object areas in the truck image when foreign objects are present in the truck image;

[0046] The fault detection module is used to query the components to be inspected of the railway freight car to be analyzed, locate the component area of ​​the component to be inspected in the freight car image, and perform fault detection on the component area to determine whether there are faulty components in the component to be inspected.

[0047] An anomaly identification module is used to determine the fault type of the faulty component when there is a faulty component in the component to be detected, extract the key area of ​​the faulty component according to the fault type, calculate the anomaly score of the key area, and determine the abnormal area of ​​the faulty component according to the anomaly score.

[0048] The analysis result generation module is used to fuse information from the foreign object area, the fault type, and the abnormal area to obtain the analysis results of the truck image.

[0049] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0050] At least one processor; and,

[0051] A memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores a computer program that can be executed by the at least one processor to implement the railway freight car image analysis method described above.

[0053] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the railway freight car image analysis method described above.

[0054] As can be seen, this embodiment of the invention can determine the information carrier of the railway freight car to be analyzed by acquiring its image, providing an operational object for subsequent intelligent analysis of the freight car. The trained reconstruction neural network is used to reconstruct the freight car image, taking advantage of the smaller reconstruction error of normal images and the larger reconstruction error of abnormal images. This allows for subsequent determination of whether foreign objects exist in the freight car image. Based on the reconstruction error, the presence of state foreign objects in the freight car image can be determined, obtaining the state information of the freight car image in the foreign object detection dimension. When state foreign objects are present in the freight car image, identifying the foreign object region can accurately locate the location of the foreign object. Secondly, this embodiment of the invention can determine the fault detection object of the railway freight car by querying the components to be detected in the freight car. By locating the component region of the component to be detected in the freight car image, the specific location and features of the component to be detected in the freight car image can be located. The component region is then analyzed. Fault detection, used to determine whether a component under test is faulty, allows for multi-dimensional detection of frequently occurring faults and those seriously endangering train safety. Identifying faulty components improves train safety. When a faulty component is found, determining its fault type clarifies the specific type of fault. Furthermore, this embodiment extracts key regions of the faulty component based on the fault type, providing candidate regions for precise fault location. Calculating anomaly scores for these key regions serves as the basis for precise fault location. Based on these anomaly scores, identifying anomalous regions of the faulty component ultimately pinpoints its precise fault location. Finally, information fusion of the foreign object region, fault type, and anomalous region yields the analysis results of the freight car image. This allows for decision fusion of multiple dimensions and model detection results, improving detection accuracy and real-time efficiency in practical applications. Therefore, this embodiment of the invention provides a railway freight car image analysis method, device, electronic equipment, and storage medium that improves the detection accuracy and real-time performance of railway freight car image analysis. Attached Figure Description

[0055] Figure 1 A schematic flowchart of a railway freight car image analysis method provided in an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of a railway freight car image analysis system provided in an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of the internal structure of an electronic device for implementing a railway freight car image analysis method according to an embodiment of the present invention;

[0058] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0060] This invention provides a method for analyzing railway freight car images. The executing entity of the railway freight car image analysis method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this invention: a server, a terminal, etc. In other words, the railway freight car image analysis method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0061] Reference Figure 1 The diagram shown is a flowchart illustrating a railway freight car image analysis method according to an embodiment of the present invention. In this embodiment, the railway freight car image analysis method includes:

[0062] S1. Obtain the freight car image of the railway freight car to be analyzed, reconstruct the freight car image using a trained reconstruction neural network to obtain a reconstructed image, calculate the reconstruction error between the reconstructed image and the freight car image, and determine whether there are foreign objects in the freight car image based on the reconstruction error.

[0063] In this embodiment of the invention, the freight car image is a similar and vivid description or photograph of a railway freight car, which is the most commonly used information carrier in human social activities and can be acquired through devices such as cameras, scanners, etc.

[0064] This invention reconstructs the truck image using a trained reconstruction neural network. It leverages the characteristic that normal images have smaller reconstruction errors while abnormal images have larger reconstruction errors to subsequently determine whether there are foreign objects in the truck image.

[0065] As an optional embodiment of the present invention, before reconstructing the truck image using the trained reconstruction neural network to obtain the reconstructed image, the method further includes training the reconstruction neural network, including: initializing the generator parameters and discriminator parameters in the pre-built reconstruction neural network; randomly selecting truck image samples from the pre-built truck image database and randomly selecting noise samples from a preset noise distribution; updating the generator parameters using the generator function in the pre-built reconstruction neural network based on the truck image samples and the noise samples to obtain updated generator parameters; updating the discriminator parameters using the discriminator function in the pre-built reconstruction neural network based on the truck image samples and the noise samples to obtain updated discriminator parameters; iterating the discriminator update parameters and the generator update parameters; obtaining balanced generation parameters and balanced discriminator parameters when the pre-built reconstruction neural network reaches Nash equilibrium; and generating the trained reconstruction neural network based on the balanced generation parameters and the balanced discriminator parameters.

[0066] The noise distribution refers to the shape of the noise data in the statistical graph. The Nash equilibrium is a concept of a solution in game theory, which refers to a strategy combination that satisfies the following property: under this strategy combination, unilaterally changing one's own strategy (while other players' strategies remain unchanged) will not increase one's own payoff.

[0067] Optionally, the generator function includes:

[0068]

[0069] in, This indicates updating the generator parameters, z. i Let represent the i-th generator parameter, G represent the truck image sample, D represent the noise sample, i represent the index of the generator parameter, and n represent the number of generator parameters.

[0070] Optionally, the discriminator function includes:

[0071]

[0072] Where V represents the updated discriminator parameters, x i Let z represent the parameters of the i-th discriminator, D represent the noise sample, and z represent the noise sample. i Let represent the i-th generator parameter, G represent the truck image sample, i represent the index of the discriminator parameter, and n represent the number of discriminator parameters.

[0073] Furthermore, in this embodiment of the invention, the difference between the reconstructed image and the truck image can be determined by calculating the reconstruction error between the reconstructed image and the truck image, which can be obtained through a difference algorithm.

[0074] Further, as an optional embodiment of the present invention, the step of reconstructing the truck image using a trained reconstruction neural network to obtain a reconstructed image includes: identifying the data distribution of the truck image; and generating a reconstructed image of the truck image using a generator in the trained reconstruction neural network based on pre-constructed random noise and the data distribution. Here, the data distribution is a visual description of data, using various statistical graphs to graphically represent the distribution pattern of data; it refers to the data probability distribution or frequency distribution. The random noise is noise caused by the accumulation of a large number of fluctuating disturbances randomly generated over time, and its value cannot be predicted at a given instant.

[0075] Furthermore, in this embodiment of the invention, by determining whether there are foreign objects in the truck image based on the reconstruction error, the state information of the truck image to be analyzed in the foreign object detection dimension can be obtained.

[0076] Furthermore, as an optional embodiment of the present invention, the determination of whether there is a foreign object in the truck image based on the reconstruction error can be made according to a preset error threshold. When the reconstruction error is greater than the error threshold, it is determined that there is a foreign object in the truck image; when the reconstruction error is not greater than the error threshold, it is determined that there is no foreign object in the truck image. Here, the error threshold is an abnormal critical value defined for the error.

[0077] S2. When there are foreign objects in the truck image, identify the foreign object area in the truck image.

[0078] In this embodiment of the invention, when a foreign object is present in the truck image, the location of the foreign object can be accurately determined by identifying the foreign object region in the truck image.

[0079] Further, as an optional embodiment of the present invention, the identification of foreign object regions in the truck image includes: acquiring a difference image corresponding to the reconstruction error of the truck image; performing denoising processing on the difference image to obtain a denoised image; performing image segmentation on the denoised image to obtain a segmented image; performing segmentation processing on the segmented image to obtain segmented image blocks; calculating the structural similarity between adjacent segmented image blocks; when the structural similarity is greater than a preset structural similarity threshold, performing similarity fusion processing on the segmented image blocks to obtain fused image blocks; and determining the foreign object region in the truck image based on the fused image blocks. The structural similarity threshold is used as a critical value to measure the structural similarity between images or the quality of compressed images.

[0080] Optionally, the denoising process can be implemented using median filtering, Gaussian filtering, and Markov random fields, and the image segmentation can be implemented using an adaptive threshold segmentation method.

[0081] Further, as an optional embodiment of the present invention, the calculation of structural similarity between adjacent segmented image blocks includes:

[0082] The structural similarity between adjacent segmented image blocks is calculated using the following formula:

[0083]

[0084] Where SSIM(P,Q) represents the structural similarity between adjacent segmented image blocks, P,Q represent adjacent segmented image blocks, and μ P μ represents the average gray level of the segmented image block P. Q σ represents the average gray level of the segmented image block Q. P σ represents the standard deviation of the gray levels of the segmented image block P. Q S1 represents the grayscale standard deviation of the segmented image block Q, C1 represents the brightness constant, and C2 represents the contrast constant. S3: Query the components to be detected in the railway freight car to be analyzed, locate the component region of the component to be detected in the freight car image, and perform fault detection on the component region to determine whether there are faulty components in the component to be detected.

[0085] In this embodiment of the invention, the fault detection target of the railway freight car can be determined by querying the components to be tested in the railway freight car to be analyzed.

[0086] Furthermore, in this embodiment of the invention, the component to be inspected can be located in the component region of the truck image and retrieved from a pre-built database of components to be inspected according to business needs. The database of components to be inspected is an ordered collection of structured information or data about the components to be inspected, typically stored electronically in a computer system, and may include information such as the name, type, structural composition, and fault category of the components to be inspected.

[0087] Furthermore, as an optional embodiment of the present invention, locating the component region to be detected in the truck image, which can determine the target range for fault detection, includes: preprocessing the truck image to obtain a preprocessed image; and using a target detection algorithm to detect the component region in the preprocessed image. The target detection algorithm is an algorithm for target localization and classification in computer vision, including algorithms such as SSD, R-CNN, Fast R-CNN, Faster R-CNN, and YOLO.

[0088] Furthermore, by performing fault detection on the component area, this embodiment of the invention can determine whether there are faulty components in the component to be detected. This allows for multi-dimensional detection of frequently occurring faults and faults that seriously endanger driving safety, thereby identifying faulty components and improving driving safety.

[0089] Further, as an optional embodiment of the present invention, the step of performing fault detection on the component area to determine whether the component to be detected has a faulty component includes: identifying the component name of the component area, extracting the regional features of the component area, and stitching the regional features according to the component name to obtain stitched features; extracting stitched normal features from a pre-constructed normal component image database, and calculating the similarity between the stitched features and the stitched normal features; determining that the component to be detected does not have a faulty component when the similarity is greater than a preset similarity threshold; and determining that the component to be detected has a faulty component when the similarity is not greater than the preset similarity threshold.

[0090] S4. When there is a faulty component in the component to be tested, determine the fault type of the faulty component, extract the key area of ​​the faulty component according to the fault type, calculate the anomaly score of the key area, and determine the abnormal area of ​​the faulty component according to the anomaly score.

[0091] In this embodiment of the invention, when a faulty component is found in the component to be tested, the specific type of the faulty component can be determined.

[0092] Furthermore, as an optional embodiment of the present invention, when a faulty component exists in the component to be tested, determining the fault type of the faulty component includes: acquiring the component region of the component to be tested, identifying the component type corresponding to the component to be tested; according to the component type, using a trained fault detection network to detect the component region, obtaining a fault score for the component region; and determining the fault type of the faulty component based on the fault score.

[0093] Furthermore, in this embodiment of the invention, by extracting the key areas of the faulty component according to the fault type, a candidate area can be provided for accurately locating the fault location of the component. This candidate area can be determined by the structural composition of the component and the possible areas where the fault may occur.

[0094] Furthermore, in this embodiment of the invention, the anomaly score of the key area can be used as a basis for precisely locating the fault location of the component.

[0095] Further, as an optional embodiment of the present invention, the calculation of the anomaly score of the key region includes: encoding the key region using an encoder in an autoencoder model to obtain encoded data; decoding the encoded data using a decoder in an autoencoder model to obtain decoded data; calculating a reconstruction error image between the key region and the decoded data; and calculating the anomaly score of the key region based on the reconstruction error image.

[0096] Further, as an optional embodiment of the present invention, the step of calculating the anomaly score of the key region based on the reconstructed error image includes: segmenting the reconstructed error image into error image patches, calculating the gradient magnitude similarity of the error image patches; calculating the structural similarity of the reconstructed error image based on the gradient magnitude similarity using a gradient magnitude similarity deviation algorithm, and using the structural similarity as the anomaly score of the key region. Wherein, the gradient magnitude similarity deviation algorithm is an evaluation metric algorithm for measuring the image loss between the reference image and the distorted image values.

[0097] Optionally, the anomaly score of the key region can also be calculated by calculating the similarity between the key region and a pre-built anomaly region database. Here, similarity is a measure describing the degree of similarity between two things, including distance similarity, cosine similarity, and Jaccard similarity, etc.

[0098] Furthermore, by determining the abnormal area of ​​the faulty component based on the abnormal score, the embodiment of the present invention can ultimately determine the precise fault location of the component.

[0099] Further, as an optional embodiment of the present invention, determining the abnormal region of the faulty component based on the abnormal score includes: configuring a mapping relationship between the abnormal score and a pre-built color data table; constructing a heatmap of the abnormal score based on the mapping relationship; and identifying the corresponding region of the heatmap when the color intensity of the heatmap is greater than a preset color intensity threshold, and designating the corresponding region as the abnormal region of the faulty component. Here, the heatmap is a statistical chart that displays data by coloring blocks. The color intensity threshold refers to the critical color intensity value in the heatmap for distinguishing abnormal regions.

[0100] S5. Information fusion is performed on the foreign object area, the fault type, and the abnormal area to obtain the analysis results of the truck image.

[0101] It should be understood that, through information fusion of the foreign object region, the fault type, and the abnormal region, the present invention can obtain the analysis results of the truck image, enabling decision fusion of multiple dimensions and multiple model detection results, thereby improving detection accuracy and real-time efficiency in practical applications. The analysis results of the truck image can be achieved through a decision-level fusion algorithm. This decision-level fusion algorithm is a high-level fusion algorithm that directly analyzes the local decisions formed by perceptual information from completely different types of sensors or from different environmental regions to derive the final decision.

[0102] As can be seen, this embodiment of the invention can determine the information carrier of the railway freight car to be analyzed by acquiring its image, providing an operational object for subsequent intelligent analysis of the freight car. The trained reconstruction neural network is used to reconstruct the freight car image, taking advantage of the smaller reconstruction error of normal images and the larger reconstruction error of abnormal images. This allows for subsequent determination of whether foreign objects exist in the freight car image. Based on the reconstruction error, the presence of state foreign objects in the freight car image can be determined, obtaining the state information of the freight car image in the foreign object detection dimension. When state foreign objects are present in the freight car image, identifying the foreign object region can accurately locate the location of the foreign object. Secondly, this embodiment of the invention can determine the fault detection object of the railway freight car by querying the components to be detected in the freight car. By locating the component region of the component to be detected in the freight car image, the specific location and features of the component to be detected in the freight car image can be located. The component region is then analyzed. Fault detection, used to determine whether a component under test is faulty, allows for multi-dimensional detection of frequently occurring faults and those seriously endangering train safety. Identifying faulty components improves train safety. When a faulty component is found, determining its fault type clarifies the specific type of fault. Furthermore, this embodiment extracts key regions of the faulty component based on the fault type, providing candidate regions for precise fault location. Calculating anomaly scores for these key regions serves as the basis for precise fault location. Based on these anomaly scores, identifying anomalous regions of the faulty component ultimately pinpoints its precise fault location. Finally, information fusion of the foreign object region, fault type, and anomalous region yields the analysis results of the freight car image. This allows for decision fusion of multiple dimensions and model detection results, improving detection accuracy and real-time efficiency in practical applications. Therefore, this embodiment of the invention provides a railway freight car image analysis method, device, electronic equipment, and storage medium that improves the detection accuracy and real-time performance of railway freight car image analysis.

[0103] like Figure 2 The diagram shown is a functional block diagram of the railway freight car image analysis device of the present invention.

[0104] The railway freight car image analysis system 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the railway freight car image analysis system may include a foreign object detection module 101, a foreign object recognition module 102, a fault detection module 103, an anomaly recognition module 104, and an analysis result generation module 105. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0105] In this embodiment, the functions of each module / unit are as follows:

[0106] The foreign object determination module 101 is used to acquire the freight car image of the railway freight car to be analyzed, reconstruct the freight car image using a trained reconstruction neural network to obtain a reconstructed image, calculate the reconstruction error between the reconstructed image and the freight car image, and determine whether there is a foreign object in the freight car image based on the reconstruction error.

[0107] The foreign object identification module 102 is used to identify the foreign object region in the truck image when there is a foreign object in the truck image;

[0108] The fault detection module 103 is used to query the components to be detected of the railway freight car to be analyzed, locate the component area of ​​the component to be detected in the freight car image, and perform fault detection on the component area to determine whether there are faulty components in the component to be detected.

[0109] The anomaly identification module 104 is used to determine the fault type of the faulty component when there is a faulty component in the component to be detected, extract the key area of ​​the faulty component according to the fault type, calculate the anomaly score of the key area, and determine the abnormal area of ​​the faulty component according to the anomaly score.

[0110] The analysis result generation module 105 is used to fuse information from the foreign object area, the fault type, and the abnormal area to obtain the analysis result of the truck image.

[0111] In detail, the modules in the railway freight car image analysis system 100 described in this embodiment of the invention employ the same methods as described above. Figure 1 The method used is the same as the railway freight car image analysis method described above, and can produce the same technical effect, so it will not be repeated here.

[0112] like Figure 3The diagram shown is a schematic diagram of the electronic device 1 for implementing the railway freight car image analysis method of the present invention.

[0113] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program, such as a railway freight car image analysis program, stored in the memory 11 and capable of running on the processor 10.

[0114] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device 1, connecting various components of the electronic device 1 via various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing a task balancing consumption program) and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0115] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 can include both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a railway freight car image analysis program, but also to temporarily store data that has been output or will be output.

[0116] The communication bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0117] The communication interface 13 is used for communication between the aforementioned electronic device 1 and other devices, including a network interface and an employee interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish a communication connection between the electronic device 1 and other electronic devices 1. The employee interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual employee interface.

[0118] Figure 3 Only electronic device 1 with components is shown; those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0119] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0120] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in terms of the scope of the patent invention.

[0121] The railway freight car image analysis program stored in the memory 11 of the electronic device 1 is a combination of multiple computer programs. When run in the processor 10, it can achieve the following:

[0122] The process involves acquiring images of the freight cars to be analyzed, reconstructing the freight car images using a trained reconstruction neural network to obtain reconstructed images, calculating the reconstruction error between the reconstructed images and the freight car images, and determining whether there are any foreign objects in the freight car images based on the reconstruction error.

[0123] When a foreign object is present in the truck image, the foreign object region in the truck image is identified;

[0124] The system queries the components to be inspected in the railway freight car to be analyzed, locates the component area of ​​the component to be inspected in the freight car image, performs fault detection on the component area, and determines whether there are faulty components in the component to be inspected.

[0125] When a faulty component is found in the component to be inspected, the fault type of the faulty component is determined. Based on the fault type, the key area of ​​the faulty component is extracted, the anomaly score of the key area is calculated, and the anomaly area of ​​the faulty component is determined based on the anomaly score.

[0126] Information is fused from the foreign object area, the fault type, and the abnormal area to obtain the analysis results of the truck image.

[0127] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0128] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0129] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device 1, can perform the following:

[0130] The process involves acquiring images of the freight cars to be analyzed, reconstructing the freight car images using a trained reconstruction neural network to obtain reconstructed images, calculating the reconstruction error between the reconstructed images and the freight car images, and determining whether there are any foreign objects in the freight car images based on the reconstruction error.

[0131] When a foreign object is present in the truck image, the foreign object region in the truck image is identified;

[0132] The system queries the components to be inspected in the railway freight car to be analyzed, locates the component area of ​​the component to be inspected in the freight car image, performs fault detection on the component area, and determines whether there are faulty components in the component to be inspected.

[0133] When a faulty component is found in the component to be inspected, the fault type of the faulty component is determined. Based on the fault type, the key area of ​​the faulty component is extracted, the anomaly score of the key area is calculated, and the anomaly area of ​​the faulty component is determined based on the anomaly score.

[0134] Information is fused from the foreign object area, the fault type, and the abnormal area to obtain the analysis results of the truck image.

[0135] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0136] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0138] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0139] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0140] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0141] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0142] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for analyzing railway freight car images, characterized in that, The method includes: The process involves acquiring images of the freight cars to be analyzed, reconstructing the freight car images using a trained reconstruction neural network to obtain reconstructed images, calculating the reconstruction error between the reconstructed images and the freight car images, and determining whether there are any foreign objects in the freight car images based on the reconstruction error. When a foreign object is present in the truck image, the foreign object region in the truck image is identified; The system queries the components to be inspected in the railway freight car to be analyzed, locates the component area of ​​the component to be inspected in the freight car image, performs fault detection on the component area, and determines whether there are faulty components in the component to be inspected. When a faulty component is found in the component to be inspected, the fault type of the faulty component is determined. Based on the fault type, the key area of ​​the faulty component is extracted, the anomaly score of the key area is calculated, and the anomaly area of ​​the faulty component is determined based on the anomaly score. Information fusion is performed on the foreign object area, the fault type, and the abnormal area to obtain the analysis results of the truck image; The method of identifying foreign object regions in the truck image includes: acquiring a difference image corresponding to the reconstruction error of the truck image; performing denoising processing on the difference image to obtain a denoised image; performing image segmentation on the denoised image to obtain a segmented image; performing segmentation processing on the segmented image to obtain segmented image blocks; calculating the structural similarity between adjacent segmented image blocks; when the structural similarity is greater than a preset structural similarity threshold, performing similarity fusion processing on the segmented image blocks to obtain fused image blocks; and determining foreign object regions in the truck image based on the fused image blocks. The calculation of the structural similarity between adjacent segmented image blocks includes: calculating the structural similarity between adjacent segmented image blocks using the following formula: in, This represents the structural similarity between adjacent segmented image patches. Represents adjacent segmented image blocks. Represents segmented image blocks Average gray level, Represents segmented image blocks Average gray level, Represents segmented image blocks The standard deviation of gray level Represents segmented image blocks The standard deviation of gray level Represents the brightness constant. This represents the contrast constant.

2. The railway freight car image analysis method as described in claim 1, characterized in that, Before reconstructing the truck image using the trained reconstruction neural network to obtain the reconstructed image, the process further includes training the reconstruction neural network, including: Initialize the generator and discriminator parameters in the pre-built reconstructed neural network, randomly select truck image samples from the pre-built truck image database, and randomly select noise samples from the pre-built reconstructed neural network from a preset noise distribution. Based on the truck image samples and the noise samples, the generator parameters are updated using the generator function in the pre-constructed reconstruction neural network to obtain the updated generator parameters; The generator function includes: in, This indicates that the generator parameters are being updated. Indicates the first There are three generator parameters, where G represents the truck image sample and D represents the noise sample. Indicates the index of the generator parameter. Indicates the number of generator parameters; Based on the truck image samples and the noise samples, the discriminator parameters are updated using the discriminator function in the pre-constructed reconstructed neural network to obtain the updated discriminator parameters; The discriminator function includes: in, This indicates an update to the discriminator parameters. Indicates the first There are several discriminator parameters, where D represents the noise sample. Indicates the first There are 1 generator parameter, G, which represents the truck image sample. Indicates the index of the discriminator parameter. Indicates the number of discriminator parameters; Iterate through the discriminator update parameters and the generator update parameters. When the pre-built reconstructed neural network reaches Nash equilibrium, obtain the equilibrium generation parameters and the equilibrium discriminator parameters. Based on the equilibrium generation parameters and the equilibrium discriminator parameters, generate the trained reconstructed neural network.

3. The railway freight car image analysis method as described in claim 1, characterized in that, The step of performing fault detection on the component area to determine whether there is a faulty component in the component to be detected includes: Identify the component names in the component area, extract the regional features of the component area, and concatenate the regional features according to the component names to obtain the concatenated features; Extract the stitched normal features from the pre-constructed normal component image database, and calculate the similarity between the stitched features and the stitched normal features; When the similarity is greater than a preset similarity threshold, it is determined that the component to be detected does not have a faulty component. When the similarity is not greater than a preset similarity threshold, it is determined that the component to be detected is a faulty component.

4. The railway freight car image analysis method as described in claim 1, characterized in that, When a faulty component is found among the components to be inspected, determining the fault type of the faulty component includes: Obtain the component region of the component to be inspected, and identify the component type corresponding to the component to be inspected; Based on the type of component, a trained fault detection network is used to detect the component area and obtain a fault score for the component area. Based on the fault score, the fault type of the faulty component is determined.

5. The railway freight car image analysis method as described in claim 1, characterized in that, The step of determining the abnormal area of ​​the faulty component based on the abnormal score includes: Configure the mapping relationship between the anomaly scores and the pre-built color data table, and construct a heatmap of the anomaly scores based on the mapping relationship; When the color intensity of the heat map is greater than a preset color intensity threshold, the corresponding area of ​​the heat map is identified and the corresponding area is regarded as the abnormal area of ​​the faulty component.

6. A railway freight car image analysis device, characterized in that, The device includes: The foreign object detection module is used to acquire images of the freight cars to be analyzed, reconstruct the freight car images using a trained reconstruction neural network to obtain a reconstructed image, calculate the reconstruction error between the reconstructed image and the freight car image, and determine whether there are any foreign objects in the freight car image based on the reconstruction error. Foreign object recognition module, used to identify foreign object areas in the truck image when foreign objects are present in the truck image; The fault detection module is used to query the components to be inspected of the railway freight car to be analyzed, locate the component area of ​​the component to be inspected in the freight car image, and perform fault detection on the component area to determine whether there are faulty components in the component to be inspected. An anomaly identification module is used to determine the fault type of the faulty component when there is a faulty component in the component to be detected, extract the key area of ​​the faulty component according to the fault type, calculate the anomaly score of the key area, and determine the abnormal area of ​​the faulty component according to the anomaly score. The analysis result generation module is used to fuse information from the foreign object region, the fault type, and the abnormal region to obtain the analysis result of the truck image. The identification of the foreign object region in the truck image includes: acquiring a difference image corresponding to the reconstruction error of the truck image; denoising the difference image to obtain a denoised image; segmenting the denoised image to obtain a segmented image; further segmenting the segmented image to obtain segmented image blocks; calculating the structural similarity between adjacent segmented image blocks; when the structural similarity is greater than a preset structural similarity threshold, performing similarity fusion processing on the segmented image blocks to obtain fused image blocks; and determining the foreign object region in the truck image based on the fused image blocks. The calculation of the structural similarity between adjacent segmented image blocks includes: calculating the structural similarity between adjacent segmented image blocks using the following formula: in, This represents the structural similarity between adjacent segmented image patches. Represents adjacent segmented image blocks. Represents segmented image blocks Average gray level, Represents segmented image blocks Average gray level, Represents segmented image blocks The standard deviation of gray level Represents segmented image blocks The standard deviation of gray level Represents the brightness constant. This represents the contrast constant.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the railway freight car image analysis method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the railway freight car image analysis method as described in any one of claims 1 to 5.

Images