Methods, devices, electronic equipment and storage media for detecting defects in overhead contact system components

Abstract

This application provides a method, apparatus, electronic device, and storage medium for detecting defects in overhead contact line components. The method includes: acquiring an image of a high-speed railway overhead contact line to be inspected; if the image includes a component to be inspected, determining the area image where the component is located; and performing defect detection on the area image of the component to be inspected using preset defect categories to obtain the defect detection result for the component. This application embodiment is beneficial for improving the accuracy of cotter pin defect detection.

Description

Technical Field

[0001] This application relates to the field of image detection technology, specifically to a method, apparatus, electronic device, and storage medium for detecting defects in overhead contact line components. Background Technology

[0002] As a crucial component of electrified railways, the reliability and safety of high-speed rail overhead contact lines are a strong guarantee for people's rail travel. The structural stability of the contact line suspension system directly affects train operation safety; therefore, regular inspection and maintenance of the high-speed rail overhead contact line are necessary. The high-speed rail overhead contact line has numerous components, and these components may have various defects. Traditionally, defect detection for these components involves manual, one-by-one defect assessment and analysis, but this results in a high rate of missed detections. To reduce the missed detection rate, the current mainstream method is to use deep learning models for detection. However, the same component may have multiple defects, and these defects can interfere with each other during training or detection, leading to low accuracy in component defect detection. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and storage medium for detecting defects in overhead contact line components, which helps to improve the accuracy of defect detection in overhead contact line components.

[0004] A first aspect of this application provides a method for detecting defects in overhead contact system components, the method comprising:

[0005] Obtain the image of the high-speed railway overhead contact line to be inspected;

[0006] When the image to be detected includes the component to be detected, determine the region of the image where the component to be detected is located;

[0007] The image of the area where the component to be inspected is located is subjected to defect detection of multiple preset defect categories to obtain the defect detection results of the component to be inspected.

[0008] The component to be inspected is a cotter pin, and the defect detection results include the defect detection results for each of a preset number of defect categories. The multiple defect categories include at least two of the following:

[0009] Cotter pin closure, cotter pin angle deviation, and missing cotter pin.

[0010] As can be seen, the embodiments of this application acquire an image of the high-speed rail contact network to be inspected; if the image includes a component to be inspected, determine the area image where the component to be inspected is located; perform defect detection on the area image where the component to be inspected is located using preset defect categories, and obtain the defect detection result of the component to be inspected. In this way, during defect detection, the detection results of the component to be inspected (such as a cotter pin) in multiple defect categories can be detected, which helps to avoid missing certain defects of the component to be inspected, thereby improving the accuracy of defect detection of the component to be inspected.

[0011] In conjunction with the first aspect, in one possible implementation, defect detection of multiple preset defect categories is performed on the image of the region where the component to be detected is located to obtain the defect detection result of the component to be detected, including:

[0012] Feature extraction is performed on the image of the region where the component to be detected is located to obtain the first shared feature map;

[0013] The first shared feature map is classified into defects by using a classification network corresponding to each defect category among multiple defect categories, so as to obtain the defect detection results of the component to be detected for each defect category.

[0014] In this example, the classification network for each defect category uses the first shared feature map for defect classification, so that each classification network does not need to perform feature extraction separately before classification, which helps to save feature extraction time and reduce feature extraction overhead.

[0015] In conjunction with the first aspect, the defect detection results for each defect category include one of the following:

[0016] This category of defects exists;

[0017] There are no defects of this type;

[0018] It is impossible to determine whether this type of defect exists;

[0019] A defect exists, but it is impossible to determine whether the defect falls into that category.

[0020] It should be understood that conventional defect detection results mostly indicate whether a defect of a certain category exists. In this example, in addition to the conventional detection results, it also considers whether the image of the area where the part to be detected is located is occluded or blurred, making it impossible to determine whether a defect of that category exists or whether the part to be detected has a defect. However, if the existing defect is in the critical area of ​​the defect category, it is impossible to determine whether the existing defect belongs to that category. This achieves a more refined detection.

[0021] In conjunction with the first aspect, in one possible implementation, feature extraction is performed on the image of the region where the part to be detected is located to obtain a first shared feature map, including:

[0022] The image of the region where the component to be detected is located is convolved to obtain the first feature map;

[0023] The first feature map is downsampled for m stages to obtain feature maps at m scales. The feature maps at m scales correspond one-to-one with the downsampling for m stages, where m is an integer greater than or equal to 2.

[0024] Map the first feature map to n target feature maps in the m scale feature maps to obtain (n+1) second feature maps, where n is an integer greater than or equal to 2 and less than or equal to m;

[0025] By combining (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain the first shared feature map. The m stages of upsampling correspond one-to-one with the m stages of downsampling.

[0026] In conjunction with the first aspect, in one possible implementation, the smallest feature map among the m scale feature maps is upsampled in m stages using (n+1) second feature maps to obtain the first shared feature map, including:

[0027] The first stage of upsampling is performed on the feature map with the smallest scale;

[0028] For the upsampling of the r-th stage in m-stage upsampling, if a target second feature map exists in the (n+1) second feature maps, then the target second feature map is fused with the feature map obtained from the upsampling of the r-th stage to obtain the first fused feature map. The target second feature map is a second feature map with the same scale as the feature map obtained from the upsampling of the r-th stage, where r is an integer greater than or equal to 1 and less than or equal to (m-1).

[0029] For the first fused feature map or the feature map obtained by upsampling in the r-th stage, perform upsampling in the (r+1)-th stage until the upsampling in the m-th stage is completed;

[0030] The feature map obtained by upsampling in the m-th stage is fused with the feature maps of the same scale in the (n+1) second feature maps to obtain the second fused feature map;

[0031] The second fused feature map is post-processed to obtain the first shared feature map.

[0032] In this example, since the feature extraction process downsamples the first feature map in m stages, the dimensionality reduction caused by downsampling may lead to the loss of local information. Therefore, by mapping the first feature map and the feature maps at m scales, and combining the mapped (n+1) second feature maps, the feature map with the smallest scale in the m scale feature maps is upsampled in m stages. For example, the scale of the (n+1) second feature maps can be fused with the feature maps of the same scale obtained in the upsampling stage to make up for the information loss caused by downsampling, thereby making the information in the first shared feature map richer.

[0033] In conjunction with the first aspect, in one possible implementation, mapping is performed between the first feature map and n target feature maps from m scales, including:

[0034] For each feature map in the first feature map and the n target feature maps, perform the first reversible transformation on each feature map to obtain the feature map after the first mapping;

[0035] Perform the (s+1)th reversible transformation on the feature map after the s-th mapping to obtain the feature map after the (s+1)-th mapping, where s is an integer greater than or equal to 1;

[0036] Each feature map is mapped to a second feature map through at least two reversible transformations, and (n+1) second feature maps are included.

[0037] In this example, a two-dimensional normalized flow model is used to perform at least two reversible transformations on each feature map in the first feature map and the feature maps at m scales. Since at least two reversible transformations can be performed in an alternating manner of large and small convolutions, it is beneficial to preserve the spatial positional relationship between the first feature map and the feature maps at m scales. This spatial positional relationship is helpful for locating defects in the component to be detected.

[0038] In conjunction with the first aspect, in one possible implementation, defect detection of multiple preset defect categories is performed on the image of the region where the component to be detected is located. The defect detection result of the component to be detected is obtained through a neural network model. The neural network model is obtained by training a neural network, which includes a feature extraction network and multiple classification networks for classifying the multiple preset defect categories respectively. The neural network model is trained through the following steps:

[0039] Acquire the first sample image of the component to be inspected and the annotation data of the first sample image for multiple defect categories;

[0040] The first sample image is input into the feature extraction network for feature extraction to obtain the second shared feature map;

[0041] The second shared feature map is input into multiple classification networks for defect classification to obtain the defect detection results of the first sample image for multiple defect categories;

[0042] Based on the defect detection results of the first sample image for multiple defect categories and the annotation data of the first sample image for multiple defect categories, the first target loss is determined;

[0043] The parameters of the feature extraction network and multiple classification networks are adjusted based on the first target loss, and the first sample image is iterated to make the first target loss converge, thus obtaining the neural network model.

[0044] This example proposes an architecture where multiple classification networks share the output of a feature extraction network, ensuring that adding or removing classification networks does not affect the feature extraction network. Furthermore, using multiple labels to define and classify various defect categories achieves decoupling between defect categories, which helps reduce the influence between defect categories and annotation noise in sample images during training.

[0045] In conjunction with the first aspect, in one possible implementation, the method further includes:

[0046] If there are target defect categories to be added, then add a target classification network for the target defect categories to the neural network model;

[0047] Acquire a second sample image of the component to be inspected and annotation data of the second sample image for the target defect category;

[0048] The second sample image is input into the feature extraction network for feature extraction to obtain the third shared feature map;

[0049] The third shared feature map is input into the target classification network for defect classification, and the defect detection results of the second sample image for the target defect category are obtained.

[0050] Based on the defect detection results of the second sample image for the target defect category and the annotation data of the second sample image for the target defect category, the second target loss of the target classification network is determined;

[0051] The parameters of the target classification network are adjusted based on the second target loss, and the second sample image is iterated until the second target loss converges.

[0052] In this example, based on the architecture of multiple classification networks sharing the output of the feature extraction network, if a new target defect category is added, a new label can be defined for that target defect category, and a classification network can be trained for that target defect category using labeled second sample images. This allows for flexible expansion of defect categories and addition of classification networks. Furthermore, since the parameters of the feature extraction network and the previous multiple classification networks are fixed, there is no need to adjust the parameters of the feature extraction network and the multiple classification networks when training the target classification network, which helps improve the training efficiency of the target classification network.

[0053] A second aspect of this application provides a contact wire component defect detection device, which includes an acquisition unit and a processing unit, wherein...

[0054] The acquisition unit is used to acquire the image to be inspected of the high-speed railway catenary.

[0055] The processing unit is used to determine the region image where the part to be detected is located when the image to be detected includes the part to be detected.

[0056] The processing unit is also used to perform defect detection of multiple preset defect categories on the image of the area where the component to be detected is located, and to obtain the defect detection result of the component to be detected.

[0057] A third aspect of this application provides an electronic device including a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to perform some or all of the steps described in the first aspect of this application.

[0058] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of this application.

[0059] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package.

[0060] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A schematic diagram illustrating an application environment provided in an embodiment of this application;

[0063] Figure 2 A schematic flowchart illustrating a method for detecting defects in overhead contact line components provided in an embodiment of this application;

[0064] Figure 3 A schematic diagram of a neural network model provided in an embodiment of this application;

[0065] Figure 4 A schematic diagram illustrating a type of cotter pin defect provided in an embodiment of this application;

[0066] Figure 5 A schematic diagram of a feature extraction network provided in an embodiment of this application;

[0067] Figure 6 A schematic diagram of a flow model provided in an embodiment of this application;

[0068] Figure 7 A schematic flowchart of another method for detecting defects in contact wire components provided in an embodiment of this application;

[0069] Figure 8 A schematic diagram of a newly added classification network provided in an embodiment of this application;

[0070] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0071] Figure 10 This is a schematic diagram of a contact wire component defect detection device provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0073] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0074] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0075] To better understand the contact wire component defect detection method provided in this application embodiment, the application environment of the contact wire component defect detection method will be briefly introduced below.

[0076] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application, such as... Figure 1As shown, the system includes a terminal device 110, an electronic device 120, a high-speed rail overhead contact line 130, and an inspection vehicle 140. The terminal device 110 and the electronic device 120 are connected via a network. The terminal device 110 is used to provide images of the high-speed rail overhead contact line to be inspected, such as images of a support pillar or an image acquisition device on the inspection vehicle 140. Electronic device 120 is used to perform target detection on the received image to be detected, so as to detect the component to be detected from the image to be detected, and then segment the image of the region where the component to be detected is located. Through a trained neural network model, it performs defect detection on the image of the region where the component to be detected is located for multiple preset defect categories, and obtains the detection results of the component to be detected relative to multiple defect attributes under multiple defect categories. For example, the component to be detected may be a cotter pin. The detection result for the defect of cotter pin closure may be that the cotter pin closure defect exists, or that the cotter pin closure defect does not exist, or that the image of the region where the cotter pin is located is occluded or blurred, making it impossible to determine whether the cotter pin closure defect exists, or that the cotter pin is in the critical region of the defect category, making it impossible to determine whether the defect of this category exists. The detection result for the defect of cotter pin angle deviation may be that the cotter pin angle deviation defect exists, or that the cotter pin angle deviation defect does not exist, or that the image of the region where the cotter pin is located is occluded or blurred, making it impossible to determine whether the cotter pin angle deviation defect exists, or that the cotter pin is in the critical region of the defect category, making it impossible to determine whether the defect of this category exists, and so on.

[0077] Compared to existing solutions, this application embodiment uses a multi-label training neural network model. The electronic device 120 can obtain the detection results of the component to be detected for multiple defect categories through the neural network model, which helps to avoid the situation where a certain defect of the component to be detected is missed, thereby improving the accuracy of defect detection of the contact wire component.

[0078] It should be understood that in some embodiments, the terminal device 110 may also be a computer, smartphone, or other device. For example, the terminal device 110 may obtain a set of sample images for training a neural network model from a database and provide the set of sample images to the electronic device 120, so that the electronic device 120 can use the set of sample images to train the neural network. The electronic device 120 may be an independent physical server, a server cluster, or a distributed system. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. Alternatively, it may be a device capable of independent operation, such as an embedded device.

[0079] Please see Figure 2 , Figure 2This is a flowchart illustrating a method for detecting defects in overhead contact line components, provided in an embodiment of this application. This method is applied to electronic devices, such as... Figure 2 As shown, the method includes steps 201-203:

[0080] 201. Obtain the image of the high-speed railway overhead contact line to be inspected.

[0081] In a specific embodiment of this application, the image to be inspected of the high-speed railway catenary can be provided to the electronic equipment by the image acquisition device on the inspection vehicle. For example, during the inspection of power transmission lines, the image acquisition device can send the captured 4C image of the high-speed railway catenary to the electronic equipment. It should be understood that the image to be inspected can be provided to the electronic equipment by any device capable of acquiring images of the high-speed railway catenary, and this application does not limit the method of acquiring the image to be inspected of the high-speed railway catenary.

[0082] 202. When the image to be detected includes the component to be detected, determine the region of the image where the component to be detected is located.

[0083] In a specific embodiment of this application, for the received image to be detected, the electronic device can perform target detection on it through a target detection algorithm, such as the Faster R-CNN framework, the YOLO framework, etc. Based on the coordinates of the detection box of the component to be detected in the image to be detected, the region where the component to be detected is located can be determined from the image to be detected, and the image of the region where the component to be detected is located can be cropped.

[0084] For example, cropping the image of the region containing the component to be detected includes:

[0085] Edge detection is performed on the image to be detected to obtain at least one edge contour of at least one object in the image to be detected;

[0086] The edge contour of the component to be detected is determined from at least one edge contour;

[0087] Obtain the minimum bounding rectangle of the edge contour of the part to be detected, and determine the intersection of the minimum bounding rectangle and the detection box of the part to be detected as the region to be segmented;

[0088] The region to be segmented is cropped from the image to be detected, and the image of the region where the component to be detected is located is obtained.

[0089] Specifically, at least one object corresponds one-to-one with at least one edge profile, that is, each object has an edge profile. It should be understood that the components in the high-speed rail contact network have certain characteristics. For example, one end of the cotter pin is usually curved, and there is a certain angle between the two sides of the foot. By using these characteristics, the edge profile of the cotter pin in at least one edge profile can be determined.

[0090] 203. Perform defect detection on the image of the area where the component to be detected is located using preset defect categories to obtain the defect detection results of the component to be detected.

[0091] The image of the region containing the component to be inspected is subjected to defect detection of multiple preset defect categories to obtain the defect detection results of the component to be inspected, including:

[0092] Feature extraction is performed on the image of the region where the component to be detected is located to obtain the first shared feature map;

[0093] The first shared feature map is classified into defects by using a classification network corresponding to each defect category among multiple defect categories, so as to obtain the defect detection results of the component to be detected for each defect category.

[0094] In specific embodiments of this application, such as Figure 3 As shown, taking the cotter pin as the component to be inspected, the cotter pin image (i.e., the image of the area where the cotter pin is located) is input into a neural network model with fixed parameters. Feature extraction is performed by a feature extraction network shared by multiple classification networks to obtain a first shared feature map. This first shared feature map is then input into multiple classification networks for defect classification. For example, these multiple classification networks can be a cotter pin closure classification network and a cotter pin angle deviation classification network. The cotter pin closure classification network obtains detection results for multiple defect attributes under the defect category of cotter pin closure based on the first shared feature map; the cotter pin angle deviation classification network obtains detection results for multiple defect attributes under the defect category of cotter pin angle deviation based on the first shared feature map. For example, when the component to be inspected is a cotter pin, as... Figure 4 As shown, the various defect categories include at least two of the following: cotter pin closure, cotter pin angle deviation, and cotter pin missing. In this embodiment, the classification network corresponding to each defect category shares a first shared feature map for defect classification, so that each classification network does not need to perform feature extraction separately before classification, which helps to save feature extraction time and reduce feature extraction overhead.

[0095] For example, the detection results for each defect category include one of the following:

[0096] This category of defects exists;

[0097] There are no defects of this type;

[0098] It is impossible to determine whether this type of defect exists;

[0099] A defect exists, but it is impossible to determine whether the defect falls into that category.

[0100] For example, the detection result for the cotter pin angle deviation defect category could be: angle not in place (i.e., "present"), no angle not in place (i.e., "absent"), the cotter pin image being occluded or blurred making it impossible to determine whether the cotter pin angle is in place (i.e., "cannot determine whether the cotter pin angle deviation defect exists"), or the cotter pin being in a critical region of the defect category making it impossible to determine whether the defect of that category exists (i.e., "a defect exists, but it cannot be determined whether the existing defect belongs to that category, in short, there is ambiguity"). Because different technicians may have different classification standards for the same defect when labeling or classifying sample images, the machine may produce ambiguity in classification, indicating that the defect exists but is not the same as the defect. In this implementation, in addition to the conventional detection results, it also adds the possibility that the image of the area where the component to be detected is located is occluded or blurred, making it impossible to determine whether the defect of that category exists, and that the component to be detected has a defect, but the existing defect is in a critical region of the defect category, making it impossible to determine whether the existing defect belongs to that category. This achieves more refined detection. Furthermore, more refined defect classification can effectively reduce annotation noise in the dataset.

[0101] For example, feature extraction is performed on the image of the region where the component to be detected is located to obtain a first shared feature map, including:

[0102] The image of the region where the component to be detected is located is convolved to obtain the first feature map;

[0103] The first feature map is downsampled for m stages to obtain feature maps at m scales. The feature maps at m scales correspond one-to-one with the downsampling for m stages, where m is an integer greater than or equal to 2.

[0104] Map the first feature map to n target feature maps in the m scale feature maps to obtain (n+1) second feature maps, where n is an integer greater than or equal to 2 and less than or equal to m;

[0105] By combining (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain the first shared feature map.

[0106] Please refer to the specific embodiments in this application. Figure 5 This paper proposes a feature extraction network based on the U-Net convolutional neural network and incorporating a two-dimensional normalized flow model. Consistent with the U-Net convolutional neural network, it includes m stages of downsampling and m stages of upsampling, meaning the downsampling and upsampling parts are symmetrical. Figure 5As shown, the multiple stages include the first stage, the second stage, the third stage, and the fourth stage, respectively. The feature maps of m scales include the feature maps obtained by downsampling in the first stage, the second stage, the third stage, and the fourth stage. The feature maps obtained by downsampling in the first stage and the second stage are used as target feature maps. The first feature map, the feature maps obtained by downsampling in the first stage, and the feature maps obtained by downsampling in the second stage are mapped by a trained two-dimensional normalized flow model to obtain the probability density estimate of the feature in each feature map, that is, the probability density estimate of the feature representation in (n+1) second feature maps.

[0107] In this implementation, since the feature extraction process downsamples the first feature map in m stages, the dimensionality reduction of downsampling may lead to the loss of local information. Therefore, by mapping the first feature map and the feature maps at m scales, and combining the mapped (n+1) second feature maps, the feature map with the smallest scale in the m scale feature maps is upsampled in m stages. For example, the scale of the (n+1) second feature maps can be fused with the feature maps of the same scale obtained in the upsampling stage to make up for the information loss caused by downsampling, thereby making the information in the first shared feature map richer.

[0108] For example, by combining (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain a first shared feature map, including:

[0109] The first stage of upsampling is performed on the feature map with the smallest scale;

[0110] For the upsampling of the r-th stage in m-stage upsampling, if a target second feature map exists in the (n+1) second feature maps, then the target second feature map is fused with the feature map obtained from the upsampling of the r-th stage to obtain the first fused feature map. The target second feature map is a second feature map with the same scale as the feature map obtained from the upsampling of the r-th stage, where r is an integer greater than or equal to 1 and less than or equal to (m-1).

[0111] For the first fused feature map or the feature map obtained by upsampling in the r-th stage, perform upsampling in the (r+1)-th stage until the upsampling in the m-th stage is completed;

[0112] The feature map obtained by upsampling in the m-th stage is fused with the feature maps of the same scale in the (n+1) second feature maps to obtain the second fused feature map;

[0113] The second fused feature map is post-processed to obtain the first shared feature map.

[0114] Please continue reading Figure 5 The feature map with the smallest scale is the feature map obtained by downsampling in the fourth stage. The feature map with the smallest scale is then upsampled in the first stage. Since there is no target second feature map with the same scale as the feature map obtained by upsampling in the first stage among the (n+1) second feature maps, the feature map obtained by upsampling in the first stage is directly upsampled in the second stage. If there is a target second feature map with the same scale as the feature map obtained by upsampling in the second stage among the (n+1) second feature maps, then the feature map obtained by upsampling in the second stage is fused with the target second feature map with the same scale among the (n+1) second feature maps (i.e., the second feature map obtained by mapping the feature map obtained by downsampling in the second stage through a two-dimensional normalized flow model) to obtain the first fused feature map. The first fused feature map is upsampled in the third stage. If there is a target second feature map with the same scale as the feature map obtained by the third stage upsampling in the (n+1) second feature maps, then the feature map obtained by the third stage upsampling is fused with the target second feature map with the same scale in the (n+1) second feature maps (that is, the second feature map obtained by the mapping of the feature map obtained by the first stage downsampling through the two-dimensional normalized flow model) to obtain the second first fused feature map.

[0115] For the upsampling of the r-th stage in m-stage upsampling, if a target second feature map of the same scale exists in the (n+1) second feature maps, then the feature map obtained from the upsampling of the r-th stage is fused with the target second feature map, and the fused feature map is used for the next stage of upsampling; if no target second feature map of the same scale exists in the (n+1) second feature maps, then the feature map obtained from the upsampling of the r-th stage is directly used for the next stage of upsampling, and so on until the m-th stage of upsampling is completed. Figure 5 The fourth stage of upsampling.

[0116] For the feature map obtained by upsampling in the m-th stage, there must be a target second feature map with the same scale among the (n+1) second feature maps (i.e., the second feature map obtained by mapping the first feature map through the two-dimensional normalized flow model). Then, the two are fused to obtain the second fused feature map. The second fused feature map is then post-processed to obtain the first shared feature map.

[0117] Specifically, "fusion" can be a dot product of two feature maps. This multiplication of the probability density estimate in the second feature map with the upsampled feature map makes the defective regions more obvious, which is beneficial for subsequent classification. Post-processing can be channel compression. It should be understood that this application performs certain post-processing, including but not limited to channel compression, after each feature map fusion.

[0118] For example, mapping the first feature map to n target feature maps from m scales includes:

[0119] For each feature map in the first feature map and the n target feature maps, perform the first reversible transformation on each feature map to obtain the feature map after the first mapping;

[0120] Perform the (s+1)th reversible transformation on the feature map after the s-th mapping to obtain the feature map after the (s+1)-th mapping, where s is an integer greater than or equal to 1;

[0121] Each feature map is mapped to a second feature map through at least two reversible transformations, and (n+1) second feature maps are included.

[0122] In specific embodiments of this application, such as Figure 6 As shown, in flow models, invertible transformation blocks are typically used to map the features in the input feature map to a shape that satisfies distribution p. z The latent variable z can be used as the probability density estimate of the feature at the corresponding position in the cotter pin image. In other words, the features in the second feature map corresponding to each feature map in the first feature map and the n target feature maps are all probability density estimates. It should be understood that for an image of a normal part to be detected, its features follow a normal distribution, meaning its probability density estimate approaches the center 0 of the normal distribution. However, for an image of an abnormal part to be detected, the probability density estimate at the defect location is far from the center 0 of the normal distribution. Using the probability density estimate in the second feature map as weights and fusing it with the upsampled feature map of the same scale makes the defect more prominent and is more conducive to defect classification. For example, each classification network can also use the center 0 of the normal distribution as a threshold to segment the abnormal region in the first shared feature map, thereby obtaining the detection result of the part to be detected under the corresponding defect category.

[0123] Please continue reading Figure 6This application uses at least two reversible transformation blocks for mapping. Taking the first feature map as an example, the first feature map is mapped by the first reversible transformation block to obtain the feature map after the first mapping. The feature map after the first mapping is mapped by the second reversible transformation block to obtain the feature map after the second mapping, and so on. The feature map after the sth mapping is mapped by the (s+1)th reversible transformation block to obtain the feature map after the (s+1)th mapping. After at least two reversible transformation processes, the corresponding second feature map is obtained.

[0124] In this flow model, each reversible transform block performs an affine coupling operation on the input through an affine coupling layer. This affine coupling operation splits the input into two sub-feature maps. Two sub-networks are then used to perform a 2D convolution on one of these sub-feature maps, yielding scaling and translation coefficients. These scaling and translation coefficients are then used to linearly combine the two sub-feature maps, resulting in the output of the first affine coupling operation. Further, the output of the first affine coupling operation is concatenated with the input of the 2D convolution to obtain the output of the reversible transform block. Alternatively, the output of the first affine coupling operation can be further subjected to affine coupling operations. After multiple affine coupling operations, the output of the last affine coupling operation is concatenated with the output of the previous affine coupling operation to obtain the output of the reversible transform block. The processing flow of the affine coupling layer can be found in existing related technologies and will not be described in detail here.

[0125] For example, such as Figure 6 As shown, the reversible transform block in the flow model uses alternating 3*3 convolution and 1*1 convolution, which helps to preserve the spatial information of features in the region image where the part to be detected is located.

[0126] In this embodiment, a two-dimensional normalized flow model is used to perform at least two reversible transformations on each feature map of the first feature map and the feature maps of m scales. Since the at least two reversible transformations can be performed in an alternating manner of large and small convolutions, it is beneficial to preserve the spatial positional relationship between the first feature map and the feature maps of m scales. This spatial positional relationship is beneficial to the localization of defects in the component to be detected.

[0127] As can be seen, the embodiments of this application acquire an image of the high-speed railway catenary to be inspected; if the image includes a component to be inspected, determine the area image where the component is located; perform defect detection on the area image where the component is located using preset defect categories, and obtain the defect detection result of the component. In this way, during defect detection, the detection results of the component under inspection in multiple defect categories can be detected, which helps to avoid missing certain defects of the component and thus improves the accuracy of catenary component defect detection.

[0128] Please see Figure 7 , Figure 7 This is a schematic flowchart illustrating another method for detecting defects in overhead contact line components provided in an embodiment of this application. Figure 7 As shown, the method includes steps 701-704:

[0129] 701. Obtain the image of the high-speed railway overhead contact line to be inspected;

[0130] 702. When the image to be detected includes the component to be detected, determine the region of the image where the component to be detected is located;

[0131] 703. Extract features from the image of the region where the component to be detected is located to obtain the first shared feature map;

[0132] 704. The first shared feature map is classified into defects by using a classification network corresponding to each of the preset multiple defect categories, so as to obtain the defect detection results of the component to be detected for each defect category.

[0133] The specific implementation methods of steps 701-704 above are as follows: Figure 2 The embodiments shown have already been described, and they can achieve the same or similar beneficial effects, so they will not be repeated here.

[0134] Steps 703 and 704 are performed using a neural network model, which is obtained by training a neural network. The neural network includes a feature extraction network and multiple classification networks for classifying various defect categories. This neural network model is trained through the following steps:

[0135] Acquire the first sample image of the component to be inspected and the annotation data of the first sample image for multiple defect categories;

[0136] The first sample image is input into the feature extraction network for feature extraction to obtain the second shared feature map;

[0137] The second shared feature map is input into multiple classification networks for defect classification to obtain the defect detection results of the first sample image for multiple defect categories;

[0138] Based on the defect detection results of the first sample image for multiple defect categories and the annotation data of the first sample image for multiple defect categories, the first target loss is determined;

[0139] The parameters of the feature extraction network and multiple classification networks are adjusted based on the first target loss, and the first sample image is iterated to make the first target loss converge, thus obtaining the neural network model.

[0140] In a specific embodiment of this application, the second shared feature map is the feature map obtained by feature extraction of the first sample image through the feature extraction network during the training phase. It should be understood that the training of neural networks usually requires a certain dataset, and the first sample image can be any sample image in the dataset. The feature extraction process of the feature extraction network for the first sample image can be found in the relevant description of feature extraction of the region image where the part to be detected is located. After obtaining the detection results of the first sample image for multiple defect attributes under multiple defect categories through the classification network, the corresponding loss can be calculated based on the detection results of multiple defect attributes under each defect category and the labeled data of that defect category. The first target loss is obtained based on the multiple losses of the first sample image under multiple defect categories. For example, when the part to be detected is a cotter pin, the loss of the first sample image in the cotter pin angle deviation defect category can be used as the first target loss, or the joint loss of multiple losses of some or all defect categories can be used as the first target loss. This is not limited here.

[0141] If the first objective loss does not converge, the parameters of the feature extraction network and multiple classification networks are adjusted, and the sample image set (i.e. the first sample image) is iterated continuously until the first objective loss converges, resulting in a neural network model with fixed parameters.

[0142] In this implementation, an architecture is proposed where multiple classification networks share the output of the feature extraction network. Adding or removing classification networks does not affect the feature extraction network. Furthermore, multiple labels are used to define and classify various defect categories, achieving decoupling between defect categories and reducing the influence between defect categories and annotation noise in sample images during training.

[0143] For example, the method further includes:

[0144] If there are target defect categories to be added, then add a target classification network for the target defect categories to the neural network model;

[0145] Obtain the second sample image of the cotter pin and the annotation data of the second sample image for the target defect category;

[0146] The second sample image is input into the feature extraction network for feature extraction to obtain the third shared feature map;

[0147] The third shared feature map is input into the target classification network for defect classification, and the detection results of the second sample image for the target defect category are obtained.

[0148] Based on the detection results of the target defect category in the second sample image and the annotation data of the target defect category in the second sample image, the second target loss of the target classification network is determined.

[0149] The parameters of the target classification network are adjusted based on the second target loss, and the second sample image is iterated until the second target loss converges.

[0150] In this specific embodiment, the component to be tested is a cotter pin. If it is now necessary to define a new defect category for the cotter pin, such as a missing cotter pin, then... Figure 8 As shown, a cotter pin missing classification network is added to the original neural network model. This cotter pin missing classification network is also the target classification network. Sample images of cotter pin missing are added to the training dataset of the neural network and labeled to update the dataset. The second sample image is any image in the updated dataset, including sample images of cotter pin missing. The third shared feature map is the feature map obtained by feature extraction network from the second sample image during the training phase of the target classification network. The feature extraction process of the second sample image can also be found in the relevant description of feature extraction for the part to be detected. After obtaining the detection result of the target defect category of the second sample image, the target classification network calculates the loss between this result and the labeled data of the target defect category of the second sample image, i.e., the second target loss. If the loss does not converge, the parameters of the target classification network are adjusted while keeping the parameters of the feature extraction network and the previous multiple classification networks unchanged. The second sample image is iterated, and the second target loss is observed until the second target loss converges. It should be noted that the third shared feature map will also be entered into the previous multiple classification networks for classification.

[0151] In this implementation, based on the architecture of multiple classification networks sharing the output of the feature extraction network, if a new target defect category is added, a new label can be defined for that target defect category, and a classification network can be trained for that target defect category using labeled second sample images. This allows for flexible expansion of defect categories and addition of classification networks. Furthermore, since the parameters of the feature extraction network and the previous multiple classification networks are fixed, there is no need to adjust the parameters of the feature extraction network and the multiple classification networks when training the target classification network, which helps improve the training efficiency of the target classification network.

[0152] For examples consistent with the above embodiments, please refer to... Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, it includes a processor, input devices, output devices, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions. The program includes instructions for performing the following steps.

[0153] Obtain the image of the high-speed railway overhead contact line to be inspected;

[0154] When the image to be detected includes the component to be detected, determine the region of the image where the component to be detected is located;

[0155] The image of the area where the component to be inspected is located is subjected to defect detection of multiple preset defect categories to obtain the defect detection results of the component to be inspected.

[0156] It can be seen that, Figure 9 The electronic device shown acquires an image of the high-speed rail overhead contact line to be inspected; if the image includes a component to be inspected, the region containing the component is determined; and multiple preset defect categories are used to detect defects in the region containing the component, resulting in the defect detection result for the component. This allows for the detection of multiple defect categories of the component during defect inspection, helping to avoid missing certain defects and thus improving the accuracy of defect detection.

[0157] In one possible implementation, the processor performs defect detection on the image of the region where the component to be detected is located, applying multiple preset defect categories to obtain the defect detection results for the component to be detected, including:

[0158] Feature extraction is performed on the image of the region where the component to be detected is located to obtain the first shared feature map;

[0159] The first shared feature map is classified into defects by using a classification network corresponding to each defect category among multiple defect categories, so as to obtain the defect detection results of the component to be detected for each defect category.

[0160] In one possible implementation, the defect detection result for each defect category includes one of the following:

[0161] This category of defects exists;

[0162] There are no defects of this type;

[0163] It is impossible to determine whether this type of defect exists;

[0164] A defect exists, but it is impossible to determine whether the defect falls into that category.

[0165] In one possible implementation, the processor performs feature extraction on the region image where the part to be detected is located to obtain a first shared feature map, including:

[0166] The image of the region where the component to be detected is located is convolved to obtain the first feature map;

[0167] The first feature map is downsampled for m stages to obtain feature maps at m scales. The feature maps at m scales correspond one-to-one with the downsampling of the m stages.

[0168] Map the first feature map to n target feature maps in the m scale feature maps to obtain (n+1) second feature maps, where n is an integer greater than or equal to 2 and less than or equal to m;

[0169] By combining (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain the first shared feature map.

[0170] In one possible implementation, the processor performs m-stage upsampling on the smallest feature map among the m-scale feature maps, combining (n+1) second feature maps, to obtain a first shared feature map, including:

[0171] The first stage of upsampling is performed on the feature map with the smallest scale;

[0172] For the upsampling of the r-th stage in m-stage upsampling, if a target second feature map exists in the (n+1) second feature maps, then the target second feature map is fused with the feature map obtained from the upsampling of the r-th stage to obtain the first fused feature map. The target second feature map is a second feature map with the same scale as the feature map obtained from the upsampling of the r-th stage, where r is an integer greater than or equal to 1 and less than or equal to (m-1).

[0173] For the first fused feature map or the feature map obtained by upsampling in the r-th stage, perform upsampling in the (r+1)-th stage until the upsampling in the m-th stage is completed;

[0174] The feature map obtained by upsampling in the m-th stage is fused with the feature maps of the same scale in the (n+1) second feature maps to obtain the second fused feature map;

[0175] The second fused feature map is post-processed to obtain the first shared feature map.

[0176] In one possible implementation, the processor performs mapping on the first feature map and n target feature maps from the m scale feature maps, including:

[0177] For each feature map in the first feature map and the n target feature maps, perform the first reversible transformation on each feature map to obtain the feature map after the first mapping;

[0178] Perform the (s+1)th reversible transformation on the feature map after the s-th mapping to obtain the feature map after the (s+1)-th mapping, where s is an integer greater than or equal to 1;

[0179] Each feature map is mapped to a second feature map through at least two reversible transformations, and (n+1) second feature maps are included.

[0180] In one possible implementation, defect detection of multiple pre-defined defect categories is performed on the image of the region where the component to be detected is located. The defect detection result of the component to be detected is obtained through a neural network model. The neural network model is obtained by training a neural network, which includes a feature extraction network and multiple classification networks for classifying the multiple defect categories respectively. The processor executes the training of the neural network model, including:

[0181] Acquire the first sample image of the component to be inspected and the annotation data of the first sample image for multiple defect categories;

[0182] The first sample image is input into the feature extraction network for feature extraction to obtain the second shared feature map;

[0183] The second shared feature map is input into multiple classification networks for defect classification to obtain the defect detection results of the first sample image for multiple defect categories;

[0184] Based on the defect detection results of the first sample image for multiple defect categories and the annotation data of the first sample image for multiple defect categories, the first target loss is determined;

[0185] The parameters of the feature extraction network and multiple classification networks are adjusted based on the first target loss, and the first sample image is iterated to make the first target loss converge, thus obtaining the neural network model.

[0186] In one possible implementation, the processor is also used to perform:

[0187] If there are target defect categories to be added, then add a target classification network for the target defect categories to the neural network model;

[0188] Acquire a second sample image of the component to be inspected and annotation data of the second sample image for the target defect category;

[0189] The second sample image is input into the feature extraction network for feature extraction to obtain the third shared feature map;

[0190] The third shared feature map is input into the target classification network for defect classification, and the defect detection results of the second sample image for the target defect category are obtained.

[0191] Based on the defect detection results of the second sample image for the target defect category and the annotation data of the second sample image for the target defect category, the second target loss of the target classification network is determined;

[0192] The parameters of the target classification network are adjusted based on the second target loss, and the second sample image is iterated until the second target loss converges.

[0193] In one possible implementation, the component to be inspected is a cotter pin, and multiple defect categories include at least two of the following: cotter pin closure, cotter pin angle deviation, and cotter pin missing.

[0194] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0196] For those consistent with the above, please refer to Figure 10 , Figure 10 This is a schematic diagram of a contact wire component defect detection device provided in an embodiment of this application. Figure 10 As shown, the device includes an acquisition unit 1001 and a processing unit 1002;

[0197] The acquisition unit 1001 is used to acquire the image to be detected of the high-speed railway catenary.

[0198] Processing unit 1002 is used to determine the region image where the component to be detected is located when the image to be detected includes the component to be detected;

[0199] The processing unit 1002 is also used to perform defect detection of multiple preset defect categories on the image of the area where the component to be detected is located, and to obtain the defect detection result of the component to be detected.

[0200] It can be seen that, Figure 10The overhead contact line component defect detection device shown acquires an image of the high-speed railway overhead contact line to be inspected; if the image includes the component to be inspected, the area image containing the component is determined; and multiple preset defect categories are used to detect defects in the area image containing the component, resulting in the defect detection result for the component. This allows for the detection of multiple defect categories of the component during defect detection, helping to avoid missing certain defects and thus improving the accuracy of defect detection.

[0201] In one possible implementation, the processing unit 1002 is specifically used to perform defect detection of multiple preset defect categories on the image of the region where the component to be detected is located, and to obtain the defect detection result of the component to be detected, in order to:

[0202] Feature extraction is performed on the image of the region where the component to be detected is located to obtain the first shared feature map;

[0203] The first shared feature map is classified into defects by using a classification network corresponding to each defect category among multiple defect categories, so as to obtain the defect detection results of the component to be detected for each defect category.

[0204] In one possible implementation, the defect detection result for each defect category includes one of the following:

[0205] This category of defects exists;

[0206] There are no defects of this type;

[0207] It is impossible to determine whether this type of defect exists;

[0208] A defect exists, but it is impossible to determine whether the defect falls into that category.

[0209] In one possible implementation, in extracting features from the image of the region where the part to be detected is located to obtain a first shared feature map, the processing unit 1002 is specifically used for:

[0210] The image of the region where the component to be detected is located is convolved to obtain the first feature map;

[0211] The first feature map is downsampled for m stages to obtain feature maps at m scales. The feature maps at m scales correspond one-to-one with the downsampling of the m stages.

[0212] Map the first feature map to n target feature maps in the m scale feature maps to obtain (n+1) second feature maps, where n is an integer greater than or equal to 2 and less than or equal to m;

[0213] By combining (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain the first shared feature map.

[0214] In one possible implementation, in order to obtain the first shared feature map by upsampling the smallest feature map among the m scales of the m scales using (n+1) second feature maps, the processing unit 1002 is specifically used for:

[0215] The first stage of upsampling is performed on the feature map with the smallest scale;

[0216] For the upsampling of the r-th stage in m-stage upsampling, if a target second feature map exists in the (n+1) second feature maps, then the target second feature map is fused with the feature map obtained from the upsampling of the r-th stage to obtain the first fused feature map. The target second feature map is a second feature map with the same scale as the feature map obtained from the upsampling of the r-th stage, where r is an integer greater than or equal to 1 and less than or equal to (m-1).

[0217] For the first fused feature map or the feature map obtained by upsampling in the r-th stage, perform upsampling in the (r+1)-th stage until the upsampling in the m-th stage is completed;

[0218] The feature map obtained by upsampling in the m-th stage is fused with the feature maps of the same scale in the (n+1) second feature maps to obtain the second fused feature map;

[0219] The second fused feature map is post-processed to obtain the first shared feature map.

[0220] In one possible implementation, the processing unit 1002 is specifically used for mapping the first feature map and n target feature maps from the m-scale feature maps to:

[0221] For each feature map in the first feature map and the n target feature maps, perform the first reversible transformation on each feature map to obtain the feature map after the first mapping;

[0222] Perform the (s+1)th reversible transformation on the feature map after the s-th mapping to obtain the feature map after the (s+1)-th mapping, where s is an integer greater than or equal to 1;

[0223] Each feature map is mapped to a second feature map through at least two reversible transformations, and (n+1) second feature maps are included.

[0224] In one possible implementation, the defect detection of the region image where the component to be detected is located is performed on multiple preset defect categories, and the defect detection result of the component to be detected is obtained by a neural network model. The neural network model is obtained by training a neural network, which includes a feature extraction network and multiple classification networks for classifying the multiple defect categories respectively. The processing unit 1002 is also used for:

[0225] Acquire the first sample image of the component to be inspected and the annotation data of the first sample image for multiple defect categories;

[0226] The first sample image is input into the feature extraction network for feature extraction to obtain the second shared feature map;

[0227] The second shared feature map is input into multiple classification networks for defect classification to obtain the defect detection results of the first sample image for multiple defect categories;

[0228] Based on the defect detection results of the first sample image for multiple defect categories and the annotation data of the first sample image for multiple defect categories, the first target loss is determined;

[0229] The parameters of the feature extraction network and multiple classification networks are adjusted based on the first target loss, and the first sample image is iterated to make the first target loss converge, thus obtaining the neural network model.

[0230] In one possible implementation, the processing unit 1002 is further configured to:

[0231] If there are target defect categories to be added, then add a target classification network for the target defect categories to the neural network model;

[0232] Obtain the second sample image of the cotter pin and the annotation data of the second sample image for the target defect category;

[0233] The second sample image is input into the feature extraction network for feature extraction to obtain the third shared feature map;

[0234] The third shared feature map is input into the target classification network for defect classification, and the defect detection results of the second sample image for the target defect category are obtained.

[0235] Based on the defect detection results of the second sample image for the target defect category and the annotation data of the second sample image for the target defect category, the second target loss of the target classification network is determined;

[0236] The parameters of the target classification network are adjusted based on the second target loss, and the second sample image is iterated until the second target loss converges.

[0237] In one possible implementation, the component to be inspected is a cotter pin, and multiple defect categories include at least two of the following: cotter pin closure, cotter pin angle deviation, and cotter pin missing.

[0238] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the contact wire component defect detection methods described in the above method embodiments.

[0239] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps of any of the contact wire component defect detection methods described in the above method embodiments.

[0240] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0241] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0242] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0243] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0244] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0245] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0246] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0247] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting defects in overhead contact line components, characterized in that, The method includes: Obtain the image of the high-speed railway overhead contact line to be inspected; If the image to be detected includes a component to be detected, the region image where the component to be detected is located is determined; Defect detection of multiple preset defect categories is performed on the image of the area where the component to be detected is located to obtain the defect detection result of the component to be detected; The step of performing defect detection on the image of the region where the component to be detected is located, based on preset defect categories, to obtain the defect detection result of the component to be detected, includes: Feature extraction is performed on the image of the region where the component to be detected is located to obtain a first shared feature map; The first shared feature map is classified using a classification network corresponding to each of the multiple defect categories to obtain the defect detection result of the component to be detected for each defect category; The step of extracting features from the image of the region where the component to be detected is located to obtain a first shared feature map includes: Convolution is performed on the image of the region where the component to be detected is located to obtain a first feature map; The first feature map is downsampled for m stages to obtain feature maps at m scales. The feature maps at m scales correspond one-to-one with the downsampling for the m stages, where m is an integer greater than or equal to 2. A two-dimensional normalized flow model is used to map the first feature map and n target feature maps from the m scale feature maps to obtain (n+1) second feature maps, where n is an integer greater than or equal to 2 and less than or equal to m. The feature representations in the (n+1) second feature maps are probability density estimates. The two-dimensional normalized flow model is used to perform at least two reversible transformations on each feature map in the first feature map and the n target feature maps. The at least two reversible transformations are performed by alternating between 3*3 convolutions and 1*1 convolutions. By combining the (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain the first shared feature map.

2. The method according to claim 1, characterized in that, The defect detection result for each defect category includes one of the following: This category of defects exists; There are no defects of this type; It is impossible to determine whether this type of defect exists; A defect exists, but it is impossible to determine whether the defect falls into that category.

3. The method according to claim 1, characterized in that, The step of combining the (n+1) second feature maps and upsampling the smallest feature map among the m scales in m stages to obtain the first shared feature map includes: The feature map with the smallest scale is upsampled in the first stage; For the upsampling of the r-th stage in the m-th stage, if there is a target second feature map in the (n+1) second feature maps, then the target second feature map is fused with the feature map obtained by the upsampling of the r-th stage to obtain a first fused feature map. The target second feature map is a second feature map with the same scale as the feature map obtained by the upsampling of the r-th stage, where r is an integer greater than or equal to 1 and less than or equal to (m-1). For the first fused feature map or the feature map obtained by upsampling in the r-th stage, perform upsampling in the (r+1)-th stage until the upsampling in the m-th stage is completed; The feature map obtained by upsampling in the m-th stage is fused with the feature maps of the same scale in the (n+1) second feature maps to obtain the second fused feature map; The second fused feature map is post-processed to obtain the first shared feature map.

4. The method according to claim 1, characterized in that, The mapping of the first feature map and the n target feature maps among the m scale feature maps includes: For each feature map in the first feature map and the n target feature maps, perform the first reversible transformation process on each feature map to obtain the feature map after the first mapping; Perform the (s+1)th reversible transformation on the feature map after the s-th mapping to obtain the feature map after the (s+1)-th mapping, where s is an integer greater than or equal to 1; Each feature map is mapped to a second feature map through at least two reversible transformation processes, and the (n+1) second feature maps include the second feature map.

5. The method according to any one of claims 1-4, characterized in that, The defect detection of the region image where the component to be detected is located, based on preset defect categories, to obtain the defect detection result of the component to be detected, is performed by a neural network model. This neural network model is obtained by training a neural network, which includes a feature extraction network and multiple classification networks for classifying the preset defect categories. The neural network model is trained through the following steps: Acquire a first sample image of the component to be inspected and annotation data of the first sample image for the various defect categories; The first sample image is input into the feature extraction network for feature extraction to obtain a second shared feature map; The second shared feature map is input into the multiple classification networks respectively for defect classification, and the defect detection results of the first sample image for the multiple defect categories are obtained. Based on the defect detection results of the first sample image for the multiple defect categories and the annotation data of the first sample image for the multiple defect categories, a first target loss is determined; The parameters of the feature extraction network and the plurality of classification networks are adjusted according to the first target loss, and the first sample image is iterated to make the first target loss converge, thereby obtaining the neural network model.

6. The method according to claim 5, characterized in that, The method further includes: If there are target defect categories to be added, then add a target classification network for those target defect categories to the neural network model; Acquire a second sample image of the component to be inspected and annotation data of the second sample image for the target defect category; The second sample image is input into the feature extraction network for feature extraction to obtain a third shared feature map; The third shared feature map is input into the target classification network for defect classification to obtain the defect detection result of the second sample image for the target defect category; Based on the defect detection results of the second sample image for the target defect category and the annotation data of the second sample image for the target defect category, the second target loss of the target classification network is determined; The parameters of the target classification network are adjusted according to the second target loss, and the second sample image is iterated to make the second target loss converge.

7. The method according to claim 6, characterized in that, The component to be inspected is a cotter pin, and the various defect categories include at least two of the following: Cotter pin closure, cotter pin angle deviation, and missing cotter pin.

8. A device for detecting defects in overhead contact line components, characterized in that, The device includes an acquisition unit and a processing unit, wherein... The acquisition unit is used to acquire the image to be detected of the high-speed railway catenary. The processing unit is configured to determine the region image where the component to be detected is located when the image to be detected includes the component to be detected. The processing unit is also used to perform defect detection of multiple preset defect categories on the image of the area where the component to be detected is located, and to obtain the defect detection result of the component to be detected; In terms of performing defect detection on the image of the region where the component to be detected is located, according to preset defect categories, and obtaining the defect detection result of the component to be detected, the processing unit is specifically used for: Feature extraction is performed on the image of the region where the component to be detected is located to obtain a first shared feature map; The first shared feature map is classified using a classification network corresponding to each of the multiple defect categories to obtain the defect detection result of the component to be detected for each defect category; In extracting features from the image of the region where the component to be detected is located to obtain a first shared feature map, the processing unit is specifically used for: Convolution is performed on the image of the region where the component to be detected is located to obtain a first feature map; The first feature map is downsampled for m stages to obtain feature maps at m scales. The feature maps at m scales correspond one-to-one with the downsampling for the m stages, where m is an integer greater than or equal to 2. A two-dimensional normalized flow model is used to map the first feature map and n target feature maps from the m scale feature maps to obtain (n+1) second feature maps, where n is an integer greater than or equal to 2 and less than or equal to m. The feature representations in the (n+1) second feature maps are probability density estimates. The two-dimensional normalized flow model is used to perform at least two reversible transformations on each feature map in the first feature map and the n target feature maps. The at least two reversible transformations are performed by alternating between 3*3 convolutions and 1*1 convolutions. By combining the (n+1) second feature maps, the smallest feature map among the m scale feature maps is upsampled in m stages to obtain the first shared feature map.

9. An electronic device, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-7.

Images