Component absence detection method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a component missing detection method and device, electronic equipment and storage medium, the method comprises: obtaining the contact network corresponding to the image to be processed;Based on the preset relative position information between the required preset local component and the corresponding preset structured component of the contact network, the image to be processed is detected to obtain the absolute position information of the target local component predicted to exist in the image to be processed and the first category of the target local component;Wherein, the target local component belongs to the sub-component of the target structured component included in the image to be processed;Based on the image to be processed, the absolute position information and the first category of the target local component, the component missing result corresponding to the image to be processed is determined.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and in particular, to a component missing detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the continuous development of high-speed rail construction, the safety and reliability of the power supply system of high-speed rail are increasingly required. Among them, the detection and maintenance of the catenary has become an important part of high-speed rail operation and maintenance. The high-speed rail operation and maintenance task includes a task of detecting whether the components (such as bolts, nuts, etc.) on the catenary are missing.

[0003] Generally, a detector with prior knowledge detects the collected catenary image to determine whether there is a component missing on the catenary. However, the above manual investigation method has the problems of long time consumption, low efficiency, and easy to miss detection. SUMMARY

[0004] Therefore, the present disclosure at least provides a component missing detection method and device, electronic equipment and storage medium.

[0005] In a first aspect, the present disclosure provides a component missing detection method, comprising:

[0006] obtaining a to-be-processed image corresponding to a catenary;

[0007] detecting the to-be-processed image based on preset relative position information between a preset local component required by the catenary and a corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component; wherein the target local component belongs to a sub-component of a target structured component included in the to-be-processed image;

[0008] determining a component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component, and the first category.

[0009] Since there is a structural correlation between the structured component on the catenary and the included local component, the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component can be obtained by detecting the to-be-processed image based on preset relative position information between the preset local component required by the catenary and the corresponding preset structured component. In this way, the absolute position information of the target local component that should exist on the target structured component and the first category of the target local component are more accurately and efficiently predicted by using the target structured component included in the to-be-processed image. Then, the component missing result corresponding to the to-be-processed image can be determined based on the to-be-processed image, the absolute position information and the first category of the target local component, thereby realizing automatic detection of component missing, improving the efficiency and accuracy of component missing detection, and reducing the missed detection rate.

[0010] In a possible implementation, the determining of the component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information and the first category of the target local component includes:

[0011] intercepting a local image corresponding to the target local component from the to-be-processed image based on the absolute position information of the target local component;

[0012] performing target detection on the local image based on the first category of the target local component to determine whether the target local component exists in the local image;

[0013] if it is detected that the target local component does not exist in any local image, determining that the component missing result corresponding to the to-be-processed image is that the target local component corresponding to the any local image is missing.

[0014] In the above implementation, the local image corresponding to the target local component is intercepted from the to-be-processed image based on the absolute position information of the target local component, so that the image information other than the local image in the to-be-processed image is filtered out, thereby avoiding interference of the other image information on the detection of the target local component and improving the detection accuracy of the target local component when the target detection is performed on the local image. Furthermore, the component missing result corresponding to the to-be-processed image can be more accurately determined according to the detection result of the target local component of each local image, thereby improving the accuracy of component missing detection.

[0015] In a possible implementation, the method further includes:

[0016] in a case where the component missing result indicates that the target local component is missing, determining missing component information corresponding to the target local component;

[0017] generate the warning information for the target local component based on the missing component information corresponding to the target local component.

[0018] Here, when it is detected that the to-be-processed image has a missing target local component, missing component information corresponding to the target local component can be determined, and warning information for the target local component can be generated based on the missing component information corresponding to the target local component. The content of the warning information is relatively rich and flexible, so that the management personnel can accurately maintain the overhead contact line according to the warning information, and the operation safety of the overhead contact line is improved.

[0019] In a possible implementation, the detecting the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component to obtain the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component comprises:

[0020] detecting the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component to obtain a target feature map corresponding to the to-be-processed image and initial position information of the target local component predicted to exist on the to-be-processed image;

[0021] obtaining the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component based on the initial position information and the target feature map.

[0022] Here, the to-be-processed image can be first detected based on the preset relative position information between the preset local component and the corresponding preset structured component, for example, initial positioning detection is performed to obtain a target feature map corresponding to the to-be-processed image and initial position information of the target local component predicted to exist on the to-be-processed image; then, repositioning detection is performed based on the initial position information and the target feature map to obtain the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component; through multiple detection processes, the generated absolute position information is relatively accurate.

[0023] In a possible implementation, the obtaining the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component based on the initial position information and the target feature map comprises:

[0024] cutting a local feature map corresponding to the target local component from the target feature map based on the initial position information;

[0025] performing at least one first feature extraction on the local feature map to obtain a first target local feature map;

[0026] adjust the initial position information based on the first target local feature map to obtain absolute position information of the target local component predicted to exist in the to-be-processed image; and

[0027] determine a first category of the target local component based on the first target local feature map.

[0028] Here, based on the initial position information, a local feature map corresponding to the target local component is intercepted from the target feature map, the feature information of the target local component accounts for a large proportion in the local feature map, and at least one first feature extraction is performed on the local feature map to obtain a first target local feature map. Based on the first target local feature map, the absolute position information and the first category of the target local component can be obtained more accurately.

[0029] In a possible implementation, after the to-be-processed image corresponding to the catenary is acquired, the method further includes:

[0030] detecting the to-be-processed image based on preset relative position information between a preset local component and a corresponding preset structured component to determine relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs, and a second category of the target structured component;

[0031] The method further includes:

[0032] In a case where the component absence result indicates absence of the target local component, based on at least one of the relative position information between the target local component and the target structured component to which the target local component belongs, the absolute position information of the target local component, the first category of the target local component, and the second category of the target structured component, the missing component information corresponding to the target local component is determined.

[0033] Here, the content of the missing component information is relatively rich, and subsequent generation of warning information for the target local component based on the missing component information corresponding to the target local component enables the generated warning information to accurately report the missing target local component, facilitating maintenance of the missing target local component by the management personnel.

[0034] In a possible implementation, the detecting the to-be-processed image based on preset relative position information between a preset local component and a corresponding preset structured component to determine relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs, and a second category of the target structured component includes:

[0035] detect the target image based on preset relative position information between the preset local component and the corresponding preset structured component, to determine initial relative position information between the target local component and the target structured component to which the target local component belongs in the target image;

[0036] cut a local feature map corresponding to the target local component from the target feature map based on the initial position information;

[0037] perform at least one second feature extraction on the local feature map to obtain a second target local feature map;

[0038] adjust the initial relative position information based on the second target local feature map to obtain relative position information between the target local component and the target structured component to which the target local component belongs in the target image; and

[0039] determine a second category of the target structured component based on the second target local feature map.

[0040] In a possible implementation, after obtaining the target image corresponding to the catenary, the method further includes:

[0041] adjust the brightness of the target image to generate an adjusted target image;

[0042] detect the target image based on preset relative position information between the preset local component and the corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the target image and a first category of the target local component, including:

[0043] detect the adjusted target image based on preset relative position information between the preset local component and the corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the adjusted target image and a first category of the target local component.

[0044] To alleviate the problem of insufficient exposure that may exist in the target image, the brightness of the target image can be adjusted to generate an adjusted target image, so that subsequent target detection can be performed on the adjusted target image, and the component detection accuracy can be improved.

[0045] In a possible implementation, in a case where the to-be-processed image corresponds to a target line identification of the catenary, the to-be-processed image is detected based on preset relative position information between preset local components and corresponding preset structured components required by the catenary to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, and the method comprises the following steps:

[0046] target relative position information matched with the target line identification is determined from preset relative position information between preset local components and corresponding preset structured components respectively corresponding to different line identifications of the catenary;

[0047] the to-be-processed image is detected based on target relative position information between preset local components and corresponding preset structured components matched with the target line identification to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0048] In view of the fact that structural correlation relationships between structured components and included local components on different line sections can be different, in order to more accurately detect local components of the catenary on each line section, a line identification can be set, different preset relative position information is set for different line identifications. Then, target relative position information matched with the target line identification can be determined according to a target line identification of the catenary corresponding to the to-be-processed image, and the to-be-processed image is detected based on the target relative position information to more accurately predict absolute position information and a first category of the target local component.

[0049] In a possible implementation, the absolute position information and the first category of the target local component are determined based on a trained target neural network; the target neural network is trained according to the following steps:

[0050] a sample image of the catenary is obtained, the sample image containing annotation data, the annotation data including sample absolute position information of at least one local component, a first sample category corresponding to the local component, sample relative position information between each local component and a structured component to which the local component belongs, and a second sample category corresponding to the structured component;

[0051] the sample image is input into a neural network to be trained to obtain prediction data corresponding to the sample image, wherein the prediction data is data matched with the annotation data;

[0052] the neural network to be trained is trained based on the annotation data and the prediction data until a trained neural network converges, and the target neural network is obtained.

[0053] Here, the sample absolute position information of at least one local component, the first sample category corresponding to the local component, the sample relative position information between each local component and the structured component to which the local component belongs, and the second sample category corresponding to the structured component can be included in the annotation data. The data content of the annotation data is rich, and the relative position information and the second sample category are used as the supervision signal of the neural network to be trained. After obtaining the prediction data, the neural network to be trained can be trained more accurately based on the annotation data and the prediction data, and the performance of the target neural network obtained by training is improved.

[0054] The effects of the following devices, electronic devices, and the like are described in the description of the above method, and will not be repeated here.

[0055] In a second aspect, the present disclosure provides a component absence detection device, comprising:

[0056] The acquisition module is configured to acquire a to-be-processed image corresponding to the overhead contact system.

[0057] The detection module is configured to detect the to-be-processed image based on preset relative position information between a preset local component required by the overhead contact system and a corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component; wherein the target local component belongs to a sub-component of a target structured component included in the to-be-processed image.

[0058] The determination module is configured to determine a component absence result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component, and the first category.

[0059] In a possible implementation, when determining the component absence result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component, and the first category, the determination module is configured to:

[0060] Based on the absolute position information of the target local component, a local image corresponding to the target local component is intercepted from the to-be-processed image;

[0061] Based on the first category of the target local component, target detection is performed on the local image to determine whether the target local component exists in the local image.

[0062] If it is detected that the target local component does not exist in any local image, it is determined that the component absence result corresponding to the to-be-processed image is that the target local component corresponding to the any local image is absent.

[0063] In a possible implementation, the device further comprises a warning module configured to:

[0064] In a case where the component absence result indicates a target local component absence, determining absence component information corresponding to the target local component;

[0065] Based on the absence component information corresponding to the target local component, generating warning information for the target local component.

[0066] In a possible implementation, when the detection module, based on preset relative position information between a preset local component required by the catenary and a corresponding preset structured component, detects the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, is configured to:

[0067] Based on preset relative position information between a preset local component and a corresponding preset structured component, detecting the to-be-processed image to obtain a target feature map corresponding to the to-be-processed image and initial position information of a target local component predicted to exist on the to-be-processed image;

[0068] Based on the initial position information and the target feature map, obtaining absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0069] In a possible implementation, when the detection module, based on the initial position information and the target feature map, obtains absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, is configured to:

[0070] Based on the initial position information, intercepting a local feature map corresponding to the target local component from the target feature map;

[0071] Performing at least one first feature extraction on the local feature map to obtain a first target local feature map;

[0072] Based on the first target local feature map, adjusting the initial position information to obtain absolute position information of a target local component predicted to exist in the to-be-processed image; and

[0073] Based on the first target local feature map, determining a first category of the target local component.

[0074] In a possible implementation, the detection module is further configured to: based on preset relative position information between a preset local component and a corresponding preset structured component, detect the to-be-processed image to determine relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs, and a second category corresponding to the target structured component.

[0075] The warning module is further configured to: in a case where it is detected that the component absence result indicates absence of the target local component, based on at least one of the relative position information between the target local component and the target structured component to which the target local component belongs, absolute position information of the target local component, the first category of the target local component, and the second category of the target structured component, determine absence component information corresponding to the target local component.

[0076] In a possible implementation, when the detection module detects the to-be-processed image based on preset relative position information between a preset local component and a corresponding preset structured component to determine relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs, and a second category corresponding to the target structured component, the detection module is configured to:

[0077] detect the to-be-processed image based on preset relative position information between a preset local component and a corresponding preset structured component to determine initial relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs.

[0078] based on the initial position information, intercept a local feature map corresponding to the target local component from the target feature map;

[0079] perform at least one second feature extraction on the local feature map to obtain a second target local feature map;

[0080] based on the second target local feature map, adjust the initial relative position information to obtain relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs; and

[0081] based on the second target local feature map, determine a second category of the target structured component.

[0082] In a possible implementation, after the to-be-processed image corresponding to the catenary is acquired, the apparatus further includes an adjustment module configured to:

[0083] adjust brightness of the to-be-processed image to generate an adjusted to-be-processed image;

[0084] The detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0085] The detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0086] In a possible implementation, when the to-be-processed image corresponds to a target line identifier of the catenary, the detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0087] The detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0088] The detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0089] In a possible implementation, the absolute position information and the first category of the target local component are determined based on a trained target neural network; and the device further includes a training module configured to train the target neural network according to the following steps:

[0090] The detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0091] The detection module is configured to, based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, detect the to-be-processed image to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0092] Based on the labeled data and the predicted data, the neural network to be trained is trained until the trained neural network converges, and the target neural network is obtained.

[0093] In a third aspect, the present disclosure provides an electronic device, comprising a processor, a memory, and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating through the bus when the electronic device is running, and the machine readable instructions being executed by the processor to perform the steps of the component missing detection method according to the first aspect or any of the embodiments.

[0094] In a fourth aspect, the present disclosure provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to perform the steps of the component missing detection method according to the first aspect or any of the embodiments.

[0095] In order to make the above objectives, features and advantages of the present disclosure more apparent, the following will specifically describe preferred embodiments in conjunction with the accompanying drawings, and a detailed description is made as follows. BRIEF DESCRIPTION OF DRAWINGS

[0096] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments, the drawings herein are incorporated into the description and form a part of the description, the drawings show the embodiments consistent with the present disclosure, and are used to illustrate the technical solutions of the present disclosure together with the description. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0097] Figure 1 A flowchart of a component missing detection method provided by an embodiment of the present disclosure is shown;

[0098] Figure 2 A schematic diagram of a detection frame of a preset local component and a preset structured component in a component missing detection method provided by an embodiment of the present disclosure is shown;

[0099] Figure 3 A flowchart of another component missing detection method provided by an embodiment of the present disclosure is shown;

[0100] Figure 4 An architectural schematic diagram of a component missing detection device provided by an embodiment of the present disclosure is shown;

[0101] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. Detailed Implementation

[0102] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0103] Overhead contact line inspection and maintenance has become an important part of high-speed rail operation and maintenance. This task includes inspecting the overhead contact line for missing components (such as bolts and nuts). Typically, inspectors with prior knowledge of the system check captured images of the overhead contact line to determine if any parts are missing. However, this manual inspection method is time-consuming, inefficient, and prone to omissions.

[0104] To improve the efficiency and accuracy of detecting missing parts in overhead contact lines, this disclosure provides a method, apparatus, electronic device, and storage medium for detecting missing parts.

[0105] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0106] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0107] To make the present disclosure embodiments be easily understood, firstly, a component missing detection method disclosed by the present disclosure embodiments is introduced in detail. An execution subject of the component missing detection method provided by the present disclosure embodiments is generally a computer device with certain computing capability, which for example includes a terminal device or a server or other processing device. The server for example can include a cloud server, a local server; the terminal device for example can include a user equipment (User Equipment, UE), a mobile device, a computing device, etc. In some possible implementation manners, the component missing detection method can be realized by a processor calling computer readable instructions stored in a memory.

[0108] Referring to Figure 1 Fig. 1 is a flowchart of a component missing detection method provided by the present disclosure embodiments, and the method includes S101-S103, where:

[0109] S101, acquiring a to-be-processed image corresponding to a catenary;

[0110] S102, detecting the to-be-processed image based on preset relative position information between a preset local component required by the catenary and a corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component; where the target local component belongs to a sub-component of a target structured component included in the to-be-processed image;

[0111] S103, determining a component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component and the first category.

[0112] Since there is a structural association relationship between the structured components on the catenary and the included local components, the to-be-processed image acquired can be detected based on preset relative position information between a preset structured component and a preset local component included in the preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component; where the target local component belongs to a sub-component of a target structured component included in the to-be-processed image, to realize more accurate and efficient reasoning of the absolute position information of the target local component predicted to exist on the target structured component and the first category of the target local component by using the target structured component included in the to-be-processed image; and then the component missing result corresponding to the to-be-processed image can be determined based on the to-be-processed image, the absolute position information of the target local component and the first category, to realize automatic detection of component missing, improve the efficiency and accuracy of component missing detection, and reduce the missed detection rate.

[0113] The following specifically describes S101-S103.

[0114] For S101:

[0115] For example, the high-definition imaging device arranged on the roof of the inspection vehicle can be used to collect the corresponding image of the catenary to be processed, and then the subject can obtain the corresponding image of the catenary to be processed. Specifically, the corresponding image of the catenary to be processed can be provided by the 4C inspection system. The system relies on the inspection vehicle to inspect the high-speed railway line. When the sensing device on the vehicle detects the high-speed railway line column, it triggers multiple groups of high-definition imaging devices installed in different directions on the roof of the inspection vehicle to shoot and collect high-speed railway line pictures (i.e. catenary pictures), thereby ensuring that no pictures are missed while avoiding dead angles as much as possible.

[0116] In implementation, after the high-definition imaging device collects the image corresponding to the catenary, the image corresponding to the catenary with low resolution can be screened out, and the screened image can be used as the image corresponding to the catenary to be processed.

[0117] In the process of collecting the image corresponding to the catenary to be processed, the target line identifier corresponding to the catenary can be determined, and the target line identifier can be associated with the image corresponding to the catenary to be processed. The target line identifier corresponding to the catenary can be set according to actual conditions. For example, the target line identifier can be: Jing-Guang railway S11 section, etc.; or the line identifier can be: JGS11, etc.

[0118] Generally, in order to avoid affecting the normal operation of the railway, the inspection vehicle will generally be inspected at night and the image corresponding to the catenary to be processed will be collected. Due to poor light at night, the image to be processed may have insufficient exposure. Based on this, in the embodiment of the present disclosure, after obtaining the image corresponding to the catenary to be processed, the method further comprises: adjusting the brightness of the image to be processed to generate an adjusted image to be processed. For example, the brightness of the image to be processed can be adjusted using gamma brightness enhancement, histogram equalization, etc. to generate an adjusted image to be processed. Then in S102, the adjusted image to be processed can be detected based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, to obtain the absolute position information of the target local component predicted to exist in the adjusted image to be processed and the first category of the target local component.

[0119] In order to alleviate the problem of insufficient exposure of the image to be processed, the brightness of the image to be processed can be adjusted to generate an adjusted image to be processed, so that subsequent target detection can be performed on the adjusted image to be processed, and the component detection accuracy can be improved.

[0120] For S102:

[0121] The preset local components and corresponding preset structured components required by the contact network can be each structured component required by the normal operation of the contact network and the local components included in each structured component. The preset structured components and the preset local components can be set according to actual conditions, and generally, the preset local components are sub-components of the preset structured components. For example, the preset local components can include at least one of a bolt, a nut, and a cotter pin; the preset structured components can include at least one of a column top cover plate, a column top base, an arm wrist base, a pressing plate fitting, a sleeve single ear, and a sleeve double ear. Alternatively, the preset local components can be a sleeve single ear, and the structured component can be a positioning ring; the preset local components can be an insulator iron mold pressing plate, and the structured component can be an insulator fitting, and the like.

[0122] Since the relative positional relationship between the preset local components and the corresponding preset structured components on the contact network is determined, for example, for the preset structured component: the column top cover plate, the nut exists on the column top cover plate, and the relative positional relationship between the nut and the column top cover plate is determined. Therefore, for each preset structured component on the contact network, the preset relative positional information between the preset local components included in each preset structured component and the preset structured component can be extracted, and the preset relative positional information between the preset local components required by the contact network and the corresponding preset structured components can be obtained.

[0123] The preset relative positional information can include the distance between each edge of the first detection frame of the preset local component and the corresponding edge of the second detection frame of the preset structured component, and the ratio between the size of the second detection frame. Figure 2 It can be seen that the first detection frame 21 of the preset local component and the second detection frame 22 of the preset structured component are included in the figure, the distance between the upper side of the first detection frame 22 and the upper side of the second detection frame 22 is y1, the distance between the lower side of the first detection frame 22 and the lower side of the second detection frame 22 is y2, the distance between the left side of the first detection frame 22 and the left side of the second detection frame 22 is x1, and the distance between the right side of the first detection frame 22 and the right side of the second detection frame 22 is x2; therefore, the preset relative positional information can include [x1 / w, y1 / h, x2 / w, y2 / h].

[0124] For example, according to target detection on the to-be-processed image, the target structured component included in the to-be-processed image can be determined, and then according to the preset relative positional information between the preset local components included in the preset structured component and the preset structured component, the absolute positional information of the target local component that should exist on the target structured component included in the to-be-processed image can be determined, and the first category of the target local component can be determined.

[0125] Or, a sample image associated with preset relative position information between a preset local component and a preset structured component to which the preset local component belongs can be obtained; the target neural network is trained by using the sample image; and then the trained target neural network is used to detect the to-be-processed image, and absolute position information of a target local component that should exist in the to-be-processed image and a first category of the target local component are determined according to at least part of feature information of the detected target structured component.

[0126] Since the target neural network is trained by using the preset relative position information, the target neural network can determine the absolute position information of the target local component and the first category according to at least part of the feature information of the detected target structured component and the preset relative position information.

[0127] The target local component that should exist in the to-be-processed image is a local component required for normal operation of the overhead contact system, but the target local component that should exist in the to-be-processed image may exist or may not exist.

[0128] When the target local component does not exist in the to-be-processed image (that is, the target local component is missing in the to-be-processed image), the absolute position information of the target local component that should exist in the to-be-processed image and the first category can also be obtained according to at least part of the feature information of the target structured component and the preset relative position information. The absolute position information of the target local component can include coordinate information of a center point of a detection frame of the target local component, a size of the detection frame, coordinate information of four vertices of the detection frame, and the like.

[0129] In a high-speed rail line, the structure and size of the same target structured component on different line sections can be different; for example, the target structured component one included in a first line section can include a split pin, and the target structured component two included in a second line section can include a bolt; or the target local component on the first line section is located at the upper right corner of the target structured component, and the target local component on the second line section is located at the upper left corner of the target structured component, that is, the relative position information between the target local component and the target structured component to which the target local component belongs is different on the first line section and the second line section.

[0130] Based on this, in an optional implementation, when the to-be-processed image corresponds to a target line identifier of an overhead contact system, the to-be-processed image is detected based on preset relative position information between a preset local component required by the overhead contact system and a corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, which can include:

[0131] Step A1, determining target relative position information matched with the target line identification from the preset relative position information between the preset local component and the corresponding preset structured component corresponding to different line identifications required by the catenary;

[0132] Step A2, detecting the to-be-processed image based on the target relative position information between the preset local component and the corresponding preset structured component matched with the target line identification, to obtain absolute position information of the target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0133] In implementation, for each line identification, preset relative position information between a preset local component and a preset structured component to which the preset local component belongs can be determined, to obtain preset relative position relationships between preset local components and preset structured components to which the preset local components belong corresponding to different line identifications. Then, target relative position relationships matched with a target line identification can be determined according to the target line identification associated with the to-be-processed image.

[0134] For example, the line identifications include line identification 1, line identification 2, and line identification 3. The line identification 1 corresponds to preset relative position information 1 between a preset local component and a preset structured component 1 to which the preset local component belongs, and preset relative position information 2 between the preset local component and a preset structured component 2 to which the preset local component belongs. The line identification 2 corresponds to preset relative position information 3 between a preset local component and a preset structured component 1 to which the preset local component belongs, and preset relative position information 2 between the preset local component and a preset structured component 2 to which the preset local component belongs. The line identification 3 corresponds to preset relative position information 4 between a preset local component and a preset structured component 1 to which the preset local component belongs, and preset relative position information 5 between the preset local component and a preset structured component 2 to which the preset local component belongs.

[0135] When the target line identification is the line identification 3, the preset relative position information 4 between the preset local component and the preset structured component 1 to which the preset local component belongs, and the preset relative position information 5 between the preset local component and the preset structured component 2 to which the preset local component belongs, can be determined as target relative position information matched with the target line identification.

[0136] Then, the to-be-processed image can be detected based on the target relative position relationships matched with the target line identification, to obtain absolute position information of the target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0137] In view of the fact that the structural correlation between the structured component and the included local component can be different on different line sections, in order to more accurately detect the local component of the catenary on each line section, a line identifier can be set, different preset relative position information is set for different line identifiers. Then, according to the target line identifier of the catenary corresponding to the to-be-processed image, the target relative position information matched with the target line identifier is determined, and based on the target relative position information, the to-be-processed image is detected to more accurately predict the absolute position information of the target local component and the first category.

[0138] In an optional implementation, the detection of the to-be-processed image based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component to obtain the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component can include:

[0139] Step B1, detecting the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component to obtain the target feature map corresponding to the to-be-processed image and the initial position information of the target local component predicted to exist in the to-be-processed image;

[0140] Step B2, obtaining the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component based on the initial position information and the target feature map.

[0141] Here, the to-be-processed image can be first detected based on the preset relative position information between the preset local component and the corresponding preset structured component, such as initial positioning detection, to obtain the target feature map corresponding to the to-be-processed image and the initial position information of the target local component predicted to exist in the to-be-processed image. Then, repositioning detection is performed based on the initial position information and the target feature map to obtain the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component. Through multiple detection processes, the generated absolute position information is more accurate.

[0142] In the embodiments of the present disclosure, the target neural network can be trained based on the training sample associated with the preset relative position information; the target neural network is used to detect the to-be-processed image to generate the target feature map corresponding to the to-be-processed image and obtain the initial position information of the target local component predicted to exist in the to-be-processed image. The initial position information can include the coordinate information of the sample frame of the target local component on the target feature map. The target feature map can be a feature map generated after the target neural network performs multiple feature extractions on the to-be-processed image.

[0143] The initial position information and the target feature map can be used to obtain absolute position information of a target local component predicted to exist in the image to be processed and a first category of the target local component.

[0144] In an optional implementation, in step B2, the absolute position information of the target local component predicted to exist in the image to be processed and the first category of the target local component can be obtained based on the initial position information and the target feature map.

[0145] In step B21, a local feature map corresponding to the target local component is intercepted from the target feature map based on the initial position information.

[0146] In step B22, at least one first feature extraction is performed on the local feature map to obtain a first target local feature map.

[0147] In step B23, the initial position information is adjusted based on the first target local feature map to obtain the absolute position information of the target local component predicted to exist in the image to be processed, and the first category of the target local component is determined based on the first target local feature map.

[0148] In implementation, the local feature map corresponding to the target local component can be intercepted from the target feature map based on the initial position information. Then, one or more first feature extractions can be performed on the local feature map by using a target neural network to obtain a first target local feature map. Then, the initial position information is adjusted based on the first target local feature map to obtain the absolute position information of the target local component predicted to exist in the image to be processed, and the first category of the target local component is determined based on the first target local feature map.

[0149] Here, the local feature map corresponding to the target local component is intercepted from the target feature map based on the initial position information, and the feature information of the target local component accounts for a large proportion in the local feature map. Then, at least one first feature extraction is performed on the local feature map to obtain a first target local feature map. Based on the first target local feature map, the absolute position information and the first category of the target local component can be obtained more accurately.

[0150] In an optional implementation, after the image to be processed corresponding to the overhead contact system is obtained, the method further includes: based on preset relative position information between a preset local component and a corresponding preset structured component, detecting the image to be processed to determine relative position information between the target local component predicted to exist in the image to be processed and a target structured component to which the target local component belongs, and a second category of the target structured component.

[0151] In implementation, after obtaining the to-be-processed image corresponding to the catenary, the to-be-processed image can be detected according to the preset relative position information between the preset local component and the corresponding preset structured component, to determine the relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs, and the second category corresponding to the target structured component.

[0152] In a possible implementation, the detection of the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component, to determine the relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs, and the second category corresponding to the target structured component can include:

[0153] Step 1, detecting the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component, to determine the initial relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs;

[0154] Step 2, based on the initial position information, intercepting a local feature map corresponding to the target local component from the target feature map;

[0155] Step 3, performing at least one second feature extraction on the local feature map to obtain a second target local feature map;

[0156] Step 4, based on the second target local feature map, adjusting the initial relative position information to obtain the relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs, and determining the second category of the target structured component based on the second target local feature map.

[0157] In implementation, a backbone network and two network detection branches connected to the backbone network can be set in the target neural network. The to-be-processed image is input into the target neural network, the backbone network performs feature extraction on the to-be-processed image to generate an intermediate feature map. Then the intermediate feature map is input into the two network detection branches respectively.

[0158] The first detection branch can detect the intermediate feature map to obtain a target feature map corresponding to the to-be-processed image and initial position information of a target local component predicted to exist on the to-be-processed image; and the second detection branch can detect the intermediate feature map to determine the initial relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs.

[0159] The initial relative position information can be: [x 11w1, y 11 h1, x 21 w1, y 21 h1, x 11 is a first distance from a left side of the target local component bounding box to a left side of the target structured component bounding box; y 11 is a second distance from an upper side of the target local component bounding box to an upper side of the target structured component bounding box; x 21 is a third distance from a right side of the target local component bounding box to a right side of the target structured component bounding box; y 21 is a fourth distance from a lower side of the target local component bounding box to a lower side of the target structured component bounding box; w1 is a width of the target structured component bounding box; and h1 is a height of the target structured component bounding box.

[0160] The local feature map corresponding to the target local component can also be intercepted from the target feature map based on the initial position information of the target local component.

[0161] The first target local feature map is obtained by performing at least one first feature extraction on the local feature map by using a first detection branch in the target neural network; the initial position information is adjusted by using the first target local feature map to obtain absolute position information of the target local component predicted to exist in the to-be-processed image; and the first category of the target local component is determined based on the first target local feature map.

[0162] The second target local feature map is obtained by performing at least one second feature extraction on the local feature map by using a second detection branch in the target neural network; the initial relative position information is adjusted by using the second target local feature map to obtain relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs; and the second category of the target structured component is determined based on the second target local feature map.

[0163] Further, the method further includes: in a case where it is detected that the component absence result indicates that the target local component is absent, determining, based on at least one of the relative position information between the target local component and the target structured component to which the target local component belongs, the absolute position information of the target local component, the first category of the target local component, and the second category of the target structured component, missing component information corresponding to the target local component.

[0164] The relative position information, the absolute position information, the first category, and the second category can be determined as the missing component information corresponding to the target local component. The missing component information can be selected as needed, for example, the missing component information can further include a target line identifier.

[0165] Here, the content of the missing part information is rich, and subsequent to the target local part corresponding to the missing part information, the warning information for the target local part is generated, so that the generated warning information can accurately report the missing target local part, and the maintenance personnel can maintain the missing target local part.

[0166] For S103:

[0167] In implementation, according to the absolute position information of the target local part, it can be determined whether there is an object corresponding to the first category at the position in the to-be-processed image matching the absolute position information; if there is, it is determined that the component missing result corresponding to the to-be-processed image is not missing; if there is not, it is determined that the component missing result corresponding to the to-be-processed image is missing.

[0168] In an optional implementation, the determination of the component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local part, and the first category comprises:

[0169] Step C1, based on the absolute position information of the target local part, the local image corresponding to the target local part is intercepted from the to-be-processed image;

[0170] Step C2, based on the first category of the target local part, target detection is performed on the local image to determine whether the target local part exists in the local image;

[0171] Step C3, if it is detected that the target local part does not exist in any local image, it is determined that the component missing result corresponding to the to-be-processed image is that the target local part corresponding to the any local image is missing.

[0172] According to the absolute position information of the target local part, the local image corresponding to the target local part can be intercepted from the to-be-processed image. Then, based on the first category of the target local part, the component detection neural network corresponding to the target local part can be determined; and the target detection neural network corresponding to the target local part is used to perform target detection on the local image to determine whether the target local part exists in the local image, and obtain the detection result corresponding to the local image. For example, the first detection result that the local image exists the target local part, or the second detection result that the local image does not exist the target local part.

[0173] When the preset local component includes an eye bolt, a bolt, or a nut, the training can obtain: a component detection neural network corresponding to the eye bolt, which is used to detect whether the eye bolt exists in the image; a component detection neural network corresponding to the bolt, which is used to detect whether the bolt exists in the image; and a component detection neural network corresponding to the nut, which is used to detect whether the nut exists in the image. For example, if the first category of the target local component is the eye bolt, the component detection neural network corresponding to the eye bolt can be selected to perform target detection on the local image, to determine whether the target local component exists in the local image, and to obtain a detection result corresponding to the local image.

[0174] When the number of local images is multiple, if the target local component exists in each local image, it is determined that the component missing result corresponding to the to-be-processed image is that there is no component missing. If it is detected that any local image does not have the target local component, it is determined that the component missing result corresponding to the to-be-processed image is that the target local component of any local image is missing. For example, the component missing result can be that the target local component is missing.

[0175] In the above embodiment, by using the absolute position information of the target local component, the local image corresponding to the target local component is intercepted from the to-be-processed image, and other image information except the local image is filtered out, so that the detection of the target local component is not interfered by other image information, and the detection accuracy of the target local component in the target detection of the local image is improved. Furthermore, the component missing result corresponding to the to-be-processed image can be accurately determined according to the detection result of the target local component of each local image, and the accuracy of the component missing detection is improved.

[0176] In an optional embodiment, the method further includes: in the case where the component missing result indicates that the target local component is missing, determining missing component information corresponding to the target local component; and generating warning information for the target local component based on the missing component information corresponding to the target local component.

[0177] In implementation, when the component missing result indicates that the target local component is missing, the missing component information corresponding to the target local component can be determined. For example, the missing component information can include absolute position information of the target local component, the first category of the target local component, the second category of the target structured component to which the target local component belongs, relative position information between the target local component and the target structured component to which the target local component belongs, a target line identifier of the overhead contact system in which the target local component is located, and the like.

[0178] Further, the missing component information corresponding to the target local component can be used to generate warning information for the target local component. For example, the warning information can be "missing nut on JGS11-column top cover plate".

[0179] Here, when it is detected that the target local component is missing in the image to be processed, the missing component information corresponding to the target local component can be determined, and the warning information for the target local component can be generated according to the missing component information. The content of the warning information is rich and flexible, so that the management personnel can accurately maintain the overhead contact line according to the warning information, and the operation safety of the overhead contact line is improved.

[0180] In an optional implementation, the absolute position information of the target local component and the first category are determined based on a trained target neural network; and the target neural network is trained according to the following steps:

[0181] In step D1, a sample image corresponding to the overhead contact line is obtained, and the sample image includes annotation data. The annotation data includes sample absolute position information of at least one local component, a first sample category corresponding to the local component, sample relative position information between each local component and a structured component to which the local component belongs, and a second sample category corresponding to the structured component.

[0182] In step D2, the sample image is input into a neural network to be trained to obtain prediction data corresponding to the sample image, wherein the prediction data is data matched with the annotation data.

[0183] In step D3, the neural network to be trained is trained based on the annotation data and the prediction data until the trained neural network converges, and the target neural network is obtained.

[0184] In implementation, the sample image can be input into the neural network to be trained to obtain prediction data corresponding to the sample image, wherein the data types included in the prediction data are consistent with the annotation data. That is, the prediction data can include predicted absolute position information of at least one local component, a first predicted category corresponding to the local component, predicted relative position information between each local component and a structured component to which the local component belongs, and a second predicted category corresponding to the structured component.

[0185] Further, the loss value of the neural network to be trained can be determined according to the annotation data and the prediction data, the network parameters of the neural network to be trained can be adjusted according to the loss value, and the target neural network is obtained until the trained neural network (i.e., the neural network after parameter adjustment) converges.

[0186] Traditional neural networks typically detect objects in images using WYSIWYG visual features. In other words, they detect objects within the image. The target neural network proposed in this disclosure, however, after detecting structured components, uses the structural relationships between the structured components and their constituent local components to infer the location and category of the local components that should be present within the structured components. It then performs object detection on the image. If the image contains the local components that should be present, then the local components are not missing; if the image does not contain the local components that should be present, then the local components are missing. This achieves automatic detection of missing components in images. Applying neural networks to missing component detection scenarios improves the efficiency and accuracy of missing component detection.

[0187] Here, the labeled data may include the absolute position information of at least one local component, the first sample category corresponding to the local component, the relative position information of each local component and its corresponding structured component, and the second sample category corresponding to the structured component. The labeled data is rich in content, and the relative position information and the second sample category are used as supervision signals for the neural network to be trained. After obtaining the prediction data, the neural network to be trained can be trained more accurately based on the labeled data and the prediction data, thereby improving the performance of the target neural network.

[0188] For example, in combination Figure 3 The method for detecting missing components is described in detail. This method may include:

[0189] S301, Obtain the image to be processed corresponding to the overhead contact line.

[0190] S302, adjust the brightness of the image to be processed to generate the adjusted image to be processed.

[0191] S303: Using the trained target neural network, perform the first-path detection on the image to be processed to obtain the target feature map corresponding to the image to be processed, as well as the initial position information of the predicted target local parts on the image to be processed.

[0192] S304. Using the trained target neural network, a second-path detection is performed on the image to be processed to obtain the initial relative position information between the predicted local target components and their respective structured components in the image to be processed.

[0193] S305, based on the initial position information, extract the local feature map corresponding to the local component of the target from the target feature map.

[0194] S306, one, performing at least one first feature extraction on the local feature map to obtain a first target local feature map; adjusting the initial position information based on the first target local feature map to obtain absolute position information of the target local component predicted to exist in the to-be-processed image; and determining a first category of the target local component based on the first target local feature map; two, performing at least one second feature extraction on the local feature map to obtain a second target local feature map; adjusting the initial relative position information based on the second target local feature map to obtain relative position information between the target local component predicted to exist in the to-be-processed image and the target structured component to which the target local component belongs; and determining a second category of the target structured component based on the second target local feature map.

[0195] S307, based on the absolute position information of the target local component, cutting a local image corresponding to the target local component from the to-be-processed image.

[0196] S308, based on the first category of the target local component, performing target detection on the local image to determine whether the target local component exists in the local image.

[0197] S309, if it is detected that the target local component does not exist in any local image, determining that the component missing result corresponding to the to-be-processed image is that the target local component corresponding to any local image is missing; if it is detected that the target local component exists in each local image, determining that the component missing result corresponding to the to-be-processed image is that there is no component missing.

[0198] S310, in the case where the component missing result indicates that the target local component is missing, determining missing component information corresponding to the target local component; and generating warning information for the target local component based on the missing component information corresponding to the target local component.

[0199] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0200] Based on the same concept, the embodiment of the disclosure also provides a component missing detection device, as shown in Figure 4 The architecture schematic diagram of the component missing detection device provided by the embodiment of the disclosure includes an acquisition module 401, a detection module 402, and a determination module 403, and specifically:

[0201] The acquisition module 401 is configured to acquire a to-be-processed image corresponding to a catenary;

[0202] The detection module 402 is configured to detect the to-be-processed image based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component; the target local component belongs to a sub-component of a target structured component included in the to-be-processed image.

[0203] The determination module 403 is configured to determine a component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component, and the first category.

[0204] In a possible implementation, when determining the component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component, and the first category, the determination module 403 is configured to:

[0205] cut a local image corresponding to the target local component from the to-be-processed image based on the absolute position information of the target local component;

[0206] perform target detection on the local image based on the first category of the target local component, to determine whether the target local component exists in the local image;

[0207] if it is detected that the target local component does not exist in any local image, determine that the component missing result corresponding to the to-be-processed image is that the target local component corresponding to the any local image is missing.

[0208] In a possible implementation, the apparatus further includes an alert module 404, configured to:

[0209] in a case where the component missing result indicates that the target local component is missing, determine missing component information corresponding to the target local component;

[0210] generate alert information for the target local component based on the missing component information corresponding to the target local component.

[0211] In a possible implementation, when detecting the to-be-processed image based on the preset relative position information between the preset local component required by the catenary and the corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, the detection module 402 is configured to:

[0212] detect the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component, to obtain a target feature map corresponding to the to-be-processed image and initial position information of a target local component predicted to exist on the to-be-processed image;

[0213] obtain absolute position information of the target local component predicted to exist in the to-be-processed image and a first category of the target local component based on the initial position information and the target feature map.

[0214] In a possible implementation, when the detection module 402 obtains the absolute position information of the target local component predicted to exist in the to-be-processed image and the first category of the target local component based on the initial position information and the target feature map, the detection module 402 is configured to:

[0215] cut a local feature map corresponding to the target local component from the target feature map based on the initial position information;

[0216] perform at least one first feature extraction on the local feature map to obtain a first target local feature map;

[0217] adjust the initial position information based on the first target local feature map to obtain the absolute position information of the target local component predicted to exist in the to-be-processed image; and

[0218] determine the first category of the target local component based on the first target local feature map.

[0219] In a possible implementation, the detection module 402 is further configured to detect the to-be-processed image based on the preset relative position information between the preset local component and the corresponding preset structured component, to determine relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs, and a second category of the target structured component;

[0220] The warning module 404 is further configured to, in a case where it is detected that the component absence result indicates absence of the target local component, determine absence component information corresponding to the target local component based on at least one of the relative position information between the target local component and the target structured component to which the target local component belongs, the absolute position information of the target local component, the first category of the target local component, and the second category of the target structured component.

[0221] In a possible implementation, the detection module 402, when detecting the to-be-processed image based on preset relative position information between a preset local component and a corresponding preset structured component, determining initial relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs, and a second category corresponding to the target structured component, is configured to:

[0222] detecting the to-be-processed image based on preset relative position information between a preset local component and a corresponding preset structured component, to determine initial relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs;

[0223] cutting, based on the initial position information, a local feature map corresponding to the target local component from the target feature map;

[0224] performing at least one second feature extraction on the local feature map to obtain a second target local feature map;

[0225] adjusting, based on the second target local feature map, the initial relative position information to obtain relative position information between the target local component predicted to exist in the to-be-processed image and a target structured component to which the target local component belongs; and

[0226] determining, based on the second target local feature map, a second category of the target structured component.

[0227] In a possible implementation, after obtaining the to-be-processed image corresponding to the overhead contact system, the apparatus further includes an adjustment module 405, configured to:

[0228] adjusting brightness of the to-be-processed image to generate an adjusted to-be-processed image;

[0229] The detection module 402, when detecting the to-be-processed image based on preset relative position information between a preset local component required by the overhead contact system and a corresponding preset structured component, obtaining absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, is configured to:

[0230] detecting the adjusted to-be-processed image based on preset relative position information between a preset local component required by the overhead contact system and a corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the adjusted to-be-processed image and a first category of the target local component.

[0231] In a possible implementation, in a case where the to-be-processed image corresponds to a target line identification of the catenary, when the detection module 402 detects the to-be-processed image based on preset relative position information between preset local components and corresponding preset structured components required by the catenary, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component, the apparatus is configured to:

[0232] determine target relative position information matched with the target line identification from preset relative position information between preset local components and corresponding preset structured components respectively corresponding to different line identifications required by the catenary;

[0233] detect the to-be-processed image based on target relative position information between preset local components and corresponding preset structured components matched with the target line identification, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component.

[0234] In a possible implementation, the absolute position information and the first category of the target local component are determined based on a trained target neural network; and the apparatus further includes a training module 406 configured to train the target neural network according to the following steps:

[0235] obtain a sample image corresponding to the catenary and containing annotation data, the annotation data including sample absolute position information of at least one local component, a first sample category corresponding to the local component, sample relative position information between each local component and a structured component to which the local component belongs, and a second sample category corresponding to the structured component;

[0236] input the sample image into a neural network to be trained, to obtain prediction data corresponding to the sample image, wherein the prediction data is data matched with the annotation data;

[0237] train the neural network to be trained based on the annotation data and the prediction data, until the trained neural network converges, to obtain the target neural network.

[0238] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes templates that can be used to perform the methods described in the above method embodiments, and specific implementations can refer to the descriptions of the above method embodiments. For brevity, they will not be described here.

[0239] Based on the same technical concept, the embodiments of the present disclosure further provide an electronic device. Refer to Figure 5As shown, a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is provided, including a processor 501, a memory 502, and a bus 503. The memory 502 is used to store execution instructions, including an internal memory 5021 and an external memory 5022; the internal memory 5021 is also called an internal memory, used to temporarily store operation data in the processor 501 and data exchanged with the external memory 5022 such as a hard disk, the processor 501 exchanges data with the external memory 5022 through the internal memory 5021, and when the electronic device 500 is running, the processor 501 and the memory 502 communicate through the bus 503, so that the processor 501 executes the following instructions:

[0240] obtaining a to-be-processed image corresponding to the catenary;

[0241] detecting the to-be-processed image based on preset relative position information between a preset local component required by the catenary and a corresponding preset structured component, to obtain absolute position information of a target local component predicted to exist in the to-be-processed image and a first category of the target local component; wherein the target local component belongs to a sub-component of a target structured component included in the to-be-processed image;

[0242] determining a component missing result corresponding to the to-be-processed image based on the to-be-processed image, the absolute position information of the target local component, and the first category.

[0243] The specific processing procedure of the processor 501 can refer to the description of the above method embodiments, which will not be described here.

[0244] In addition, an embodiment of the present disclosure further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, the steps of the component missing detection method described in the above method embodiments are executed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0245] An embodiment of the present disclosure further provides a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the component missing detection method described in the above method embodiments. For details, refer to the above method embodiments, which will not be described here.

[0246] The above computer program product can be specifically implemented by hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied as a computer storage medium, and in another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc.

[0247] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here. In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0248] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0249] In addition, the functional units in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0250] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present disclosure. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0251] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for detecting missing components, characterized in that, include: Obtain the image to be processed corresponding to the overhead contact line; Based on the preset relative position information between the preset local components and corresponding preset structured components required by the contact network, the image to be processed is detected to obtain the absolute position information of the predicted target local components in the image to be processed and the first category of the target local components; wherein, the target local component belongs to a sub-component of the target structured component included in the image to be processed; the preset local components and corresponding preset structured components required by the contact network are: each structured component that should exist for normal operation of the contact network and the local components included in each structured component; the relative positional relationship between the preset local components and their respective preset structured components on the contact network is determined; Based on the image to be processed, the absolute position information of the target local component, and the first category, determine the component missing result corresponding to the image to be processed; Based on the preset relative position information between the preset local components and corresponding preset structured components required by the contact network, the image to be processed is detected to obtain the absolute position information of the predicted target local components in the image to be processed and the first category of the target local components, including: Based on the preset relative position information between preset local components and corresponding preset structured components, the image to be processed is detected to obtain the target feature map corresponding to the image to be processed, and the initial position information of the predicted target local components on the image to be processed. Based on the initial location information and the target feature map, the absolute location information of the predicted target local parts in the image to be processed and the first category of the target local parts are obtained.

2. The method according to claim 1, characterized in that, The step of determining the missing component result corresponding to the image to be processed based on the image to be processed, the absolute position information of the target local component, and the first category includes: Based on the absolute position information of the target local component, a local image corresponding to the target local component is extracted from the image to be processed; Based on the first category of the target local component, target detection is performed on the local image to determine whether the target local component exists in the local image; If the target local component is not detected in any local image, then the component missing result corresponding to the image to be processed is determined to be the missing target local component corresponding to any local image.

3. The method according to claim 1, characterized in that, The method further includes: If the component missing result indicates that a target local component is missing, determine the missing component information corresponding to the target local component; Based on the missing component information corresponding to the target local component, a warning message is generated for the target local component.

4. The method according to claim 1, characterized in that, The step of obtaining the absolute location information of the predicted target local parts in the image to be processed and the first category of the target local parts based on the initial location information and the target feature map includes: Based on the initial position information, a local feature map corresponding to the local component of the target is extracted from the target feature map; Perform at least one first feature extraction on the local feature map to obtain a first target local feature map; Based on the first target local feature map, the initial position information is adjusted to obtain the absolute position information of the predicted target local components in the image to be processed; and Based on the first target local feature map, a first category of the target local component is determined.

5. The method according to claim 1, characterized in that, After acquiring the image to be processed corresponding to the overhead contact line, the method further includes: Based on the preset relative position information between preset local components and corresponding preset structured components, the image to be processed is detected to determine the relative position information between the predicted target local component and its corresponding target structured component in the image to be processed, and the second category corresponding to the target structured component. The method further includes: If the detection result indicates that the target local component is missing, the missing component information corresponding to the target local component is determined based on at least one of the following: the relative position information between the target local component and the target structured component to which it belongs, the absolute position information of the target local component, the first category of the target local component, and the second category of the target structured component.

6. The method according to claim 5, wherein detecting the image to be processed based on preset relative position information between preset local components and corresponding preset structured components, and determining the relative position information between the predicted target local component and its corresponding target structured component in the image to be processed, and the second category corresponding to the target structured component, comprises: Based on the preset relative position information between preset local components and corresponding preset structured components, the image to be processed is detected to determine the initial relative position information between the predicted target local component and its corresponding target structured component in the image to be processed. Based on the initial position information, a local feature map corresponding to the local component of the target is extracted from the target feature map; Perform at least one second feature extraction on the local feature map to obtain a second target local feature map; Based on the second target local feature map, the initial relative position information is adjusted to obtain the relative position information between the predicted target local component and its corresponding target structured component in the image to be processed; and Based on the second target local feature map, the second category of the target structured component is determined.

7. The method according to any one of claims 1 to 3, characterized in that, After acquiring the image to be processed corresponding to the overhead contact line, the method further includes: The brightness of the image to be processed is adjusted to generate an adjusted image to be processed; Based on the preset relative position information between the preset local components and corresponding preset structured components required by the contact network, the image to be processed is detected to obtain the absolute position information of the predicted target local components in the image to be processed and the first category of the target local components, including: Based on the preset relative position information between the preset local components and the corresponding preset structured components required by the overhead contact system, the adjusted image to be processed is detected to obtain the absolute position information of the predicted target local components and the first category of the target local components in the adjusted image to be processed.

8. The method according to any one of claims 1 to 3, characterized in that, When the image to be processed corresponds to the target line identifier of the overhead contact line, the image to be processed is detected based on the preset relative position information between the preset local components required by the overhead contact line and the corresponding preset structured components, to obtain the absolute position information of the predicted target local components in the image to be processed and the first category of the target local components, including: From the preset relative position information between preset local components and corresponding preset structured components corresponding to different line identifiers required by the overhead contact system, determine the target relative position information that matches the target line identifier; Based on the target relative position information between the preset local component and the corresponding preset structured component that matches the target line identifier, the image to be processed is detected to obtain the absolute position information of the predicted target local component in the image to be processed and the first category of the target local component.

9. The method according to any one of claims 1 to 3, characterized in that, The absolute position information and first category of the target local component are determined based on the trained target neural network; the target neural network is trained according to the following steps: Obtain sample images containing labeled data corresponding to the overhead contact line. The labeled data includes the absolute position information of at least one local component, the first sample category corresponding to the local component, the relative position information of each local component and its corresponding structured component, and the second sample category corresponding to the structured component. The sample image is input into the neural network to be trained to obtain the prediction data corresponding to the sample image, wherein the prediction data is the data that matches the labeled data; Based on the labeled data and the predicted data, the neural network to be trained is trained until the trained neural network converges, thus obtaining the target neural network.

10. A component missing detection device, characterized in that, include: The acquisition module is used to acquire the image to be processed corresponding to the overhead contact line; The detection module is used to detect the image to be processed based on the preset relative position information between preset local components and corresponding preset structured components required by the contact network, to obtain the absolute position information of the predicted target local components in the image to be processed and the first category of the target local components; wherein, the target local component belongs to a sub-component of the target structured component included in the image to be processed; the preset local components and corresponding preset structured components required by the contact network are: each structured component that should exist for normal operation of the contact network and the local components included in each structured component; the relative positional relationship between the preset local component and its corresponding preset structured component on the contact network is determined; The determining module is used to determine the missing component result corresponding to the image to be processed based on the image to be processed, the absolute position information of the target local component, and the first category; The detection module, based on the preset relative position information between the preset local components and corresponding preset structured components required by the contact network, detects the image to be processed to obtain the absolute position information of the predicted target local components and the first category of the target local components in the image to be processed, and is used for: Based on the preset relative position information between preset local components and corresponding preset structured components, the image to be processed is detected to obtain the target feature map corresponding to the image to be processed, and the initial position information of the predicted target local components on the image to be processed. Based on the initial location information and the target feature map, the absolute location information of the predicted target local parts in the image to be processed and the first category of the target local parts are obtained.

11. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the component missing detection method as described in any one of claims 1 to 9 are performed.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the component missing detection method as described in any one of claims 1 to 9.

Images