A detection method, device, apparatus and computer readable storage medium

Abstract

The application discloses a detection method, device and equipment and a computer readable storage medium. The detection method comprises the following steps: obtaining a to-be-detected image, a trained first detection model and a trained second detection model; the first detection model is used for global detection of the to-be-detected image; the second detection model is used for local detection of the to-be-detected image; the to-be-detected image is detected by using the trained first detection model to obtain a first detection result; a target region in the to-be-detected image is determined, and the target region is detected by using the trained second detection model to obtain a second detection result; and the first detection result and the second detection result are fused to obtain a target detection result corresponding to the to-be-detected image.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and includes, but is not limited to, a detection method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] With the continuous development of science and technology, high-speed rail has developed rapidly due to its advantages such as high passenger capacity, short travel time, good safety, high punctuality rate, energy saving and environmental protection, which has greatly promoted economic development and scientific research progress.

[0003] However, high-speed rail has high construction costs, high technical requirements, and strict construction standards, which also brings complexity and difficulty to management and maintenance. In particular, the overhead contact system of high-speed rail is a special type of power transmission line that is erected along the railway line to supply power to electric locomotives.

[0004] In practice, monitoring the condition of high-speed rail overhead contact lines requires high-precision imaging inspection of system components. This is often done manually or through a holistic inspection method. The large amount of data collected during a single inspection by a patrol vehicle leads to lengthy analysis times and poor real-time performance. Furthermore, the data collection environment is often not a laboratory or production line-specific setting, and the images are taken at night. Combined with the influence of shooting distance, angle, and obstructions, image quality is difficult to guarantee, increasing the difficulty of defect analysis. Therefore, images collected by a patrol vehicle often take months to analyze, and the analysis quality is inconsistent and efficiency is low. Summary of the Invention

[0005] In view of the above, embodiments of this application provide a detection method, apparatus, device, and computer-readable storage medium.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a detection method, including:

[0008] The system acquires an image to be detected, a trained first detection model, and a trained second detection model; the first detection model is used to perform global detection on the image to be detected; the second detection model is used to perform local detection on the image to be detected.

[0009] The trained first detection model is used to detect the image to be detected, and a first detection result is obtained;

[0010] The target region in the image to be detected is determined, and the trained second detection model is used to detect the target region to obtain a second detection result;

[0011] The first detection result and the second detection result are fused to obtain the target detection result corresponding to the image to be detected.

[0012] This application provides a detection device, including:

[0013] The acquisition module is used to acquire the image to be detected, a trained first detection model, and a trained second detection model; the first detection model is used to perform global detection on the image to be detected; the second detection model is used to perform local detection on the image to be detected.

[0014] The first detection module is used to detect the image to be detected using the trained first detection model to obtain a first detection result;

[0015] The second detection module is used to determine the target region in the image to be detected, and to detect the target region using the trained second detection model to obtain a second detection result;

[0016] The fusion module is used to fuse the first detection result and the second detection result to obtain the target detection result corresponding to the image to be detected.

[0017] This application provides a detection device, which includes at least:

[0018] Processor; and

[0019] Memory for storing computer programs that can run on the processor;

[0020] The computer program, when executed by the processor, implements the aforementioned detection method.

[0021] This application provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described detection method.

[0022] This application provides a detection method, apparatus, device, and computer-readable storage medium. The detection method includes: firstly, acquiring an image to be detected, a trained first detection model, and a trained second detection model, wherein the first detection model is capable of performing overall and global detection on the image to be detected; the second detection model is capable of performing partial and local detection on the image to be detected; next, using the trained first detection model to detect the image to be detected, thereby obtaining a first detection result representing the overall detection result; then, further determining a target region of the image to be detected, and using the trained second detection model to detect the target region, thereby obtaining a second detection result representing the partial detection result of the target region; finally, fusing the first detection result and the second detection result to obtain a target detection result that includes both the overall and regional detection results of the image to be detected. This allows for overall and local detection of the image to be detected through these two detection models, i.e., comprehensive detection of the image to be detected, thus improving detection accuracy. Furthermore, since two detection models are used, compared to a single overall detection model, these two detection models are less complex, thereby simplifying the detection process and improving real-time detection performance. Attached Figure Description

[0023] In the accompanying drawings (which are not necessarily drawn to scale), similar reference numerals may describe similar parts in different views. The drawings illustrate, by way of example and not limitation, the various embodiments discussed herein.

[0024] Figure 1 A schematic diagram of an implementation flow of the detection method provided in this application embodiment;

[0025] Figure 2 A schematic diagram of an implementation flow of the rapid detection method provided in this application embodiment;

[0026] Figure 3 A schematic diagram illustrating an implementation flow of the hierarchical alarm method provided in this application embodiment;

[0027] Figure 4 This is a schematic diagram illustrating an implementation process of the training and detection model provided in an embodiment of this application;

[0028] Figure 5 This is a schematic diagram illustrating an implementation process for determining target detection results provided in an embodiment of this application;

[0029] Figure 6 This is a schematic diagram illustrating an implementation process for classifying and displaying target detection results according to an embodiment of this application.

[0030] Figure 7 This is a schematic flowchart illustrating an implementation of the detection method provided in an embodiment of this application.

[0031] Figure 8 This is a schematic diagram illustrating the location of different types of defects provided in the embodiments of this application;

[0032] Figure 9 A schematic diagram illustrating another implementation process for displaying target detection results by classification, as provided in an embodiment of this application;

[0033] Figure 10A A schematic diagram of an image to be detected provided in an embodiment of this application;

[0034] Figure 10B A schematic diagram of a brightened image to be detected provided in an embodiment of this application;

[0035] Figure 10C Another schematic diagram of the brightened image to be detected provided in an embodiment of this application;

[0036] Figure 11 This is a schematic diagram of the module composition in the hierarchical early warning system provided in the embodiments of this application.

[0037] Figure 12 This is a schematic diagram of the composition of the detection device provided in the embodiments of this application;

[0038] Figure 13 This is a schematic diagram of another component structure of the detection device provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0041] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0043] To address the problems existing in related technologies, this application provides a detection method that can be applied to a detection device. The method provided in this embodiment can be implemented by a computer program, which, when executed, completes each step of the detection method. In some embodiments, the computer program can control the processor in the detection device to perform the execution. Figure 1 This is a schematic diagram illustrating an implementation flow of the detection method provided in an embodiment of this application, such as... Figure 1 As shown, the detection method includes:

[0044] Step S101: Obtain the image to be detected, the trained first detection model, and the trained second detection model.

[0045] Here, the image to be detected can be an image of the object to be detected acquired by an image acquisition device. In this embodiment, the image to be detected can be an image of a high-speed rail contact network. The image to be detected can also be an image of a machine tool, manufacturing equipment, etc. By detecting and analyzing the image to be detected, it is possible to determine whether the object to be detected has defects.

[0046] In this embodiment, the first detection model is used for global detection of the image to be detected, and the second detection model is used for local detection of the image to be detected. That is, when using the first detection model, detection is mainly performed on the entire image to be detected; when using the second detection model, detection is mainly performed on key areas of the image to be detected. Taking the overhead contact line as an example, these key areas can be the corresponding image areas such as the contact ring, dropper clamp, and current-carrying ring. The trained first detection model refers to the model obtained by training a preset first detection model using samples, and the trained second detection model refers to the model obtained by training a preset second detection model using samples.

[0047] In practice, the trained first detection model can be an artificial intelligence algorithm model such as a trained neural network model, a trained Bayesian network model, or a trained genetic algorithm model. This trained first detection model can determine the corresponding first detection result based on the image to be detected, thereby achieving the purpose of automatic and intelligent global detection. Similarly, the trained second detection model can also be an artificial intelligence algorithm model such as a trained neural network model, a trained Bayesian network model, or a trained genetic algorithm model. This trained second detection model can determine the corresponding second detection result based on the target region of the image to be detected, thereby achieving the purpose of automatic and intelligent region detection.

[0048] Step S102: Use the trained first detection model to detect the image to be detected and obtain the first detection result.

[0049] Here, the image to be detected is input into the trained first detection model, which performs global detection on the image and outputs the first detection result. For example, taking the trained first detection model as a neural network model, the image to be detected can be processed layer by layer through input layer neurons, hidden layer neurons, and output layer neurons, and finally the first detection result is output through the output layer.

[0050] Taking the overhead contact line as an example, suppose there are large foreign objects such as bird nests in the contact line, or the conduit is broken. If the image of the contact line is input into the trained first detection model, the trained first detection model can output the defects such as large foreign objects like bird nests, or output the defects such as broken conduits.

[0051] In some embodiments, since the defect type of large foreign objects such as bird nests is an unknown defect type, while the defect type of broken conduit is a known defect type, the trained first detection model may include a trained first sub-detection model and a trained second sub-detection model. The trained first sub-detection model is used to detect unknown defects, obtaining detection results for unknown defects; while the trained second sub-detection model is used to detect known defects, obtaining detection results for known defects. Finally, the detection results for unknown defects and the detection results for known defects are determined as the first detection result.

[0052] Step S103: Determine the target region in the image to be detected, and use the trained second detection model to detect the target region to obtain the second detection result.

[0053] Here, the target area of ​​the image to be detected can be a key area or an area that needs attention in the object to be detected. Taking the overhead contact line as an example, the target area can be the image area corresponding to the heart ring, dropper clamp, current-carrying ring, etc.

[0054] In some embodiments, the target region can be determined from the image to be detected by feature extraction and comparison. In actual implementation, the image to be detected can first be segmented according to a set shape to obtain each segmented sub-image to be detected. The set shape can be a circle, rectangle, ellipse, etc. Next, feature extraction is performed on each sub-image to be detected to obtain each feature vector. Then, the difference information between each feature vector and the reference feature vector is determined. If there is a target difference information that is less than the difference threshold, the region of the sub-image to be detected corresponding to the target difference information is determined as the target region.

[0055] Based on this, relevant information about the target region is input into the trained second detection model to obtain a second detection result that characterizes the local defects in the image to be detected.

[0056] Step S104: The first detection result and the second detection result are fused to obtain the target detection result corresponding to the image to be detected.

[0057] In this embodiment, the first detection result and the second detection result can be spliced ​​together to obtain the spliced ​​detection result. Then, the spliced ​​detection result is deredundant, which is equivalent to removing duplicate detection results to obtain the deredundant detection result. Finally, the deredundant detection result is determined as the target detection result corresponding to the image to be detected.

[0058] In this embodiment, the process first acquires the image to be detected, a trained first detection model, and a trained second detection model. The first detection model performs overall, global detection on the image, while the second detection model performs partial, local detection. Next, the trained first detection model is used to detect the image, yielding a first detection result representing the overall detection result. Then, the target region of the image is determined, and the trained second detection model is used to detect this target region, yielding a second detection result representing the partial detection result of the target region. Finally, the first and second detection results are fused to obtain a target detection result that includes both the overall and regional detection results of the image. This allows for comprehensive detection of the image, improving detection accuracy, by using two detection models, which are less complex than a single overall detection model. Furthermore, the use of two detection models simplifies the detection process and improves real-time performance.

[0059] In some embodiments, to mitigate the serious consequences of defects such as conduit breakage or damage, a lightweight detection model is used to quickly identify whether such defects exist in the image to be inspected. If conduit breakage or damage is confirmed in the image, an alarm message is quickly issued. Figure 2 As shown, the detection method further includes steps S201 to S205:

[0060] Step S201: Obtain the trained third detection model.

[0061] Here, the third detection model is used to quickly detect the image to be detected. This third detection model can be a lightweight neural network model, which has a simple model structure and can quickly obtain detection results.

[0062] In this embodiment of the application, the third detection model is used to detect defects that have a significant impact and a high defect level.

[0063] Step S202: Use the trained third detection model to detect the image to be detected and obtain the third detection result.

[0064] Taking the trained third detection model as a lightweight neural network model as an example, the image to be detected is input into the lightweight neural network model. The image to be detected is processed layer by layer by the input layer neurons, hidden layer neurons, and output layer neurons. Finally, the third detection result is output through the output layer. This third result can characterize whether there are defects with a large impact and a high defect level in the image to be detected.

[0065] Step S203: Determine whether there is a first defect result in the third detection result that matches the target level defect.

[0066] Here, the target level defect can refer to a high-level defect. For example, the target level could be a broken or damaged conduit. This high-level defect can cause serious adverse consequences, such as causing a high-speed train to suddenly change speed or lose control.

[0067] If it is determined that there is a first defect result in the third detection result that matches the target level defect, it indicates that there is a high-level defect in the third detection result, which needs to be taken seriously immediately, and then proceed to step S204; if it is determined that there is no first defect result in the third detection result that matches the target level defect, it indicates that there is no high-level defect in the third detection result, which does not need to be taken seriously immediately, then return to step S202 and continue to detect the image to be detected.

[0068] Step S204: Generate a first alarm message based on the first defect result.

[0069] Here, the first alarm message can be in the form of at least one of character, voice, or video. This application embodiment does not limit the form of the first alarm message.

[0070] For example, if the first defect is a broken conduit, the first alarm message generated could be "A broken conduit exists in image X, posing a significant risk. Please arrange repairs as soon as possible."

[0071] Step S205: Output the first alarm message.

[0072] Here, the first alarm message can be output in the form of characters through a pop-up window, in the form of voice through a sound output device, or in the form of light through a light-emitting device.

[0073] In some embodiments, the detection device can also establish a communication connection with the terminal via a communication link. Based on this, the first alarm message can also be output through the terminal's output device to achieve the purpose of alarm.

[0074] It should be noted that steps S201 to S205 can be executed synchronously with steps S101 to S104, and there is no specific order between them. That is, global and local detection of the image to be detected is performed, and fast detection of the image to be detected is also performed at the same time.

[0075] Through steps S201 to S205 above, a trained third detection model is first obtained. This detection model is a lightweight and fast detection model that can quickly detect the image to be detected. Then, the trained third detection model is used to detect the image to be detected to obtain the third detection result. If it is determined that there is a first defect result in the third detection result that matches the target level defect, it indicates that there is a high-level defect in the third detection result. An alarm message will be generated based on the third detection result, and an alarm will be issued based on the alarm message. This achieves rapid detection and timely alarm for target level defects, improves detection efficiency, and reduces loss of life and property.

[0076] In some embodiments, after obtaining the target detection results, alarms can be issued in a tiered manner, as referenced. Figure 3 Following step S104, the detection method further includes steps S105 to S109:

[0077] S105, determine whether there is a second defect result in the target detection result that matches the target level defect.

[0078] Here, the target level defect can still refer to a high-level defect. Since the target level defect has a significant impact, it is also necessary to determine whether there is a second defect result in the target detection result that matches the target level. If it is determined that there is a second defect result in the target detection result that matches the target level defect, it indicates that there is a high-level defect in the target detection result that needs to be taken seriously immediately, and then proceed to step S106; if it is determined that there is no second defect result in the target detection result that matches the target level defect, it indicates that there is no high-level defect in the target detection result that does not need to be taken seriously immediately, and then proceed to step S108 to continue to determine whether there is a third defect result in the target detection result that is not a target level defect.

[0079] In this embodiment, the second defect result is different from the first defect result. That is, if a high-level first defect has been detected by the third detection module, after the target detection result determines the high-level defect, a comparison method will be used to determine whether the high-level defect in the target detection result includes the first defect. If the high-level defect in the target detection result does not include the first defect, then the high-level defect in the target detection result will be determined as the second defect. If the high-level defect in the target detection result includes the first defect, in order to avoid repeated alarms for the same defect, the first defect will be removed, and the removed defect will be determined as the second defect.

[0080] Step S106: Generate a second alarm message based on the second defect result.

[0081] At this point, it is determined that there is a second defect result in the target detection result that matches the target level defect. In actual implementation, the implementation method of step S106 is similar to that of step S204. Therefore, the implementation method of step S106 can refer to the implementation method of step S204.

[0082] Step S107: Output the second alarm message.

[0083] In actual implementation, the implementation method of step S107 is similar to that of step S205. Therefore, the implementation method of step S107 can refer to the implementation method of step S205.

[0084] Step S108: Determine whether there is a third defect result other than the target-level defect in the target detection result.

[0085] If it is determined that there is no second defect result in the target detection result that matches the target level defect, then it is necessary to continue to determine whether there is a third defect result other than the target level defect in the target detection result.

[0086] In this embodiment, since the object to be detected has various defects, some of which have serious adverse consequences while others have minor ones, the defects of the object to be detected are classified into different levels, such as high-level defects, medium-level defects, and low-level defects. As described in the above embodiment, high-level defects can be identified as target-level defects. Taking the inclusion of high-level, medium-level, and low-level defects as an example, when implementing step S108, it is further determined whether medium-level and low-level defects exist in the target detection result. If medium-level and low-level defects exist in the target detection result, the process proceeds to step S109; if medium-level and low-level defects do not exist in the target detection result, the process returns to step S108.

[0087] Step S109: Determine the defect level corresponding to the third defect result.

[0088] Continuing with the example above, medium-level defects can include bulges, bends, and lack of stress; low-level defects can include contamination and electrical connection clamp failures. Based on this, the defects included in the third defect result are matched with each reference defect included in the medium-level and low-level defects. If the defects match, the match is considered successful, and the defect level corresponding to the reference defect is determined as the defect level corresponding to the third defect result.

[0089] Step S110: Store the image to be detected, the third defect result, and the defect level in the defect result database.

[0090] Here, a correspondence is established between the image to be detected, the third detection result, and the defect level. Since there is no defect in the third defect result that matches the target level defect at this time, there is no need to immediately issue an alarm. Therefore, the image to be detected, the third defect result, the defect level, and the correspondence between them are stored in the defect result database for easy viewing later. Of course, alarms can also be issued in order of defect level from high to low.

[0091] Through steps S105 to S110 above, it is first determined whether there is a second defect result in the target detection result that matches the target level defect. If it is determined that there is a second defect result in the target detection result that matches the target level defect, it indicates that there is a high-level defect in the target detection result, which needs to be taken seriously immediately. Then, a second alarm message is generated based on the second defect result and the second alarm message is output. If it is determined that there is no second defect result in the target detection that matches the target level defect, it is further determined whether there is a third defect result other than the target level defect in the target detection result. If it is determined that there is a third defect result other than the target level defect in the detection result, the defect level corresponding to the third defect result will be determined. Finally, the image to be detected, the third detection result, and the defect level are stored in the database for later viewing, or an alarm can be issued together after the detection is completed.

[0092] In practice, defects can be categorized into unknown defects and known defects based on their type. To improve detection efficiency, different detection models are used for different types of defects. The trained first detection model can include a trained first sub-detection model and a trained second sub-detection model. The trained first sub-detection model is used to detect unknown defects, while the trained second sub-detection model is used to detect known defects. This allows for the use of different detection models for different types of defects, improving detection efficiency and real-time performance.

[0093] In this embodiment of the application, step S102 can be implemented through the following steps:

[0094] Step 1: Use the trained first sub-detection model to detect the image to be detected, and obtain the fourth detection result; use the trained second sub-detection model to detect the image to be detected, and obtain the fifth detection result.

[0095] In this step, the image to be detected is input into the trained first sub-detection model and the trained second sub-detection model respectively. The trained first sub-detection model detects unknown types of defects in the image to be detected, thereby obtaining a fourth detection result that characterizes whether the image to be detected contains unknown types of defects. The trained second sub-detection model detects known types of defects in the image to be detected, thereby obtaining a fifth detection result that characterizes whether the image to be detected contains known types of defects.

[0096] Step two: The fourth and fifth test results are determined as the first test result.

[0097] In this step, the fourth and fifth detection results are combined to obtain the first detection result; or the set of the fourth and fifth detection results is determined and the set is determined as the first detection result.

[0098] In some embodiments, known type defects are generally high-level defects. When the target level defect refers to a high-level defect, such as a broken or damaged conduit, it belongs to both known type defects and target level defects. Therefore, the trained third detection model in the above embodiments and the second sub-detection model here can be the same model. That is, this model can quickly detect both target level defects and known type defects, thus improving the model's functionality, reducing the number of detection models, and improving detection timeliness.

[0099] In this embodiment of the application, through the above two steps, the global detection of the image to be detected is divided into two detections according to different types of defects: the detection of unknown types of defects using a trained first sub-detection model, and the detection of known types of defects using a trained second sub-detection model. Thus, the detection efficiency, detection accuracy, and detection comprehensiveness are improved through these two sub-detection models.

[0100] In some embodiments, on the one hand, various detection models are trained to obtain trained detection models; on the other hand, if the detection results do not match the actual detection results during the detection process, the trained detection models can be iteratively updated based on the actual detection results to obtain updated detection models, such as... Figure 4 As shown, the detection method also includes:

[0101] Step S401: Obtain the preset first sub-detection model, the preset second sub-detection model, the preset second detection model, the preset third detection model, the positive sample image, and the negative sample image, respectively.

[0102] Here, a preset first sub-detection model, a preset second sub-detection model, a preset second detection model, a preset third detection model, positive sample images, and negative sample images can be obtained from a dedicated server or a general-purpose server. The preset first sub-detection model, the preset second sub-detection model, the preset second detection model, and the preset third detection model can be artificial intelligence algorithm models such as neural network models, Bayesian network models, and genetic algorithm models. Different detection models can be the same type of algorithm model or different types of algorithm models; this embodiment does not limit this. Positive sample images represent images without defects, that is, normal images; negative sample images represent images with defects, that is, abnormal sample images.

[0103] Step S402: Train the preset first sub-detection model based at least on positive sample images to obtain the trained first sub-detection model.

[0104] Here, the preset first detection sub-model is used to detect unknown types of defects. For example, the preset first detection sub-model is used to determine whether there are unknown types of defects such as bird nests or large foreign objects in the image to be detected. Since the defects are of unknown types and it is impossible to know what the defects are, in this embodiment, the preset first sub-detection model can be trained based on positive sample images to obtain a trained first sub-detection model. The trained first sub-detection model can determine whether the image to be detected is an image without defects. If the image to be detected is an image without defects, it indicates that there are no unknown types of defects in the image to be detected; if the image to be detected is not an image without defects, it indicates that there are unknown types of defects in the image to be detected.

[0105] Step S403: Train the preset second sub-detection model, the preset second detection model, and the preset third detection model based on at least the negative sample images to obtain the trained second sub-detection model, the trained second detection model, and the trained third detection model.

[0106] Here, the preset second sub-detection model is used to detect known types of defects, such as pipe breakage or damage; the preset second detection model is used to detect local defects, such as faults in the heart ring, dropper clamp, or current-carrying ring; the preset third detection model is used to quickly detect the image to be detected, and to detect whether the image to be detected contains a target-level defect, such as pipe breakage or damage.

[0107] Based on this, a pre-defined second sub-detection model can be trained using negative examples including known types of defects, and also using positive examples and negative examples including known types of defects, to obtain a trained second sub-detection model used to detect whether known types of defects exist in the image to be detected. Similarly, a pre-defined second detection model can be trained using negative examples including local defects, and also using positive examples and negative examples including local defects, to obtain a trained second detection model used to detect whether local defects exist in the image to be detected. Likewise, a pre-defined third detection model can be trained using negative examples including target-level defects, and also using positive examples and negative examples including target-level defects, to obtain a trained third detection model used to detect whether target-level defects exist in the image to be detected.

[0108] Step S404: Obtain the actual detection results for the image to be detected.

[0109] Here, the actual detection result refers to the actual situation of the image to be detected after on-site confirmation, and the actual detection result for the image to be detected can be obtained based on the input operation.

[0110] Step S405: Determine whether the target detection result is consistent with the actual detection result.

[0111] Here, if the target detection result is consistent with the actual detection result, it indicates that the detection result obtained based on the trained detection model is accurate, reflecting that the trained detection model is appropriately selected, and then return to step S404; if the target detection result is inconsistent with the actual detection result, it indicates that the detection result obtained based on the trained detection model is inaccurate, reflecting that the trained detection model is not appropriately selected, and it is necessary to continue training the trained detection model to obtain a suitable detection model, and then proceed to step S406.

[0112] Step S406: Based on the image to be detected and the actual detection results, continue to train the trained first detection model, the trained second detection model and the trained third detection model to obtain the updated first detection model, the updated second detection model and the updated third detection model.

[0113] If the target detection result is inconsistent with the actual detection result, it is necessary to train the trained detection model based on the image to be detected and the actual detection result, optimize the detection model, and obtain an updated detection model to solve the problem of inaccurate detection results. Thus, the updated detection model can obtain accurate detection results and improve the detection accuracy. The detection model includes a first detection model, a second detection model, and a third detection model.

[0114] Step S407: Defect detection is performed using the updated first detection model, the updated second detection model, and the updated third detection model.

[0115] Here, after obtaining the updated first detection model, the updated second detection model, and the updated third detection model through step S406, since the updated model can obtain detection results with better accuracy, the updated detection model is used for defect detection in subsequent detection.

[0116] Through steps S401 to S407, pre-defined detection models are first obtained. Then, these models are trained based on samples to obtain trained detection models used for defect detection on the image to be detected. During the detection process, if the actual detection result and the target detection result are found to be inconsistent, the trained detection model will be further trained based on the image to be detected and the actual detection result to obtain an updated detection model. This updated detection model can detect results with higher accuracy, overcoming the problem of inconsistency between the actual and target detection results, and ensuring consistency between the actual and target detection results.

[0117] In some embodiments, such as Figure 5 As shown, step S104, "fusing the first detection result and the second detection result to obtain the target detection result corresponding to the image to be detected," can be achieved through the following steps S1041 to S1043:

[0118] Step S1041: The first detection result and the second detection result are spliced ​​together to obtain the spliced ​​detection result.

[0119] Here, the combination of the first and second detection results can be defined as a splicing process, and the resulting combination is the spliced ​​detection result.

[0120] Step S1042: Perform redundancy removal processing on the spliced ​​detection results to obtain redundancy-removed detection results.

[0121] Here, the first and second trained detection models may detect the same defect, resulting in duplicate detection results. To avoid duplicate detection results, the concatenated detection results are deredundant, resulting in deredundant detection results that do not contain duplicate detection results.

[0122] Step S1043: Determine the redundancy-removed detection result as the target detection result.

[0123] Here, the redundancy-free detection results can be directly used as the target detection results for the image to be detected.

[0124] Through the above steps S1041 to S1043, the first detection result of the overall detection and the second detection result of the local detection are stitched together, and redundancy removal is performed after stitching, thereby obtaining the target detection result of the image to be detected, improving the completeness and simplicity of the target detection result.

[0125] In some embodiments, to facilitate a holistic understanding of the test results, such as Figure 6As shown, the detection method also includes:

[0126] Step S601: Obtain the set of detection results within the target duration.

[0127] Here, the target duration can be 1 day, 2 days, 5 days, etc., and the set of detection results within the target duration can be obtained from the detection device by reading the command.

[0128] Step S602: Determine the defect type corresponding to each detection result in the detection result set.

[0129] Here, each reference defect that matches each test result is identified, and the reference defect type corresponding to each reference defect is determined as the defect type corresponding to each test result.

[0130] Step S603: Based on the anomaly type corresponding to each detection result, determine the statistical frequency corresponding to each anomaly type.

[0131] Here, the number of times each anomaly type is counted can be determined by counting.

[0132] Step S604: Display each anomaly type and the corresponding number of occurrences.

[0133] Here, various anomaly types and their corresponding statistical frequencies can be displayed using bar charts, pie charts, line charts, etc., to achieve the purpose of grouping and displaying various detection results according to defect type.

[0134] In this embodiment of the application, through the above steps S601 to S604, the detection results within the target duration are first obtained, and the defect type corresponding to each detection result is determined; then, the statistical frequency corresponding to each defect type is determined; finally, each defect type and its corresponding statistical frequency are visualized and displayed, so as to achieve the purpose of grouping and displaying each detection result according to the defect type.

[0135] Based on the above embodiments, this application also provides a detection method, which is applied to the detection of high-speed railway catenary from the whole map to the part. Figure 7 Here is another implementation flowchart of the detection method provided in the embodiments of this application, referred to Figure 7 The testing process includes steps one through five:

[0136] Step 1: Detect unknown anomalies across the entire image

[0137] First, perform full-image anomaly detection using an anomaly detection algorithm, refer to... Figure 8This is equivalent to using a detection model to detect 81 defects in the entire image and determine whether there are obvious defects such as bird nests, large foreign objects, or large-area damage. These defects are unknown. The characteristics of unknown anomalies are that they are very obvious or cover a large area. The anomaly detection algorithm corresponds to the first sub-detection model trained in the above embodiment.

[0138] Secondly, the anomaly detection method can be trained using normal samples, where normal samples correspond to the positive sample images in the above embodiments.

[0139] Step 2: Detection of known defects in the entire image.

[0140] First, we use an object detection algorithm to perform full-image object defect detection, referring to... Figure 8 Determine whether there are any specified obvious defects 82. These defects are known and occur frequently. They are characterized by being obvious and having a high defect level, requiring special attention. Among them, the defect levels of known defects, from high to low, include: various types of pipe breakage and damage; bulging, bending, and failure to bear force; contamination and other obvious defects.

[0141] Secondly, the anomaly detection algorithm can be trained based on defect-annotated samples, where defect-annotated samples correspond to negative sample images in the above embodiments.

[0142] Step 3: Defect detection of key components.

[0143] First, the Region of Interest (ROI) of the component is located, and then anomaly detection is performed based on the ROI. This part focuses on handling defects that could not be detected in step two; some of these defects are of a high severity level, while others are relatively low.

[0144] Among them, reference Figure 8 Key component area 83 includes: combined areas that are not clearly defined, such as the core rings at the upper and lower ends of the dropper, dropper clamps, current-carrying rings, nuts, and crimping pipe areas; small component areas, such as the center anchor clamp, electrical connection clamp, and other clamp areas; and other areas of the dropper system.

[0145] Secondly, the technologies employed include: first, accurately locating the component and outputting the ROI; then, using an anomaly detection algorithm that can detect anomalies based on the ROI.

[0146] Step 4: Output the final result.

[0147] Here, based on the results of steps one through three, the final result is output after integration. Integration can be fusion or redundancy removal.

[0148] Step 5: Adaptive interactive display.

[0149] Here, as Figure 9 As shown, examples of defect types such as fracture, bulge, and breakage are displayed in groups. The webpage displays images grouped according to defect type. Additionally, some images are very dark, such as... Figures 10A to 10C As shown, users can manually adjust the image brightness to better see the defects. Figure 10A The collected images, Figure 10B The image after brightening. Figure 10C In order to be in Figure 10B The image was brightened again based on the previous image, and then compared. Figure 10A , Figure 10B as well as Figure 10C As can be seen, by increasing the image brightness, the captured object to be detected can be displayed more clearly, making it easier to view and detect.

[0150] Through steps one through five, the progressive defect detection scheme moves from the overall to the local, focusing on defects from large to small, enabling comprehensive and efficient detection of all defects and reducing missed detections. Steps one through three are independent and can be performed simultaneously, resulting in fast processing speed and significant savings in manpower and time costs. Furthermore, the anomaly detection algorithm is trained using only normal samples, while the actual number of normal samples far exceeds the number of anomaly samples, providing ample training data to ensure model stability and thus guarantee stable algorithm performance.

[0151] In some embodiments, computer vision is used to replace manual inspection for the inspection of high-speed rail catenary, thereby accelerating the inspection speed and improving accuracy. However, in such scenarios with large data volumes, even with the acceleration provided by computer vision, the inspection results per cycle still cannot achieve real-time performance. Therefore, in the embodiments of this application, such as Figure 11 As shown, tiered early warning systems can be implemented using the following modules:

[0152] First, a risk database 1101 is built to store various defects detected by the real-time detection algorithm in the background.

[0153] Next, using the rapid detection algorithm 1102, high-risk defects such as fractures and damages are detected. The detection results are stored in the risk database 1101 and transmitted to the risk processing platform 1103 in real time. Subsequently, an early warning notification is issued and assigned to maintenance workers for on-site maintenance 1104.

[0154] Then, a detailed inspection of defects outside the high-risk level is performed, including a full-map inspection 1105 and a detailed local inspection 1107 based on key components 1106. Based on the inspection results, the risk level is matched and stored in the risk database 1101. Various different detection algorithms may be used during the detailed inspection of defects outside the high-risk level, operating serially or in parallel, so the speed is relatively slow and the results may not be synchronized.

[0155] This tiered early warning system allows for the prompt inspection and repair of high-risk defects, while low-risk defects are processed in batches, improving both inspection efficiency and safety. Results can be obtained simply by monitoring the risk database in real time, eliminating the need to constantly monitor the results of the background detection algorithm. Furthermore, the content of the risk database is traceable.

[0156] Finally, data backtracking 1108 refers to establishing a closed-loop mechanism after the entire inspection cycle is completed, performing defect verification and adjusting manual maintenance tasks. Defect verification involves comparing the actual on-site conditions inspected by maintenance workers with the inspection results, storing the comparison results in a database for algorithm developers to optimize the corresponding detection algorithms. Adjusting manual maintenance tasks involves adjusting manual maintenance tasks based on the frequency of defect occurrence; for components with high failure rates, investigating whether the components themselves have problems, and increasing the maintenance frequency for high failure rate intervals.

[0157] The hierarchical early warning method in this application embodiment can quickly respond to high-risk systems, ensure rapid repair, and conduct detailed data testing after rapid detection to identify and address any deficiencies. Furthermore, all data information and test results will be saved into the database, enabling overall backtracking and the formation of a closed loop.

[0158] Based on the foregoing embodiments, this application provides a detection device. The modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a CPU, a microprocessor unit (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0159] This application embodiment further provides a detection device, Figure 12 This is a schematic diagram of the composition of the detection device provided in the embodiments of this application, such as... Figure 12 As shown, the detection device 1200 includes:

[0160] The acquisition module 1201 is used to acquire the image to be detected, the trained first detection model, and the trained second detection model; the first detection model is used to perform global detection on the image to be detected; the second detection model is used to perform local detection on the image to be detected.

[0161] The first detection module 1202 is used to detect the image to be detected using the trained first detection model to obtain a first detection result;

[0162] The second detection module 1203 is used to determine the target region in the image to be detected, and to detect the target region using the trained second detection model to obtain a second detection result;

[0163] The fusion module 1204 is used to fuse the first detection result and the second detection result to obtain the target detection result corresponding to the image to be detected.

[0164] In some embodiments, the acquisition module 1201 is further configured to acquire a trained third detection model, the third detection model being used for rapid detection of the image to be detected; the detection device 1200 further includes:

[0165] The third detection module is used to detect the image to be detected using the trained third detection model to obtain a third detection result;

[0166] The first generation module is used to determine that there is a first defect result in the third detection result that matches the target level defect, and to generate a first alarm message based on the first defect result.

[0167] The first output module is used to output the first alarm message.

[0168] In some embodiments, the detection device 1200 further includes:

[0169] The second generation module is used to determine that there is a second defect result in the target detection result that matches the target level defect, and to generate a second alarm message based on the second defect result, wherein the second defect result is different from the first defect result;

[0170] The second output module is used to output the second alarm message.

[0171] In some embodiments, the detection device 1200 further includes:

[0172] The first determining module is used to determine that there is a third defect result in the target detection result other than the target level defect, and to determine the defect level corresponding to the third defect result;

[0173] The storage module is used to store the image to be detected, the third defect result, and the defect level into the defect result database.

[0174] In some embodiments, the trained first detection model includes a trained first sub-detection model and a trained second sub-detection model, wherein the trained first sub-detection model is used to detect defects of unknown types, and the trained second sub-detection model is used to detect defects of known types; the first detection module 1202 includes:

[0175] The first detection submodule is used to detect the image to be detected using the trained first sub-detection model to obtain a fourth detection result; and to detect the image to be detected using the trained second sub-detection model to obtain a fifth detection result.

[0176] The first determining submodule is used to determine the fourth detection result and the fifth detection result as the first detection result.

[0177] In some embodiments, the acquisition module 1201 is further configured to acquire a preset first sub-detection model, a preset second sub-detection model, a preset second detection model, a preset third detection model, a positive sample image, and a negative sample image, wherein the positive sample image represents an image without defects, and the negative sample image represents an image with defects; the detection device 1200 further includes:

[0178] The first obtaining module is used to train the preset first sub-detection model based at least on the positive sample images to obtain the trained first sub-detection model;

[0179] The second obtaining module is used to train the preset second sub-detection model, the preset second detection model, and the preset third detection model based at least on the negative sample images, so as to obtain the trained second sub-detection model, the trained second detection model, and the trained third detection model.

[0180] In some embodiments, the acquisition module 1201 is further configured to acquire the actual detection result for the image to be detected; the detection device 1200 further includes:

[0181] The training module is used to continue training the trained first detection model, the trained second detection model, and the trained third detection model based on the image to be detected and the actual detection result if the target detection result is inconsistent with the actual detection result, so as to obtain the updated first detection model, the updated second detection model, and the updated third detection model.

[0182] The fourth detection module is used to perform defect detection using the updated first detection model, the updated second detection model, and the updated third detection model.

[0183] In some embodiments, the fusion module 1204 includes:

[0184] The splicing submodule is used to splice the first detection result and the second detection result to obtain the spliced ​​detection result.

[0185] The redundancy removal submodule is used to perform redundancy removal processing on the spliced ​​detection results to obtain redundancy-removed detection results;

[0186] The second determining submodule is used to determine the redundancy-removed detection result as the target detection result.

[0187] In some embodiments, the acquisition module 1201 is further configured to acquire a set of detection results within a target duration; the detection device 1200 further includes:

[0188] The second determining module is used to determine the defect type corresponding to each detection result in the detection result set;

[0189] The third determining module is used to determine the statistical frequency corresponding to each defect type based on the defect type corresponding to each detection result;

[0190] The display module is used to display each defect type and the corresponding statistical count for each defect type.

[0191] It should be noted that the description of the detection device in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects. For technical details not disclosed in this device embodiment, please refer to the description of the method embodiment in this application for understanding.

[0192] It should be noted that, in the embodiments of this application, if the above-described system upgrade method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0193] Accordingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the detection method provided in the above embodiments.

[0194] This application provides a detection device. Figure 13 This is a schematic diagram of the composition structure of the detection equipment provided in the embodiments of this application, such as... Figure 13As shown, the detection device 1300 includes: a processor 1301, at least one communication bus 1302, a user interface 1303, at least one external communication interface 1304, and a memory 1305. The communication bus 1302 is configured to enable communication between these components. The user interface 1303 may include a display screen, and the external communication interface 1304 may include standard wired and wireless interfaces. The processor 1301 is configured to execute a program for a detection method stored in the memory to implement the detection method provided in the above embodiment.

[0195] The descriptions of the above-described testing equipment and storage medium embodiments are similar to those of the above-described method embodiments, and have similar beneficial effects. For technical details not disclosed in the testing equipment and storage medium embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0196] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0197] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0198] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0199] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0200] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0201] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0202] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an AC to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0203] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A detection method, the method comprising: Acquire the image to be detected, the trained first detection model, and the trained second detection model; The first detection model is used to perform global detection on the image to be detected; The second detection model is used to perform local detection on the image to be detected; the trained first detection model includes a trained first sub-detection model and a trained second sub-detection model, the trained first sub-detection model is used to perform full-image anomaly detection for unknown types of defects, and the trained second sub-detection model is used to perform full-image target defect detection for known types of defects; The trained first detection model is used to detect the image to be detected, and a first detection result is obtained; the first detection result is the detection result for the unknown type of defect and the detection result for the known type of defect; The target region in the image to be detected is determined, and the trained second detection model is used to detect the target region to obtain a second detection result; The first detection result and the second detection result are fused to obtain the target detection result corresponding to the image to be detected. The method further includes: A trained third detection model is obtained, which is used for rapid detection of the image to be detected; The trained third detection model is used to detect the image to be detected, and a third detection result is obtained; A first defect result matching the target level defect is determined in the third detection result, and a first alarm message is generated based on the first defect result; the first alarm message is used to characterize that the image to be detected has a high-level defect; Output the first alarm message; Specifically, the global and local detection of the image to be detected is performed using the first and second detection models simultaneously with the rapid detection of the image to be detected using the third detection model.

2. The method according to claim 1, further comprising: Determine that there is a second defect result in the target detection result that matches the target level defect, and generate a second alarm message based on the second defect result, wherein the second defect result is different from the first defect result; Output the second alarm message.

3. The method according to claim 2, further comprising: Determine that a third defect result exists in the target detection results, in addition to the defect of the target level, and determine the defect level corresponding to the third defect result; The image to be detected, the third defect result, and the defect level are stored in the defect result database.

4. The method according to claim 1, wherein the step of using the trained first detection model to detect the image to be detected and obtaining a first detection result includes: The trained first sub-detection model is used to detect the image to be detected, and a fourth detection result is obtained; The trained second sub-detection model is used to detect the image to be detected, and a fifth detection result is obtained; The fourth and fifth detection results are determined as the first detection result.

5. The method according to claim 4, further comprising: A preset first sub-detection model, a preset second sub-detection model, a preset second detection model, a preset third detection model, a positive sample image, and a negative sample image are obtained respectively, wherein the positive sample image represents an image without defects, and the negative sample image represents an image with defects; The preset first sub-detection model is trained based on at least the positive sample images to obtain the trained first sub-detection model; The preset second sub-detection model, the preset second detection model, and the preset third detection model are trained based on at least the negative sample images to obtain the trained second sub-detection model, the trained second detection model, and the trained third detection model.

6. The method according to claim 5, further comprising: Obtain the actual detection results for the image to be detected; If the target detection result is inconsistent with the actual detection result, based on the image to be detected and the actual detection result, the trained first detection model, the trained second detection model and the trained third detection model are trained again to obtain the updated first detection model, the updated second detection model and the updated third detection model. Defect detection is performed using the updated first detection model, the updated second detection model, and the updated third detection model.

7. The method according to claim 1, wherein the first detection result and the second detection result are fused to obtain the target detection result corresponding to the image to be detected, comprising: The first detection result and the second detection result are spliced ​​together to obtain the spliced ​​detection result. The spliced ​​detection results are subjected to redundancy removal processing to obtain redundancy-free detection results; The redundancy-removed detection result is determined as the target detection result.

8. The method according to any one of claims 1 to 7, further comprising: Obtain the set of detection results within the target duration; Determine the defect type corresponding to each detection result in the detection result set; Based on the defect type corresponding to each detection result, determine the statistical frequency corresponding to each defect type; Displays each defect type and the corresponding number of times each defect type is counted.

9. A detection device, the detection device comprising: The acquisition module is used to acquire the image to be detected, the trained first detection model, and the trained second detection model; The first detection model is used to perform global detection on the image to be detected; the second detection model is used to perform local detection on the image to be detected; the trained first detection model includes a trained first sub-detection model and a trained second sub-detection model, the trained first sub-detection model is used to perform full-image anomaly detection for unknown types of defects, and the trained second sub-detection model is used to perform full-image target defect detection for known types of defects; The first detection module is used to detect the image to be detected using the trained first detection model to obtain a first detection result; the first detection result is the detection result for the unknown type of defect and the detection result for the known type of defect; The second detection module is used to determine the target region in the image to be detected, and to detect the target region using the trained second detection model to obtain a second detection result; The fusion module is used to fuse the first detection result and the second detection result to obtain the target detection result corresponding to the image to be detected. The acquisition module is also used to acquire a trained third detection model, which is used to perform fast detection on the image to be detected. The third detection module is used to detect the image to be detected using the trained third detection model to obtain a third detection result; The first generation module is used to determine that there is a first defect result in the third detection result that matches the target level defect, and to generate a first alarm message based on the first defect result. The first alarm message is used to indicate that the image to be detected has a high-level defect; The first output module is used to output the first alarm message; Specifically, the global and local detection of the image to be detected is performed using the first and second detection models simultaneously with the rapid detection of the image to be detected using the third detection model.

Images