Bridge rubber bearing disease identification method based on target detection and multi-label classification

Abstract

This invention relates to a method for identifying bridge rubber bearing defects based on target detection and multi-label classification, comprising: acquiring original images of bridge rubber bearings; extracting rubber bearings from the original images using a trained target detection network to obtain multiple prediction boxes; extracting rubber bearing images corresponding to each prediction box from the original images based on the prediction boxes; classifying the rubber bearing images using a trained defect classification network; and coupling the overall state vector and defect label vector to obtain the final identification result of the rubber bearings. Through a cascaded framework of "target detection-defect classification," defect identification is decoupled into two stages: localization and classification. This effectively overcomes interference from small targets and background, and innovatively proposes a two-layer classification network of "overall state + defect label," ensuring the mutual exclusivity of "intact / uncertain / defect" conclusions, completely avoiding contradictory outputs, and significantly improving identification accuracy, logical rigor, and engineering practicality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for identifying defects in bridge rubber bearings based on target detection and multi-label classification. Background Technology

[0002] Bridge rubber bearings are critical force-transmitting and deformation-carrying components connecting the superstructure and substructure, and their service condition directly affects the bridge's load-bearing performance and operational safety. Under the influence of repeated vehicle loads, temperature and humidity cycles, ultraviolet aging, and corrosive media, rubber bearings are prone to surface defects such as cracking, hollowing, and aging, which may lead to a decrease in load-bearing and deformation capacity, creating safety hazards. Therefore, the appearance and condition inspection of bearings is an important part of the regular inspection and maintenance assessment of bridges, and relevant standards are constantly increasing the frequency and coverage of inspections.

[0003] Currently, bearing inspection in engineering still relies mainly on manual visual inspection and experience-based judgment, which suffers from low efficiency, strong subjectivity, and insufficient consistency of results, making it difficult to meet the needs of large-scale and routine operation and maintenance of bridge groups. Although some research in recent years has introduced deep learning for bearing defect identification, directly applying general detection or classification methods to field scenarios still faces difficulties.

[0004] First, rubber bearings are typically small-scale in images, with strong background interference, and are often accompanied by imaging problems such as occlusion, stains, shadows, and reflections, leading to false positives, false negatives, or positioning errors, affecting the reliability of subsequent assessments. Second, existing methods often treat defects such as cracks and hollow areas as independent targets for box-level localization and identification. While this can achieve a certain degree of automation, it does not fully match the goal of "primarily judging the overall condition of the bearing" in actual maintenance needs. Furthermore, multi-defect localization and annotation are costly and complex to train, and can easily lead to unstable model generalization when the sample size is limited and the defect morphology is varied. Third, the image sources and scales vary significantly. Directly scaling to a fixed input size can easily cause geometric and textural distortions, weakening the fine-grained features of defect identification. Fourth, multiple types of bearing defects often coexist. Without reasonable representation and output constraints, conflicting prediction results can easily occur, affecting the usability of the project.

[0005] Therefore, there is an urgent need for an intelligent identification method for rubber bearing defects that can reliably locate and suppress backgrounds in complex environments, and is applicable to working conditions with multiple defects coexisting and provides stable and consistent output. Summary of the Invention

[0006] To address the technical problems of low recognition accuracy, conflicting output results, and significant background interference in existing technologies, this invention provides a method for identifying bridge rubber bearing defects based on target detection and multi-label classification. The technical solution is as follows:

[0007] On the one hand, a method for identifying bridge rubber bearing defects based on object detection and multi-label classification is provided. This method includes: acquiring an original image of the bridge rubber bearing; using a trained object detection network to extract the rubber bearing from the original image, obtaining multiple prediction boxes; based on the prediction boxes, extracting the rubber bearing image corresponding to each prediction box from the original image; using a trained defect classification network to classify the rubber bearing image, wherein the defect classification network is configured to simultaneously output an overall state vector and a defect label vector, where each component of the overall state vector represents a mutually exclusive overall state of the bearing, and each component of the defect label vector independently represents the probability of existence of different defect categories; and coupling the overall state vector and the defect label vector to calculate the final identification result of the rubber bearing.

[0008] The beneficial effects of the technical solution provided by the embodiments of the present invention include at least the following: by designing a cascaded "target detection-disease classification" intelligent recognition framework, the complex problem of bridge rubber bearing defect identification is decoupled into two stages: precise positioning and fine classification. First, the method uses a target detection network to robustly locate the bearing from a complex background, effectively overcoming the interference of small target scale and cluttered background in the field image. Second, it innovatively adopts a two-layer output classification network structure of "overall status + defect label", which fundamentally ensures the logical mutual exclusivity of the three conclusions of "intact / uncertain / defect", completely avoiding contradictory outputs such as "both intact and cracked" that may occur in existing methods, making the identification results more consistent with the actual judgment logic of engineering. It significantly improves the accuracy, logical rigor, and engineering practicality of intelligent identification of bearing defects. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart of the first bridge rubber bearing defect identification method based on target detection and multi-label classification provided in the embodiments of the present invention;

[0011] Figure 2 This is a flowchart of the second method for identifying bridge rubber bearing defects based on target detection and multi-label classification provided in this embodiment of the invention;

[0012] Figure 3 This is a flowchart of a prediction box filtering method provided in an embodiment of the present invention;

[0013] Figure 4 This is a flowchart of an embodiment of the present invention for expanding a prediction box;

[0014] Figure 5 This is a flowchart of an uncertain state component correction provided by an embodiment of the present invention;

[0015] Figure 6 This is a decision flowchart of the final identification result of a rubber bearing provided by an embodiment of the present invention;

[0016] Figure 7 This is an original image provided in an embodiment of the present invention;

[0017] Figure 8 This is a schematic diagram of a prediction box and its confidence level provided in an embodiment of the present invention;

[0018] Figure 9 This is an extracted rubber support image provided in an embodiment of the present invention;

[0019] Figure 10 This is a schematic diagram of a final identification result provided by an embodiment of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0021] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0022] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0023] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0024] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0025] Please see Figure 1 This invention provides a method for identifying defects in bridge rubber bearings based on target detection and multi-label classification. The processing flow of this method may include steps S101 to S105.

[0026] Step S101: Obtain the original image of the bridge rubber bearing. The original image is as follows: Figure 7 As shown, this step aims to collect actual surface images of the rubber bearings on the bridge as the data input source for the subsequent intelligent recognition process. When training the model, the collected original images should cover as many different lighting conditions, shooting angles, bearing models, and various typical defects as possible to ensure the generalization ability of the constructed system. Realistic and diverse on-site image data lays a solid foundation for training a highly robust deep learning model and is a key prerequisite for moving from the laboratory to engineering applications.

[0027] Step S102: Using the trained object detection network, extract the rubber supports from the original image to obtain multiple predicted bounding boxes. The object detection network in this step can be, for example, a single-stage detector such as the YOLO series (YOLOv5, YOLOv8, YOLOv11), SSD, or a two-stage detector such as Faster R-CNN. The specific network selection can be based on a trade-off between detection speed and accuracy; its backbone feature extraction network can be ResNet, CSPDarknet, etc.

[0028] Understandably, the original image needs to be preprocessed to fit the input size of the object detection network. Preprocessing may include, for example, (1) scaling the image proportionally according to the original aspect ratio so that the long side of the scaled image does not exceed the preset input size M; (2) filling the scaled image with boundary values ​​to expand the image size to M×M. Among them, the boundary filling uses fixed pixel values ​​to ensure that the geometric shape of the target is not stretched or deformed, thereby improving the feature consistency under cross-device and cross-resolution conditions.

[0029] Understandably, the object detection network here is trained, and the training process includes: First, collecting and labeling a large number of bridge site images containing rubber bearings to construct a detection dataset, with the labeling information being the bounding boxes of the bearing locations. Then, the dataset is divided, and the selected object detection network model is iteratively trained using the training set. The network parameters are optimized through backpropagation to minimize the localization and classification errors between the predicted and ground truth boxes. Finally, the model performance is evaluated using a validation set until satisfactory detection accuracy is achieved. Data augmentation (such as rotation, scaling, and color dithering) is often used during training to improve the model's robustness.

[0030] Please see Figure 2Optionally, in step S102 above, the target detection network outputs a predicted bounding box, and generally also outputs the confidence level that the predicted bounding box belongs to the rubber bearing, such as... Figure 8 As shown, the initial predicted bounding boxes output by the target detection network typically contain a large number of redundant boxes with low confidence, inaccurate localization, or high overlap. Directly feeding all predicted boxes into the subsequent classification network would not only introduce a large number of low-quality or duplicate samples, severely impacting classification efficiency and accuracy, but also lead to duplicate judgments of the same scaffold. Therefore, it is necessary to filter and deduplicate the initial predicted boxes to ensure that the input to the disease classification network consists of high-quality, non-duplicative candidate regions. This is a crucial preprocessing step to ensure the reliability of the entire system's identification. Therefore, this method also includes step S106: filtering multiple predicted boxes based on confidence and at least one preset quality rule to obtain filtered predicted boxes. After filtering, the subsequent step S103 is performed; that is, step S103 is based on the filtered predicted boxes.

[0031] Please see Figure 4 Optionally, multiple prediction boxes are filtered based on confidence level and at least one preset quality rule, including at least one of steps S301 to S304.

[0032] Step S301: Based on a preset confidence threshold, initially filter out prediction boxes with a confidence level greater than or equal to the threshold. For example, the preset confidence threshold is τ. s For a single original image, an object detection network may output multiple predicted bounding boxes. For those boxes with a confidence level below a confidence threshold τ... s Those can be discarded.

[0033] Step S302: Perform non-maximum suppression on the initially screened prediction boxes to remove overlapping redundant prediction boxes. For example, an overlap threshold τ can be preset. iou For overlapping prediction boxes, first calculate the Intersection over Union (IoU) ratio (the ratio of the intersection area to the union area of ​​the two boxes), then compare this IoU ratio with the overlap threshold τ. iou Compare if IoU ≥ τ iou If the confidence level is higher in the two prediction frames, then the one with the higher confidence level will be retained.

[0034] Step S303: Based on the comparison between the short side length of the predicted bounding box and the preset size threshold, filter out predicted bounding boxes whose short side length is less than the size threshold. Since there are generally fixed requirements when capturing the original image, the size of the rubber support in the captured original image is usually not too small. Therefore, this step introduces a preset size threshold τ. min The shorter side length of the prediction box is min(w,h)≥τ min The predicted bounding box is preserved, where w and h are the width and height of the predicted bounding box, respectively.

[0035] Step S304: Based on the comparison between the proportion of the predicted bounding box in the original image and a preset proportion threshold, filter out predicted bounding boxes whose proportion is less than the proportion threshold. The predicted bounding box corresponds to the rubber support, and its proportion in the original image is (w×h) / (M×M). For example, a preset proportion threshold τ can be set. ratio Only when the aforementioned proportion is greater than or equal to the proportion threshold τ ratio The prediction box is only retained when the time is right.

[0036] Theoretically, if the target detection network can achieve near-perfect recognition accuracy, the screening step can be eliminated. However, due to limitations in complex field environments and model generalization capabilities, existing models inevitably produce noise in their output. By introducing a screening process based on confidence thresholds, non-maximum suppression, and size / proportion rules, low-quality predicted boxes with low confidence, ambiguous localization, excessively small size, or insufficient effective region proportion can be effectively filtered out. This process ensures that each candidate box entering the disease classification stage has high confidence and uniquely corresponds to a potential rubber support, thus providing high-quality input for subsequent high-precision disease identification.

[0037] Step S103: Based on the predicted bounding boxes, extract the rubber bearing images corresponding to each predicted bounding box from the original image. The rubber bearing images are as follows: Figure 9 As shown, it can be understood that if the filtering process of step S106 is executed after step S102, then step S103 and subsequent steps are all performed on the filtered prediction boxes.

[0038] Please see Figure 4 Optionally, step S103 includes: S401, enlarging the prediction box by a preset expansion factor to obtain an expanded box, wherein the expansion factor is calculated by the following formula:

[0039] ,

[0040] In the formula, γ is the expansion coefficient, γ min and γ max These are the minimum and maximum values ​​of the preset expansion coefficients, respectively, and s is the confidence level of the predicted box;

[0041] S402. Based on the expanded frame, extract the corresponding rubber support image from the original image.

[0042] This step employs a "confidence-driven adaptive expansion and pruning" strategy to intelligently enlarge the original predicted bounding boxes to obtain a more complete image of the region of interest (ROI) for the rubber bearing. The core of this strategy lies in the negative correlation between the expansion coefficient γ and the detection box confidence s: for low-confidence boxes, a larger expansion is performed to retain more contextual information and aid classification; for high-confidence boxes, a smaller expansion is performed to focus on the target and suppress redundant background. This strategy cleverly balances the contradiction between "including the complete target" and "reducing background interference" and can adaptively address detection uncertainty. Subsequently, the pruned region is uniformly scaled to a fixed size, providing the disease classification network with standardized input that is consistent in size, highlights the subject, and has controlled background interference, significantly improving the effectiveness of disease feature extraction and the stability of the classification model.

[0043] Understandably, if the expanded bounding box extends beyond the original image boundary, the excess portion can be cropped or padded. Furthermore, the corresponding rubber bearing image extracted from the original image must also undergo preprocessing to unify its size to M×M (or other preset sizes) to meet the input size requirements of the subsequent disease classification network.

[0044] Step S104: Classify the rubber bearing image using the trained disease classification network. The disease classification network is configured to output an overall state vector and a disease label vector simultaneously. Each component of the overall state vector represents the mutually exclusive overall state of the bearing, and each component of the disease label vector independently represents the probability of existence of different disease categories.

[0045] Understandably, the disease classification network is also trained, and its training process is basically the same as that of the target detection network, so it will not be described in detail here.

[0046] Understandably, disease classification networks can be lightweight networks such as the MobileNet series and ShuffleNet, or efficient networks such as the EfficientNet series and RegNet. Network selection requires comprehensive consideration of computational resources, inference speed, and classification accuracy. The network head needs to be modified to a two-layer output structure to generate the overall state vector and disease label vector separately, achieving mutual exclusion and label unification.

[0047] Optionally, the overall state vector includes at least three components: intact state, uncertain state, and diseased state, and the disease label vector includes at least three components: cracking, hollowing, and aging. Specifically, the first layer outputs the overall state vector [P]. N ,P U ,P D ], corresponding to the intact state, uncertain state, and damaged state, respectively, and Softmax is used to make it satisfy P N +P U +PD =1. The second layer outputs the disease label vector [P] A ,P B ,P C These correspond to cracking, hollowing, and aging, respectively, and are output using Sigmoid.

[0048] In this step, the disease classification network employs a carefully designed two-layer output structure to achieve a multi-granular, conflict-free description of the support status. The first layer outputs mutually exclusive overall states (intact / uncertain / disease) through the Softmax function, providing a primary and unique macroscopic assessment of the support's health status. The second layer independently outputs the probability of the existence of multiple disease labels through the Sigmoid function, allowing for an objective description of "coexistence of multiple diseases." This structure logically decouples "state determination" from "disease enumeration" while linking them informationally. This not only makes the network output more aligned with engineers' thinking habits (first determining whether there is a problem, then determining what the problem is), but also eliminates the logical contradiction between state and label at the model architecture level, laying a structural foundation for outputting stable and interpretable recognition results.

[0049] Step S105: Couple the overall state vector with the disease label vector to obtain the final identification result of the rubber bearing.

[0050] Optionally, step S105 includes: associating the overall state with the disease label using the following formula to obtain the constrained disease output probability:

[0051] , , ,

[0052] In the formula, [P A ,P B ,P C ] represents the disease label vector, P D This refers to the disease state component in the overall state vector. The constrained output probabilities of the defects are represented by subscripts A, B, and C, corresponding to the components of cracking, hollowing, and aging, respectively. This step transforms the abstract network output into final probabilities with clear physical meaning through mathematical coupling calculations. Specifically, each defect probability in the defect label vector is coupled to the "defect state" component P in the overall state vector. D Multiplication. This operation contains a profound logical constraint: only when the network as a whole determines that the support is in a "diseased state" (P... D Only when the value is high will the specific disease probability be significantly retained and output; conversely, if the network is judged as "intact" or "uncertain" (P... DIf the value is low or zero, the probability of all diseases will be suppressed to near zero. This enforces the engineering common sense that "no disease state means no specific disease" at the post-processing level, ensuring the absolute logical consistency of the final identification results.

[0053] Optionally, the total loss function of the disease classification network during training is:

[0054] ,

[0055] ,

[0056] In the formula, L base For the mixed loss term with class weights, L imp The consistency loss term is derived from constructing an asymmetric prior relation based on the degradation law of rubber bearings, where λ is the weighting coefficient. When training the disease classification network, the total loss function L is composed of the base loss L0. base And prior consistency loss L imp Weighted composition. L base This is responsible for driving the network to learn the basic mapping from image features to states and labels, and mitigating the data imbalance problem through class weights. The innovative introduction of L... imp The loss term incorporates domain knowledge (asymmetric prior) that "cracking or hollowing of rubber bearings is usually accompanied by material aging, but aging can occur independently." This loss term penalizes unreasonable scenarios during training where cracking / hollowing is predicted but aging is not, thus guiding the network to learn label associations that conform to physical degradation laws. This method of injecting domain knowledge into model training in the form of a differentiable loss function significantly enhances the physical plausibility and interpretability of the model's output.

[0057] Optionally, step S105 further includes: performing consistency correction on the constrained disease output probability according to the following formula:

[0058] This formula represents the probability of "aging" defects after constraints. Perform a consistency correction, updating it to itself and the "fracture" probability. "Empty drum" probability The maximum value among the three. This is because, based on the degradation mechanism of rubber bearings, "cracking" and "hollowing" are usually manifestations of material "aging" reaching a certain stage. Therefore, when the model provides evidence supporting the existence of "cracking" or "hollowing," it should simultaneously support the existence of "aging." This correction adds a safeguard during the inference stage, ensuring that the final output set of defect labels satisfies this physical prior under all circumstances, further improving the reliability and engineering credibility of the results.

[0059] Please see Figure 5Optionally, step S105 may further include steps S501 and S502.

[0060] Step S501: Based on the width, height, confidence level, and preset scale normalization threshold of the predicted bounding box, obtain the quality index of the region of interest. This step can be expressed by the formula:

[0061] ,

[0062] In the formula, w, h, and s represent the width, height, and confidence level of the prediction box, respectively, and τ size q is the preset scale normalization threshold, and q is the quality index of the region of interest.

[0063] Step S502: Based on the quality index of the region of interest and the uncertain state components in the overall state vector, obtain the corrected uncertain state components. This step can be expressed by the formula:

[0064] .

[0065] Steps S501 and S502 together constitute the "cross-task consistency constraint" mechanism. Step S501 integrates the confidence s of the detection box and the normalized scale. A comprehensive quality index q is generated to quantify the reliability and sharpness of the ROI of the current input classification network. A higher q value indicates better ROI quality. Step S502 uses q to evaluate the probability P of the original "uncertain" state of the classification network. U Make corrections and update it to P. U The larger of (1-q) and (q). The core logic is: when the ROI quality is low (small q, then large 1-q), regardless of the model's own judgment, the system should tend to output "uncertain" to reflect the caution of the conclusion. This mechanism transmits the quality perception of the upstream detection stage to the downstream classification decision, enabling the system to proactively reduce the confidence of strong conclusions such as "intact" or "specific defects" when facing low-quality input. This effectively reduces the risk of misjudgment in difficult scenarios such as blurred images and excessively small targets, improving the overall robustness of the system's output.

[0066] In step S104, the disease classification network outputs two vectors: the overall state vector [P]. N ,P U ,P D ] and disease label vector [P A ,P B ,P C In the steps included in S105 above, these parameters are constrained and corrected, and the final overall state vector is obtained as follows: Disease label vector is .

[0067] Please see Figure 6 Optionally, step S105 further includes the following decision-making step: S601, if the intact state component value P in the overall state vector is... N If the value is the maximum, the rubber support is determined to be in good condition, and the probability of the defect output after all constraints is zero; S602, if the value of the uncertain state component in the overall state vector is the maximum... If the value is maximum, the rubber support is determined to be in an uncertain state, and the probability of the defect output after all constraints is zero; S603, if the defect state component value P in the overall state vector is maximum. D If the value is the maximum, then based on the output probability of the disease after each constraint, the specific disease label is determined and output. When outputting the disease label, the value of each component in each disease label vector can be compared with a preset threshold. If it is greater than the preset threshold, the corresponding label is output. Figure 10 This illustrates one possible final identification result.

[0068] Steps S601-S603 constitute a clear and definitive final decision rule. Steps S601 and S602 stipulate that when the overall state is determined to be "intact" or "uncertain," regardless of the original value of the defect label vector, the probability output of all specific defects is forcibly set to zero. This strictly ensures the engineering logic of "not reporting specific defects in non-defect states" at the decision-making level, avoiding any possible misleading output. Step S603 stipulates that only when the overall state is clearly determined to be "defect," the system outputs one or more specific defect labels such as "cracking," "hollowing out," or "aging" based on the constraints and corrected defect probability vector, through threshold comparison and other methods. These three decision-making steps are interconnected, ultimately transforming the probability output of the neural network into clear, conflict-free engineering conclusions that can be directly used for maintenance decisions (such as "support intact," "image unclear, needs re-inspection," "support defects: cracking and aging"), greatly improving the end-user usability of the method.

[0069] In summary, this invention provides an end-to-end, automated, and highly reliable intelligent solution for identifying bridge rubber bearing defects, from image input to defect reporting. This method creatively cascades target detection and multi-label classification technologies and deeply integrates several innovative mechanisms tailored to this specific engineering problem: adaptive expansion and pruning to suppress background interference; a two-layer network structure and coupled computation to ensure logical consistency; physical prior loss and inference correction to improve the rationality of results; and cross-task quality constraints to enhance system robustness. Compared with existing technologies, this solution not only significantly improves the accuracy of bearing location and defect identification in complex environments, but more importantly, its output strictly conforms to the actual logic and physical laws of engineering, fundamentally solving the core pain points of existing methods such as easy output conflicts and poor engineering usability. This method provides an efficient and reliable technical tool for large-scale, routine, and intelligent inspection of bridge bearings, possessing significant engineering application value and market prospects.

[0070] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0071] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0072] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0073] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0074] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0076] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0077] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0079] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying defects in bridge rubber bearings based on target detection and multi-label classification, characterized in that, The method includes: Obtain the original image of the bridge rubber bearing; Using a trained object detection network, the rubber supports in the original image are extracted to obtain multiple prediction boxes; Based on the prediction boxes, the rubber support images corresponding to each prediction box are extracted from the original image; The rubber bearing image is classified using a trained disease classification network. The disease classification network is configured to output an overall state vector and a disease label vector simultaneously. Each component of the overall state vector represents a mutually exclusive overall state of the bearing, and each component of the disease label vector independently represents the probability of existence of different disease categories. The overall state vector and the disease label vector are coupled and calculated to obtain the final identification result of the rubber bearing.

2. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 1, characterized in that, The target detection network outputs multiple predicted bounding boxes and their confidence scores; The method further includes: Based on the confidence level and at least one preset quality rule, the multiple prediction boxes are filtered to obtain the filtered prediction boxes. The step of extracting the rubber support image corresponding to each prediction box from the original image based on the prediction box is performed based on the filtered prediction boxes.

3. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 2, characterized in that, Based on the confidence level and at least one preset quality rule, the multiple prediction boxes are filtered, including at least one of the following steps: Based on the preset confidence threshold, prediction boxes with a confidence level greater than or equal to the confidence threshold are initially selected. Non-maximum suppression is performed on the initial screening of the predicted boxes to remove overlapping redundant predicted boxes. Based on the comparison between the short side length of the predicted bounding box and the preset size threshold, predicted bounding boxes with a short side length less than the size threshold are filtered out. Based on the comparison between the proportion of the predicted bounding box in the original image and a preset proportion threshold, predicted bounding boxes with a proportion less than the proportion threshold are filtered out.

4. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 1, characterized in that, Based on the predicted bounding boxes, the rubber bearing images corresponding to each predicted bounding box are extracted from the original image, including: The prediction box is enlarged by a preset expansion factor to obtain an expanded box, wherein the expansion factor is calculated by the following formula: ， In the formula, γ is the expansion coefficient, γ min and γ max These are the minimum and maximum values ​​of the preset expansion coefficients, respectively, and s is the confidence level of the predicted box; Based on the expanded frame, the corresponding rubber support image is extracted from the original image.

5. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 1, characterized in that, The overall state vector includes at least three components: intact state, uncertain state, and diseased state. The disease label vector includes at least three components: cracking, hollowing, and aging.

6. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 1, characterized in that, The overall state vector and the defect label vector are coupled and calculated to obtain the final identification result of the rubber bearing, including: The overall state is associated with the disease label using the following formula to obtain the constrained disease output probability: ， ， ， In the formula, [P A ,P B ,P C ] represents the disease label vector, P D This refers to the disease state component in the overall state vector. The output probability of the disease after constraint is given, with subscripts A, B, and C corresponding to the three components of cracking, hollowing, and aging, respectively.

7. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 6, characterized in that, The total loss function of the disease classification network during training is: ， ， In the formula, L base For the mixed loss term with class weights, L imp λ is the consistency loss term obtained by constructing an asymmetric prior relation based on the degradation law of rubber bearings, where λ is the weighting coefficient.

8. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 6, characterized in that, The final identification result of the rubber bearing is obtained by coupling the overall state vector with the disease label vector. This also includes: The constrained disease output probability is adjusted for consistency using the following formula: 。 9. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 1, characterized in that, The final identification result of the rubber bearing is obtained by coupling the overall state vector with the disease label vector. This also includes: Based on the width, height, confidence level, and preset scale normalization threshold of the predicted bounding box, the quality index of the region of interest is obtained. Based on the quality index of the region of interest and the uncertain state components in the overall state vector, the corrected uncertain state components are obtained.

10. The method for identifying bridge rubber bearing defects based on target detection and multi-label classification according to claim 5, characterized in that, The overall state vector and the disease label vector are coupled and calculated to obtain the final identification result of the rubber bearing. The calculation also includes the following decision steps: If the intact state component value in the overall state vector is the largest, then the rubber support is determined to be in an intact state, and the probability of the defect output after all constraints is zero. If the value of the uncertain state component in the overall state vector is the largest, then the rubber support is determined to be in an uncertain state, and the probability of the defect output after all constraints is zero. If the disease state component value in the overall state vector is the largest, then based on the disease output probability after each constraint, the specific disease label is determined and output.

Images