Image recognition method, system and equipment for bridge plate-type rubber support

Abstract

The invention provides an image recognition method, system and equipment for a bridge plate-type rubber support. The image recognition method comprises the following steps: shooting an image with the bridge plate-type rubber support; the image is input into a semantic segmentation model, a segmentation mask of the image is obtained, and the segmentation mask is provided with a plurality of mask areas including the bridge plate type rubber support; screening the plurality of mask areas by adopting a multi-stage screening method, and positioning the area position of the bridge plate-type rubber support in the image; the multi-stage screening method comprises the steps of spatial position screening, shape information screening, texture feature screening, color information screening and relative spatial position screening. The sensitivity of a single feature is reduced through a multi-stage screening method, the recognition accuracy of the bridge plate-type rubber support under the complex working conditions of uneven illumination, local shielding, background interference and the like is improved, the recognition effect in the complex environment is further improved in combination with auxiliary recognition of a manual click mechanism, and the burden of manual recognition is reduced.

Description

Technical Field

[0001] This invention relates to an image recognition method, specifically, to an image recognition method, system, and device for bridge plate rubber bearings. Background Technology

[0002] In bridge structures, plate rubber bearings serve as critical load-bearing components, responsible for force transmission and support between the superstructure and substructure. Damage to or failure of these bearings can lead to cracking of the bridge deck and damage to the piers, thus affecting the overall structural safety of the bridge. Therefore, timely and accurate identification and detection of defects in plate rubber bearings are crucial for ensuring the long-term stable operation of bridges.

[0003] With the widespread application of image recognition technology in the detection of defects in building and bridge components, image processing-based defect detection of bridge slab rubber bearings has gradually become a hot research and practical trend. Since computers cannot directly identify the location of bridge slab rubber bearings like the human eye, accurate identification of these bearings is essential before defect detection and classification. Without this identification, subsequent defect classification and damage quantification cannot be performed. Therefore, bridge slab rubber bearing identification is not only a prerequisite for defect detection but also a crucial step in improving the automation level and reliability of the detection process.

[0004] In recent years, some progress has been made in the field of bridge plate rubber bearing identification by introducing deep learning technology. Although the fully automatic identification method based on deep learning technology has significant effects in specific scenarios, the identification capability of the fully automatic identification method depends on the data quality. Bridge plate rubber bearings often face problems such as occlusion, uneven lighting and background interference, making it difficult to accurately identify bridge plate rubber bearings in complex scenarios, which has certain limitations. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a semi-automatic image recognition method, system, and device for bridge plate rubber bearings that integrates multi-level screening and manual selection. This invention aims to solve the problem of inaccurate identification of bridge plate rubber bearings under complex conditions such as uneven lighting and partial occlusion by screening multiple features of the bearings, including shape, texture, color, and spatial relationships. Furthermore, the invention combines manual selection to compensate for the deficiencies of automated methods in complex scenarios, effectively improving recognition accuracy and robustness.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The first aspect of this invention provides an image recognition method for bridge plate rubber bearings, comprising the following steps: Images of bridge-type rubber bearings were captured. The image is input into a semantic segmentation model to obtain a segmentation mask for the image. The segmentation mask has several mask regions, including the bridge plate rubber bearing. A multi-level filtering method is used to filter the several mask regions to obtain the mask region to which the bridge plate rubber bearing belongs, that is, to locate the location of the bridge plate rubber bearing region in the image. The multi-level filtering method includes first performing a rapid coarse screening, then performing a fine screening, and finally performing a relative screening. The rapid coarse screen includes: Spatial location filtering: calculate the contact amount between each mask region and the image edge; if a mask region is in contact with the image edge, then filter out the mask region with that contact edge. Shape information filtering involves extracting the outline of each mask region, calculating the aspect ratio of the mask region, filtering out mask regions whose shape and aspect ratio conform to the characteristics of bridge plate rubber bearings, and filtering out mask regions that do not meet the requirements. The fine-grained screening includes: Texture feature filtering: Extract texture features from the image, filter out masked regions whose texture features meet the preset threshold range, and filter out masked regions whose texture features do not meet the characteristics of bridge plate rubber bearings. Color information filtering involves calculating the average RGB channel value of each mask region in the image, filtering out mask regions whose RGB average value meets the preset threshold range, and removing mask regions whose RGB channels do not conform to the color characteristics of bridge plate rubber bearings. The relative filtering includes: Relative spatial location filtering extracts the relative spatial location features between masked regions, filters out regions whose relative spatial location features do not meet the preset filtering rules or thresholds, and selects masked regions that meet the relative spatial location features of bridge plate rubber bearings.

[0007] The multi-level screening method first performs rapid coarse screening of several mask regions through spatial location screening and shape information screening to eliminate a large number of interference regions that do not meet expectations, thereby reducing the computational load of fine screening and relative screening. Then, it further refines the positioning and screening of mask regions through texture feature screening and color information screening, which have relatively high computational complexity. Finally, it locates the mask region to which the bridge plate rubber bearing belongs through relative spatial location screening.

[0008] As a preferred technical solution of the present invention, the specific steps of the shape information filtering include: Polygon approximation is performed on the contours of the masked region with 3 to 5 vertices to obtain new vertices as polygonal contours. Then, the masked regions with quadrilateral shapes that are close to the bridge plate rubber bearing are selected from them by quadrilateral filtering. Calculate the minimum bounding rectangle width and height of the selected mask region, as well as the aspect ratio, and select mask regions with aspect ratios between 1 and 6; The expression for calculating the width of the minimum bounding rectangle is: w = max(x) i )-min(x i ); The expression for calculating the height of the minimum bounding rectangle is: h = max(y i )-min(y i ); The expression for calculating the aspect ratio is: AR (Aspect Ratio) = max(w / h, h / w); Where, x i Let y be the row coordinate of the i-th pixel in the mask region. i Let be the column coordinates of the i-th pixel in the mask region.

[0009] During the polygon approximation and quadrilateral filtering process, partially occluded bridge slab rubber bearings can still be accurately identified; by statistically analyzing the aspect ratio of the bridge slab rubber bearing area, an aspect ratio screening threshold range is set to effectively remove non-bridge slab rubber bearing areas with abnormal aspect ratios.

[0010] As a preferred embodiment of the present invention, the specific steps of the relative spatial position screening include: Dilatation is performed on each mask region. If there are overlapping areas between masks, the centroid ordinate is calculated, and the mask region with the smallest centroid ordinate is retained. The expression for calculating the centroid's ordinate is: ; Among them, C y It is the ordinate of the centroid, x i It is the x-coordinate of the pixel in the mask region, y i It is the ordinate of the pixel in the mask region. It is the sum of the x-coordinates of all pixels within the mask area. It is the sum of the products of the horizontal and vertical coordinates of all pixels; If there is no overlap between mask regions, select the two masks with the lowest ordinates, calculate the difference in their centroid ordinates and the area ratio. If the difference in their centroid ordinates is less than a set threshold of 30 and the area ratio is less than a set threshold of 2, then both masks are retained. Otherwise, only the mask with the largest area is retained as the final result. If the image has only one mask, that mask is directly retained. If there is no mask information in the image, no mask is retained. The expression for calculating the difference in the centroid's ordinate is: ΔC y =|C y1 -C y2 |; The formula for calculating the area ratio is: Area Ratio = max(A1 / A2, A2 / A1). Among them, C y1 and C y2 These are the centroid ordinates of the two masks, ΔC. y A1 represents the difference in the centroid ordinates of the two masks, and A2 represents the area of ​​the two masks, i.e., the number of pixels within the region.

[0011] By analyzing the relative spatial characteristics between the bridge slab rubber bearing and the pad stone and irregularly shaped stones, the pad stone masking area located directly below the bridge slab rubber bearing, as well as the irregularly shaped stone masking area, are filtered out.

[0012] As a preferred technical solution of the present invention, the specific steps of the spatial location screening include: Calculate the amount of contact between each mask region and the top, bottom, left, or right edge of the image; If any one of the top edge contact amount, bottom edge contact amount, left edge contact amount, or right edge contact amount is greater than 0, it indicates that the mask area is in contact with the image edge. If and only if all four parameters of the top edge contact amount, bottom edge contact amount, left edge contact amount, and right edge contact amount are 0, it indicates that the mask area is completely inside the image.

[0013] Bridge plate rubber bearings, as the primary identification target, typically do not contact the edges of the captured image; even if they do, it's because the image may contain multiple bridge plate rubber bearings, and the portion in contact with the edge is usually not part of the main identification target. By filtering the spatial location of masked areas in the image, masked areas in contact with the image edges can be effectively filtered out.

[0014] As a preferred embodiment of the present invention, the specific steps of color information filtering include: The mean value of the RGB channels of each mask region in the image is calculated and compared with a pre-set threshold range. The mean value of the R channel is 22 to 162, the mean value of the G channel is 22 to 162, and the mean value of the B channel is 17 to 137. When the mean value of all RGB channels is within the set range, the mask is a mask region that conforms to the color characteristics of bridge plate rubber bearings.

[0015] By statistically analyzing the color mean distribution of bridge plate rubber bearings, a selection threshold range for the average pixel values ​​of the R, G, and B channels was set, effectively removing non-bridge plate rubber bearing areas with abnormal color characteristics.

[0016] As a preferred embodiment of the present invention, the texture feature filtering includes: The gray-level co-occurrence matrix method is used to extract texture features of the image, including contrast, dissimilarity, consistency, energy, correlation, and angular second moment. The values ​​of the six key texture features are then compared with pre-set threshold ranges. The threshold ranges for contrast feature are 1.269 to 90.080, dissimilarity feature is 0.052 to 1.454, consistency feature is 0.868 to 0.995, energy feature is 0.834 to 0.994, correlation feature is 0.791 to 0.986, and angular second moment is 0.696 to 0.988.

[0017] By statistically analyzing the numerical distribution range of six key texture features of bridge plate rubber bearings, a screening threshold range for the six key texture features was set, effectively removing non-bridge plate rubber bearing areas whose texture features differ significantly from those of bridge plate rubber bearings.

[0018] As a preferred technical solution of the present invention, when performing multi-level screening, the candidate mask areas are first screened by spatial location features, then by shape features to roughly screen out the candidate mask areas, then by texture features to further filter the candidate mask areas, then by color features to finely screen the candidate mask areas, and finally by relative spatial location information to screen out the mask areas of bridge plate rubber bearings.

[0019] As a preferred technical solution of the present invention, after multi-level screening, it also includes manual selection to assist in identification. The manual selection is performed on the area of ​​the bridge plate rubber bearing that is difficult to be automatically screened in the complex environment image. The corresponding mask area is matched according to the manual selection position to supplement and improve the accuracy of the multi-level screening method.

[0020] When bridge plate rubber bearings are severely obstructed, lack sufficient lighting, or suffer from severe damage, making it impossible to automatically and accurately identify the bridge plate rubber bearing area in complex environment images, manual point selection can be used as an auxiliary method to improve the recognition performance in complex environment images.

[0021] A second aspect of the present invention provides an image recognition system for bridge plate rubber bearings, comprising: The image acquisition module captures images of bridge plate rubber bearings. The feature extraction module inputs the image into the semantic segmentation model to obtain the segmentation mask of the image, the segmentation mask having several mask regions including the bridge plate rubber bearing; The image recognition module uses a multi-level filtering method to filter the several mask regions to obtain the mask region to which the bridge plate rubber bearing belongs, that is, to locate the location of the bridge plate rubber bearing region in the image. The multi-level filtering method includes first performing a rapid coarse screening, then performing a fine screening, and finally performing a relative screening. The rapid coarse screen includes: Spatial location filtering: calculate the contact amount between each mask region and the image edge; if a mask region is in contact with the image edge, then filter out the mask region with that contact edge. Shape information filtering involves extracting the outline of each mask region, calculating the aspect ratio of the mask region, filtering out mask regions whose shape and aspect ratio conform to the characteristics of bridge plate rubber bearings, and filtering out mask regions that do not meet the requirements. The fine-grained screening includes: Texture feature filtering: Extract texture features from the image, filter out masked regions whose texture features meet the preset threshold range, and filter out masked regions whose texture features do not meet the characteristics of bridge plate rubber bearings. Color information filtering involves calculating the average RGB channel value of each mask region in the image, filtering out mask regions whose RGB average value meets the preset threshold range, and removing mask regions whose RGB channels do not conform to the color characteristics of bridge plate rubber bearings. The relative filtering includes: Relative spatial location filtering extracts the relative spatial location features between masked regions, filters out regions whose relative spatial location features do not meet the preset filtering rules or thresholds, and selects masked regions that meet the relative spatial location features of bridge plate rubber bearings.

[0022] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image recognition method for bridge plate rubber bearings described in the first aspect.

[0023] This invention has outstanding substantive features and significant progress compared to the prior art. Specifically, this invention, through statistical analysis of multiple features such as the shape, texture, color, and spatial relationship of bridge plate rubber bearings, sets corresponding feature screening thresholds. By reducing the sensitivity of a single feature, it improves the accuracy of identifying bridge plate rubber bearings under complex working conditions such as uneven lighting, partial occlusion, and background interference. Combined with the auxiliary identification mechanism of manual selection, it further improves the identification effect in complex environments and reduces the burden of manual identification. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a numerical distribution diagram of the length-to-width ratio of the bridge plate rubber bearing of the present invention; Figure 3 This is a numerical distribution diagram of the bridge plate rubber bearing of the present invention on six key texture features; Figure 4 This is a color mean distribution diagram of the bridge plate rubber bearing of the present invention; Figure 5 This is a diagram showing the results of stages (a)-(d) of the multi-level screening and identification process of this invention; Figure 6 This is a diagram showing the results of stages (e)-(h) of the multi-level screening and identification process of this invention; Figure 7 This is a result image of manual point selection supplementary identification in the scenario of the bridge plate rubber bearing with steel plate of the present invention; Figure 8 The image shows the result of manual selection and supplementary identification in a scenario where the bridge plate rubber bearing of the present invention is severely obstructed. Figure 9 The result of manual selection and supplementary recognition in a dimly lit environment according to the present invention; Figure 10 A schematic diagram illustrating the principle of manual selection in this invention. Detailed Implementation

[0025] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0026] To facilitate understanding of the technical solutions provided in this application, the technical terms involved in the embodiments of this application are explained below.

[0027] SAM (Segment Anything Model) is a general-purpose artificial intelligence model released by Meta in April 2023, focusing on image segmentation tasks. Its technical architecture employs an encoder-decoder design, fusing image features with cue information (such as points, boxes, and text) to achieve efficient segmentation of arbitrary objects. SAM uses a three-stage architecture: Image encoder: Extracts image features based on Vision Transformer (ViT); Prompt encoder: Handles interactive prompts for user input (such as clicks, selections); Mask decoder: Generates high-quality segmentation masks and evaluates confidence levels.

[0028] Example 1 In this embodiment of the application, in order to identify bridge plate rubber bearings, an image recognition method for bridge plate rubber bearings is proposed: First, SAM (Segment Anything Model) is used to perform multi-target segmentation on the bridge image, including the supports, to initially obtain the supports and surrounding areas. To address oversegmentation, a region merging algorithm is used to eliminate unnecessary small areas within the supports, ensuring the integrity of the support structure. Based on this, a multi-layered filtering mechanism is constructed: in the spatial dimension, segmented areas located at the image edges are removed based on the actual spatial distribution of the supports; in the shape dimension, polygon contour approximation and aspect ratio thresholding are used as dual constraints to retain areas that conform to the support shape characteristics; in the texture and color dimensions, Gray-Level Co-occurrence Matrix (GLCM) texture features and RGB three-channel statistical features are extracted, and non-support texture and color interference areas are excluded using preset thresholds. Furthermore, to address feature differences across different images, a relative filtering mechanism is introduced to more accurately identify the support areas. Finally, a manual interactive point-and-click function is designed as a supplementary strategy. When automatic identification encounters severe occlusion or abnormal lighting interference, manual point-and-clicking of target locations can quickly correct missed and false detection areas.

[0029] Specifically, such as Figure 1 As shown, the image recognition method for bridge plate rubber bearings includes the following steps: Step S1: Capture an image dataset containing bridge plate rubber bearings.

[0030] Specifically, photographs of bridge plate rubber bearings at the bridge inspection site were collected regularly to construct a dataset of bridge plate rubber bearings. The data acquisition equipment used professional cameras to ensure high-resolution image quality. The dataset contains 400 images of bridge plate rubber bearings of various types, covering a variety of complex environments and visual conditions, including the following situations: Regarding environmental interference, some images were difficult to analyze due to cluttered environments and a lot of gravel around the bridge plate rubber bearings. Regarding lighting conditions, some images were poorly lit, and the color of the bridge plate rubber bearing was similar to the background, which affected the contrast and detail of the images. Regarding the integrity of the viewpoint, some images only show part of the structure of the bridge plate rubber bearing due to limited viewpoint or other occlusion factors, making it impossible to fully observe its condition. Regarding structural damage, some images also show severe damage to bridge slab rubber bearings. The damage or corrosion of the bridge slab rubber bearings is quite obvious, reflecting structural problems that may occur during long-term use.

[0031] Step S2: Input the image into the semantic segmentation model to obtain the segmentation mask of the image. The segmentation mask has several mask regions, including the bridge plate rubber bearing.

[0032] Specifically, the semantic segmentation model can use the Segment Anything Model (SAM model) to perform multi-target image segmentation on bridge images, including bridge plate rubber bearings. After the image is input into the SAM model, the SAM model automatically segments the image into multiple regions, including bridge plate rubber bearings, and initially obtains potential targets in and around the bridge plate rubber bearings.

[0033] When segmenting images, you can choose to use the default parameters of the SAM model to automatically generate segmentation masks, ensuring accuracy when processing multiple images on a large scale. Although adjusting the segmentation parameters individually can improve the image, due to the diversity of datasets, parameter adjustments may be effective for some images but have a negative impact on others.

[0034] Step S3: Merge the over-segmented mask regions inside the support.

[0035] In some images after multi-target image segmentation, over-segmentation occurs in the support region, meaning that multiple small masked regions exist within the support, which are actually part of the support. To further optimize the segmentation effect, these over-segmented masked regions need to be merged. The core idea of ​​this merging process is to calculate the intersection and area between the masks and, according to certain merging conditions, merge similar masks into a larger, more accurate region.

[0036] However, the support may be divided into areas with and without steel plates due to the presence of steel plates, leading to ambiguity during merging. To avoid incorrectly merging the support body and the area with steel plates, merging parameters are set so that the two masks are only merged into a new mask when the intersection of the two mask areas completely covers one of the masks themselves and the area ratio of the two is greater than 2. This effectively merges small areas while ensuring accurate identification of the support body.

[0037] Step S4: Use a multi-level automatic filtering method to filter the several mask regions to obtain the mask region to which the bridge plate rubber bearing belongs, that is, locate the location of the bridge plate rubber bearing region in the image.

[0038] The multi-level screening method includes first performing rapid coarse screening, then fine screening, and finally relative screening; the rapid coarse screening includes spatial location screening and shape information screening; the fine screening includes texture feature screening and color information screening; and the relative screening includes relative spatial location screening.

[0039] In the multi-level screening process, to determine appropriate thresholds for aspect ratio, texture features, and color features, the support regions in the dataset were first manually selected, and statistical analysis was performed on the aspect ratio, texture features, and color features of these regions, such as... Figures 2 to 4 As shown. Based on these statistics, corresponding feature thresholds were set, and only features of the mask that fall within this range will be retained.

[0040] Filter areas whose spatial location touches the image edge Generally, the support, being the primary subject of the photograph, will not touch the edges of the image; even if it does, it's because the image may contain multiple supports, and the portion touching the edge is usually not part of the main research area. Therefore, the spatial location of the supports in the image can be used for filtering, effectively eliminating masked areas that touch the image edges.

[0041] Specifically, it is necessary to calculate the amount of contact between each mask region and the image edge, and determine whether each mask region is in contact with the top, bottom, left, or right edge of the image. If the mask region is in contact with the image edge, the mask region with that contact edge is filtered out.

[0042] The pixel matrix expression for the masked region is: ; The expression for calculating the top edge contact amount is: ; The formula for calculating the bottom edge contact amount is: ; The expression for calculating the contact amount at the left edge is: ; The expression for calculating the contact amount at the right edge is: ; Where H is the height of the image, W is the width of the image, i is the row index of the image, in the range i∈[0,H-1], j is the column index of the image, in the range j∈[0,W-1], M is the second-order matrix of the mask region, and M(i,j) represents the pixel value of the mask in the image; If any one of the top edge contact amount, bottom edge contact amount, left edge contact amount, or right edge contact amount is greater than 0, it indicates that the mask area is in contact with the image edge. If and only if all four parameters of the top edge contact amount, bottom edge contact amount, left edge contact amount, and right edge contact amount are 0, it indicates that the mask area is completely inside the image.

[0043] When filtering by spatial location, the goal is to filter out masked regions that touch the image edges. However, due to gaps between masks during segmentation, some regions may visually touch the image edges, but in reality, the mask and the image edges are not perfectly aligned. By expanding the filtering range and adjusting parameters to check the contact between the masked region and the image edges in the ten rows or ten columns before and after it, the contact situation can be more accurately determined, avoiding the omission of edge regions.

[0044] Filter areas where the shape information differs significantly from that of the support. In image processing, a plate rubber bearing should ideally appear as a quadrilateral region with four vertices. However, due to potential image quality issues caused by noise or occlusion, some vertices of the bearing may not be accurately identified, or the bearing itself may be partially occluded, resulting in a shape with fewer than four vertices. By extending the quadrilateral filtering—that is, for masked regions with 3 to 5 vertices—extracting the contour of each segmentation mask and approximating the contour with polygons, masked regions that approximate a quadrilateral shape are selected.

[0045] In the process of expanding quadrilateral filtering, polygons with 3 to 5 vertices are retained to ensure that partially occluded supports can still be accurately identified. For some supports that appear hexagonal due to severe occlusion, a more lenient polygon approximation method is adopted by increasing the polygon contour approximation accuracy parameter, for example, setting the contour perimeter to 2%, thereby avoiding the incorrect exclusion of bridge plate rubber bearings under occlusion conditions.

[0046] Aspect ratio analysis is performed on the mask region after extended quadrilateral filtering. Figure 2 The image shows the aspect ratio distribution of bridge plate rubber bearings. 99% of the aspect ratio values ​​are within 6, and only a very small number of images have aspect ratios that exceed this range, indicating that the aspect ratio of bridge plate rubber bearings shows a relatively consistent distribution in the overall sample.

[0047] Specifically, first calculate the width and height of the minimum bounding rectangle for each profile, and then calculate the aspect ratio based on the width and height of the rectangle. Finally, filter out the mask areas with aspect ratios between 1 and 6 to effectively remove non-bridge plate rubber bearings with abnormal aspect ratios.

[0048] The expression for calculating the width of the minimum bounding rectangle is: w = max(x) i )-min(x i ); The expression for calculating the height of the minimum bounding rectangle is: h = max(y i )-min(y i ); The expression for calculating the aspect ratio is: AR (Aspect Ratio) = max(w / h, h / w); Where, x i Let y be the row coordinate of the i-th pixel in the mask region. i Let be the column coordinates of the i-th pixel in the mask region.

[0049] Filter areas where the texture information differs significantly from that of the support. The Gray-Level Co-occurrence Matrix (GLCM) analysis method is used to extract texture features from the image to assist in the selection of support regions. The GLCM is constructed based on the spatial relationship between adjacent pixels in the image. It obtains multiple texture features by calculating the joint gray-level distribution of pixels at different distances and directions. These features can effectively reflect the texture characteristics of the image. Figure 3 The numerical distribution ranges of six key texture features for bridge plate rubber bearings are presented. These features include contrast, dissimilarity, consistency, energy, correlation, and angular second moment. Specifically, the threshold ranges for contrast are 1.269 to 90.080, dissimilarity is 0.052 to 1.454, consistency is 0.868 to 0.995, energy is 0.834 to 0.994, correlation is 0.791 to 0.986, and angular second moment is 0.696 to 0.988.

[0050] Specifically, the gray-level co-occurrence matrix method is used to extract the texture features of the image. The maximum and minimum values ​​of the numerical distribution of the six key texture features are used as the detection threshold. The texture features of the masked region are then compared with the detection threshold to select masked regions whose texture features meet the preset threshold range. Masked regions whose texture features do not meet the characteristics of bridge plate rubber bearings are filtered out.

[0051] The expression for calculating contrast is: ; The expression for calculating dissimilarity is: ; The expression for calculating consistency is: ; The expression for calculating energy is: ; The expression for calculating correlation is: ; The expression for calculating the second moment of an angle is: ; Where i and j represent the row and column indices of the image, respectively, and represent the gray values ​​of two pixels; P(i, j) represents the normalized probability that a pixel with gray level i and a pixel with gray level j co-occur in the image under a specific spatial relationship (such as direction and distance); μ x μ y These represent the average grayscale values ​​in the row and column directions, respectively; σx , σ y These are the standard deviations of the rows and columns, respectively.

[0052] Filter areas where the color information differs significantly from that of the support. Since the support area usually exhibits a gray tone, color information analysis is introduced for screening in order to exclude non-support areas that are predominantly black or pure white. Figure 4 This reflects the mean color distribution of bridge plate rubber bearings. Statistical analysis shows that the color distribution of bridge plate rubber bearings is not concentrated in extreme black (close to 0) or white (close to 255) areas, but is relatively uniform, indicating that the color of bridge plate rubber bearings has a certain stability in different images. The mean value ranges from 22 to 162 for the R channel, 22 to 162 for the G channel, and 17 to 137 for the B channel.

[0053] Specifically, the mean RGB channel values ​​of each masked region in the image are calculated, including the average pixel values ​​of the red (R), green (G), and blue (B) channels of each masked region in each image; masked regions whose R, G, and B channels meet the preset mean range are selected, and masked regions whose RGB channels do not conform to the color characteristics of bridge plate rubber bearings are filtered out.

[0054] The expression for calculating the average pixel value of the red channel is: ; The expression for calculating the average pixel value of the green channel is: ; The formula for calculating the average pixel value of the blue channel is: ; Among them, avg r avg g avg b These represent the average pixel values ​​of the mask region in the R, G, and B channels, respectively, where N is the total number of pixels in the mask region (i.e., the mask area), and R... i G i B i These are the intensity values ​​of the i-th pixel in the R, G, and B channels, respectively, within the mask area.

[0055] Image ordinate relative spatial position filtering The pad bearing area is typically located directly beneath the bridge slab rubber bearing. To address the relative spatial relationship between the bridge slab rubber bearing and the pad bearing, each mask is first expanded to strengthen its boundary. Then, bitwise operations are used to detect overlapping areas between the masks. If overlapping areas exist between the masks, their centroid ordinates are calculated, and the mask area with the smallest centroid ordinate is retained, thus completing the accurate selection of the pad bearing area, which is typically located directly beneath the bridge slab rubber bearing.

[0056] The expression for calculating the centroid's ordinate is: ; Among them, C y It is the ordinate of the centroid, x i It is the x-coordinate of the pixel in the mask region, y i It is the ordinate of the pixel in the mask region. It is the sum of the x-coordinates of all pixels within the mask area. It is the sum of the products of the horizontal and vertical coordinates of all pixels; For irregularly shaped stone areas scattered around bridge plate rubber bearings, where there is no overlap between the mask areas, filtering can be performed using relative spatial location and area information. Specifically, the segmented regions of all masks in each image are extracted, and their area and centroid ordinate are calculated. Then, they are sorted in ascending order of centroid ordinate, prioritizing the selection of the two masks with the lowest ordinates. The centroid difference and area ratio of the two masks are then calculated. If the difference in centroid ordinates is less than a set threshold of 30 and the area ratio is less than a set threshold of 2, both masks are retained. Otherwise, only the mask with the largest area is retained as the final result. If the image has only one mask, that mask is directly retained. If the image has no mask information, no mask is retained.

[0057] The expression for calculating the difference in the centroid's ordinate is: ΔC y =|C y1 -C y2 |; The formula for calculating the area ratio is: Area Ratio = max(A1 / A2, A2 / A1); Among them, C y1 and C y2 These are the centroid ordinates of the two masks, ΔC. y A1 represents the difference in the centroid ordinates of the two masks, and A2 represents the area of ​​the two masks, i.e., the number of pixels within the region.

[0058] It should be noted that when performing multi-level filtering, the following order should be followed: first, quick coarse filtering; then, fine filtering; and finally, relative filtering. The multi-level filtering process ends after all four filtering methods—spatial location filtering, shape information filtering, texture feature filtering, color information filtering, and relative spatial location filtering—have been performed once.

[0059] In some exemplary embodiments, when performing multi-level filtering, the following preferred filtering order can be adopted: First, spatial location filtering is used to initially filter out masked areas that are in contact with the image edges, reducing the amount of subsequent calculations. Secondly, a quick coarse screening is performed by filtering shape information to eliminate a large number of interference areas whose shape features do not meet expectations, and to screen out candidate mask areas. The candidate mask regions are further filtered by texture feature screening to remove mask regions whose texture features do not meet expectations. Then, color information filtering is used to finely filter the candidate mask regions, and mask regions whose RGB channel mean does not meet expectations are filtered out. Finally, relative spatial location filtering is used to finely filter the image internally, in order to exclude and retain only a few remaining candidate regions, thus achieving the final identification of bridge plate rubber bearings.

[0060] Step S5: Manually click to select the support region of the segmented image.

[0061] Due to factors such as severe shading, insufficient sunlight, or damage, multi-level screening methods may not be able to automatically and accurately identify the areas of bridge plate rubber bearings. In such cases, manual selection is necessary to assist in identification. Figure 10 As shown, through an interactive image display and selection interface, users can click on the area of ​​the bridge plate rubber bearing in the segmented image using the mouse. The program handles mouse events through callback functions and matches the corresponding mask area based on the click location. For the selected area, the program extracts the outline of the area and draws it on the original image to highlight the area of ​​the bridge plate rubber bearing. After image processing is complete, the image with the outline is saved to a specified directory for recognition rate statistics.

[0062] Verification Experiment In this validation experiment, the evaluation of the recognition performance for bridge plate rubber bearing regions was based on four main evaluation metrics: TP (True Positive), FP (False Positive), TN (True Negative), and FN (False Negative). TP represents regions correctly identified as bridge plate rubber bearings; FP represents regions incorrectly identified as bridge plate rubber bearings but as background; FN represents regions incorrectly identified as background but as background; and TN represents regions correctly identified as background. Based on these fundamental values, precision, recall, accuracy, F1 score, and overall accuracy (OA) were further calculated.

[0063] Precision measures the proportion of areas predicted as bridge plate rubber bearings that are actually bridge plate rubber bearings, reflecting the extent of misidentification. The formula for calculating precision is: Precision = TP / (TP + FP); Recall measures the proportion of all actual bridge plate rubber bearing areas that are correctly identified as bridge plate rubber bearings, reflecting the degree of missed detection. The formula for calculating recall is: Recall = TP / (TP + FN); Precision comprehensively evaluates the proportion of correctly identified regions out of the total regions; F1 score, on the other hand, is a harmonic mean of precision and recall, and is particularly suitable for evaluating imbalanced datasets. The expression for calculating the F1 score is: F1=(2×Recall×Precision) / (Recall+Precision); Overall accuracy is used to evaluate the algorithm's overall performance across all regions. A comprehensive analysis of these metrics allows for a thorough evaluation of the accuracy and reliability of the bridge slab rubber bearing region identification algorithm. The formula for calculating overall accuracy is: Overall Accuracy = (TP + TN) / (TP + FN + FP + TN); Table 1. Location Identification Results of Bridge Plate Rubber Bearings

[0064] The dataset of 400 images was divided into two parts based on environmental complexity (200 images for standard environments and 200 images for complex environments). As shown in Table 1, in standard environment images with good lighting and intact structures, the proposed multi-level screening method effectively identified 163 pedestal regions (TPs), with precision, recall, overall accuracy (OA), and F1 score of 0.815, 0.663, 0.700, and 0.731, respectively, reducing workload by approximately 70% compared to traditional manual bounding box selection. However, in complex environment images with severe occlusion, structural defects, or low lighting, the recognition performance deteriorated, with false positives (FP = 147) and false negatives (FN = 139), and precision and recall were only 0.226 and 0.236, respectively. By introducing manual selection to assist in recognition, the precision and recall rates were improved to 0.760 and 0.849, respectively. In complex scenarios, false positives and false negatives were effectively suppressed, enabling high recognition accuracy and stability in practical engineering, while significantly reducing the burden of manual recognition.

[0065] Figure 5 and Figure 6The results of each stage of the multi-level screening and recognition process are shown: (a) the original image; (b) the SAM model initially segmented multiple regions, but the segmented regions were numerous and fragmented; (c) through region merging, the recognition of the overall structure of the bridge plate rubber bearing was significantly enhanced; (d) through spatial location screening, some redundant regions located at the edge of the image were eliminated; (e)-(h) by combining four feature dimensions such as shape, texture, and color, the region screening logic was gradually optimized, and finally the effective recognition of the target region of the bridge plate rubber bearing was achieved.

[0066] Figure 7 , Figure 8 and Figure 9 The results of manual selection and supplementary identification are shown, among which, Figure 7 For the scenario of bridge plate rubber bearings with steel plates, the identification results are supplemented by manual selection. Figure 8 For scenarios where bridge plate rubber bearings are severely obscured, the identification results are supplemented by manual selection. Figure 9 For scenes with dim ambient light, the results of recognition are supplemented by manual selection.

[0067] Example 2 Embodiment 2 of the present invention provides an image recognition system for bridge plate rubber bearings, comprising: The image acquisition module captures images of bridge plate rubber bearings.

[0068] The feature extraction module inputs the image into the semantic segmentation model to obtain the segmentation mask of the image, the segmentation mask having several mask regions including the bridge plate rubber bearing.

[0069] The image recognition module uses a multi-level filtering method to filter the several mask regions to obtain the mask region to which the bridge plate rubber bearing belongs, that is, to locate the location of the bridge plate rubber bearing region in the image.

[0070] The multi-level screening method includes: Spatial location filtering: calculate the contact amount between each mask region and the image edge; if a mask region is in contact with the image edge, then filter out the mask region with that contact edge. Shape information filtering involves extracting the outline of each mask region, calculating the aspect ratio of the mask region, filtering out mask regions whose shape and aspect ratio conform to the characteristics of bridge plate rubber bearings, and filtering out mask regions that do not meet the requirements. Texture feature filtering: Extract texture features from the image, filter out masked regions whose texture features meet the preset threshold range, and filter out masked regions whose texture features do not meet the characteristics of bridge plate rubber bearings. Color information filtering involves calculating the average RGB channel value of each mask region in the image, filtering out mask regions whose RGB average value meets the preset threshold range, and removing mask regions whose RGB channels do not conform to the color characteristics of bridge plate rubber bearings. Relative spatial location filtering extracts the relative spatial location features between masked regions, filters out regions whose relative spatial location features do not meet the preset filtering rules or thresholds, and selects masked regions that meet the relative spatial location features of bridge plate rubber bearings.

[0071] Example 3 Embodiment 3 of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image recognition method for bridge plate rubber bearings described in Embodiment 1.

[0072] In an alternative embodiment, the electronic device further includes an input device and a display device, and is connected to a memory and a processor via a bus or other means.

[0073] Memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules. The processor executes various functional applications and data processing of electronic devices by running the software programs, instructions, and modules stored in memory. Input devices are used to receive images captured with bridge-plate rubber bearings, and display devices are used to output the processed results through displays and other devices.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. An image recognition method for bridge plate rubber bearings, characterized in that, Includes the following steps: Images of bridge-type rubber bearings were captured. The image is input into a semantic segmentation model to obtain a segmentation mask for the image. The segmentation mask has several mask regions, including the bridge plate rubber bearing. A multi-level filtering method is used to filter the several mask regions to obtain the mask region to which the bridge plate rubber bearing belongs, that is, to locate the location of the bridge plate rubber bearing region in the image. The multi-level filtering method includes first performing a rapid coarse screening, then performing a fine screening, and finally performing a relative screening. The rapid coarse screen includes: Spatial location filtering: Calculate the contact amount between each mask region and the image edge. If a mask region is in contact with the image edge, filter out the mask region with that contact edge. Shape information filtering involves extracting the outline of each mask region, calculating the aspect ratio of the mask region, filtering out mask regions whose shape and aspect ratio conform to the characteristics of bridge plate rubber bearings, and filtering out mask regions that do not meet the requirements. The fine-grained screening includes: Texture feature filtering: Extract texture features from the image, filter out masked regions whose texture features meet the preset threshold range, and filter out masked regions whose texture features do not meet the characteristics of bridge plate rubber bearings. Color information filtering involves calculating the average RGB channel value of each mask region in the image, filtering out mask regions whose RGB average value meets the preset threshold range, and removing mask regions whose RGB channels do not conform to the color characteristics of bridge plate rubber bearings. The relative filtering includes: Relative spatial location filtering extracts the relative spatial location features between masked regions, filters out regions whose relative spatial location features do not meet the preset filtering rules or thresholds, and selects masked regions that meet the relative spatial location features of bridge plate rubber bearings.

2. The image recognition method for bridge plate rubber bearings according to claim 1, characterized in that, The specific steps for filtering the shape information include: Polygon approximation is performed on the contours of the masked region with 3 to 5 vertices to obtain new vertices as polygonal contours. Then, the masked regions with quadrilateral shapes that are close to the bridge plate rubber bearing are selected from them by quadrilateral filtering. Calculate the minimum bounding rectangle width and height of the selected mask region, as well as the aspect ratio, and select mask regions with aspect ratios between 1 and 6; The expression for calculating the width of the minimum bounding rectangle is: w = max(x) i )-min(x i ); The expression for calculating the height of the minimum bounding rectangle is: h = max(y i )-min(y i ); The expression for calculating the aspect ratio is: AR (Aspect Ratio) = max(w / h, h / w); Where, x i Let y be the row coordinate of the i-th pixel in the mask region. i Let be the column coordinates of the i-th pixel in the mask region.

3. The image recognition method for bridge plate rubber bearings according to claim 1, characterized in that, The specific steps for relative spatial location filtering include: Dilatation is performed on each mask region. If there are overlapping areas between masks, the centroid ordinate is calculated, and the mask region with the smallest centroid ordinate is retained. The expression for calculating the centroid's ordinate is: ; Among them, C y It is the ordinate of the centroid, x i It is the x-coordinate of the pixel in the mask region, y i It is the ordinate of the pixel in the mask region. It is the sum of the x-coordinates of all pixels within the mask area. It is the sum of the products of the horizontal and vertical coordinates of all pixels; If there is no overlap between mask regions, select the two masks with the lowest ordinates, calculate the difference in their centroid ordinates and the area ratio. If the difference in their centroid ordinates is less than a set threshold of 30 and the area ratio is less than a set threshold of 2, then both masks are retained. Otherwise, only the mask with the largest area is retained as the final result. If the image has only one mask, that mask is directly retained. If there is no mask information in the image, no mask is retained. The expression for calculating the difference in the centroid's ordinate is: ΔC y =|C y1 -C y2 |; The formula for calculating the area ratio is: Area Ratio = max(A1 / A2, A2 / A1). Among them, C y1 and C y2 These are the centroid ordinates of the two masks, ΔC. y A1 represents the difference in the centroid ordinates of the two masks, and A2 represents the area of ​​the two masks, i.e., the number of pixels within the region.

4. The image recognition method for bridge plate rubber bearings according to claim 1, characterized in that, The specific steps for spatial location filtering include: Calculate the amount of contact between each mask region and the top, bottom, left, or right edge of the image; If any one of the top edge contact amount, bottom edge contact amount, left edge contact amount, or right edge contact amount is greater than 0, it indicates that the mask area is in contact with the image edge. If and only if all four parameters of the top edge contact amount, bottom edge contact amount, left edge contact amount, and right edge contact amount are 0, it indicates that the mask area is completely inside the image.

5. The image recognition method for bridge plate rubber bearings according to claim 1, characterized in that, The specific steps for filtering color information include: The mean value of the RGB channels of each mask region in the image is calculated and compared with a pre-set threshold range. The mean value of the R channel is 22 to 162, the mean value of the G channel is 22 to 162, and the mean value of the B channel is 17 to 137. When the mean value of all RGB channels is within the set range, the mask is a mask region that conforms to the color characteristics of bridge plate rubber bearings.

6. The image recognition method for bridge plate rubber bearings according to claim 1, characterized in that, The texture feature filtering includes: The gray-level co-occurrence matrix method is used to extract texture features of the image, including contrast, dissimilarity, consistency, energy, correlation, and angular second moment. The values ​​of the six key texture features are then compared with pre-set threshold ranges. The threshold ranges for contrast feature are 1.269 to 90.080, dissimilarity feature is 0.052 to 1.454, consistency feature is 0.868 to 0.995, energy feature is 0.834 to 0.994, correlation feature is 0.791 to 0.986, and angular second moment is 0.696 to 0.

988.

7. The image recognition method for bridge plate rubber bearings according to any one of claims 1-6, characterized in that: When performing multi-level screening, the first step is to perform preliminary screening using spatial location features. The second step is to use shape features to roughly screen out candidate mask areas. The third step is to use texture features to further filter the candidate mask areas. The fourth step is to use color features to finely screen the candidate mask areas. Finally, the mask areas for bridge plate rubber bearings are screened out using relative spatial location information.

8. The image recognition method for bridge plate rubber bearings according to claim 1, characterized in that: After multi-level screening, manual selection is also included to assist in identification. Manual selection is performed on the areas of bridge plate rubber bearings that are difficult to automatically screen in complex environment images. The corresponding mask area is matched according to the manually selected position to supplement and improve the accuracy of the multi-level screening method.

9. An image recognition system for bridge plate rubber bearings, characterized in that, include: The image acquisition module captures images of bridge plate rubber bearings. The feature extraction module inputs the image into the semantic segmentation model to obtain the segmentation mask of the image, the segmentation mask having several mask regions including the bridge plate rubber bearing; The image recognition module uses a multi-level filtering method to filter the several mask regions to obtain the mask region to which the bridge plate rubber bearing belongs, that is, to locate the location of the bridge plate rubber bearing region in the image. The multi-level filtering method includes first performing a rapid coarse screening, then performing a fine screening, and finally performing a relative screening. The rapid coarse screen includes: Spatial location filtering: calculate the contact amount between each mask region and the image edge; if a mask region is in contact with the image edge, then filter out the mask region with that contact edge. Shape information filtering involves extracting the outline of each mask region, calculating the aspect ratio of the mask region, filtering out mask regions whose shape and aspect ratio conform to the characteristics of bridge plate rubber bearings, and filtering out mask regions that do not meet the requirements. The fine-tuning includes: Texture feature filtering: Extract texture features from the image, filter out masked regions whose texture features meet the preset threshold range, and filter out masked regions whose texture features do not meet the characteristics of bridge plate rubber bearings. Color information filtering involves calculating the average RGB channel value of each mask region in the image, filtering out mask regions whose RGB average value meets the preset threshold range, and removing mask regions whose RGB channels do not conform to the color characteristics of bridge plate rubber bearings. The relative filtering includes: Relative spatial location filtering extracts the relative spatial location features between masked regions, filters out regions whose relative spatial location features do not meet the preset filtering rules or thresholds, and selects masked regions that meet the relative spatial location features of bridge plate rubber bearings.

10. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an image recognition method for bridge plate rubber bearings as described in any one of claims 1 to 8.

Images