An AI vision-based tower crane structural member damage data automatic identification method

Abstract

The application discloses a kind of tower crane structural member damage data automatic identification method based on AI vision, comprising the following steps: collecting and pre-processing the visual data of tower crane structural member;Standardized structural member visual data set is positioned and divided into parts;Based on tower crane structural member type and structural member part, construct damage semantic rule set;Structural member part visual fragment set is input into damage identification model, and output damage semantic identification result;Correlation encapsulation is generated structural member damage data set;Structural member damage data set is subjected to engineering safety semantic check and consistency constraint processing;Data damage cognitive result is output as structural member damage data output set and sent to safety management interface.The application fuses artificial intelligence vision and damage semantic rule constraint, realizes the accurate identification of tower crane structural member damage and engineering semantic consistency determination, with the advantages of high identification reliability and strong engineering applicability.

Description

Technical Field

[0001] This invention relates to the field of engineering structure inspection, and in particular to an automatic identification method for damage data of tower crane structural components based on AI vision. Background Technology

[0002] Tower cranes, as key equipment in large-scale engineering construction, are prone to damage such as cracks, corrosion, and structural deformation due to their structural components being subjected to high loads, alternating stresses, and complex environments for extended periods. Current methods for structural component damage detection mainly rely on manual inspections or visual recognition methods based on traditional image processing and general deep learning models. These methods analyze collected image or video data to detect and label damaged areas. While these methods improve detection efficiency to some extent, they typically only focus on local appearance features of the damage and lack systematic modeling of structural component types, locations, and engineering semantic constraints.

[0003] In existing technologies, damage identification results are mostly output as isolated detection boxes or segmentation results, failing to combine the tower crane structural topology, location semantics, and engineering safety rules for consistency verification. This leads to instability in the identification results regarding the rationality of damage categories, the accuracy of spatial location, and the determination of severity. Furthermore, existing methods often lack a unified constraint mechanism for damage semantic rules, severity grading standards, and engineering safety requirements, making it difficult to generate data-driven damage recognition results that can be directly used in safety management systems. This limits their practical application value in engineering safety assessment and decision support.

[0004] Therefore, how to provide an automatic identification method for damage data of tower crane structural components based on AI vision is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an automatic identification method for damage data of tower crane structural components based on AI vision. This invention integrates artificial intelligence vision with damage semantic rule constraints to achieve accurate identification of damage to tower crane structural components and determination of engineering semantic consistency, and has the advantages of high identification reliability and strong engineering applicability.

[0006] An automatic identification method for damage data of tower crane structural components based on AI vision according to an embodiment of the present invention includes the following steps:

[0007] Collect and preprocess visual data of tower crane structural components to generate a standardized visual dataset of structural components;

[0008] Based on the structural topology and structural component types of tower cranes, structural component localization and part division are performed on the standardized structural component visual dataset to obtain a set of visual fragments of structural component parts.

[0009] A set of damage semantic rules is constructed based on the structural component types and locations of tower cranes;

[0010] Input the set of visual fragments of structural components into the damage recognition model and output the damage semantic recognition result;

[0011] The structural component identifier, part identifier, and damage semantic recognition results are associated and encapsulated to generate a structural component damage dataset.

[0012] Perform engineering safety semantic verification and consistency constraint processing on the structural component damage dataset to generate data-driven damage cognition results;

[0013] The data-driven damage perception results are output as a structural component damage data output set and sent to the safety management interface.

[0014] Optionally, the preprocessing includes timestamp alignment, camera distortion correction, illumination normalization, motion blur suppression, noise filtering, and resolution unification. The visual data includes structural component image data or structural component video frame data. The types of structural components include standard tower sections, lifting booms, counterweight booms, and connecting nodes.

[0015] Optionally, obtaining the set of visual fragments of the structural component specifically includes:

[0016] A structural component topology relationship table is established based on the structural topology of the tower crane. The structural component topology relationship table includes a structural component type field, a connection node field, a connection relationship field, and a spatial relative position field.

[0017] Based on the structural component type and structural component part definition, establish the structural component positioning and part mapping relationship, and configure part boundary description parameters and part geometric prior parameters for each part set;

[0018] Perform structural component localization processing on the standardized structural component visual dataset and output a set of candidate structural component localization results.

[0019] Candidate result filtering and overlap resolution are performed on the candidate location result set of structural components, and the location results of structural components whose location confidence meets the threshold condition are retained as the target structural component location result set;

[0020] Based on the structural component positioning and part mapping relationship, part segmentation processing is performed on each target structural component positioning result in the target structural component positioning result set to obtain a set of structural component part visual segments.

[0021] Generate a structural component identifier and a part identifier for each structural component visual segment in the structural component visual segment set, and write the structural component identifier and part identifier into the metadata field of the structural component visual segment set.

[0022] Optionally, the construction of the damage semantic rule set specifically includes:

[0023] Obtain the set of structural component types and the set of structural component parts for tower cranes, and use the set of structural component types and the set of structural component parts as the basic constraint set for constructing the damage semantic rule set;

[0024] Establish a set of damage semantic categories, and set cracks, corrosion and structural deformation as category entries respectively. Write a category code field and a category name field for each category entry, and write the fields into the damage semantic rule set.

[0025] Establish a set of damage semantic attributes, set damage location, damage geometric scale and damage severity as attribute entries, and write value type field and unit field for each attribute entry and write them into the damage semantic rule set;

[0026] Establish a set of damage semantic constraints, generate constraint entries between structural component types and damage semantic categories based on the set of structural component types and the set of damage semantic categories, generate constraint entries between structural component parts and damage semantic categories based on the set of structural component parts and the set of damage semantic categories, and write the corresponding constraint entries into the damage semantic rule set;

[0027] Based on the damage samples and structural parts set marked by historical inspections, morphological features are extracted. Based on the statistical distribution results of the morphological features, the corresponding threshold intervals are determined. The threshold intervals are written into the morphological threshold field to form damage morphological constraint rule entries and written into the damage semantic rule set.

[0028] Based on the quantification results and severity levels corresponding to the damage geometric scale parameters and damage location parameters, scale threshold ranges and level mapping fields are set for each category to obtain a damage severity grading rule entry table and write it into the damage semantic rule set.

[0029] Optionally, the output of the damage semantic recognition result specifically includes:

[0030] Read the visual fragments of structural components, structural component identifiers and part identifiers from the set of visual fragments of structural components, and perform resolution unification and pixel value normalization on the visual fragments of structural components to obtain the model input feature map;

[0031] The input feature map of the model is divided into image blocks. Each image block is mapped into a block embedding vector and then a positional encoding is superimposed to obtain the initial visual feature sequence.

[0032] Based on the part identifiers corresponding to the visual segments of structural parts, semantic partitioning is performed on the initial visual feature sequence to obtain part feature subsequences divided by part. Each part feature subsequence is then input into a hierarchical Transformer encoder to perform self-attention calculation and feedforward mapping to generate partition features.

[0033] The damage recognition model is an improved model built on the SegFormer network. The improvement includes performing feature partitioning and attention constraint processing based on structural component identification in the encoding process, performing morphological constraint modulation processing based on damage semantic rule set in the multi-scale feature processing stage, and performing severity classification rule matching processing based on damage semantic rule set in the output result processing stage.

[0034] During the self-attention computation process, a location constraint mask is applied to the partition features to restrict the attention interaction between the partition features corresponding to different structural parts, and outputs a multi-scale coding feature set for each coding stage.

[0035] Damage morphology modulation parameters are generated based on damage morphology constraint rules in the damage semantic rule set. The damage morphology modulation parameters are then applied to the multi-scale encoded feature set to perform weighted modulation, resulting in a modulated multi-scale feature set.

[0036] The modulated multi-scale feature set is input into the multi-scale feature fusion layer. Alignment and fusion processing are performed on the features at each scale to generate a fused feature representation. The fused feature representation is then input into the segmentation decoding head, and the damage semantic segmentation result and damage semantic category label are output.

[0037] Based on the damage semantic segmentation results, connected component extraction and geometric quantity calculation are performed to obtain damage location parameters and damage geometric scale parameters. Then, according to the damage semantic category label, the corresponding damage severity grading rule entry is selected from the damage semantic rule set. The damage geometric scale parameters and damage location parameters are input into the selected severity grading rule entry to determine the damage severity parameters.

[0038] The damage semantic category label, damage location parameter, damage geometric scale parameter and damage severity parameter are associated and encapsulated to output damage semantic recognition results consistent with the damage semantic rule set.

[0039] Optionally, the generation of the structural component damage dataset specifically includes:

[0040] Read the structural component identifier and part identifier from the metadata field of the visual fragment set of structural component parts, and read the damage semantic category label, damage location parameter, damage geometric scale parameter and damage severity parameter corresponding to each damage item from the damage semantic recognition result;

[0041] Traverse each damage entry in the damage semantic recognition results, generate a structured damage data record for each damage entry, and write the structural component identifier field and the location identifier field into the structured damage data record.

[0042] Write the damage semantic category label into the damage semantic category field, and write the damage location parameter into the damage location field. The damage location field includes a center location parameter field and a boundary location parameter field.

[0043] Write the damage geometry parameter into the damage geometry field, and write the damage severity parameter into the damage severity field;

[0044] The acquisition timestamp information associated with the structural component identifier is obtained and written into the structured damage data record as a timestamp field. The structured damage data records are then sorted in ascending order of timestamp field value and aggregated to generate a structural component damage dataset.

[0045] Optionally, the generation of the data-driven impairment cognition results specifically includes:

[0046] Based on the set of damage semantic constraints in the damage semantic rule set, a consistency check is performed on the damage semantic category field of each structured damage data record in the structural component damage dataset, and the result is written to the category consistency check field.

[0047] Based on the mapping relationship between the structural component location and its parts, a spatial consistency check is performed on the damage location field, and the result is written to the spatial consistency check field.

[0048] Perform a classification consistency check on the injury severity field according to the set of injury severity classification rules, and write the result to the classification consistency check field;

[0049] The category consistency verification field, spatial consistency verification field, and hierarchical consistency verification field are associated and encapsulated with the corresponding structured damage data records to generate data-driven damage cognition results.

[0050] Optionally, obtaining the structural component damage data output set specifically includes:

[0051] The structured damage data records associated with the three types of consistency verification fields are read from the data-driven damage cognition results. The structural component identification field, location identification field, damage semantic category field, damage location field, damage geometric scale field, damage severity field and timestamp field are read from the structured damage data records as the basic field set for outputting structural component damage data.

[0052] Based on the set of basic fields, generate corresponding structural component damage data output records for each data-driven damage cognition record;

[0053] Based on the data structure definition of the safety management interface, the structural component damage data output records are collected and encapsulated into a structural component damage data output set, and an output batch identifier field is written into the structural component damage data output set.

[0054] The structural component damage data output set is written into the damage data storage unit, and at the same time, the structural component damage data output set is sent to the safety management interface.

[0055] The beneficial effects of this invention are:

[0056] This invention achieves semantic consistency recognition and data representation of damage to tower crane structural components by introducing a joint modeling mechanism that integrates structural component type, structural component location, and damage semantic rule set. Compared to existing technologies that rely solely on appearance features for damage detection, this invention explicitly incorporates structural component topological relationships, location segmentation results, and damage semantic constraints into the recognition and judgment process. This establishes a clear engineering semantic association between damage semantic category, damage location, damage geometric scale, and damage severity, effectively reducing misidentification and semantic conflicts caused by scene complexity, similar parts, or texture interference, and improving the reliability and consistency of damage recognition results at the engineering semantic level.

[0057] This invention achieves deep coupling between model inference results and engineering safety rules by constructing a damage semantic rule set and introducing corresponding semantic constraint mechanisms in the encoding, multi-scale feature processing, and result output stages of the damage identification model. On the one hand, feature partitioning and attention constraint processing based on structural component identification effectively suppress interference from irrelevant features between different structural component parts, improving the stability of damage semantic segmentation and category determination. On the other hand, the joint application of damage morphology constraint rules and damage severity grading rules establishes a clear rule mapping relationship between damage geometric scale parameters, spatial location parameters, and severity levels, thereby ensuring that the damage severity determination results conform to engineering safety semantics and historical experience constraints, enhancing the engineering credibility of the identification results.

[0058] Furthermore, based on the damage semantic recognition results, this invention further processes the damage recognition results through engineering safety semantic verification and consistency constraints to form a data-driven damage cognition result. This result is then standardized and output as a structural component damage data output set, enabling the damage recognition results to be directly connected to the safety management interface or damage data storage unit. Through this data-driven and structured processing method, this invention not only achieves a complete closed loop of tower crane structural component damage from visual perception and semantic recognition to engineering cognition, but also provides a standardized and traceable data foundation for subsequent safety assessments, risk warnings, and operation and maintenance decisions. This significantly enhances the application value and scalability of structural component damage detection technology in actual engineering safety management scenarios. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Figure 1 This is a flowchart of an AI vision-based automatic identification method for damage data of tower crane structural components proposed in this invention.

[0061] Figure 2 This is a flowchart illustrating the construction of a damage semantic rule set for an AI vision-based automatic identification method for tower crane structural components, as proposed in this invention.

[0062] Figure 3 This is a flowchart of the damage semantic recognition process for an AI vision-based automatic identification method for tower crane structural components proposed in this invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0064] refer to Figures 1-3 An automatic identification method for damage data of tower crane structural components based on AI vision includes the following steps:

[0065] Visual data of tower crane structural components are collected and preprocessed to generate a standardized visual dataset of structural components.

[0066] Based on the structural topology and structural component types of tower cranes, structural component localization and part division are performed on the standardized structural component visual dataset to obtain a set of visual fragments of structural component parts.

[0067] A set of damage semantic rules is constructed based on the structural component types and locations of tower cranes;

[0068] Input the set of visual fragments of structural components into the damage recognition model, and output the damage semantic recognition result that is consistent with the damage semantic rule set;

[0069] The structural component identifier, part identifier, and damage semantic recognition results are associated and encapsulated to generate a structural component damage dataset.

[0070] Perform engineering safety semantic verification and consistency constraint processing on the structural component damage dataset to generate data-driven damage cognition results;

[0071] The data-driven damage perception results are output as a structural component damage data output set and sent to the safety management interface.

[0072] In this embodiment, the visual data includes structural component image data or structural component video frame data. The types of structural components include standard tower sections, lifting booms, counterweight booms, and connecting nodes. The preprocessing includes timestamp alignment, camera distortion correction, illumination normalization, motion blur suppression, noise filtering, and resolution unification.

[0073] In this embodiment, obtaining the set of visual fragments of the structural component specifically includes:

[0074] A structural component topology relationship table is established based on the structural topology of the tower crane. The structural component topology relationship table includes a structural component type field, a connection node field, a connection relationship field, and a spatial relative position field.

[0075] Based on the structural component type and structural component part definition, a structural component positioning and part mapping relationship is established, and part boundary description parameters and part geometric prior parameters are configured for each part set. The structural component positioning and part mapping relationship includes mapping entries from structural component type to part set. The part set includes one or more of the following: rod surface area, weld line area, connecting plate area, and bolt hole area.

[0076] Perform structural component localization processing on a standardized structural component visual dataset and output a set of candidate structural component localization results. The candidate structural component localization results include candidate box boundaries, structural component type prediction results, and localization confidence.

[0077] The structural component localization process includes: detecting structural component candidate boxes on structural component image data or structural component video frame data in a standardized structural component visual dataset to generate a set of structural component candidate boxes; extracting corresponding visual features for each structural component candidate box and performing structural component type recognition based on a structural component type feature model to obtain a structural component type prediction result; performing consistency verification on the structural component type prediction result according to the structural component type and spatial relative position constraints defined in the structural component topology table, and eliminating structural component candidate boxes that do not meet the structural topology constraints; calculating a first confidence value based on the confidence level of the structural component type prediction result, calculating a second confidence value based on the clarity, integrity, and stability of the structural component candidate box boundary, and performing a weighted summation calculation on the first confidence value and the second confidence value according to preset weights to obtain the localization confidence of the corresponding structural component candidate box; associating and encapsulating the candidate box boundary, structural component type prediction result, and localization confidence to form a structural component candidate localization result, and summarizing them to obtain a set of structural component candidate localization results;

[0078] Candidate result filtering and overlap resolution are performed on the candidate location result set of structural components, and the location results of structural components whose location confidence meets the threshold condition are retained as the target structural component location result set;

[0079] Obtaining the target structural component location result set includes: sorting the candidate structural component location results in the candidate location result set according to their location confidence scores; performing preliminary screening on the sorted candidate structural component location results based on a location confidence threshold, removing candidate structural component location results whose location confidence scores do not meet the location confidence threshold condition; calculating the overlap rate between candidate boxes for each candidate structural component location result that passed the preliminary screening, wherein the overlap rate is obtained by calculating the ratio of the overlapping area of ​​two candidate boxes to the union area of ​​the two candidate boxes; performing overlap resolution processing on candidate structural component location results with overlapping relationships based on the overlap rate threshold, wherein when the overlap rate corresponding to any two candidate structural component location results is greater than the overlap rate threshold, selecting the candidate structural component location result corresponding to the maximum location confidence score from these two candidate structural component location results, and deleting the other candidate structural component location result; and summarizing the candidate structural component location results after the overlap resolution processing to obtain the target structural component location result set.

[0080] Based on the structural component positioning and part mapping relationship, part segmentation processing is performed on each target structural component positioning result in the target structural component positioning result set to obtain a set of structural component part visual segments.

[0081] The part segmentation process includes: based on the structural component positioning and part mapping relationship, reading the candidate box boundary information and structural component type information corresponding to each target structural component positioning result from the target structural component positioning result set; defining the target structural component candidate region based on the candidate box boundary information; determining the part set corresponding to the target structural component based on the structural component type information; for each part in the part set, generating a part region mask in the target structural component candidate region based on the corresponding part boundary description parameters and part geometric prior parameters; under the constraint of the part region mask, cropping the corresponding part visual region fragment from the target structural component candidate region; summarizing the visual region fragments corresponding to each part to obtain a structural component part visual fragment set;

[0082] For each structural component visual segment in the set of structural component visual segments, a structural component identifier and a part identifier are generated, and the structural component identifier and part identifier are written into the metadata field of the set of structural component visual segments. The structural component identifier is obtained by combining the acquisition timestamp information, the acquisition viewpoint identifier information and the structural component type sequence number information, and the part identifier is obtained by combining the structural component type information and the part category sequence number information.

[0083] In this embodiment, the construction of the damage semantic rule set specifically includes:

[0084] Obtain the set of structural component types and the set of structural component parts for tower cranes, and use the set of structural component types and the set of structural component parts as the basic constraint set for constructing the damage semantic rule set;

[0085] Establish a set of damage semantic categories, and set cracks, corrosion and structural deformation as category entries respectively. Write a category code field and a category name field for each category entry, and write the fields into the damage semantic rule set.

[0086] Establish a set of damage semantic attributes, set damage location, damage geometric scale and damage severity as attribute entries, and write value type field and unit field for each attribute entry and write them into the damage semantic rule set;

[0087] Establish a set of damage semantic constraints, generate constraint entries between structural component types and damage semantic categories based on the set of structural component types and the set of damage semantic categories, generate constraint entries between structural component parts and damage semantic categories based on the set of structural component locations and the set of damage semantic categories, and write the corresponding constraint entries into the damage semantic rule set. The constraint entries between structural component types and damage semantic categories are used to define the set of damage semantic categories that are allowed to appear for each structural component type, and the constraint entries between structural component locations and damage semantic categories are used to define the set of damage semantic categories that are allowed to appear for each structural component location.

[0088] Based on the damage samples and structural parts set marked by historical inspections, morphological features are extracted. Based on the statistical distribution results of the morphological features, the corresponding threshold intervals are determined. The threshold intervals are written into the morphological threshold field to form damage morphological constraint rule entries and written into the damage semantic rule set. The damage morphological constraint rule entries include morphological feature fields, morphological threshold fields and morphological consistency judgment fields. The morphological feature fields include crack connection length, crack width, rust coverage area ratio, rust texture roughness, structural deformation bending degree and structural deformation offset distance.

[0089] Based on the quantization results and severity levels corresponding to the damage geometric scale parameters and damage location parameters, scale threshold intervals and level mapping fields are set for each category to obtain a damage severity grading rule entry table and write it into the damage semantic rule set. The damage severity grading rule entry table includes a severity level set and severity determination rule entries. The severity level set includes level 1, level 2, and level 3. The severity determination rule entries include a geometric scale threshold interval field and a level mapping field. The geometric scale threshold interval field is jointly determined by the spatial range corresponding to the damage geometric scale and damage location. The level mapping field is used to map the damage semantic recognition results that meet the corresponding threshold interval conditions to the corresponding severity level.

[0090] In this embodiment, the output of the damage semantic recognition result specifically includes:

[0091] Read the visual fragments of structural components, structural component identifiers and part identifiers from the set of visual fragments of structural components, and perform resolution unification and pixel value normalization on the visual fragments of structural components to obtain the model input feature map;

[0092] The input feature map of the model is divided into image blocks. Each image block is mapped into a block embedding vector and then a positional encoding is superimposed to obtain the initial visual feature sequence.

[0093] Based on the part identifiers corresponding to the visual segments of structural parts, semantic partitioning is performed on the initial visual feature sequence to obtain part feature subsequences divided by part. Each part feature subsequence is then input into a hierarchical Transformer encoder to perform self-attention calculation and feedforward mapping to generate partition features.

[0094] The damage recognition model is an improved model built on the SegFormer network. The improvement includes performing feature partitioning and attention constraint processing based on structural component identification in the encoding process, performing morphological constraint modulation processing based on damage semantic rule set in the multi-scale feature processing stage, and performing severity classification rule matching processing based on damage semantic rule set in the output result processing stage.

[0095] The generation of the partition features includes: assigning a corresponding part identifier to each visual feature element in the initial visual feature sequence based on the part identifier corresponding to the visual segment of the structural component; grouping the visual feature elements in the initial visual feature sequence according to the part identifier, and classifying visual feature elements with the same part identifier into the same part feature set; performing a rearrangement process on the visual feature elements in each part feature set according to their spatial order in the original initial visual feature sequence to form a part feature subsequence; inputting each part feature subsequence into the encoding layer of the hierarchical Transformer encoder, performing a weighted association calculation based on feature correlation on the visual feature elements in the part feature subsequence in the encoding layer to update the feature representation of each visual feature element, and performing an element-wise mapping transformation on the updated feature representation to obtain the part encoding features of the corresponding part; and summarizing the part encoding features corresponding to each part to obtain a partition feature set including part differentiation information.

[0096] During the self-attention computation process, a location constraint mask is applied to the partition features to restrict the attention interaction between the partition features corresponding to different structural parts, and outputs a multi-scale coding feature set for each coding stage.

[0097] The output of the multi-scale encoded feature set includes: in each coding layer of the hierarchical Transformer encoder, using partition features as input features for self-attention calculation, and generating corresponding part constraint masks based on part identifiers; during the self-attention calculation process, using the part constraint masks to constrain the attention weights, allowing only partition features with the same part identifier to participate in the attention weight calculation; performing weighted aggregation on the partition features based on the constrained attention weights to obtain the constrained attention features corresponding to each coding layer; using the constrained attention features output by each coding layer as the encoded feature output of that coding layer; and aggregating the encoded features output by each coding layer according to the hierarchical order of the coding layers to obtain a multi-scale encoded feature set including features from different coding layers.

[0098] Damage morphology modulation parameters are generated based on damage morphology constraint rules in the damage semantic rule set. The damage morphology modulation parameters are then applied to the multi-scale encoded feature set to perform weighted modulation, resulting in a modulated multi-scale feature set.

[0099] The process of obtaining the modulated multi-scale feature set includes: reading damage morphology constraint rule entries from the damage semantic rule set and parsing them to obtain the morphology constraint parameters corresponding to each damage morphology; generating corresponding damage morphology modulation parameters for each scale coding feature in the multi-scale coding feature set based on the morphology constraint parameters; performing scale-by-scale matching between the damage morphology modulation parameters and the corresponding scale coding features, and performing weighted operations on the coding features according to the modulation parameters; reorganizing the scale coding features after the weighted operations to obtain the multi-scale feature representation modulated by damage morphology constraints; and collecting the modulated multi-scale feature representations to obtain the modulated multi-scale feature set.

[0100] The modulated multi-scale feature set is input into the multi-scale feature fusion layer. Alignment and fusion processing are performed on the features at each scale to generate a fused feature representation. The fused feature representation is then input into the segmentation decoding head, and the damage semantic segmentation result and damage semantic category label are output.

[0101] The process of obtaining the fused feature representation includes: inputting each scale feature in the modulation multi-scale feature set into the multi-scale feature fusion layer; performing spatial resolution alignment processing on each scale feature according to a preset scale alignment relationship to make each scale feature have a consistent spatial size; performing scale-by-scale accumulation and feature mapping processing on each scale feature after spatial alignment in the corresponding scale order to obtain the fused feature representation under a unified semantic space, and using the fused feature representation as the input feature of the segmentation decoding head.

[0102] The output of the damage semantic segmentation result includes: inputting the fused feature representation into the segmentation decoding head, performing layer-by-layer feature mapping and spatial reconstruction processing on the fused feature representation to generate a pixel-level feature map corresponding to the spatial resolution of the visual fragment of the input structural part; performing category discrimination mapping on the pixel-level feature map to assign a corresponding damage semantic category label to each pixel position to obtain a pixel-level damage semantic segmentation map, and outputting the pixel-level damage semantic segmentation map as the damage semantic segmentation result.

[0103] The output of the damage semantic category label specifically includes: based on the damage semantic segmentation result, traversing the damage semantic category determination result corresponding to each pixel position in the pixel-level damage semantic segmentation map, extracting the damage semantic category appearing within the visual segment of the current structural component; summarizing the extracted damage semantic categories to form a damage semantic category label corresponding to the visual segment of the current structural component.

[0104] Based on the damage semantic segmentation results, connected component extraction and geometric quantity calculation are performed to obtain damage location parameters and damage geometric scale parameters. Then, according to the damage semantic category label, the corresponding damage severity grading rule entry is selected from the damage semantic rule set. The damage geometric scale parameters and damage location parameters are input into the selected severity grading rule entry to determine the damage severity parameters.

[0105] The damage location parameters are obtained by: performing connected component extraction processing on adjacent pixels belonging to the same damage semantic category in the pixel-level damage semantic segmentation map based on the damage semantic segmentation results to obtain a set of damage connected components; determining the pixel location range of each damage connected component in the visual segment of the structural component; calculating the center location parameter and boundary location parameter of the damage connected component based on the pixel location range; and outputting the center location parameter and boundary location parameter as the damage location parameters of the corresponding damage.

[0106] The damage geometric scale parameters are obtained by: for each damage connected component in the damage connected component set, calculating the number of pixels covered by the damage connected component based on the pixel distribution range of each damage connected component in the pixel-level damage semantic segmentation map; determining the length scale parameter, width scale parameter, and area scale parameter of the damage connected component in the visual segment of the structural part based on the pixel distribution range of the damage connected component; and outputting the length scale parameter, width scale parameter, and area scale parameter as the damage geometric scale parameters of the corresponding damage.

[0107] The determination of the injury severity parameter includes: based on the injury semantic category label, reading the injury severity grading rule entries corresponding to the injury semantic category label from the injury semantic rule set; using the injury geometric scale parameter and injury location parameter of the corresponding injury as rule input parameters, and matching them with the geometric scale threshold range in the injury severity grading rule entries respectively; determining the severity level mapping field that meets the conditions based on the matching result, and outputting the severity level corresponding to the severity level mapping field as the injury severity parameter of each injury;

[0108] The damage semantic category label, damage location parameter, damage geometric scale parameter and damage severity parameter are associated and encapsulated to output damage semantic recognition results consistent with the damage semantic rule set.

[0109] In this embodiment, the generation of the structural component damage dataset specifically includes:

[0110] Read the structural component identifier and part identifier from the metadata field of the visual fragment set of structural component parts, and read the damage semantic category label, damage location parameter, damage geometric scale parameter and damage severity parameter corresponding to each damage item from the damage semantic recognition result;

[0111] Traverse each damage entry in the damage semantic recognition results, generate a structured damage data record for each damage entry, and write the structural component identifier field and the location identifier field into the structured damage data record.

[0112] Write the damage semantic category label into the damage semantic category field, and write the damage location parameter into the damage location field. The damage location field includes a center location parameter field and a boundary location parameter field. The horizontal center value of the center location parameter field is obtained by summing the left boundary value and the right boundary value of the boundary location parameter field and dividing by two. The vertical center value of the center location parameter field is obtained by summing the upper boundary value and the lower boundary value of the boundary location parameter field and dividing by two.

[0113] The damage geometry parameters are written into the damage geometry field, and the damage severity parameters are written into the damage severity field. The damage geometry field includes a length scale parameter field, a width scale parameter field, and an area scale parameter field.

[0114] The acquisition timestamp information associated with the structural component identifier is obtained and written into the structured damage data record as a timestamp field. The structured damage data records are then sorted in ascending order of timestamp field value and aggregated to generate a structural component damage dataset.

[0115] In this embodiment, the generation of the data-driven impairment cognition results specifically includes:

[0116] Based on the set of damage semantic constraints in the damage semantic rule set, a consistency check is performed on the damage semantic category field of each structured damage data record in the structural component damage dataset, and the result is written to the category consistency check field.

[0117] The consistency verification of the damage semantic category field includes: for each structured damage data record, parsing the structural component identifier field to obtain structural component type information, parsing the part identifier field to obtain structural component part information, performing consistency verification on the damage semantic category field based on the pre-stored set of damage semantic categories allowed for structural component types and the set of damage semantic categories allowed for structural component parts in the damage semantic constraint set, and writing the verification result into the category consistency verification field.

[0118] Based on the mapping relationship between the structural component location and its parts, a spatial consistency check is performed on the damage location field, and the result is written to the spatial consistency check field.

[0119] The spatial consistency verification of the damage location field includes: for structured damage data records where the category consistency verification field is consistent, reading the part boundary description parameters corresponding to the structure type information and structure part information according to the structural component positioning and part mapping relationship, and determining the spatial boundary range of the structural component part based on the part boundary description parameters; extracting the center position parameter field and the boundary position parameter field from the damage location field, determining whether the horizontal center value of the center position parameter field is located between the left and right boundary values ​​of the spatial boundary range, and simultaneously determining whether the vertical center value of the center position parameter field is located between the upper and lower boundary values ​​of the spatial boundary range, and writing the determination result into the spatial consistency verification field;

[0120] Perform a classification consistency check on the injury severity field according to the set of injury severity classification rules, and write the result to the classification consistency check field;

[0121] The graded consistency verification of the damage severity field includes: for structured damage data records where the spatial consistency verification field is consistent, selecting the corresponding damage severity grading rule entry from the damage severity grading rule set based on the damage semantic category field, reading the length scale parameter field, width scale parameter field, and area scale parameter field in the damage geometric scale field, and reading the center position parameter field and boundary position parameter field in the damage location field, matching the above parameters with the geometric scale threshold interval field in the selected damage severity grading rule entry to obtain the rule matching severity level, and comparing the rule matching severity level with the damage severity field for consistency, and writing the comparison result into the graded consistency verification field;

[0122] The category consistency verification field, spatial consistency verification field, and hierarchical consistency verification field are associated and encapsulated with the corresponding structured damage data records to generate data-driven damage cognition results.

[0123] In this embodiment, obtaining the structural component damage data output set specifically includes:

[0124] The structured damage data records associated with the three types of consistency verification fields are read from the data-driven damage cognition results. The structural component identification field, location identification field, damage semantic category field, damage location field, damage geometric scale field, damage severity field and timestamp field are read from the structured damage data records as the basic field set for outputting structural component damage data.

[0125] Based on the set of basic fields, generate corresponding structural component damage data output records for each data-driven damage cognition record;

[0126] The generation of the structural component damage data output record includes: based on each field in the basic field set, writing the structural component identifier field into the structural component identifier field output position, writing the part identifier field into the part identifier field output position, writing the damage semantic category field into the damage semantic category field output position, writing the damage location field into the damage location field output position, writing the damage geometric scale field into the damage geometric scale field output position, writing the damage severity field into the damage severity field output position, and writing the timestamp field into the timestamp field output position, forming a structural component damage data output record that corresponds one-to-one with the data-driven damage cognition record;

[0127] Based on the data structure definition of the safety management interface, the structural component damage data output records are collected and encapsulated into a structural component damage data output set, and an output batch identifier field is written into the structural component damage data output set.

[0128] The structural component damage data output set is written into the damage data storage unit, and at the same time, the structural component damage data output set is sent to the safety management interface.

[0129] Example 1: To verify the feasibility of this invention in practice, it was applied to a tower crane structural safety monitoring scenario at a large construction site. This construction site is located in a complex climate environment with long-term high-altitude operations. During continuous operation, the tower crane is prone to damage such as cracks, corrosion, or structural deformation in parts such as the standard tower sections, boom, counterweight boom, and connecting nodes. Existing methods relying on manual inspection or single visual detection often suffer from low inspection frequency, inaccurate positioning, strong subjectivity in damage assessment, and a lack of structural safety semantic verification, making it difficult to provide timely and reliable support for on-site safety management decisions.

[0130] In this scenario, multi-angle industrial cameras are deployed around key structural components of the tower crane to periodically collect images and video frames of the components. The collected visual data is then processed using the method described in this invention. The system first performs time alignment, distortion correction, and illumination normalization on the collected visual data to form standardized visual data of the structural components. Based on this, and considering the structural topology of the tower crane, the system automatically locates the structural components in the visual data and divides them into parts according to their type, obtaining visual segments of corresponding rod surfaces, weld lines, connecting plates, and bolt holes. Subsequently, the system constructs a damage semantic rule set based on the structural component type and location information, incorporating allowed damage categories, damage attributes, and severity grading rules into the subsequent recognition process.

[0131] In the damage identification stage, visual fragments of each structural component are input into the improved damage identification model. During the encoding process, the model introduces location-aware feature partitioning and attention constraints, enabling the differentiation of features across different locations during computation. Simultaneously, in multi-scale feature processing, damage semantic rules are combined to constrain and modulate morphological features, thereby improving the stability of identifying minute cracks, localized corrosion, and slight deformations. The model's output damage semantic identification results not only include the damage category but also simultaneously generate damage location, geometric scale, and severity information. Subsequently, the system structurally encapsulates the identification results and performs engineering safety semantic verification based on damage semantic rules, automatically eliminating abnormal results that do not conform to structural logic or spatial constraints, thus forming reliable, data-driven damage recognition results.

[0132] In practical applications, the system runs continuously and outputs the processed structural component damage data to the on-site safety management platform. Compared to traditional manual inspection methods, this invention can cover more structural component parts within the same time period and complete identification and recording when early signs of damage appear, effectively shortening the time from damage occurrence to information acquisition by management personnel. On-site management personnel can promptly grasp changes in the safety status of key parts of the tower crane based on the structured damage data output by the system, thereby arranging maintenance or adjusting usage plans in advance and reducing the possibility of safety risks caused by accumulated structural component damage. The above application process shows that this invention has good applicability and stability in complex construction environments and can effectively solve the problems of untimely and inaccurate identification of tower crane structural component damage and lack of semantic constraints.

[0133] Table 1. Performance Comparison of the Invention and Traditional Recognition Methods

[0134] Damage identification accuracy (%) 88.4 92.1 Damage missed detection rate (%) 7.6 5.9 Damage false detection rate (%) 6.8 5.6 Average processing time for a single structural component (s) 2.35 1.92 Damage localization deviation (mm) 18.6 12.4 Consistency rate of severity assessment (%) 83.2 89.5 Percentage of available results (%) 86.9 94.2

[0135] As can be clearly seen from Table 1, the method of the present invention is superior to the traditional method in many indicators.

[0136] Regarding the accuracy of damage identification, the traditional method achieves 88.4%, while the method of this invention improves it to 92.1%. This improvement mainly stems from the fact that this invention introduces a structural topology-part mapping relationship in the structural component positioning and part division stage, enabling the damage identification model to have clear structural semantic constraints at the input stage, thus reducing the interference of non-target areas on the identification results.

[0137] Regarding the damage false negative rate, the traditional method has a false negative rate of 7.6%, while the method of this invention reduces it to 5.9%. This reduction is directly related to the introduction of location-aware feature partitioning and attention constraints into the damage identification model of this invention. By semantically partitioning the features of different structural components, the model can more stably capture subtle damage in local areas such as weld lines and bolt holes, thereby reducing omissions.

[0138] Regarding the damage false detection rate, the traditional method has a false detection rate of 6.8%, while the method of this invention reduces it to 5.6%. This improvement mainly stems from the dual constraint effect of the damage semantic rule set in both the identification and post-processing stages. The model output must simultaneously satisfy the semantic consistency conditions among structural component type, location, and damage category, effectively filtering out false detections caused by factors such as background noise and abnormal lighting.

[0139] The traditional method takes 2.35 seconds to process a single structural component, while the method of this invention takes 1.92 seconds. Although this invention introduces more semantic processing steps, it improves the overall processing efficiency by performing targeted processing on visual segments after structural component positioning and part segmentation, reducing redundant calculations in irrelevant areas.

[0140] The damage location deviation was reduced from 18.6 mm in the traditional method to 12.4 mm. This improvement is mainly due to the introduction of part boundary description parameters and geometric prior parameters in the part division stage of this invention, which restricts the calculation of the damage location to a reasonable spatial range of the structural part, thereby improving the positioning accuracy.

[0141] Regarding the consistency rate of damage severity determination, the traditional method achieves 83.2%, while the method of this invention improves it to 89.5%. This improvement stems from the introduction of a set of damage severity grading rules in the post-processing stage, which matches the damage's geometric scale and spatial location information with predefined engineering semantic rules, reducing the subjectivity of severity determination.

[0142] Finally, regarding the percentage of usable results, the traditional method achieves 86.9%, while the method of this invention reaches 94.2%. This result indicates that the data-driven damage recognition results processed by engineering safety semantic verification and consistency constraints are more in line with the requirements of actual engineering use, can be directly received and utilized by the safety management interface, and reduce the workload of manual review and secondary screening.

[0143] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatic identification of damage data of tower crane structural components based on AI vision, characterized in that, Includes the following steps: Collect and preprocess visual data of tower crane structural components to generate a standardized visual dataset of structural components; Based on the structural topology and structural component types of tower cranes, structural component localization and part division are performed on the standardized structural component visual dataset to obtain a set of visual fragments of structural component parts. The specific process of obtaining the visual fragment set of the structural component includes: A structural component topology relationship table is established based on the structural topology of the tower crane. The structural component topology relationship table includes a structural component type field, a connection node field, a connection relationship field, and a spatial relative position field. Based on the structural component type and structural component part definition, establish the structural component positioning and part mapping relationship, and configure part boundary description parameters and part geometric prior parameters for each part set; Perform structural component localization processing on the standardized structural component visual dataset and output a set of candidate structural component localization results. Candidate result filtering and overlap resolution are performed on the candidate location result set of structural components, and the location results of structural components whose location confidence meets the threshold condition are retained as the target structural component location result set; Based on the structural component positioning and part mapping relationship, part segmentation processing is performed on each target structural component positioning result in the target structural component positioning result set to obtain a set of structural component part visual segments. Generate a structural component identifier and a part identifier for each structural component visual segment in the set of structural component visual segments, and write the structural component identifier and part identifier into the metadata field of the set of structural component visual segments; A set of damage semantic rules is constructed based on the structural component types and locations of tower cranes; Input the set of visual fragments of structural components into the damage recognition model and output the damage semantic recognition result; The output of the damage semantic recognition result specifically includes: Read the visual fragments of structural components, structural component identifiers and part identifiers from the set of visual fragments of structural components, and perform resolution unification and pixel value normalization on the visual fragments of structural components to obtain the model input feature map; The input feature map of the model is divided into image blocks. Each image block is mapped into a block embedding vector and then a positional encoding is superimposed to obtain the initial visual feature sequence. Based on the part identifiers corresponding to the visual segments of structural parts, semantic partitioning is performed on the initial visual feature sequence to obtain part feature subsequences divided by part. Each part feature subsequence is then input into a hierarchical Transformer encoder to perform self-attention calculation and feedforward mapping to generate partition features. The damage recognition model is an improved model built on the SegFormer network. The improvement includes performing feature partitioning and attention constraint processing based on structural component identification in the encoding process, performing morphological constraint modulation processing based on damage semantic rule set in the multi-scale feature processing stage, and performing severity classification rule matching processing based on damage semantic rule set in the output result processing stage. During the self-attention computation process, a location constraint mask is applied to the partition features to restrict the attention interaction between the partition features corresponding to different structural parts, and outputs a multi-scale coding feature set for each coding stage. Damage morphology modulation parameters are generated based on damage morphology constraint rules in the damage semantic rule set. The damage morphology modulation parameters are then applied to the multi-scale encoded feature set to perform weighted modulation, resulting in a modulated multi-scale feature set. The modulated multi-scale feature set is input into the multi-scale feature fusion layer. Alignment and fusion processing are performed on the features at each scale to generate a fused feature representation. The fused feature representation is then input into the segmentation decoding head, and the damage semantic segmentation result and damage semantic category label are output. Based on the damage semantic segmentation results, connected component extraction and geometric quantity calculation are performed to obtain damage location parameters and damage geometric scale parameters. Then, according to the damage semantic category label, the corresponding damage severity grading rule entry is selected from the damage semantic rule set. The damage geometric scale parameters and damage location parameters are input into the selected severity grading rule entry to determine the damage severity parameters. The damage semantic category label, damage location parameter, damage geometric scale parameter and damage severity parameter are associated and encapsulated to output damage semantic recognition results consistent with the damage semantic rule set; The structural component identifier, part identifier, and damage semantic recognition results are associated and encapsulated to generate a structural component damage dataset. Perform engineering safety semantic verification and consistency constraint processing on the structural component damage dataset to generate data-driven damage cognition results; The data-driven damage perception results are output as a structural component damage data output set and sent to the safety management interface.

2. The method for automatic identification of damage data of tower crane structural components based on AI vision according to claim 1, characterized in that, The preprocessing includes timestamp alignment, camera distortion correction, illumination normalization, motion blur suppression, noise filtering, and resolution unification. The visual data includes structural component image data or structural component video frame data. The types of structural components include standard tower sections, lifting booms, counterweight booms, and connecting nodes.

3. The method for automatic identification of damage data of tower crane structural components based on AI vision according to claim 1, characterized in that, The construction of the damage semantic rule set specifically includes: Obtain the set of structural component types and the set of structural component parts for tower cranes, and use the set of structural component types and the set of structural component parts as the basic constraint set for constructing the damage semantic rule set; Establish a set of damage semantic categories, and set cracks, corrosion and structural deformation as category entries respectively. Write a category code field and a category name field for each category entry, and write the fields into the damage semantic rule set. Establish a set of damage semantic attributes, set damage location, damage geometric scale and damage severity as attribute entries, and write value type field and unit field for each attribute entry and write them into the damage semantic rule set; Establish a set of damage semantic constraints, generate constraint entries between structural component types and damage semantic categories based on the set of structural component types and the set of damage semantic categories, generate constraint entries between structural component parts and damage semantic categories based on the set of structural component parts and the set of damage semantic categories, and write the corresponding constraint entries into the damage semantic rule set; Based on the damage samples and structural parts set marked by historical inspections, morphological features are extracted. Based on the statistical distribution results of the morphological features, the corresponding threshold intervals are determined. The threshold intervals are written into the morphological threshold field to form damage morphological constraint rule entries and written into the damage semantic rule set. Based on the quantification results and severity levels corresponding to the damage geometric scale parameters and damage location parameters, scale threshold ranges and level mapping fields are set for each category to obtain a damage severity grading rule entry table and write it into the damage semantic rule set.

4. The method for automatic identification of damage data of tower crane structural components based on AI vision according to claim 1, characterized in that, The generation of the structural component damage dataset specifically includes: Read the structural component identifier and part identifier from the metadata field of the visual fragment set of structural component parts, and read the damage semantic category label, damage location parameter, damage geometric scale parameter and damage severity parameter corresponding to each damage item from the damage semantic recognition result; Traverse each damage entry in the damage semantic recognition results, generate a structured damage data record for each damage entry, and write the structural component identifier field and the location identifier field into the structured damage data record. Write the damage semantic category label into the damage semantic category field, and write the damage location parameter into the damage location field. The damage location field includes a center location parameter field and a boundary location parameter field. Write the damage geometry parameter into the damage geometry field, and write the damage severity parameter into the damage severity field; The acquisition timestamp information associated with the structural component identifier is obtained and written into the structured damage data record as a timestamp field. The structured damage data records are then sorted in ascending order of timestamp field value and aggregated to generate a structural component damage dataset.

5. The method for automatic identification of damage data of tower crane structural components based on AI vision according to claim 1, characterized in that, The generation of the data-driven impairment cognition results specifically includes: Based on the set of damage semantic constraints in the damage semantic rule set, a consistency check is performed on the damage semantic category field of each structured damage data record in the structural component damage dataset, and the result is written to the category consistency check field. Based on the mapping relationship between the structural component location and its parts, a spatial consistency check is performed on the damage location field, and the result is written to the spatial consistency check field. Perform a classification consistency check on the injury severity field according to the set of injury severity classification rules, and write the result to the classification consistency check field; The category consistency verification field, spatial consistency verification field, and hierarchical consistency verification field are associated and encapsulated with the corresponding structured damage data records to generate data-driven damage cognition results.

6. The method for automatic identification of damage data of tower crane structural components based on AI vision according to claim 1, characterized in that, The specific steps to obtain the structural component damage data output set include: The structured damage data records associated with the three types of consistency verification fields are read from the data-driven damage cognition results. The structural component identification field, location identification field, damage semantic category field, damage location field, damage geometric scale field, damage severity field and timestamp field are read from the structured damage data records as the basic field set for outputting structural component damage data. Based on the set of basic fields, generate corresponding structural component damage data output records for each data-driven damage cognition record; Based on the data structure definition of the safety management interface, the structural component damage data output records are collected and encapsulated into a structural component damage data output set, and an output batch identifier field is written into the structural component damage data output set. The structural component damage data output set is written into the damage data storage unit, and at the same time, the structural component damage data output set is sent to the safety management interface.

Images