An aerial-ground integrated image data intelligent analysis method and system

Abstract

The application discloses an air-ground integrated image data intelligent analysis method and system, relates to the general image data processing field, and constructs a space mapping data field according to air-ground integrated image data corresponding to a bridge in the method; a three-dimensional geometric hypothesis of a physical defect is generated at a target space position of a three-dimensional structured model; the three-dimensional geometric hypothesis is projected to corresponding multiple two-dimensional image planes based on a space mapping relationship, and multiple two-dimensional synthetic views containing the physical defect are generated; image similarity between the multiple two-dimensional synthetic views and air-ground integrated image data is calculated, and a confidence score is obtained; when the confidence score is greater than a preset threshold, it is confirmed that the physical defect exists at the target space position; a structural function attribute of the target space position is extracted from the three-dimensional structured model, and an analysis result is generated in combination with a type of the physical defect. The application improves the accuracy of defect identification in the bridge inspection field according to air-ground integrated image data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the general field of image data processing, and particularly relates to an air-ground integrated image data intelligent analysis method and system. BACKGROUND

[0002] In the field of bridge structure health monitoring and maintenance, unmanned aerial vehicle (UAV) aerial inspection has become an important technical means, which can efficiently obtain macro image data of the main structure of the bridge, such as the bridge deck, the main cable and the top of the bridge tower. However, there are inherent limitations in single aerial view or long-distance side-view perspective, which makes it difficult to conduct detailed exploration on the key load-bearing components and vulnerable areas of the bridge, such as the supports, expansion joints and small cracks on the surface of the pier column of the bridge, and the corrosion condition of the connection of the steel structure. These microscopic defects hidden in the aerial blind area are exactly the core elements for early warning of bridge structure risks and ensuring traffic safety, resulting in that the conclusion relying only on UAV inspection is often incomplete and has insufficient reliability.

[0003] In the related art, a three-dimensional digital twin modeling technology based on photogrammetry can be used. This technology fuses the UAV oblique photography image and the high-definition image taken from multiple angles on the ground, uses algorithms such as Structure from Motion (SfM) and Multi-View Stereo (MVS), and generates a refined three-dimensional model with high-fidelity texture that completely corresponds to the geometry of the physical bridge. This digital twin model integrates every visible surface of the bridge, including the bridge deck, supports and other areas that are difficult for the UAV to directly photograph, into a unified virtual space that can be fully explored and zoomed in or out, thereby achieving full coverage of bridge information at the geometric level and solving the problem of incomplete information.

[0004] However, the related art is difficult to use the consistency of multi-view images to confirm a blurred suspected crack, and is also difficult to evaluate its potential risk level in combination with its specific position in the three-dimensional structure of the bridge (such as the tension area of the main beam), thereby reducing the accuracy of defect identification in the field of bridge inspection according to air-ground integrated image data. SUMMARY

[0005] The present application provides an air-ground integrated image data intelligent analysis method and system for improving the accuracy of defect identification in the field of bridge inspection according to air-ground integrated image data.

[0006] In a first aspect, the present application provides an air-ground integrated image data intelligent analysis method, which constructs a spatial mapping data field according to the air-ground integrated image data corresponding to the bridge, and the spatial mapping data field is used to establish a one-to-one spatial mapping relationship between the three-dimensional structured model of the bridge and the pixel coordinates of the air-ground integrated image data.

[0007] generating a three-dimensional geometric hypothesis of the physical defect at the target spatial position of the three-dimensional structured model;

[0008] projecting the three-dimensional geometric hypothesis to corresponding multiple two-dimensional image planes based on the spatial mapping relationship to generate multiple two-dimensional synthetic views containing the physical defect;

[0009] calculating image similarity between the multiple two-dimensional synthetic views and the integrated image data to obtain a confidence score representing the credibility of the three-dimensional geometric hypothesis;

[0010] confirming that the physical defect exists at the target spatial position when the confidence score is greater than a preset threshold;

[0011] extracting a structural functional attribute of the target spatial position from the three-dimensional structured model, and generating an analysis result containing defect position information, defect type and risk level in combination with the type of the physical defect.

[0012] By adopting the above technical solutions, the one-to-one correspondence between the bridge three-dimensional structured model and the integrated image data pixel coordinates is realized by constructing a spatial mapping data field, so that the physical defect geometric hypothesis generated in the three-dimensional space can be accurately projected to multiple two-dimensional image planes. Based on the similarity comparison between the multiple two-dimensional synthetic views generated by projection and the actual image data, the accuracy of the physical defect hypothesis can be verified. When the similarity reaches a preset threshold, in combination with the structural functional attribute of the target position and the defect type, the physical defect can be accurately positioned and the risk level can be evaluated, thereby improving the accuracy of defect detection. At the same time, by introducing the structural functional attribute analysis, the influence of the defect on the structural safety can be more comprehensively evaluated.

[0013] In combination with some embodiments of the first aspect, in some embodiments, the spatial mapping data field is constructed according to the integrated image data corresponding to the bridge, specifically including:

[0014] establishing a three-dimensional structured model containing geometric parameters and structural parameters based on design drawings of the bridge;

[0015] determining camera pose parameters when collecting the integrated image data by using a visual odometry technology;

[0016] extracting feature points in the integrated image data based on the camera pose parameters;

[0017] establishing a spatial mapping relationship according to the matching relationship between the feature points and control points in the three-dimensional structured model to obtain the spatial mapping data field.

[0018] By adopting the technical scheme, the camera pose parameters are determined through the visual odometry technology, and the space mapping relationship is established based on the feature point matching, so that the accurate correspondence between the bridge three-dimensional structured model and the image data is realized. Since the geometric parameters and the structural parameters in the design drawing are used to establish the three-dimensional model, the consistency of the model and the actual structure is ensured. The visual odometry technology can accurately obtain the camera position and pose information at the time of image acquisition, and in combination with the feature point matching, a stable and reliable space mapping relationship can be established, thereby improving the accuracy of the space mapping.

[0019] In combination with some embodiments of the first aspect, in some embodiments, the three-dimensional geometric hypothesis of the physical defect is generated at the target space position of the three-dimensional structured model, and specifically includes:

[0020] The generative adversarial network is trained based on the historical bridge defect data, and the generative adversarial network includes a generative network and a discriminative network. The generative network is used to generate three-dimensional geometric features of different types of defects, and the discriminative network is used to evaluate the authenticity of the three-dimensional geometric features;

[0021] The three-dimensional structured model is divided into a plurality of detection regions, and the size of each detection region is determined according to a preset maximum size of the physical defect, and there is an overlapping region between adjacent detection regions;

[0022] A defect feature library is constructed, and the defect feature library contains different types of defects;

[0023] In each detection region, a plurality of different types of physical defect hypotheses are generated based on the generative adversarial network and the defect feature library;

[0024] The structural stress distribution features and material characteristic parameters of each detection region are extracted;

[0025] The compatibility score between each physical defect hypothesis and the structural stress distribution features and material characteristic parameters of the detection region is calculated;

[0026] The physical defect hypothesis with the highest compatibility score is taken as the three-dimensional geometric hypothesis.

[0027] By adopting the technical scheme, the generative adversarial network is used to generate the physical defect hypothesis in combination with the defect feature library, and the rationality of the defect hypothesis is evaluated by considering the structural stress distribution features and material characteristic parameters. The discriminative network in the generative adversarial network can evaluate the authenticity of the generated defect, thereby ensuring the rationality of the generated defect. By calculating the compatibility score between the defect hypothesis and the structural features, the most reasonable defect hypothesis is selected as the three-dimensional geometric hypothesis, thereby improving the accuracy of the defect detection. This method combines the deep learning technology with the structural mechanics knowledge, thereby ensuring the authenticity of the defect morphology and ensuring the compliance of the defect position and the structural stress features, thereby reducing the possibility of false detection.

[0028] In some embodiments in combination with the first aspect, after generating the analysis result containing the defect position information, the defect type and the risk level in combination with the type of the physical defect, the method further comprises:

[0029] determining the area not containing the physical defect as the intact surface area;

[0030] extracting the image texture corresponding to the intact surface area from the empty-ground integrated image data through the spatial mapping data field;

[0031] calculating the gradient direction of each pixel point in the image texture to generate a texture direction field, the texture direction field being used to represent the micro-topography trend of the intact surface area;

[0032] calculating a theoretical stress direction field of the intact surface area according to the three-dimensional structured model, the theoretical stress direction field being used to represent the stress transmission direction when the structure is under stress;

[0033] calculating the direction vector included angle of the texture direction field and the theoretical stress direction field at the corresponding position, and defining the area with the direction vector included angle less than a preset included angle threshold as an abnormal correlation area;

[0034] labeling the abnormal correlation area on the three-dimensional structured model, and determining the abnormal correlation area as a potential structural weak area.

[0035] By using the above technical solution, the potential structural weak area can be identified by analyzing the relationship between the image texture direction field and the theoretical stress direction field of the intact surface area. Since the micro-topography of the structure surface is often correlated with the internal stress distribution, by comparing the included angle of the texture direction and the theoretical stress direction, the stress transmission abnormal area can be found. This method not only can detect the physical defects that have already appeared, but also can predict the area where damage may occur. By labeling the abnormal correlation area on the three-dimensional structured model, the spatial distribution of the potential risk area is intuitively displayed. This texture stress correlation analysis method is helpful to realize the preventive maintenance of the bridge structure.

[0036] In some embodiments in combination with the first aspect, the calculation of the theoretical stress direction field of the intact surface area according to the three-dimensional structured model specifically comprises:

[0037] obtaining the surface shape parameters of the intact surface area, the surface shape parameters including the height variation and the inclination angle of the curved surface;

[0038] calculating the gradient variation direction of the surface shape parameters, the gradient variation direction representing the maximum variation trend of the curved surface at each point;

[0039] determining the stress flow direction according to the gradient variation direction, the stress flow direction representing the stress transmission path when the structure is under load;

[0040] Generate a theoretical stress direction field based on the stress flow direction, and the theoretical stress direction field is used to represent the stress transmission direction of the structure under stress.

[0041] By using the above technical solution, the surface shape parameters of the intact surface area are obtained, the gradient change direction of the surface shape parameters is calculated, the stress flow direction is determined according to the gradient change direction, and a theoretical stress direction field is generated based on the stress flow direction. The stress transmission direction of the structure under stress can be accurately represented. Since the surface shape parameters contain the height variation and inclination angle information of the curved surface, the maximum change direction of the structure surface at each point can be reflected by calculating the gradient change direction. This change direction is closely related to the stress transmission path inside the structure. The stress flow direction determined based on this is more in line with the actual stress characteristics of the structure, and the generated theoretical stress direction field can more accurately reflect the stress state of each part of the structure. This method of deriving stress characteristics based on geometric features improves the accuracy of the theoretical stress direction field.

[0042] In combination with some embodiments of the first aspect, in some embodiments, after determining the abnormal association area as a potential structural weak area, the method further comprises:

[0043] Calculate the distribution density of the abnormal association area, and the distribution density represents the proportion of the abnormal association area in a unit area;

[0044] Analyze the spatial continuity of the abnormal association area, and the spatial continuity represents the connection degree between adjacent abnormal association areas;

[0045] Determine the weak area expansion trend according to the distribution density and the spatial continuity, and the weak area expansion trend represents the development direction of the potential structural weak area;

[0046] Based on the weak area expansion trend, a risk warning is given to the potential structural weak area.

[0047] By using the above technical solution, the distribution density of the abnormal association area is calculated and the spatial continuity thereof is analyzed, and the weak area expansion trend is determined based on the two characteristics, thereby realizing the quantitative evaluation of the development trend of the potential structural weak area. The distribution density reflects the concentration degree of the structural defects, and the spatial continuity represents the connection state of the defects. The combination of the two parameters can comprehensively describe the spatial distribution characteristics of the weak area. When the distribution density of the abnormal association area is high and has good spatial continuity, it indicates that there is significant association between these areas, and the weak area is likely to continue to expand along this association. By analyzing the expansion trend, the development direction of the weak area can be predicted, thereby realizing early warning of the safety risk of the structure and reducing the possibility of structural failure.

[0048] In some embodiments of the first aspect, in some embodiments, the weak zone expansion trend is determined according to the distribution density and the spatial continuity, specifically comprising:

[0049] The distribution density of the abnormal correlation area is divided into a plurality of density level intervals;

[0050] The change rate of the spatial continuity in different directions is calculated, and a gradual change direction of the distribution density from low to high is determined;

[0051] The weak zone expansion trend is determined based on a matching result corresponding to a direction with the largest change rate in the gradual change direction.

[0052] By adopting the above technical solution, by dividing the distribution density of the abnormal correlation area into a plurality of level intervals, calculating the change rate of the spatial continuity in different directions, and combining the gradual change direction of the distribution density, an accurate weak zone expansion trend determination method is established. The division of the density level interval makes the difference of the distribution characteristics more clear, and the directional analysis of the change rate reveals the dominant direction of the spatial continuity. By comparing the gradual change direction of the distribution density with the direction with the largest change rate of the spatial continuity, the most possible expansion direction of the weak zone can be found. This multi-dimensional feature-based analysis method improves the accuracy of the weak zone expansion trend determination, and makes the risk warning more targeted.

[0053] In a second aspect, the embodiments of the present application provide an air-ground integrated image data intelligent analysis system, which comprises one or more processors and a memory; the memory is coupled with the one or more processors, and the memory is used to store computer program codes, the computer program codes comprising computer instructions, and the one or more processors invoke the computer instructions to enable the system to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0054] In a third aspect, the embodiments of the present application provide a computer readable storage medium comprising instructions, when the instructions are run on a system, the system performs the method described in the first aspect and any possible implementation manner of the first aspect.

[0055] In a fourth aspect, the embodiments of the present application provide a computer program product, when the computer program product is run on a system, the system performs the method described in any possible implementation manner of the first aspect.

[0056] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0057] 1. The application provides an intelligent analysis method for aerial-ground integrated image data. By constructing a spatial mapping data field, a one-to-one correspondence between the bridge three-dimensional structured model and the aerial-ground integrated image data pixel coordinates is achieved, so that the physical defect geometric hypothesis generated in the three-dimensional space can be accurately projected onto multiple two-dimensional image planes. Based on the similarity comparison between the multiple two-dimensional synthetic views generated by projection and the actual image data, the accuracy of the physical defect hypothesis can be verified. When the similarity reaches a preset threshold, combined with the structural functional properties and defect types of the target position, the physical defect can be accurately positioned and risk level evaluated, improving the accuracy of defect detection. At the same time, by introducing structural functional property analysis, the influence of defects on structural safety can be more comprehensively evaluated.

[0058] 2. The application provides an intelligent analysis method for aerial-ground integrated image data. By analyzing the relationship between the image texture direction field of the intact surface area and the theoretical stress direction field, potential structural weak areas can be identified. Since the micro-topography of the structural surface often correlates with the internal stress distribution, by comparing the angle between the texture direction and the theoretical stress direction, abnormal stress transmission areas can be found. This method not only detects existing physical defects, but also predicts areas where damage may occur. By marking abnormal correlation areas on the three-dimensional structured model, the spatial distribution of potential risk areas is intuitively displayed. This texture stress correlation analysis method helps to achieve preventive maintenance of bridge structures.

[0059] 3. The application provides an intelligent analysis method for aerial-ground integrated image data. By calculating the distribution density of abnormal correlation areas and analyzing their spatial continuity, and based on these two characteristics to determine the weak area expansion trend, quantitative evaluation of the development trend of potential structural weak areas is achieved. The distribution density reflects the concentration of structural defects, and the spatial continuity represents the connection state of the defects. The combination of these two parameters can comprehensively describe the spatial distribution characteristics of the weak area. When the distribution density of abnormal correlation areas is high and has good spatial continuity, it indicates that there is significant correlation between these areas, and the weak area is likely to continue to expand along this correlation. By analyzing this expansion trend, the development direction of the weak area can be predicted, thereby achieving early warning of structural safety risks and reducing the possibility of structural failure. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a flowchart of an intelligent analysis method for aerial-ground integrated image data in an embodiment of the application.

[0061] Figure 2 is another flowchart of an intelligent analysis method for aerial-ground integrated image data in an embodiment of the application.

[0062] Figure 3is a schematic structural diagram of an entity device of an aerial-ground integrated image data intelligent analysis system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the present application, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used in the present application, refers to any or all possible combinations of one or more of the listed items.

[0064] Hereinafter, the terms "first" and "second" are only for the purpose of description and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0065] Hereinafter, an embodiment is used in conjunction with Figure 1 An aerial-ground integrated image data intelligent analysis method in an embodiment of the present application is described.

[0066] Please refer to Figure 1 is a flowchart of an aerial-ground integrated image data intelligent analysis method in an embodiment of the present application.

[0067] S101, constructing a spatial mapping data field according to aerial-ground integrated image data corresponding to the bridge;

[0068] The system constructs a spatial mapping data field according to aerial-ground integrated image data corresponding to the bridge, and the spatial mapping data field is used to establish a one-to-one spatial mapping relationship between the three-dimensional structured model of the bridge and the pixel coordinates of the aerial-ground integrated image data. Specifically, it includes: establishing a three-dimensional structured model containing geometric parameters and structural parameters based on the design drawings of the bridge; determining the camera pose parameters when collecting the aerial-ground integrated image data by using the visual odometry technology; extracting feature points in the aerial-ground integrated image data based on the camera pose parameters; establishing a spatial mapping relationship according to the matching relationship between the feature points and the control points in the three-dimensional structured model, and obtaining the spatial mapping data field.

[0069] This step involves 3D modeling and spatial mapping of the bridge. The system first needs to acquire the bridge's design drawings, including its geometric and structural parameters, and then uses this information to create a 3D structured model of the bridge. The 3D structured model is a digital representation of the bridge's physical structure, accurately reflecting its spatial dimensions and structural features. In addition to design drawings, the system can also acquire 3D data of the bridge through techniques such as 3D scanning and oblique photogrammetry, further improving the model's accuracy and detail.

[0070] While constructing the 3D structured model, the system also needs to acquire integrated air-ground image data of the bridge. Integrated air-ground images refer to aerial and ground-based imagery of the bridge collected using drones, vehicle-mounted equipment, etc., containing rich texture and color information. To register the image data with the 3D model, the system employs visual odometry technology. By analyzing changes in feature points in the image sequence, it estimates the camera's motion trajectory and attitude parameters, thereby determining the 3D spatial position of each frame. Next, the system extracts feature points from the integrated air-ground images, such as corners and edges, and matches them with control points of the 3D structured model, establishing a mapping relationship between 2D pixel coordinates and 3D spatial coordinates. In this way, each image pixel can correspond to a specific location in the 3D model, forming a complete spatial mapping data field.

[0071] S102. Generate three-dimensional geometric assumptions about physical defects at the target spatial location of the three-dimensional structured model;

[0072] The system generates 3D geometric hypotheses of physical defects at target spatial locations within a 3D structured model. Specifically, this includes: training a generative adversarial network (GAN) based on historical bridge defect data. The GAN comprises a generator network and a discriminator network. The generator network generates 3D geometric features for different types of defects, while the discriminator network evaluates the realism of these features. The 3D structured model is divided into multiple detection regions, each with a size determined by a preset maximum physical defect size, and overlapping areas exist between adjacent regions. A defect feature library is constructed, containing different types of defects. Within each detection region, multiple different types of physical defect hypotheses are generated based on the GAN and the defect feature library. Structural stress distribution characteristics and material property parameters are extracted from each detection region. A compatibility score is calculated between each physical defect hypothesis and the structural stress distribution characteristics and material property parameters of the detection region. The physical defect hypothesis with the highest compatibility score is selected as the 3D geometric hypothesis.

[0073] This step is to generate a three-dimensional geometric hypothesis of physical defects at a specific location of the bridge three-dimensional model using the generative adversarial network (GAN) technique. The system first trains a generative adversarial network model based on historical bridge defect data. The historical defect data includes three-dimensional geometric features, location information, structural parameters, and other information of different types of defects, which can be obtained through manual annotation or automatic extraction. The system uses these data to train the generative adversarial network, enabling it to learn the intrinsic characteristics and distribution rules of defects.

[0074] The generative adversarial network consists of two parts: the generative network and the discriminative network. The generative network receives random noise as input and generates three-dimensional geometric features similar to real defects through operations such as deconvolution and upsampling. The discriminative network receives generated features and real features and determines whether the input features come from the real distribution through operations such as convolution and pooling. The two networks interact during training, with the generative network generating realistic defect features and the discriminative network distinguishing between generated and real features, ultimately achieving the goal of generating defect features that are difficult to distinguish from real ones.

[0075] To improve the efficiency and accuracy of defect generation, the system uses a hierarchical detection strategy. First, the three-dimensional structured model is divided into multiple detection regions of the same size, with each region's size determined by the preset maximum defect size, such as 1m x 1m x 1m. A certain overlap, such as 20%, is required between adjacent detection regions to avoid defects being segmented. Then, the system constructs a defect feature library containing three-dimensional geometric features of different types of defects, such as cracks, voids, and rust. These features can be extracted from historical data or designed manually by experts.

[0076] For each detection region, the system first randomly selects some defect features from the defect feature library as conditional input for the generative network. At the same time, the system also generates some random noise as random input for the generative network. The generative network generates multiple physical defect hypotheses of different types and shapes based on the conditional input and random input, with each hypothesis being a three-dimensional geometric model containing information such as the location, size, and shape of the defect.

[0077] To evaluate the reasonableness of these defect hypotheses, the system needs to consider the actual structural state of the detection region. Specifically, the system extracts the structural stress distribution features and material property parameters of the detection region from the three-dimensional structured model. The structural stress distribution features describe the stress size and direction of each point in the region, and the material property parameters describe the physical properties of the materials in the region, such as the elastic modulus and Poisson's ratio. The system compares each defect hypothesis with the extracted features and calculates their compatibility scores. The compatibility score considers the degree of agreement between the defect hypothesis and the actual structural state, with a higher score indicating a more reasonable defect hypothesis.

[0078] Finally, the system selects the defect hypothesis with the highest compatibility score as the three-dimensional geometric hypothesis for the detection region. This hypothesis not only matches the general characteristics of the defect but also aligns with the actual structural state, having a higher degree of credibility. By repeating the above process for all detection regions, the system ultimately obtains the three-dimensional geometric hypothesis of the physical defect for the entire bridge model.

[0079] S103, projecting the three-dimensional geometric hypothesis to the corresponding multiple two-dimensional image planes based on the spatial mapping relationship to generate multiple two-dimensional synthetic views containing the physical defect;

[0080] This step is to project the three-dimensional defect geometric hypothesis generated in the previous step onto the two-dimensional image plane to generate a synthetic view containing the physical defect. The system first needs to determine the corresponding position of the three-dimensional defect hypothesis in the integrated image according to the spatial mapping data field. Since each image pixel has a mapping relationship with the three-dimensional model, the system can convert the three-dimensional coordinates of the defect hypothesis into image coordinates to obtain the projection area of the defect in the image.

[0081] In order to generate realistic defect synthetic views, the system needs to consider factors such as the geometric shape, texture characteristics, and lighting conditions of the defect. For simple defects such as cracks, rust, etc., the system can directly draw the corresponding patterns and textures within the projection area; for complex defects such as cavities, shedding, etc., the system can use three-dimensional rendering techniques such as ray tracing, volume rendering, etc. to generate defect images with realistic lighting effects and occlusion relationships. At the same time, in order to improve the visual coherence of the synthetic view, the system also needs to perform smooth transition processing on the defect edge to make it naturally blend with the surrounding background.

[0082] Since the integrated image is collected from multiple perspectives, each defect hypothesis will correspond to multiple image planes. The system needs to generate corresponding defect projections in images of different perspectives and maintain consistency between projections. In order to achieve this, the system can use multi-view geometry principles such as epipolar constraint, scale-invariant features, etc. to jointly optimize the defect projections so that their positions, shapes, textures, etc. remain consistent under different perspectives. In this way, the system can generate multiple two-dimensional synthetic views containing the physical defect, providing intuitive visual reference for subsequent defect analysis.

[0083] S104, calculating the image similarity between the multiple two-dimensional synthetic views and the integrated image data to obtain a confidence score representing the credibility of the three-dimensional geometric hypothesis;

[0084] This step is to evaluate the similarity between the generated defect synthetic view and the actual image data, and obtain the confidence score of the defect hypothesis. The system first needs to select a suitable image similarity measurement method, such as normalized cross correlation (Normalized Cross Correlation), structural similarity index (Structural Similarity Index), etc. These methods can measure the similarity between two images from different angles, such as pixel value difference, structural similarity, etc.

[0085] For each defect synthetic view, the system finds its corresponding projection area in the integrated image data, and extracts the image block of this area. Then, the system calculates the similarity score between the synthetic view and the image block. The higher the similarity score, the closer the synthetic view is to the actual image, and the higher the credibility of the defect hypothesis. Considering the differences in image quality and occlusion under different viewing angles, the system needs to weight the similarity scores of multiple viewing angles and obtain the final confidence score.

[0086] To improve the reliability of the confidence score, the system can introduce a multi-scale analysis strategy. Specifically, the system first scales the synthetic view and the image block to different scale levels, then calculates the similarity score at each scale, and finally integrates the scores of different scales to obtain a more robust confidence score. This method can effectively reduce the influence of image noise and detail changes on similarity calculation, and improve the accuracy of evaluation results.

[0087] S105, when the confidence score is greater than the preset threshold, confirming that the physical defect exists in the target space position;

[0088] This step is to determine whether the physical defect actually exists according to the confidence score. The system first needs to set a confidence threshold as the standard for judging whether the defect exists. This threshold can be set according to historical data and expert experience, or can be adjusted adaptively through cross-validation and other methods.

[0089] When the confidence score of a certain defect hypothesis is greater than the preset threshold, the system considers that the defect actually exists in the corresponding target space position and needs to be further analyzed and processed. Conversely, if the confidence score is lower than the threshold, the system considers that the defect hypothesis is not established and can be ignored.

[0090] S106, extracting the structural function attribute of the target space position from the three-dimensional structured model, and combining the type of the physical defect to generate an analysis result containing defect position information, defect type and risk level.

[0091] This step is to conduct a comprehensive analysis of the confirmed physical defects and generate the final analysis results. The system first extracts the structural function attributes of the location where the defect is located from the three-dimensional structured model, such as the component it belongs to, the stress situation, material properties, etc. These attribute information is very important for evaluating the severity and impact range of the defect.

[0092] Next, the system combines the type of defect (such as cracks, holes, etc.) and the structural function attributes to conduct risk assessment of the defect. Different types of defects have different effects on the structure, and corresponding evaluation standards and algorithms need to be used. For example, for crack-type defects, the system can calculate the length, width, and depth of the crack, and consider the stress state of the part where the crack is located to evaluate the degree of impact on the structural safety; for hole-type defects, the system can analyze the size, location, and integrity of the surrounding material to determine whether it will cause local instability or damage to the structure.

[0093] According to the results of risk assessment, the system can classify defects into different risk levels, such as high risk, medium risk, low risk, etc. The classification criteria can refer to relevant industry standards and standards. For defects of different risk levels, the system can also give corresponding treatment suggestions and maintenance schemes, such as reinforcement, replacement, monitoring, etc., for reference by maintenance personnel.

[0094] Finally, the system integrates the defect location information, defect type, risk level, and treatment suggestions, etc. into the final analysis results. The analysis results can be presented in the form of reports, charts, etc. to help maintenance personnel intuitively understand the health status of the bridge and timely discover and handle safety hazards. At the same time, the system can also store the analysis results in the database as historical records and the basis for big data analysis, providing data support for the long-term maintenance and management of the bridge.

[0095] In the above embodiment, the one-to-one correspondence between the bridge three-dimensional structured model and the pixel coordinates of the aerial-terrestrial integrated image data is achieved by constructing the spatial mapping data field, so that the physical defect geometry hypothesis generated in the three-dimensional space can be accurately projected onto multiple two-dimensional image planes. The similarity comparison between the multiple two-dimensional synthetic views generated based on the projection and the actual image data can verify the accuracy of the physical defect hypothesis. When the similarity reaches the preset threshold, combined with the structural function attributes and defect types of the target location, the physical defect can be accurately positioned and risk level evaluated, improving the accuracy of defect detection. At the same time, by introducing structural function attribute analysis, the impact of defects on structural safety can be more comprehensively evaluated.

[0096] The above technical solution realizes accurate detection and positioning of physical defects of a bridge, but relying only on detection and evaluation of existing defects can not be able to discover potential structural safety hazards in time. Therefore, the application also provides a potential structural weak area identification method based on intact surface area analysis. The method can discover potential weak areas when structural defects have not yet fully appeared by analyzing the correlation between image texture features of the intact surface area and a theoretical stress field, thereby realizing early warning of structural safety risks. The following describes the potential structural weak area identification method based on intact surface area analysis in the embodiments of the application in combination with Figure 2 The potential structural weak area identification method based on intact surface area analysis in the embodiments of the application is described as follows:

[0097] Please refer to Figure 2 A flowchart of the potential structural weak area identification method based on intact surface area analysis in the embodiments of the application is shown in FIG. 2.

[0098] S201, determining an area not containing a physical defect as an intact surface area;

[0099] This step is to label an area not containing a physical defect as an intact surface area on the basis of existing defect detection results. The system first analyzes a three-dimensional structured model to find an area not appearing in the physical defect hypothesis generated in S102, i.e., an area with a complete surface and no obvious damage. These areas can have potential structural weak problems and need to be further analyzed, although they have not yet shown obvious defect features.

[0100] The system can use spatial indexing, area division, and other techniques to quickly locate and extract the intact surface area. Meanwhile, to ensure the accuracy of analysis, the system can further screen and merge the intact area, eliminate areas with too small areas and irregular shapes, and merge adjacent intact areas into larger analysis units to reduce computational complexity.

[0101] S202, extracting image texture corresponding to the intact surface area from the aerial-terrestrial integrated image data through a spatial mapping data field;

[0102] This step is to extract texture information of the intact surface area from the aerial-terrestrial integrated image data by using the established spatial mapping relationship. The system first finds corresponding image pixel coordinates in the spatial mapping data field according to the three-dimensional spatial coordinates of the intact area determined in S201. Since the spatial mapping data field records a one-to-one correspondence between the three-dimensional model and the image, the system can accurately map the intact area to the image space.

[0103] Then, the system extracts an image block of a certain size centered at the mapped pixel coordinates as the texture information of the intact region. The size of the image block can be set according to actual needs and computing performance, and is usually selected to cover the main features of the intact region, such as 64x64 pixels or 128x128 pixels. In order to obtain as much texture information of the region as possible, the system can repeat the extraction process on images of different viewing angles to obtain multiple image blocks.

[0104] Finally, the system pre-processes the extracted image blocks, such as grayscale, denoising, etc., to facilitate subsequent feature analysis. At the same time, the system can also splice and align the image blocks according to their positional relationship in the three-dimensional space to obtain a complete texture map of the intact region.

[0105] S203, calculate the gradient direction of each pixel point in the image texture to generate a texture direction field;

[0106] The system calculates the gradient direction of each pixel point in the image texture to generate a texture direction field, which is used to represent the micro-topography of the intact surface region.

[0107] This step is to perform gradient analysis on the extracted intact region texture map to obtain a texture direction field reflecting the micro-topography characteristics of the region. The gradient is the direction of the fastest change of pixel value in the image, which can be used to describe the texture direction and edge information of the image. The system calculates the gradient size and direction of each pixel point in the texture map in the horizontal and vertical directions to obtain the gradient vector of the pixel point.

[0108] Common gradient calculation methods include Sobel operator, Prewitt operator, Canny operator, etc. Taking the Sobel operator as an example, two 3x3 convolution kernels are used to convolve the image to obtain the horizontal and vertical gradients respectively:

[0109] Gx = [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]] * A

[0110] Gy = [[-1, -2, -1], [0, 0, 0], [1, 2, 1]] * A

[0111] Where A is the image region, and Gx and Gy are the horizontal and vertical gradients respectively. The size and direction of the gradient can be calculated by:

[0112] G = sqrt(Gx^2 + Gy^2)

[0113] theta = arctan(Gy / Gx)

[0114] The system calculates the gradient size and direction of each pixel point of the texture map and quantizes the gradient direction to obtain a direction field with the same size as the texture map. Each element in the direction field is an angle value representing the texture direction of the pixel point at that position. Due to the influence of factors such as light and noise, the direction field may have some interference and discontinuity. In order to improve the stability and robustness of the direction field, the system can perform smoothing and regularization processing on it, such as using a Gaussian filter to reduce noise and using a Markov random field model to globally optimize the direction.

[0115] After the above processing, the system obtains the texture direction field of the intact surface area, which is used to represent the micro-topographic features of the area. The consistency and continuity of the direction in the texture direction field can reflect the organizational structure and stress state of the surface material, providing a basis for subsequent analysis of weak structure areas.

[0116] S204, calculating a theoretical stress direction field of the intact surface area according to the three-dimensional structured model;

[0117] The system calculates the theoretical stress direction field of the intact surface area according to the three-dimensional structured model. The theoretical stress direction field is used to represent the stress transmission direction when the structure is under stress. Specifically, the surface shape parameters of the intact surface area are obtained, including the height variation and inclination angle of the curved surface. The gradient variation direction of the surface shape parameters is calculated, which represents the maximum variation direction of the curved surface at each point. The stress flow direction is determined according to the gradient variation direction, which represents the stress transmission path when the structure is under load. The theoretical stress direction field is generated based on the stress flow direction, which is used to represent the stress transmission direction when the structure is under stress.

[0118] This step is to calculate the stress distribution characteristics of the intact surface area under ideal conditions based on its geometric shape and stress state, and obtain the theoretical stress direction field. Unlike the actual physical stress field, the theoretical stress direction field is a mathematical model of stress distribution based on material mechanics and structural mechanics theory under simplified assumptions. Although there is some difference from the actual situation, it can reflect the basic laws and trends of structure stress.

[0119] The system first extracts the shape parameters of the intact surface area from the three-dimensional structured model, including the size, thickness, curvature, etc. of the area. These parameters can be obtained by geometric measurement and calculation of the three-dimensional model. Then, the system establishes a mathematical description model of the area based on its shape parameters, such as representing the curved surface in the form of a parameter equation, an implicit equation, etc.

[0120] After obtaining the mathematical model of the region, the system begins to calculate its theoretical stress distribution. Specifically, the system first applies a virtual unit load to the region, such as applying a unit pressure or tension on the boundary of the region. Then, the system calculates the stress components of each point in the region under the load according to the material constitutive equation and the balance equation. For simple geometric shapes and load conditions, the stress distribution can be directly solved using analytical methods; for complex cases, numerical methods such as finite element method, boundary element method, etc. are needed to discretize the region into multiple elements and then solve the stress distribution on the elements.

[0121] Through the above calculation, the system obtains the theoretical stress tensor of each point on the intact surface region. The stress tensor is a 3x3 matrix that describes the normal stress and shear stress components in three orthogonal directions at that point. In order to obtain the stress direction field, the system needs to perform principal stress analysis on the stress tensor, i.e. to solve the eigenvalues and eigenvectors of the stress tensor. The eigenvalues represent the magnitude of the principal stress, and the eigenvectors represent the direction of the principal stress. The system takes the eigenvector of the maximum principal stress as the theoretical stress direction of the point, and combines the stress directions of all points to obtain the theoretical stress direction field of the intact surface region.

[0122] S205, calculate the direction vector included angle of the texture direction field and the theoretical stress direction field at the corresponding position, and define the region with a direction vector included angle less than a preset included angle threshold as an abnormal correlation region;

[0123] This step is to find the region with abnormal correlation between the texture direction field and the theoretical stress direction field by comparing the differences between the two fields, as the potential structural weak area. Under normal circumstances, the texture direction of the intact surface region should be basically consistent with its stress direction, i.e. the microstructure of the material will arrange along the direction of stress transmission. Therefore, if there is a large deviation between the texture direction and the theoretical stress direction, it may mean that there is a potential structural problem in this region, such as material degradation, internal damage, etc.

[0124] The system first registers the texture direction field and the theoretical stress direction field, making them aligned in the same coordinate system. Since both direction fields are defined on the intact surface region and have a mapping relationship with the three-dimensional structured model, they can be unified to the same coordinate system through interpolation and other methods.

[0125] Then, the system traverses each element in the direction field and calculates the included angle between the texture direction vector and the theoretical stress direction vector. The included angle can be calculated by the dot product of the two vectors:

[0126] cos(theta) = (v1 · v2) / (|v1| × |v2|)

[0127] where v1 and v2 are the texture direction vector and the theoretical stress direction vector respectively, and theta is the included angle between the two vectors, taking a value in the range of [0, pi]. The smaller the included angle, the closer the two directions; the larger the included angle, the more deviated the two directions.

[0128] The system compares the calculated included angle with a preset included angle threshold value to find the area where the included angle is less than the threshold value. The selection of the included angle threshold value needs to be determined according to actual engineering experience and theoretical analysis, and is usually a small angle value, such as 10° or 20°. The area where the included angle is less than the threshold value means that the texture direction is basically consistent with the theoretical stress direction, and belongs to the normal correlation area; while the area where the included angle is greater than the threshold value means that the texture direction deviates significantly from the theoretical stress direction, and belongs to the abnormal correlation area.

[0129] In order to improve the reliability of the identification of the abnormal correlation area, the system can further screen and merge the preliminary identification results. For example, the abnormal areas with small area and scattered distribution are removed, and adjacent abnormal areas are merged into larger connected areas. At the same time, other prior knowledge such as structural stress characteristics and material properties can be combined to verify and correct the abnormal correlation area to reduce misjudgment.

[0130] S206, marking the abnormal correlation area on the three-dimensional structured model, and determining the abnormal correlation area as a potential structural weak area.

[0131] This step is to map the identified abnormal correlation area back to the three-dimensional structured model and determine it as a potential structural weak area. Although the abnormal correlation area has not shown obvious physical defects in the macroscopic view, there may be micro-damage or deterioration inside, which poses a potential threat to the safety and durability of the structure and needs to be paid enough attention and disposed.

[0132] The system first converts the spatial coordinates of the abnormal correlation area into position information in the three-dimensional structured model, that is, through the previously established spatial mapping relationship, the abnormal area is mapped from the image space to the model space. Then, the system marks the abnormal correlation area on the three-dimensional model in a conspicuous way, such as using special colors, symbols, labels, etc., to facilitate intuitive viewing and positioning.

[0133] For the marked abnormal correlation area, the system determines it as a potential weak area of the structure. Unlike physical defects, potential weak areas have not yet developed to the macroscopically visible damage stage, but have shown abnormal characteristics of microstructure and stress. If not disposed in time, the potential weak area may further deteriorate into an actual defect under the long-term action of external load and environmental factors, eventually endangering the safety of the structure.

[0134] Therefore, after identifying the potential structural weak area, the system also needs to give corresponding treatment suggestions according to its severity and influence range. For weak areas with lower severity, regular monitoring and evaluation can be recommended to track their development trend; for weak areas with higher severity, maintenance and reinforcement measures such as surface treatment, sticking reinforcement, replacement repair, etc. need to be taken in time to eliminate safety hazards.

[0135] At the same time, the system can also enter the identified potential weak area information into the health file of the structure as important historical data to support subsequent maintenance decision-making and big data analysis. By continuously tracking and analyzing the evolution law of potential weak areas, the degradation trend of the structure can be discovered early, and the early warning and prevention of structural safety can be realized.

[0136] In the above embodiment, by analyzing the relationship between the image texture direction field of the intact surface area and the theoretical stress direction field, the potential structural weak area can be identified. Since the micro-topography of the structure surface is often related to the internal stress distribution, by comparing the angle between the texture direction and the theoretical stress direction, the abnormal stress transmission area can be found. This method not only can detect the physical defects that have already appeared, but also can predict the areas where damage may occur. By marking the abnormal correlation area on the three-dimensional structured model, the spatial distribution of the potential risk area is intuitively displayed. This method based on texture stress correlation analysis helps to realize the preventive maintenance of bridge structure.

[0137] Further, in another embodiment, after determining the abnormal correlation area as a potential structural weak area, further comprising: calculating the distribution density of the abnormal correlation area, the distribution density representing the proportion of the abnormal correlation area in a unit area;

[0138] Analyzing the spatial continuity of the abnormal correlation area, the spatial continuity representing the connection degree between adjacent abnormal correlation areas;

[0139] Determining the weak area expansion trend according to the distribution density and the spatial continuity, the weak area expansion trend representing the development direction of the potential structural weak area, specifically including: dividing the distribution density of the abnormal correlation area into multiple density level intervals; calculating the change rate of the spatial continuity along different directions, and determining the gradual change direction of the distribution density from low to high; determining the weak area expansion trend based on the matching result corresponding to the gradual change direction and the direction with the largest change rate;

[0140] Based on the weak area expansion trend, the potential structural weak area is given a risk warning.

[0141] Firstly, the system calculates the distribution density of the abnormal correlation area, that is, the proportion of the abnormal correlation area in the unit area. The system divides the intact surface area into regular grid cells, and calculates the area proportion of the abnormal correlation area in each cell to obtain the density matrix reflecting the spatial distribution characteristics of the weak area. The area with larger numerical value in the matrix represents that the abnormal correlation area is dense and the weak degree is high, while the area with smaller numerical value represents that the abnormal correlation area is sparse and the weak degree is light.

[0142] Secondly, the system analyzes the spatial continuity of the abnormal correlation area, that is, the connection degree between adjacent areas. The system uses the graph theory method to abstract the abnormal correlation area as the node of the graph and the connection relationship between adjacent areas as the edge of the graph to construct an undirected graph model. By calculating parameters such as the number of connected components and the minimum spanning tree, the overall continuity of the weak area is evaluated. At the same time, the system also calculates the variation rate of spatial continuity in different directions to reflect the connectivity variation characteristics of the weak area in each direction.

[0143] Then, the system comprehensively considers the distribution density and spatial continuity to determine the expansion trend of the weak area. The system divides the distribution density matrix into different interval levels to analyze the gradual change characteristics of the density in space and determine the dominant direction from the low-density area to the high-density area. At the same time, the directional variation rate of spatial continuity is matched with the density gradual change direction to find the area with consistent direction and maximum variation rate as the key direction of the expansion of the weak area.

[0144] Finally, the system gives a risk warning according to the expansion trend of the weak area. The system predicts the development of the weak area in the future period of time, identifies the area with high risk level, and gives the corresponding warning information such as risk level, key position, and recommended measures. The warning information is sent to the management personnel in time through three-dimensional visualization, report pushing, and other ways. At the same time, the system can also optimize the inspection and monitoring scheme to intensively strengthen the on-site investigation of the weak area.

[0145] In the above embodiment, the distribution density of the abnormal correlation area is calculated and the spatial continuity thereof is analyzed, and the expansion trend of the weak area is determined based on the two characteristics, thereby realizing the quantitative evaluation of the development trend of the potential structural weak area. The distribution density reflects the concentration degree of the structural defects, and the spatial continuity represents the connection state of the defects. The combination of the two parameters can comprehensively describe the spatial distribution characteristics of the weak area. When the distribution density of the abnormal correlation area is high and has good spatial continuity, it indicates that there is significant correlation between these areas, and the weak area is likely to continue to expand along this correlation. By analyzing the expansion trend, the development direction of the weak area can be predicted, thereby realizing the early warning of the structural safety risk and reducing the possibility of structural failure.

[0146] The system in the embodiments of the present application is described from the perspective of hardware processing below. Please refer to Figure 3 FIG. 1 is a schematic diagram of an entity device structure of an air-ground integrated image data intelligent analysis system provided by the embodiments of the present application.

[0147] It should be noted that Figure 3 The structure of the system shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0148] As Figure 3 shown, the system includes a central processing unit (CPU) 301 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or loaded from a storage portion 308 into a random access memory (RAM) 303, such as performing the method in the above embodiments. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, the ROM 302 and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0149] The following components are connected to the I / O interface 305: an input portion 306 including a camera, an infrared sensor and the like; an output portion 307 including a liquid crystal display (LCD), a speaker and the like; a storage portion 308 including a hard disk and the like; and a communication portion 309 including a network interface card such as a LAN (Local Area Network) card, a modem and the like. The communication portion 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory and the like is mounted on the drive 310 as needed, so that a computer program read therefrom is installed into the storage portion 308 as needed.

[0150] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising computer programs for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present application are executed.

[0151] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, in which a computer readable program is carried. Such a propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above.

[0152] The computer program product of the present application can be a computer program embodied on a tangible medium or transmitted from a storage medium to a computer or a processor. The computer program product can be a plug-in, a cartridge or an external interface with the computer or processor. The computer program product can also be loaded onto a computer or a processor to produce a specific machine, thereby producing the machine implemented process for the computer or processor. The computer program product interfaces with the computer or processor to perform the operations described in the embodiments of the present application.

[0153] As another aspect, the present application also provides a computer readable storage medium, which can be included in the system described in the above embodiments, or can exist separately without being assembled into the system. The storage medium carries one or more computer programs, which, when executed by a processor of a system, cause the system to implement the method provided in the above embodiments.

[0154] The above described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; even though the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0155] In the above embodiments, according to the context, the term "when" can be interpreted as "if" or "after" or "in response to determining" or "in response to detecting". Similarly, according to the context, the phrase "upon determining" or "if detecting (the stated condition or event)" can be interpreted as "if determining" or "in response to determining" or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)".

[0156] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk) and the like.

[0157] Those of ordinary skill in the art understand that all or part of the processes in the above embodiments can be implemented by a computer program to instruct the relevant hardware, which can be stored in a computer readable storage medium. The program can include the processes of the above method embodiments when executed. The aforementioned storage medium includes ROM or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

Claims

1. An aerial-ground integrated image data intelligent analysis method, characterized in that, The method comprises the following steps: constructing a spatial mapping data field according to the corresponding air-ground integrated image data of the bridge, the spatial mapping data field being used to establish a one-to-one spatial mapping relationship between a three-dimensional structured model of the bridge and pixel coordinates of the air-ground integrated image data; generating a three-dimensional geometric hypothesis of a physical defect at a target spatial position of the three-dimensional structured model; projecting the three-dimensional geometric hypothesis to corresponding multiple two-dimensional image planes based on the spatial mapping relationship to generate multiple two-dimensional synthetic views containing the physical defect; calculating image similarity between the multiple two-dimensional synthetic views and the air-ground integrated image data to obtain a confidence score representing credibility of the three-dimensional geometric hypothesis; when the confidence score is greater than a preset threshold, confirming that the physical defect exists at the target spatial position; extracting a structural functional attribute of the target spatial position from the three-dimensional structured model and generating an analysis result containing defect position information, defect type and risk level in combination with the type of the physical defect; determining a sound surface area not containing the physical defect; extracting an image texture corresponding to the sound surface area from the air-ground integrated image data through the spatial mapping data field; calculating a gradient direction of each pixel point in the image texture to generate a texture direction field, the texture direction field being used to represent a micro-topography trend of the sound surface area; calculating a theoretical stress direction field of the sound surface area according to the three-dimensional structured model, the theoretical stress direction field being used to represent a stress transmission direction when the structure is stressed; calculating a directional vector included angle between the texture direction field and the theoretical stress direction field at a corresponding position, and defining a region with the directional vector included angle less than a preset included angle threshold as an abnormal correlation region; marking the abnormal correlation region on the three-dimensional structured model, and determining the abnormal correlation region as a potential structural weak area.

2. The method of claim 1, wherein, The method of constructing a spatial mapping data field according to the corresponding air-ground integrated image data of the bridge specifically comprises: establishing a three-dimensional structured model containing geometric parameters and structural parameters based on design drawings of the bridge; determining camera pose parameters when the air-ground integrated image data is collected by using a visual odometry technology; extracting feature points in the air-ground integrated image data based on the camera pose parameters; establishing a spatial mapping relationship according to a matching relationship between the feature points and control points in the three-dimensional structured model to obtain a spatial mapping data field.

3. The method of claim 1, wherein, The method of generating a three-dimensional geometric hypothesis of a physical defect at a target spatial position of the three-dimensional structured model specifically comprises: training a generative adversarial network based on historical bridge defect data, the generative adversarial network comprising a generative network and a discriminative network, the generative network being used to generate three-dimensional geometric features of different types of defects, and the discriminative network being used to evaluate authenticity of the three-dimensional geometric features; dividing the three-dimensional structured model into multiple detection regions, a size of each detection region being determined according to a maximum size of a preset physical defect, and there being an overlapping region between adjacent detection regions; constructing a defect feature library containing different types of defects. In each of the detection regions, multiple different types of physical defect hypotheses are generated based on the generative adversarial network and the defect feature library; The structural stress distribution features and material characteristic parameters of each of the detection regions are extracted; The compatibility scores between each of the physical defect hypotheses and the structural stress distribution features and the material characteristic parameters of the detection regions are calculated; The physical defect hypothesis with the highest compatibility score is taken as the three-dimensional geometric hypothesis.

4. The method of claim 1, wherein, The method further comprises: The surface shape parameters of the intact surface region are obtained, including the height variation and the inclination angle of the curved surface; The gradient variation direction of the surface shape parameters is calculated, representing the maximum variation direction of the curved surface at each point; The stress flow direction is determined according to the gradient variation direction, representing the stress transmission path when the structure is under load; The theoretical stress direction field is generated based on the stress flow direction, which is used to represent the stress transmission direction when the structure is under load.

5. The method of claim 1, wherein, After determining the potential structural weak area by the abnormal correlation region, the method further comprises: The distribution density of the abnormal correlation region is calculated, representing the proportion of the abnormal correlation region in a unit area; The spatial continuity of the abnormal correlation region is analyzed, representing the connection degree between adjacent abnormal correlation regions; The weak area expansion trend is determined according to the distribution density and the spatial continuity, representing the development direction of the potential structural weak area; The risk warning of the potential structural weak area is performed based on the weak area expansion trend.

6. The method of claim 5, wherein, The weak area expansion trend is determined according to the distribution density and the spatial continuity, specifically comprising: The distribution density of the abnormal correlation region is divided into multiple density level intervals; The change rate of the spatial continuity in different directions is calculated, and the gradual change direction of the distribution density from low to high is determined; The weak area expansion trend is determined based on the matching result corresponding to the direction with the maximum change rate in the gradual change direction.

7. An air-ground integrated image data intelligent analysis system, characterized in that, The system comprises: One or more processors and a memory; the memory is coupled with the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors invoke the computer instructions to enable the system to perform the method of any one of claims 1-6.

8. A computer-readable storage medium comprising instructions, characterized in that, When the instructions run on the system, the system is enabled to perform the method of any one of claims 1-6.

9. A computer program product, characterised in that, When the computer program product runs on the system, the system is enabled to perform the method of any one of claims 1-6.

Images