Bridge construction quality and safety intelligent identification method and system based on multi-modal large model

Abstract

This invention relates to an intelligent identification method for bridge construction quality and safety based on a multimodal large model, comprising: receiving and preprocessing construction images uploaded by users through a front-end interactive module; extracting technical quality features and safety behavior features; determining the category of hidden dangers and obtaining the basic weights and model confidence levels; performing time-series evolution analysis; calculating a comprehensive risk index based on a multidimensional dynamic weighting method; determining the risk level; and generating a structured report containing rectification suggestions. This invention achieves integrated intelligent identification of bridge construction quality and safety, breaking through the limitations of traditional single-dimensional detection, providing managers with a global and intuitive basis for decision-making, significantly improving the efficiency and accuracy of supervision; realizing adaptive dynamic assessment and trend early warning of risks; and uniquely adding a time-series evolution analysis step, which makes the assessment results closer to the actual engineering situation and has a forward-looking early warning capability, transforming passive rectification into proactive prevention and control.

Description

Technical Field

[0001] This invention relates to the field of intelligent supervision technology for construction engineering, and in particular to an intelligent identification method and system for bridge construction quality and safety based on a multimodal large model. Background Technology

[0002] Currently, quality control and safety management during bridge construction mainly rely on manual inspections and post-construction rectification, which generally suffers from low regulatory coverage, strong subjectivity, and information lag. Although some construction sites have deployed video monitoring systems, their functions are mostly limited to video acquisition and storage, lacking the ability to deeply understand the semantics of image content, and thus unable to automatically identify specific technical violations and safety hazards such as insufficient rebar spacing or failure to wear safety belts while working at heights.

[0003] In recent years, multimodal large-scale models have made breakthroughs in the fields of image and text understanding and scene reasoning, making it possible to build intelligent monitoring systems. However, existing technical solutions mostly focus on single scenarios (such as safety helmet recognition only), lacking customized modeling for complex and dynamic scenarios such as bridge construction. In addition, existing systems usually treat quality issues and safety issues separately, failing to achieve collaborative analysis and comprehensive judgment of the two.

[0004] Therefore, there is an urgent need for a system that can integrate multimodal perception and professional knowledge to achieve integrated intelligent identification of quality and safety. Summary of the Invention

[0005] To address the issues of low efficiency in manual inspections, insufficient intelligence in existing monitoring systems, and fragmented quality and safety analysis, the primary objective of this invention is to provide a multimodal large-scale model-based intelligent identification method for bridge construction quality and safety. This method enables integrated intelligent identification of bridge construction quality and safety, adaptive dynamic risk assessment and trend early warning, and significantly improves the speed and standardization of on-site rectification response.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for intelligent identification of bridge construction quality and safety based on a multimodal large model, the method comprising the following sequential steps:

[0007] (1) Receive construction images uploaded by users through the front-end interaction module and perform preprocessing to obtain preprocessed images;

[0008] (2) The back-end intelligent analysis engine calls the first multimodal large model to extract the technical quality features of the bridge construction entity in the preprocessed image, and at the same time calls the second multimodal large model to extract the safety behavior features of the construction personnel and the working environment in the preprocessed image;

[0009] (3) Semantically match the extracted technical quality features and safety behavior features with the bridge construction domain knowledge graph built into the backend intelligent analysis engine to determine the hazard category and obtain the basic weight W corresponding to the hazard category. base and model confidence C conf ;

[0010] (4) Perform time-series evolution analysis: Obtain the historical image sequence of the same monitoring point collected and stored by the front-end interaction module at different time points, register the images of adjacent time points, and extract the size and quantity parameters from the feature regions corresponding to the hidden danger categories determined in step (3), and calculate the rate of change of size or quantity between adjacent time points. If the rate of change Exceeding the preset deterioration threshold Then the trend intensification coefficient is generated. and >1; if the rate of change Not exceeding the preset deterioration threshold Then let =1; The size and quantity parameters include linear dimensions, area, and count;

[0011] (5) Calculate the comprehensive risk index S based on the multidimensional dynamic weighting method: Obtain the current environmental parameters through environmental sensors deployed at the construction site or external meteorological service interfaces, and determine the time period parameters by parsing the construction image acquisition timestamps, and generate environmental correction factors E respectively. env and time period correction factor T period ; Set the basic weight W base Model confidence level C conf Environmental Correction Factor E env Time period correction factor T period As an input variable, if step (4) generates a trend intensification coefficient and If the value is greater than 1, then the synchronous superposition trend aggravation coefficient will be applied. Perform the multiplication operation to obtain the comprehensive risk index S;

[0012] (6) Compare the comprehensive risk index S with the preset grading threshold, determine the risk level, and generate a structured report containing rectification suggestions.

[0013] In step (4), the rate of change Calculate using the following formula: For potential hazards related to linear dimensions, ;

[0014] For area-related hazards For potential hazards related to counting, ; (1);

[0015] in, , , These are the linear size characteristic value, area characteristic value, and count characteristic value corresponding to the current time point t, respectively. , , These are the linear size characteristic value, area characteristic value, and count characteristic value corresponding to the previous adjacent time point t-1, respectively; The time interval between two adjacent time points;

[0016] k is the magnification factor, 0.5 ≤ k ≤ 2; ( ) indicates taking the maximum value between the value inside the parentheses and 0;

[0017] When the rate of change Greater than the preset deterioration threshold hour, Calculate according to formula (1); when the rate of change Less than or equal to the preset deterioration threshold hour, The value is 1.

[0018] In step (5), the environmental correction factor E env and time period correction factor T period The setting method is as follows:

[0019] If the current environmental parameters indicate rain, strong winds, or heavy fog, then the environmental correction factor E will be adjusted. env Set it to be greater than 1; otherwise, set the environmental correction factor E. env Set to 1;

[0020] If the current time period parameter indicates a nighttime construction period, then the time period correction factor T will be adjusted. period Set it to greater than 1; otherwise, adjust the time period correction factor T. period Set to 1.

[0021] In step (5), the multidimensional dynamic weighting method specifically refers to: weighting the basic weights W... base Model confidence level C conf Environmental Correction Factor E env Time period correction factor T period The comprehensive risk index S is obtained by multiplying the trend aggravation coefficient α together with the trend aggravation coefficient α.

[0022] ;

[0023] The environmental correction factor E env and time period correction factor T period As a dynamic variable: when severe weather or nighttime construction periods are detected, E will be... envor T period Setting it to a value greater than 1 causes the comprehensive risk index S to increase non-linearly under severe working conditions, thereby increasing the risk level; when normal weather or daytime construction periods are detected, E... env and T period All values ​​are 1.

[0024] In step (6), determining the risk level specifically refers to:

[0025] Multiple risk threshold ranges are preset, corresponding to extremely low, low, medium, high, and extremely high risk levels, respectively;

[0026] The threshold range into which the calculated comprehensive risk index S falls is taken as the final risk level, and it is visualized on the user interface using graphic symbols and color coding.

[0027] In step (6), the structured report includes a problem description, risk level, corresponding index of standard provisions, and rectification suggestions; the rectification suggestions are dynamically generated from a pre-set measures library based on the current construction environment parameters and risk level.

[0028] Another objective of this invention is to provide a system for intelligent identification of bridge construction quality and safety based on a multimodal large model, comprising:

[0029] The front-end interaction module is used to receive construction images uploaded by users and display the analysis results;

[0030] The backend intelligent analysis engine is used to analyze construction images;

[0031] The report generation module is used to generate structured analysis reports;

[0032] The time-series trend analysis unit is used to acquire historical image sequences of the same monitoring point, calculate the rate of change of hidden danger characteristics through image registration and feature comparison, and generate a trend aggravation coefficient α when the rate of change exceeds the preset deterioration threshold and transmit it to the risk dynamic rating unit.

[0033] The front-end interaction module is used to receive construction images uploaded by users and display the analysis results;

[0034] The backend intelligent analysis engine includes:

[0035] The first multimodal large model is used to identify technical quality issues.

[0036] The second multimodal large model is used to identify safety behaviors and environmental risks;

[0037] The knowledge graph matching unit has a built-in standard and specification library and a hidden danger feature knowledge graph in the field of bridge construction. It is used to map and match the semantic descriptions output by the first multimodal large model and the second multimodal large model with the standard and specification library to determine the specific category and basic attributes of the hidden danger.

[0038] The risk dynamic rating unit is used to calculate a comprehensive risk index based on the specific category and basic attributes of the hidden danger, combined with the current construction environment parameters and time parameters, through a multi-dimensional dynamic weighted algorithm, and to determine the risk level based on the comprehensive risk index.

[0039] As can be seen from the above technical solution, the beneficial effects of this invention are as follows: First, it achieves integrated intelligent identification of bridge construction quality and safety, breaking through the limitations of traditional single-dimensional detection: This invention adopts a dual-engine architecture with a first multimodal large model and a second multimodal large model working in parallel, extracting technical quality features and safety behavior features from construction images respectively, and performing semantic matching and fusion through a unified bridge construction knowledge graph, providing managers with a global and intuitive decision-making basis, significantly improving the efficiency and accuracy of supervision. Second, it introduces environmental correction factors, time-period correction factors, and time-series evolution analysis to achieve adaptive dynamic assessment and trend warning of risks: This invention dynamically acquires environmental parameters and time-period parameters, automatically calculates correction factors greater than 1, and uses multiplicative superposition to make the comprehensive risk index grow non-linearly, thereby automatically raising the warning level under adverse conditions; Third, this invention uniquely adds a time-series evolution analysis step, by calculating the rate of change of hidden danger features in historical images of the same monitoring point, generating a trend aggravation coefficient and participating in the risk index calculation. When a hidden danger deteriorates rapidly, the system not only outputs the current risk level, but also actively issues a trend deterioration warning, making the assessment results closer to the actual engineering situation and possessing... Fourth, it establishes an end-to-end structured report generation mechanism, significantly improving the speed and standardization of on-site rectification response: This invention not only outputs risk levels, but also automatically generates structured reports containing problem descriptions, risk levels, corresponding standard clause indexes, and dynamically matched rectification suggestions. Risk levels are intuitively displayed on the user interface through graphic symbols and color coding. This closed-loop design greatly reduces the time spent on manual analysis, consulting standards, and writing rectification notices, enabling frontline personnel to quickly understand the severity of problems and take targeted measures, effectively reducing the accident rate and improving the standardization and intelligence level of on-site quality and safety management. Attached Figure Description

[0040] Figure 1 This is a system structure block diagram of the present invention;

[0041] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0042] like Figure 2 As shown, a method for intelligent identification of bridge construction quality and safety based on a multimodal large model is presented. This method includes the following steps in sequence:

[0043] (1) Receive construction images uploaded by users through the front-end interaction module and perform preprocessing to obtain preprocessed images;

[0044] (2) The back-end intelligent analysis engine calls the first multimodal large model to extract the technical quality features of the bridge construction entity in the preprocessed image, and at the same time calls the second multimodal large model to extract the safety behavior features of the construction personnel and the working environment in the preprocessed image;

[0045] (3) Semantically match the extracted technical quality features and safety behavior features with the bridge construction domain knowledge graph built into the backend intelligent analysis engine to determine the hazard category and obtain the basic weight W corresponding to the hazard category. base and model confidence C conf ;

[0046] (4) Perform time-series evolution analysis: Obtain the historical image sequence of the same monitoring point collected and stored by the front-end interaction module at different time points, register the images of adjacent time points, and extract the size and quantity parameters from the feature regions corresponding to the hidden danger categories determined in step (3), and calculate the rate of change of size or quantity between adjacent time points. If the rate of change Exceeding the preset deterioration threshold Then the trend intensification coefficient is generated. and >1; if the rate of change Not exceeding the preset deterioration threshold Then let =1; The size and quantity parameters include linear dimensions, area, and count;

[0047] (5) Calculate the comprehensive risk index based on the multidimensional dynamic weighting method: Obtain the current environmental parameters through environmental sensors deployed at the construction site or external meteorological service interfaces, and determine the time period parameters by parsing the construction image acquisition timestamps, and generate environmental correction factors E respectively. env and time period correction factor T period ; Set the basic weight W base Model confidence level C conf Environmental Correction Factor E env Time period correction factor T period As an input variable, if step (4) generates a trend intensification coefficient and If the value is greater than 1, then the synchronous superposition trend aggravation coefficient will be applied. Perform the multiplication operation to obtain the comprehensive risk index S;

[0048] (6) Compare the comprehensive risk index S with the preset grading threshold, determine the risk level, and generate a structured report containing rectification suggestions.

[0049] In step (4), the rate of change Calculate using the following formula: For potential hazards related to linear dimensions, ;

[0050] For area-related hazards For potential hazards related to counting, ; (1);

[0051] in, , , These are the linear size characteristic value, area characteristic value, and count characteristic value corresponding to the current time point t, respectively. , , These are the linear size characteristic value, area characteristic value, and count characteristic value corresponding to the previous adjacent time point t-1, respectively; The time interval between two adjacent time points;

[0052] k is the magnification factor, 0.5 ≤ k ≤ 2; ( ) indicates taking the maximum value between the value inside the parentheses and 0;

[0053] When the rate of change Greater than the preset deterioration threshold hour, Calculate according to formula (1); when the rate of change Less than or equal to the preset deterioration threshold hour, The value is 1.

[0054] In step (5), the environmental correction factor E env and time period correction factor T period The setting method is as follows:

[0055] If the current environmental parameters indicate rain, strong winds, or heavy fog, then the environmental correction factor E will be adjusted. env Set it to be greater than 1; otherwise, set the environmental correction factor E. env Set to 1;

[0056] If the current time period parameter indicates a nighttime construction period, then the time period correction factor T will be adjusted. period Set it to greater than 1; otherwise, adjust the time period correction factor T. period Set to 1.

[0057] In step (5), the multidimensional dynamic weighting method specifically refers to: weighting the basic weights W... base Model confidence level C conf Environmental Correction Factor E env Time period correction factor T period The comprehensive risk index S is obtained by multiplying the trend aggravation coefficient α together with the trend aggravation coefficient α.

[0058] ;

[0059] The environmental correction factor E env and time period correction factor T period As a dynamic variable: when severe weather or nighttime construction periods are detected, E will be... env or T period Setting it to a value greater than 1 causes the comprehensive risk index S to increase non-linearly under severe working conditions, thereby increasing the risk level; when normal weather or daytime construction periods are detected, E... env and T period All values ​​are 1.

[0060] In step (6), determining the risk level specifically refers to:

[0061] Multiple risk threshold ranges are preset, corresponding to extremely low, low, medium, high, and extremely high risk levels, respectively;

[0062] The threshold range into which the calculated comprehensive risk index S falls is taken as the final risk level, and it is visualized on the user interface using graphic symbols and color coding.

[0063] In step (6), the structured report includes a problem description, risk level, corresponding index of standard provisions, and rectification suggestions; the rectification suggestions are dynamically generated from a pre-set measures library based on the current construction environment parameters and risk level.

[0064] like Figure 1 As shown, this system includes:

[0065] The front-end interaction module is used to receive construction images uploaded by users and display the analysis results;

[0066] The backend intelligent analysis engine is used to analyze construction images;

[0067] The report generation module is used to generate structured analysis reports;

[0068] The time-series trend analysis unit is used to acquire historical image sequences of the same monitoring point, calculate the rate of change of hidden danger characteristics through image registration and feature comparison, and generate a trend aggravation coefficient α when the rate of change exceeds the preset deterioration threshold and transmit it to the risk dynamic rating unit.

[0069] The front-end interaction module is used to receive construction images uploaded by users and display the analysis results;

[0070] The backend intelligent analysis engine includes:

[0071] The first multimodal large model is used to identify technical quality issues.

[0072] The second multimodal large model is used to identify safety behaviors and environmental risks;

[0073] The knowledge graph matching unit has a built-in standard and specification library and a hidden danger feature knowledge graph in the field of bridge construction. It is used to map and match the semantic descriptions output by the first multimodal large model and the second multimodal large model with the standard and specification library to determine the specific category and basic attributes of the hidden danger.

[0074] The risk dynamic rating unit is used to calculate a comprehensive risk index based on the specific category and basic attributes of the hidden danger, combined with the current construction environment parameters and time parameters, through a multi-dimensional dynamic weighted algorithm, and to determine the risk level based on the comprehensive risk index.

[0075] In summary, this invention achieves integrated intelligent identification of bridge construction quality and safety, breaking through the limitations of traditional single-dimensional detection. It employs a dual-engine architecture with a first multimodal large model and a second multimodal large model working in parallel, extracting technical quality features and safety behavior features from construction images respectively. Semantic matching and fusion are then performed using a unified bridge construction knowledge graph, providing managers with a comprehensive and intuitive decision-making basis, significantly improving regulatory efficiency and accuracy. Furthermore, it introduces environmental correction factors, time-period correction factors, and time-series evolution analysis to achieve adaptive dynamic risk assessment and trend warning. This invention dynamically acquires environmental and time-period parameters, automatically calculates correction factors greater than 1, and uses multiplicative superposition to make the comprehensive risk index grow non-linearly, thereby automatically raising the warning level under adverse conditions. Finally, this invention uniquely adds a time-series evolution analysis step, calculating the hidden danger characteristics in historical images of the same monitoring point. The system calculates the rate of change of a hazard, generates a trend aggravation coefficient, and participates in the risk index calculation. When a hazard rapidly deteriorates, the system not only outputs the current risk level but also proactively issues a trend deterioration warning. This makes the assessment results closer to the actual engineering situation and provides a forward-looking early warning capability, transforming passive rectification into proactive prevention and control. The system also constructs an end-to-end structured report generation mechanism, significantly improving the speed and standardization of on-site rectification response: This invention not only outputs the risk level but also automatically generates a structured report containing a problem description, risk level, corresponding standard clause index, and dynamically matched rectification suggestions. The risk level is intuitively displayed on the user interface through graphic symbols and color coding. This closed-loop design greatly reduces the time spent on manual analysis, consulting standards, and drafting rectification notices, enabling frontline personnel to quickly understand the severity of the problem and take targeted measures, effectively reducing the accident rate and improving the standardization and intelligence level of on-site quality and safety management.

[0076] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for intelligent identification of bridge construction quality and safety based on a multimodal large model, characterized in that: The method includes the following steps in sequence: (1) Receive construction images uploaded by users through the front-end interaction module and perform preprocessing to obtain preprocessed images; (2) The back-end intelligent analysis engine calls the first multimodal large model to extract the technical quality features of the bridge construction entity in the preprocessed image, and at the same time calls the second multimodal large model to extract the safety behavior features of the construction personnel and the working environment in the preprocessed image. (3) Semantically match the extracted technical quality features and safety behavior features with the bridge construction domain knowledge graph built into the backend intelligent analysis engine to determine the hazard category and obtain the basic weight W corresponding to the hazard category. base and model confidence C conf ; (4) Perform time-series evolution analysis: Obtain the historical image sequence of the same monitoring point collected and stored by the front-end interaction module at different time points, register the images of adjacent time points, and extract the size and quantity parameters from the feature regions corresponding to the hidden danger categories determined in step (3), and calculate the rate of change of size or quantity between adjacent time points. If the rate of change Exceeding the preset deterioration threshold Then the trend intensification coefficient is generated. and >1; if the rate of change Not exceeding the preset deterioration threshold Then let =1; The size and quantity parameters include linear dimensions, area, and count; (5) Calculate the comprehensive risk index S based on the multidimensional dynamic weighting method: Obtain the current environmental parameters through environmental sensors deployed at the construction site or external meteorological service interfaces, and determine the time period parameters by parsing the construction image acquisition timestamps, and generate environmental correction factors E respectively. env and time period correction factor T period ; The basic weight W base Model confidence level C conf Environmental Correction Factor E env Time period correction factor T period As an input variable, if step (4) generates a trend intensification coefficient and If the value is greater than 1, then the synchronous superposition trend aggravation coefficient will be applied. Perform the multiplication operation to obtain the comprehensive risk index S; (6) Compare the comprehensive risk index S with the preset grading threshold, determine the risk level, and generate a structured report containing rectification suggestions.

2. The intelligent identification method for bridge construction quality and safety based on a multimodal large model according to claim 1, characterized in that: In step (4), the rate of change Calculate using the following formula: For potential hazards related to linear dimensions, ; For area-related hazards ; For count-related hidden dangers, ; (1); in, , , These are the linear size characteristic value, area characteristic value, and count characteristic value corresponding to the current time point t, respectively. , , These are the linear size characteristic value, area characteristic value, and count characteristic value corresponding to the previous adjacent time point t-1, respectively; The time interval between two adjacent time points; k is the magnification factor, 0.5 ≤ k ≤ 2; ( ) indicates taking the maximum value between the value inside the parentheses and 0; When the rate of change Greater than the preset deterioration threshold hour, Calculate according to formula (1); when the rate of change Less than or equal to the preset deterioration threshold hour, The value is 1.

3. The intelligent identification method for bridge construction quality and safety based on a multimodal large model according to claim 1, characterized in that: In step (5), the environmental correction factor E env and time period correction factor T period The setting method is as follows: If the current environmental parameters indicate rain, strong winds, or heavy fog, then the environmental correction factor E will be adjusted. env Set it to be greater than 1; otherwise, set the environmental correction factor E. env Set to 1; If the current time period parameter indicates a nighttime construction period, then the time period correction factor T will be adjusted. period Set to greater than 1; Otherwise, adjust the time period correction factor T. period Set to 1.

4. The intelligent identification method for bridge construction quality and safety based on a multimodal large model according to claim 1, characterized in that: In step (5), the multidimensional dynamic weighting method specifically refers to: weighting the basic weights W... base Model confidence level C conf Environmental Correction Factor E env Time period correction factor T period The comprehensive risk index S is obtained by multiplying the trend aggravation coefficient α together with the trend aggravation coefficient α. ； The environmental correction factor E env and time period correction factor T period As a dynamic variable: when severe weather or nighttime construction periods are detected, E will be... env or T period Setting it to be greater than 1 makes the comprehensive risk index S grow non-linearly under harsh working conditions, thereby increasing the risk level; When normal weather or daytime construction periods are detected, E env and T period All values ​​are 1.

5. The intelligent identification method for bridge construction quality and safety based on a multimodal large model according to claim 1, characterized in that: In step (6), determining the risk level specifically refers to: Multiple risk threshold ranges are preset, corresponding to extremely low, low, medium, high, and extremely high risk levels, respectively; The threshold range into which the calculated comprehensive risk index S falls is taken as the final risk level, and it is visualized on the user interface using graphic symbols and color coding.

6. The intelligent identification method for bridge construction quality and safety based on a multimodal large model according to claim 1, characterized in that: In step (6), the structured report includes a problem description, risk level, corresponding index of standard provisions, and rectification suggestions; the rectification suggestions are dynamically generated from a pre-set measures library based on the current construction environment parameters and risk level.

7. A system for implementing the intelligent identification method for bridge construction quality and safety based on a multimodal large model as described in any one of claims 1 to 6, characterized in that: include: The front-end interaction module is used to receive construction images uploaded by users and display the analysis results; The backend intelligent analysis engine is used to analyze construction images; The report generation module is used to generate structured analysis reports; The time-series trend analysis unit is used to acquire historical image sequences of the same monitoring point, calculate the rate of change of hidden danger characteristics through image registration and feature comparison, and generate a trend aggravation coefficient α when the rate of change exceeds the preset deterioration threshold and transmit it to the risk dynamic rating unit. The front-end interaction module is used to receive construction images uploaded by users and display the analysis results.

8. The system according to claim 7, characterized in that: The backend intelligent analysis engine includes: The first multimodal large model is used to identify technical quality issues. The second multimodal large model is used to identify safety behaviors and environmental risks; The knowledge graph matching unit has a built-in standard and specification library and a hidden danger feature knowledge graph in the field of bridge construction. It is used to map and match the semantic descriptions output by the first multimodal large model and the second multimodal large model with the standard and specification library to determine the specific category and basic attributes of the hidden danger. The risk dynamic rating unit is used to calculate a comprehensive risk index based on the specific category and basic attributes of the hidden danger, combined with the current construction environment parameters and time parameters, through a multi-dimensional dynamic weighted algorithm, and to determine the risk level based on the comprehensive risk index.

Images