Construction site potential safety hazard intelligent judgment system and method based on AI large model

Abstract

The invention discloses a construction site potential safety hazard intelligent judgment system and method based on an AI large model. The system comprises an acquisition unit and an intelligent judgment unit. The acquisition unit is used for acquiring video data of a construction site in real time and transmitting the video data to the intelligent judgment unit; and the intelligent judgment unit is used for receiving the video data, analyzing the video data by using an AI large model so as to determine whether a potential safety hazard exists in the field or not, automatically generating a message for reminding and sending the message to a safety manager once the potential safety hazard is found, and receiving submitted rectification feedback information. According to the invention, accurate identification of more scenes, deeper safety knowledge application and continuous optimization learning can be realized, and the safety management level of a construction site is comprehensively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of construction safety, in particular to a construction site safety hazard intelligent judgment system and method based on AI large model. BACKGROUND

[0002] With the continuous development of the construction industry, the safety production management of the construction site is increasingly valued. However, the overall safety management in the construction field is still at a relatively extensive development stage, and the digitalization and intelligentization application lags significantly compared to the manufacturing and logistics industries. The current safety management mainly relies on the on-site inspection experience of management personnel to identify potential risks, but due to the complex and variable environment of the construction site and the limited knowledge reserve and energy of safety management personnel, it often leads to untimely inspection, incomplete records or incomplete inspection, thereby increasing the risk of accidents.

[0003] To improve the safety management efficiency of the construction site, the industry has begun to introduce artificial intelligence technology. For example, Guanglianda provides an AI recognition solution that can detect safety helmets, reflective clothing, etc. worn by construction personnel; the Xiaomeng AI intelligent glasses launched by Pinming have functions such as voice photographing, querying the construction knowledge base, etc. in addition to personnel clothing recognition; foreign companies have also developed wearable AR devices for application in municipal engineering and other scenarios. However, these existing products still have great limitations in terms of intelligence and scene coverage: they cannot identify the size, distance, depth, etc. of objects such as construction site components, equipment, building materials, and foundation pits, and it is difficult to automatically determine the size or distance safety hazards in complex scenarios according to building specifications. In addition, existing AI models lack deep interaction with management personnel and continuous learning capabilities, making it difficult to adapt to dynamic changes in construction site conditions and safety standards.

[0004] Therefore, to solve the above problems, a new construction site safety hazard intelligent judgment system and method is needed, which can achieve accurate identification of more scenarios, deeper application of safety knowledge, and continuous optimization learning, and comprehensively improve the safety management level of the construction site. SUMMARY

[0005] Therefore, the purpose of the present application is to overcome the defects in the prior art and provide a construction site safety hazard intelligent judgment system and method based on AI large model, which can achieve accurate identification of more scenarios, deeper application of safety knowledge, and continuous optimization learning, and comprehensively improve the safety management level of the construction site.

[0006] The construction site safety hazard intelligent judgment system based on AI large model of the present application comprises a collection unit and an intelligent judgment unit;

[0007] The collection unit is used for real-time collection and transmission of video data of the construction site to the intelligent judgment unit.

[0008] The intelligent judgment unit is configured to receive the video data and analyze the video data by using an AI large model to determine whether there is a safety hazard on the site. Once a safety hazard is found, a message is automatically generated to remind and send the message to a safety management personnel, and receive submitted rectification feedback information.

[0009] Further, the collection unit includes a camera terminal, a data transmission module, and a depth measurement algorithm module.

[0010] The camera terminal is configured to collect video data of the construction site in real time.

[0011] The data transmission module is configured to send the video data to the intelligent judgment unit.

[0012] The depth measurement algorithm module is configured to dynamically adjust the sampling frequency and video resolution according to the actual scene.

[0013] Further, the intelligent judgment unit includes a safety management collaboration platform and an AI large model. The safety management collaboration platform includes a safety management collaboration platform PC end and a safety management collaboration platform mobile end.

[0014] The safety management collaboration platform PC end includes a picture receiving module, an AI large model calling engine module, a problem marking module, a voice interaction module, a picture output module, and a safety specification module.

[0015] The picture receiving module is configured to receive camera picture streams from the collection unit in real time.

[0016] The AI large model calling engine module is configured to call the AI large model to intelligently analyze the video stream.

[0017] The problem marking module is configured to accurately locate the problem area.

[0018] The voice interaction module is configured to provide a convenient user operation interface.

[0019] The picture output module is configured to provide intuitive decision support for users.

[0020] The safety specification module is configured to provide safety standards and specification basis for users, find relevant safety operation guidance, regulatory requirements, and correct examples.

[0021] The safety management collaboration platform mobile end includes a business collaboration module, a message reminder push module, and a real-time picture display module.

[0022] The business collaboration module is configured to meet the business operation needs of the safety management personnel on the construction site.

[0023] The message reminder pushing module is used to realize timely reminding and pushing of messages.

[0024] The real-time picture display module is used to display real-time pictures with low delay.

[0025] Further, the AI large model performs scene recognition based on a GPT and a Segment Everything framework, extracts key targets and their parameters, and supports video scene analysis and safety hazard judgment.

[0026] Further, the AI large model supports intelligent collaboration between engineers and the large model, and completes the reasoning of the interaction intention of the engineer, response query, search, confirmation, and marking.

[0027] Further, the AI large model constructs a safety hazard knowledge base, optimizes the large model parameters based on a Transformer framework, updates the knowledge storage framework, and enhances the hazard identification capability.

[0028] Further, the AI large model has the ability of autonomous learning and optimization performance, and is suitable for safety specification updates and video data accumulation.

[0029] A judgment method for construction site safety hazard intelligent judgment by using a construction site safety hazard intelligent judgment system, comprising:

[0030] Object classification and identification: after receiving video data from the device, the video picture is divided into multiple regions, and the objects in each region are identified;

[0031] Safety specification reference: after object identification is completed, the constructed knowledge base is analyzed; wherein the knowledge base is a triple of conditions, targets and requirements, responsible for representing related safety specifications; the specification text of the original rule file is parsed by a language model, key information is extracted, and a rule knowledge management library is constructed; the matching algorithm is used to match the identified objects and scenes with the safety specifications in the knowledge base to determine whether there is a violation of the specifications;

[0032] Problem and hazard marking: on the basis of specification matching, potential problems and hazards are further analyzed to mark objects or behaviors that do not comply with safety specifications, and possible risks are evaluated.

[0033] Further, the video picture is divided into multiple regions, and the objects in each region are identified, which specifically includes:

[0034] A segmentation model based on EfficientSAM is constructed, the original video image is input into the network where the segmentation model is located, the object code is obtained after input, and the mask with prompt information is fused to obtain the final image after segmentation;

[0035] After the scene target is segmented, a classifier facing downstream tasks is constructed to further analyze the characteristics of each object, obtain its semantic information, and realize the identification and classification of the object without explicit labels according to the similarity of categories.

[0036] Further, the identified object and the scene are matched with safety specifications in a knowledge base by using a matching algorithm, specifically including:

[0037] The relationship between objects and the meaning of objects in a specific scene are understood by using prompt words and target analysis; the identified target is analyzed with specification articles in the knowledge base according to the existing target information and structured knowledge, and the matching between the hidden danger in the target and the specification articles is completed; the knowledge base stores rich background information and prior knowledge, and the knowledge base is continuously adjusted to obtain the best matching result.

[0038] The construction site safety hidden danger intelligent judgment system and method based on an AI large model has the advantages that the system can collect and analyze construction site data in real time, quickly, accurately and comprehensively identify safety hidden dangers by using an AI large model, realize the identification of safety hidden dangers from a single type problem of "yes or no", and expand to more detailed safety hidden danger identification including specific size problems, relative distance problems between targets and material state problems. The safety management collaboration platform automatically marks the hidden dangers in the video pictures and immediately sends a notification to the site manager, prompting the manager to take necessary measures in time, effectively reducing the risk of accidents, realizing the identification accuracy and comprehensiveness of construction site safety hidden dangers, assisting the on-site inspection work, and reducing the consumption of manpower. BRIEF DESCRIPTION OF DRAWINGS

[0039] The application will be further described below in combination with the drawings and embodiments:

[0040] Figure 1 The safety hidden danger intelligent judgment system architecture of the application is shown in the figure;

[0041] Figure 2 The safety hidden danger intelligent judgment method principle schematic diagram of the application is shown in the figure;

[0042] Figure 3 The network architecture diagram of the AI large model of the application is shown in the figure. DETAILED DESCRIPTION

[0043] The application will be further described below in combination with the drawings and embodiments:

[0044] The construction site safety hidden danger intelligent judgment system based on an AI large model is disclosed in the embodiment, which includes a collection unit and an intelligent judgment unit;

[0045] The acquisition unit is used for collecting and transmitting video data of the construction site to the intelligent judgment unit in real time.

[0046] The intelligent judgment unit is used for receiving the video data and analyzing by using an AI large model to determine whether there is a safety hazard on site, and once a safety hazard is found, a message is automatically generated and sent to the safety management personnel, and the submitted rectification feedback information is received.

[0047] The present application realizes real-time video acquisition and AI large model intelligent analysis of the construction site by the cooperation of the acquisition unit and the intelligent judgment unit, can automatically find and prompt safety hazards, and combines rectification feedback closed-loop management, significantly improves the comprehensiveness and accuracy of hazard identification, reduces manual patrol omissions; at the same time, the response speed and work efficiency of safety management are improved, and help the construction site to realize digital and intelligent safety management.

[0048] In the embodiment, the acquisition unit includes a plurality of acquisition devices, such as a camera terminal, a data transmission module, and a depth measurement algorithm module.

[0049] The camera terminal is used for collecting video data of the construction site in real time; the data transmission module is used for sending the video data to the intelligent judgment unit; and the depth measurement algorithm module is used for dynamically adjusting the sampling frequency and video resolution according to the actual scene. In order to ensure the continuous operation of the acquisition device in the construction site, mobile transmission and endurance scheme setting are needed, which will not be repeated here.

[0050] By introducing the depth measurement algorithm module in the acquisition unit, the sampling frequency and resolution can be dynamically adjusted according to the complexity of the scene, taking into account data accuracy and transmission efficiency; combined with the camera terminal and the data transmission module, multi-angle and multi-dimensional video data acquisition and efficient transmission are realized, which enhances the capture ability of details and spatial information of the construction site, and improves the accuracy and real-time performance of safety hazard determination.

[0051] In the embodiment, the intelligent judgment unit includes a safety management collaboration platform and an AI large model; the safety management collaboration platform includes a safety management collaboration platform PC and a safety management collaboration platform mobile terminal.

[0052] The PC end of the safety management collaboration platform comprises a picture receiving module, an AI large model calling engine module, a problem marking module, a voice interaction module, a picture output module and a safety specification module; the picture receiving module is configured to receive camera picture streams from the acquisition unit in real time; the AI large model calling engine module is configured to call the AI large model to intelligently analyze the video stream; the problem marking module is configured to realize accurate positioning of the problem area; the voice interaction module is configured to provide a convenient user operation interface; the picture output module is configured to provide intuitive decision support for the user; and the safety specification module is configured to provide safety standards and specification basis for the user, find relevant safety operation guidance, regulatory requirements and correct examples;

[0053] The mobile end of the safety management collaboration platform comprises a business collaboration module, a message reminding and pushing module and a real-time picture display module; the business collaboration module is configured to meet the business operation needs of the safety management personnel on the construction site; the message reminding and pushing module is configured to realize timely reminding and pushing of messages; and the real-time picture display module is configured to display real-time pictures with low delay.

[0054] By integrating the AI large model and the multifunctional safety management collaboration platform in the intelligent judgment unit, the whole-process closed-loop management from hidden danger detection, positioning and marking to specification guidance is realized; the PC end supports intelligent analysis and decision visualization, and the mobile end meets the collaboration and real-time viewing needs of the personnel on the site, thereby improving the man-machine interaction and mobility of the system and greatly enhancing the real-time performance, accuracy and compliance of hidden danger disposal.

[0055] In this embodiment, the AI large model performs scene recognition based on the GPT and Segment Everything frameworks, extracts key targets and their parameters, supports video scene analysis and safety hidden danger judgment. The AI large model supports intelligent collaboration between engineers and the large model, completes the interactive intention reasoning, response query, search, confirmation and marking of engineers. The AI large model builds a safety hidden danger knowledge base, optimizes the large model parameters based on the Transformer framework, updates the knowledge storage framework and enhances the hidden danger recognition capability. The AI large model has the ability of autonomous learning and optimization performance, and is suitable for safety specification updates and video data accumulation.

[0056] By combining the GPT and Segment Everything frameworks, the AI large model not only realizes accurate extraction of key targets and parameters, but also supports intelligent interaction and intention understanding with engineers; builds and continuously optimizes a safety hidden danger knowledge base, improves the recognition accuracy and knowledge update speed; and at the same time has the ability of autonomous learning, is suitable for specification updates and data accumulation, and greatly enhances the intelligent, adaptability and sustainable optimization capability of the system.

[0057] The intelligent determination system of the present application is a highly integrated intelligent solution, as shown in Figure 1 It consists of three core parts: acquisition device, safety management collaboration platform, and AI large model.

[0058] The acquisition device collects on-site data at the construction site and sends it to the platform through the network; the safety management collaboration platform processes the received data and interacts with the AI large model through API; the AI large model performs intelligent analysis and determination, and feeds back the safety hazard identification results to the platform, and finally presents them to the user.

[0059] Among them, the device layer is the starting point of the entire system, responsible for real-time collection of various types of data. These devices include infrared cameras, depth cameras, communication networks, etc., which can capture key information such as images and depth. The role of the device is not limited to data collection, but also includes preliminary processing of these raw data, and safe and reliable transmission to the safety management collaboration platform.

[0060] The safety management collaboration platform layer serves as middleware, undertaking key tasks of data management and application services. The platform receives raw data from devices, performs further cleaning, integration and analysis. It calls the API of the AI large model to convert data into useful information and provide support for decision-making. The platform is also responsible for managing user access permissions, handling data storage and backup, and providing a user interface to allow users to easily view and manipulate data.

[0061] The AI large model is the core of the entire system, responsible for performing advanced intelligent determination and analysis. The AI large model is usually built with EfficientSAM and CLIP, capable of handling complex recognition tasks such as presence or absence, distance, quantity relationship, material state, etc. It uses data transmitted from the platform to provide suggestions for safety hazards on the construction site through deep learning and incremental updates.

[0062] The present application also relates to a determination method for determining safety hazards on the construction site using the construction site safety hazard intelligent determination system of the above embodiment, as shown in Figure 2 , comprising:

[0063] Object classification and recognition: after receiving video data from the device, the video image is divided into multiple regions, and the objects in each region are identified;

[0064] Safety specification reference: After object recognition is completed, the constructed knowledge base is analyzed; wherein the knowledge base is a triple of conditions, targets and requirements, responsible for representing relevant safety specifications; the specification text of the original rule file is parsed through the language model, the key information is extracted, and the rule knowledge management base is constructed; the matching algorithm is used to match the safety specifications in the knowledge base according to the identified objects and scenes, to determine whether there is a violation of the specifications; wherein the industry standards or specifications can be used to construct the knowledge management base;

[0065] Problem hazard marking: on the basis of specification matching, further analyze potential problems and hazards, and mark the objects or behaviors that do not conform to the safety specifications, and evaluate the possible risks.

[0066] By introducing the three-step process of object classification recognition, safety specification reference and problem hazard marking, the structured understanding and intelligent analysis of the construction site video data are realized; combined with knowledge base matching and risk assessment, potential safety hazards can be accurately found and marked, improving the comprehensiveness and compliance of hazard determination, and significantly improving the intelligence and reliability of construction site safety management.

[0067] In this embodiment, the video picture is divided into multiple regions, and the objects in each region are identified, specifically including:

[0068] A segmentation model based on EfficientSAM is constructed, the original video image is input into the network where the segmentation model is located, the object code is obtained after the original video image is input into the network where the segmentation model is located, and the mask with prompt information is fused to obtain the final segmented image;

[0069] After the scene target is cut, a classifier for downstream tasks is constructed to further analyze the features of each object and obtain its semantic information, without explicit labels, but according to the similarity of categories to realize the recognition and classification of objects. The classifier forms an identification model.

[0070] Through the segmentation model based on EfficientSAM, the video picture is finely segmented, the object code is extracted, and the high-precision segmentation result is generated combined with the prompt information; then the identification model for downstream tasks is used to realize automatic classification and semantic recognition without labels according to the feature similarity, which improves the intelligence and adaptability of multi-target recognition in complex scenes, and enhances the accuracy and comprehensiveness of hazard determination.

[0071] In this embodiment, the matching algorithm is used to match the safety specifications in the knowledge base according to the identified objects and scenes, specifically including:

[0072] The relationship between objects and their meaning in a specific scene is understood by using cue words and target analysis. Based on existing target information and structured knowledge, traversal matching is performed to analyze the recognized target and the specification provisions in the knowledge base, and to complete the matching of potential hazards in the target and the specification provisions. The knowledge base stores rich background information and prior knowledge, thereby providing support for matching and judgment. The knowledge base is continuously adjusted to obtain the best matching result. Specification provisions can be obtained directly through language model processing or through expert screening.

[0073] By combining cue words with target analysis, the semantic understanding of object relationships and scenes is achieved. Based on a structured knowledge base and traversal matching algorithm, the recognized target is efficiently associated with specification provisions to accurately discover potential hazards. By dynamically optimizing the content of the knowledge base, the matching accuracy and adaptability are continuously improved, thereby significantly enhancing the intelligence and reliability of hazard determination.

[0074] As shown in Figure 3 , the network architecture diagram of the AI large model includes a segmentation model, an identification model, a structured knowledge base and a traversal matching algorithm, an interactive interface, and a dialogue module.

[0075] The interactive interface provides visual information display, enabling users to intuitively understand the analysis results of the model, and is also responsible for capturing user input, including voice, text, or other forms of feedback. The design of this interface needs to consider user experience and ease of use to ensure that users can easily interact with the system. The dialogue module includes a typical UI / UX dialogue system to establish a clear user profile and usage scenario, ensuring that the dialogue system can meet the needs and expectations of target users. In the user interface (UI) design, a simple and clear layout is used, with appropriate visual elements such as icons, colors, and animations to enhance the user interaction experience. In the user experience (UX) design, the dialogue is smooth and natural, and natural language processing technology is used to enable the system to understand and respond to user instructions and questions. At the same time, the dialogue system has learning ability and can be optimized according to user feedback to improve accuracy and personalized service. The dialogue system can understand user queries, instructions, or comments and convert them into inputs that the model can understand, helping the model to dynamically update and iteratively optimize.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all should be included in the scope of the claims of the present application.

Claims

1. An AI large model-based construction site safety hazard intelligent judgment system, characterized in that: The system comprises a collection unit and an intelligent judgment unit. The collection unit is configured to collect and transmit video data of a construction site to the intelligent judgment unit in real time. The intelligent judgment unit is configured to receive the video data, analyze the video data using an AI large model, determine whether there is a safety hazard on the construction site, generate a message to remind of the safety hazard and send the message to a safety management personnel, and receive feedback information on rectification submitted by the safety management personnel. 2.The AI large model-based construction site safety hazard intelligent determination system according to claim 1, characterized in that: The collection unit comprises a camera terminal, a data transmission module, and a depth measurement algorithm module. The camera terminal is configured to collect video data of a construction site in real time. The data transmission module is configured to transmit the video data to the intelligent judgment unit. The depth measurement algorithm module is configured to dynamically adjust a sampling frequency and a video resolution according to an actual scene. 3.The AI large model-based construction site safety hazard intelligent determination system according to claim 1, characterized in that: The intelligent judgment unit comprises a safety management collaboration platform and an AI large model. The safety management collaboration platform comprises a PC end and a mobile end. The PC end comprises a picture receiving module, an AI large model calling engine module, a problem marking module, a voice interaction module, a picture output module, and a safety specification module. The picture receiving module is configured to receive camera picture streams from the collection unit in real time. The AI large model calling engine module is configured to call the AI large model to intelligently analyze the video streams. The problem marking module is configured to accurately locate a problem area. The voice interaction module is configured to provide a convenient user operation interface. The picture output module is configured to provide intuitive decision support for a user. The safety specification module is configured to provide a user with safety standards and specification basis, find relevant safety operation guidance, regulatory requirements, and correct examples, and provide a user with a safety operation guidance, a regulatory requirement, and a correct example. The mobile end comprises a business collaboration module, a message reminding and pushing module, and a real-time picture display module. The business collaboration module is configured to meet the business operation needs of safety management personnel on a construction site. The message reminding and pushing module is configured to timely remind and push a message. 4.The AI large model-based construction site safety hazard intelligent determination system according to claim 3, characterized in that: The real-time picture display module is configured to display a real-time picture with low delay. 5.The AI large model-based construction site safety hazard intelligent determination system according to claim 3, characterized in that: The AI large model is based on a GPT and a Segment Everything framework to identify a scene, extract key targets and parameters, support video scene analysis, and determine a safety hazard. 6.The AI large model-based construction site safety hazard intelligent determination system according to claim 3, characterized in that: The AI large model supports intelligent collaboration between an engineer and the large model to complete the engineer's interactive intention reasoning, response query, search, confirmation, and marking. 7.The AI-large model based construction site safety hazard intelligent determination system according to claim 3, characterized in that: The AI large model builds a safety hazard knowledge base, optimizes large model parameters based on a Transformer framework, updates a knowledge storage framework, and enhances a hazard identification capability.

8. A determination method for intelligently determining construction site safety hazards by using the construction site safety hazard intelligent determination system according to any one of claims 1-7, characterized in that: The AI large model has the ability of autonomous learning and optimization, and is suitable for safety specification updates and video data accumulation. The system comprises: object classification and identification: after receiving video data from a device, the video picture is divided into multiple areas, and objects in each area are identified. Safety specification reference: After object recognition is completed, the constructed knowledge base is analyzed; wherein, the knowledge base is a triple of conditions, targets and requirements, responsible for representing relevant safety specifications; the specification text of the original rule file is parsed through a language model, key information is extracted, and a rule knowledge management base is constructed; a matching algorithm is used to match the safety specifications in the knowledge base according to the recognized objects and scenes, to determine whether there is a violation of the specifications; Problem hazard marking: on the basis of specification matching, potential problems and hazards are further analyzed, so as to mark objects or behaviors that do not conform to safety specifications, and to evaluate possible risks. 9.The AI large model-based construction site safety hazard intelligent determination method according to claim 8, characterized in that: The video picture is divided into multiple regions, and objects in each region are identified, specifically including: A segmentation model based on EfficientSAM is constructed, the original video image is input into the network where the segmentation model is located, the object code is obtained, and the mask with prompt information is fused to obtain the final segmented image; After the scene target is cut, a classifier for downstream tasks is constructed to further analyze the features of each object and obtain its semantic information, without explicit labels, but according to the similarity of categories, to realize the identification and classification of objects. 10.The AI large model-based construction site safety hazard intelligent determination method according to claim 8, characterized in that: Using a matching algorithm to match the safety specifications in the knowledge base according to the recognized objects and scenes, specifically including: Using prompt words and target analysis to understand the relationship between objects and their meaning in a specific scene; according to the existing target information and structured knowledge, traversal matching is performed, the recognized target is analyzed with the specification articles in the knowledge base, and the matching of the hazards existing in the target with the specification articles is completed; wherein, the knowledge base stores rich background information and prior knowledge, and the knowledge base is continuously adjusted to obtain the best matching result.

Images