System for monitoring safety of construction site through dome camera

Abstract

The invention discloses a system for monitoring the safety of a construction site through a dome camera, and the system comprises a dome camera module which is disposed in a key region of the construction site, and is used for collecting the video image data of the site; the visual identification module is used for identifying key objects, figures and abnormal events on the construction site; the AI large model analysis module is used for performing efficient data processing and analysis by utilizing a pre-trained large-scale AI language vision model, and identifying and predicting potential safety hazards; the natural language generation module is used for converting a visual recognition result into understandable voice information and feeding back the voice information to safety supervision personnel in real time, so that timely response is facilitated; and the intelligent alarm module is used for automatically triggering an alarm mechanism through real-time analysis of the safety condition of the construction site when dangerous behaviors or abnormal conditions are found. According to the system for monitoring the safety of the construction site through the dome camera, real-time safety monitoring, risk prediction, anomaly detection and intelligent alarm of the construction site are achieved.

Description

Technical Field

[0001] The present invention relates to the field of construction site monitoring, and more particularly to a system for monitoring construction site safety using a PTZ camera. Background Art

[0002] With the rapid development of the construction industry, the safety management of construction sites faces more and more challenges. Although traditional manual inspections and video monitoring can provide certain safety guarantees, there are still problems such as monitoring blind spots, lag in information processing, and fatigue in manual inspections. Therefore, how to improve the real-time performance, accuracy, and intelligent level of construction site safety monitoring has become an urgent problem to be solved.

[0003] As a device with high-precision image acquisition and a wide range of views, PTZ cameras have been widely used in the security field. However, existing PTZ cameras still rely on manual or traditional algorithms for monitoring, and it is difficult to identify and predict potential safety hazards on construction sites in real time and automatically, such as falling objects from heights, illegal operations by construction workers, equipment failures, etc. Summary of the Invention

[0004] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a system and monitoring method for monitoring construction site safety using a PTZ camera based on AI language vision large model technology, which can monitor the safety status of construction sites in real time and improve the monitoring efficiency and emergency response ability through AI analysis and automatic alarm mechanisms.

[0005] To achieve the above object, the present invention provides the following technical solutions: A system for monitoring construction site safety using a PTZ camera, comprising:

[0006] PTZ camera module: Set in key areas of the construction site for collecting on-site video image data; Visual recognition module: Analyze the images collected by the PTZ camera module through deep learning and image processing algorithms to identify key objects, people, and abnormal events on the construction site;

[0007] AI large model analysis module: Use a pre-trained large-scale AI language vision model for efficient data processing and analysis to identify and predict potential safety hazards;

[0008] Natural language generation module: Convert the visual recognition results into understandable voice information and provide real-time feedback to safety supervision personnel for timely response;

[0009] Intelligent alarm module: Automatically trigger the alarm mechanism when dangerous behaviors or abnormal situations are found through real-time analysis of the construction site safety status.

[0010] As a further improvement of the present invention, the specific method for the visual recognition module to analyze and recognize images is as follows: First, perform a convolution operation on the image data, scan the input image through a filter to extract local features, and then perform a non-linear mapping on the convolved feature map through an activation function; finally, use an object detection method to locate and classify the object in the image;

[0011] Among them, when classifying, candidate regions are generated through a region proposal network, and then object classification is performed.

[0012] As a further improvement of the present invention, the AI large model analysis module uses a long short-term memory network and a self-attention mechanism in deep learning for anomaly detection and behavior prediction.

[0013] As a further improvement of the present invention, the long short-term memory network includes a forget gate, an input gate, candidate memory, updated memory, an output gate, and hidden state update. The forget gate determines which information should be forgotten from the cell state; the input gate determines which new information should be added to the cell state; the candidate memory represents the new information; the output gate determines which information should be output from the cell state to the hidden state; the hidden state is the output of the LSTM and is used to be passed to the next time step.

[0014] As a further improvement of the present invention, the candidate memory specifically includes:

[0015]

[0016] Among them, represents the candidate value of the cell state at time step tt, tanh: represents the hyperbolic tangent function, which is a commonly used activation function for mapping the output of a linear combination to the interval (-1, 1), W C : represents the weight matrix of the candidate value of the cell state, which is used to perform a linear transformation on the hidden state h t-1 at the previous time step and the input x t at the current time step, [h t-1 , x t : represents concatenating the hidden state h t-1 at the previous time step and the input x t at the current time step together, b C : represents the bias term of the candidate value of the cell state, which is a constant used to adjust the output of the linear combination.

[0017] As a further improvement of the present invention, the updated memory specifically includes:

[0018]

[0019] Among them, C t: Represents the cell state at time step \(t\), \(f\) t : Represents the value of the forget gate at time step \(t\), which determines which information should be forgotten from the cell state, \(C\) t-1 : Represents the cell state at time step \(t - 1\), \(i\) t : Represents the value of the input gate at time step \(t\), which determines which new information should be added to the cell state, Represents the candidate value of the cell state at time step \(t\), which is obtained through a non - linear transformation.

[0020] As a further improvement of the present invention, the hidden state update is specifically as follows:

[0021] \(h\) t \(=\ o\) t· \(\tanh(C\) t )

[0022] Wherein, \(h\) t : Represents the hidden state at time step \(t\), \(\tanh(C\) t ): Represents the hyperbolic tangent function of the cell state \(C\) t which maps the value of the cell state to the interval \((-1, 1)\).

[0023] As a further improvement of the present invention, the calculation formula of the self - attention mechanism is as follows:

[0024]

[0025] Wherein, Attention(Q, K, V): Represents the output of the attention mechanism, Q: Represents the query vector, K: Represents the key vector, V: Represents the value vector, softmax: Represents the softmax function, which maps a vector to a probability distribution and is used to convert the calculation result into a probability distribution, \(QK\) T : Represents the matrix multiplication of the query vector \(Q\) and the transpose \(K\) of the key vector \(K\) T Represents the square root of the dimension \(d\) of the key vector \(K\) k for scaling the dot product to prevent gradient vanishing or explosion.

[0026] As a further improvement of the present invention, the natural language generation module uses a sequence - to - sequence model and a Transformer - based language model to implement language generation. The language model is trained by maximizing the conditional probability of predicting words, specifically as follows:

[0027]

[0028] Wherein, \(P(w1, w2,......, wT ):Represents the probability of the entire word sequence (or sentence), which is expressed as the product from 1 to T, P(w t |w1, w2,......, w t-1 ):Represents the conditional probability of the t-th word given the previous t - 1 words;

[0029] The sequence-to-sequence model maps the input sequence to a fixed-length context vector through an encoder and then generates the target sequence through a decoder, as follows:

[0030] h t = f encoder (x t , h t-1 )

[0031] where h t : Represents the hidden state at time step y, f encoder : Represents the encoder function, which is a non-linear function used to combine the current input and the hidden state of the previous time step to generate the hidden state of the current time step, x t : Represents the input at time step t, h t-1 : Represents the hidden state at time step t - 1; The decoder then generates the target sequence Y based on the output hidden state of the encoder:

[0032] y t = f decoder (h t )

[0033] where y t : Represents the output at time step t, f decoder : Represents the decoder function, which is a non-linear function used to convert the hidden state h t at the current time step into the output y t .

[0034] As a further improvement of the present invention, the intelligent alarm module, according to the real-time data stream and the rule-based triggering mechanism, its triggering condition can be represented by a logistic regression model:

[0035] P Alarm = σ(W·X + b)

[0036] where P Alarm : Represents the probability of an alarm, usually a value between 0 and 1, σ: Represents the sigmoid function, which is a commonly used activation function for mapping the output of a linear combination into the interval (0, 1); The formula of the sigmoid function is

[0037]

[0038] Among them, e represents the base of the natural logarithm, approximately equal to 2.71828, W represents the weight matrix, which is used to perform a linear transformation on the input vector XX, X represents the input vector, which contains the features used to predict the alarm probability, and b represents the bias term, which is a constant used to adjust the output of the linear combination.

[0039] Advantages of the present invention. The pan-tilt camera monitoring construction site safety supervision system provided by the present invention combines advanced deep learning, image processing, natural language generation, and data analysis technologies, can monitor the safety status of the construction site in real time, accurately identify and predict potential safety hazards, and through an intelligent voice feedback and alarm mechanism, convey information to relevant personnel in a timely manner, improving the intelligent and automated level of construction site safety management, and specifically achieving:

[0040] Multi-modal fusion AI analysis: Combining visual recognition and language model technologies, realizing multi-modal fusion processing of image data and text information. The AI large model can not only identify objects and actions, but also understand the scene context, thus making more accurate safety judgments.

[0041] Real-time warning and voice feedback system: Through real-time image processing and natural language generation technologies, the system can quickly convert the visual recognition results into voice information and feedback them to the safety supervision personnel, greatly improving the response speed to potential safety hazards.

[0042] Intelligent safety prediction: Using deep learning and the AI large model to predict construction site safety risks, being able to identify potential safety hazards in advance, taking preventive measures in advance, and effectively reducing the probability of accidents.

[0043] Adaptive anomaly detection and learning: This system can continuously optimize the AI large model through historical monitoring data, improve the accuracy and recognition ability of anomaly behavior detection, has an adaptive learning function, and can make dynamic adjustments according to the construction site environment and operation habits.

[0044] Integrated cloud platform management: Using the cloud platform to centrally manage all monitoring data and analysis results, which is not only convenient for remote monitoring and management, but also enables data sharing and collaboration, improving the overall and systematic nature of construction site safety supervision management. Specific implementation manners

[0045] The following will further elaborate on the present invention with the given embodiments.

[0046] A system for a pan-tilt camera to monitor construction site safety in this embodiment includes the following modules:

[0047] Pan-tilt camera module: Set in key areas of the construction site to collect on-site video image data.

[0048] Visual recognition module: Analyze the images collected by the camera through deep learning and image processing algorithms to identify key objects, people, abnormal events, etc. on the construction site.

[0049] AI large model analysis module: Use a pre-trained large-scale AI language-vision model for efficient data processing and analysis to identify and predict potential safety hazards.

[0050] Natural language generation module: Convert the visual recognition results into understandable voice information and provide real-time feedback to the safety supervision personnel for timely response.

[0051] Intelligent alarm module: Automatically trigger the alarm mechanism when dangerous behaviors or abnormal situations are detected through real-time analysis of the construction site safety conditions.

[0052] Data storage and cloud platform module: Store the monitoring data and analysis results in the cloud for subsequent query and analysis. Through the settings of the above-mentioned various modules, the safety conditions of the construction site can be monitored in real time, and the monitoring efficiency and emergency response capabilities can be improved through AI analysis and automatic alarm mechanisms.

[0053] The working methods of the above-mentioned various modules are further described in detail as follows:

[0054] The pan-tilt-zoom camera module used in this embodiment is the commonly used pan-tilt-zoom camera in the prior art. Therefore, the working process and principle thereof will not be elaborated in this embodiment. The visual recognition module mainly uses a convolutional neural network to implement the recognition and classification of image data. In this embodiment, the convolutional neural network is the core method for image processing and object detection, involving convolutional operations, pooling operations, etc., as follows:

[0055] The convolutional operation scans the input image through a filter (convolution kernel) to extract local features. The convolution formula is:

[0056]

[0057] (I * K)(x, y): Represents the result of the convolution operation, that is, the value of the convolution kernel K applied to the point (x, y) on the image I.

[0058] I(x + i, y + j): Represents the pixel value of the point (x + i, y + j) on the image I.

[0059] K(i, j): Represents the value of the point (i, j) on the convolution kernel K.

[0060] Indicates the double summation of the values of all points on the convolution kernel K, where k is the radius of the convolution kernel.

[0061] The feature map after convolution is usually non-linearly mapped through an activation function (such as ReLU), and the formula is:

[0062] f(x) = max(0, x)

[0063] f(x): represents the output of the ReLU function.

[0064] It means taking the larger value between 0 and x

[0065] The pooling layer is used to reduce the dimension of the image while maintaining important features. The commonly used pooling operation is max pooling, and its formula is:

[0066] yi , j = max m∈[i,i+2],n∈[j,j+2] x m,n

[0067] y i,j : represents the value of the output feature map at position (i, j) after the max pooling operation.

[0068] max m∈[i,i+2],n∈[j,j+2] : means taking the maximum value within the sub-region [i, i + 2] × [j, j + 2] of the input feature map.

[0069] x m,n : represents the value at position (m, n) in the input feature map.

[0070] Then object detection is carried out. Object detection methods are used to locate and classify objects in the image.

[0071] The loss function comprehensively considers the position error (coordinate regression), confidence error (whether there is an object), and class error (object classification). The loss function is expressed as:

[0072]

[0073] L: represents the coordinate loss.

[0074] λ coord : represents the weight of the coordinate loss, which is used to balance the coordinate loss with other losses (such as confidence loss and classification loss).

[0075] represents the sum over all grid cells, where SS is the size of the grid.

[0076] 1 obj (i): represents the indicator function, which is 1 if the grid cell ii contains an object, and 0 otherwise.

[0077] Represents the squared difference between the x - coordinate of the target center and the predicted x - coordinate.

[0078] Represents the squared difference between the y - coordinate of the target center and the predicted y - coordinate.

[0079] Represents the squared difference between the width of the target and the predicted width.

[0080] Represents the squared difference between the height of the target and the predicted height.

[0081] The loss function enables the model to learn how to accurately predict the position and size of the target.

[0082] After that, a region - based convolutional network is adopted to achieve the final classification. Candidate regions are generated through the region proposal network and then object classification is performed. The loss function is as follows:

[0083] L = L cls (p)+L reg (t)

[0084] L: Represents the total loss.

[0085] L cls (p): Represents the classification loss, which measures the difference between the predicted class probability p and the true class. It measures the difference between the predicted class probability distribution and the true class.

[0086] L reg (t): Represents the regression loss, which measures the difference between the predicted bounding box coordinates t and the true bounding box coordinates. It measures the difference between the predicted bounding box coordinates and the true bounding box coordinates.

[0087]

[0088] N: Represents the number of samples.

[0089] p i : Represents the true label of the i - th sample, usually a binary variable with values 0 or 1.

[0090] Represents the predicted probability of the i - th sample, with values between 0 and 1.

[0091]

[0092] Represents the predicted x - coordinate of the i - th target.

[0093] x i : Represents the true x - coordinate of the i - th target.

[0094] Represents the predicted y - coordinate of the i - th target.

[0095] y i : Represents the true y - coordinate of the i - th target.

[0096] Thus, by continuously adjusting the model through the above - mentioned loss functions, the feature extraction, target localization, and classification of image data can be achieved.

[0097] The AI large - model analysis module of this embodiment is mainly based on the long short - term memory network and self - attention mechanism technologies in deep learning, and is used for anomaly detection and behavior prediction.

[0098] 2.1 Long short - term memory network

[0099] Used to process time - series data, such as behavior changes in construction site monitoring videos. The core formula is:

[0100] Forget Gate:

[0101] f t =σ(W f ·[h t-1 , x t +b f )

[0102] f t : Represents the value of the forget gate at time step t.

[0103] σ: Represents the sigmoid function, which is a commonly used activation function for mapping the output of a linear combination to the interval (0, 1).

[0104] W f : Represents the weight matrix of the forget gate, which is used to perform a linear transformation on the hidden state h t-1 at the previous time step and the input x t at the current time step.

[0105] [h t-1 , x t : Represents concatenating the hidden state h t-1 at the previous time step and the input x t at the current time step together.

[0106] b f : Represents the bias term of the forget gate, which is a constant used to adjust the output of the linear combination.

[0107] Input Gate:

[0108] i t =σ(Wi · [h t-1 , x t + b i )

[0109] i t : represents the value of the input gate at time step t.

[0110] σ: represents the sigmoid function, a commonly used activation function that maps the output of a linear combination to the interval (0, 1).

[0111] W i : represents the weight matrix of the input gate, which is used to perform a linear transformation on the hidden state h t-1 from the previous time step and the input x t at the current time step.

[0112] [h t-1 , x t : represents concatenating the hidden state h t-1 from the previous time step and the input x t at the current time step.

[0113] b i : represents the bias term of the input gate, which is a constant used to adjust the output of the linear combination.

[0114] Candidate Memory (CellCandidate):

[0115]

[0116] represents the candidate value of the cell state at time step t.

[0117] tanh: represents the hyperbolic tangent function, a commonly used activation function that maps the output of a linear combination to the interval (-1, 1).

[0118] W C : represents the weight matrix of the candidate value of the cell state, which is used to perform a linear transformation on the hidden state h t-1 from the previous time step and the input x t at the current time step.

[0119] [h t-1 , x t : represents concatenating the hidden state h t-1 from the previous time step and the input x t at the current time step.

[0120] b C: Bias term representing candidate values of cell state, which is a constant used to adjust the output of the linear combination.

[0121] Update memory (CellState):

[0122]

[0123] C t : Represents the cell state at time step t.

[0124] f t : Represents the value of the forget gate at time step t, which determines which information should be forgotten from the cell state.

[0125] C t-1 : Represents the cell state at time step t - 1.

[0126] i t : Represents the value of the input gate at time step t, which determines which new information should be added to the cell state.

[0127] Represents the candidate value of the cell state at time step t, which is obtained through a non - linear transformation.

[0128] OutputGate:

[0129] o t =σ(W o ·[h t-1 ,x t +b o )

[0130] o t : Represents the value of the output gate at time step t.

[0131] σ: Represents the sigmoid function, which is a commonly used activation function for mapping the output of the linear combination to the interval (0, 1).

[0132] W o : Represents the weight matrix of the output gate, which is used to perform a linear transformation on the hidden state h at the previous time step t-1 and the input x at the current time step t for linear transformation.

[0133] [h t-1 ,x t : Represents concatenating the hidden state h at the previous time step t-1 and the input x at the current time step t together.

[0134] b o: Represents the bias term of the output gate, which is a constant used to adjust the output of the linear combination.

[0135] Hidden state update:

[0136] h t = o t· tanh(C t )

[0137] h t : Represents the hidden state at time step t.

[0138] tanh(C t ): Represents the hyperbolic tangent function of the cell state C t , which maps the value of the cell state to the interval (-1, 1).

[0139] Process description:

[0140] For each identified target (such as a worker), extract its feature vector, including information such as position, action, and posture. These feature vectors will be passed as input to the LSTM network.

[0141] Concatenate the extracted feature vectors h t-1 , x t , where h t-1 is the hidden state of the previous time step, and x t is the input feature vector of the current time step.

[0142] The forget gate determines which information should be forgotten from the cell state to save computational resources; the input gate determines which new information should be added to the cell state; the candidate memory is obtained through a non-linear transformation, representing new information; the cell state is a key component in the LSTM for capturing long-term dependencies; the output gate determines which information should be output from the cell state to the hidden state; the hidden state is the output of the LSTM and is used to be passed to the next time step;

[0143] Through the above steps, the LSTM network can perform real-time analysis on the behaviors of workers in the construction site monitoring video and obtain the following analysis results:

[0144] Behavior recognition: The system can real-time recognize various behaviors of workers, such as wearing safety helmets, fastening seat belts, operating equipment, entering dangerous areas, etc.

[0145] Risk prediction: The system can predict potential safety risks, such as falling objects from height, equipment failures, illegal operations, etc.

[0146] Anomaly detection: The system can detect abnormal behaviors, such as workers not wearing personal protective equipment as required, abnormal equipment operation, etc.

[0147] Specific Example

[0148] Suppose a worker is detected entering the construction area without wearing a safety helmet in the surveillance video, and the system will perform the following analysis and processing:

[0149] Feature Extraction: Extract the feature vectors of the worker entering the construction area, including position, movement, posture, etc.

[0150] Forget Gate: The forget gate will gradually forget the information of the worker in the safe area before.

[0151] Input Gate: The input gate will add the information that the worker is not wearing a safety helmet to the cell state.

[0152] Candidate Memory: Generate the eigenvalue that the worker is not wearing a safety helmet.

[0153] Update Cell State: The cell state will be updated to retain the information that the worker is not wearing a safety helmet.

[0154] Output Gate: The output gate will output the information that the worker is not wearing a safety helmet to the hidden state.

[0155] Hidden State Update: The hidden state will contain the features that the worker is not wearing a safety helmet for subsequent processing, such as alarm push.

[0156] In this way, the system can identify and process the behaviors of workers in the construction site surveillance video in real time and accurately, improving the efficiency and effectiveness of construction site safety management.

[0157] 2.2 Self-Attention Mechanism

[0158] The self-attention mechanism (Self-Attention) is mainly used to handle long-range dependence problems and is widely applied in multi-modal learning. The self-attention calculation formula is:

[0159]

[0160] Attention(Q, K, V): Represents the output of the attention mechanism.

[0161] Q: Represents the query vector.

[0162] K: Represents the key vector.

[0163] V: Represents the value vector.

[0164] softmax: Represents the softmax function, which maps a vector to a probability distribution and is used to convert the calculation result into a probability distribution.

[0165] QK T: It represents the matrix multiplication of the query vector Q and the transpose of the key vector K, i.e., KT. T of the matrix multiplication.

[0166] It represents the dimension d of the key vector K. k The square root of k is used to scale the dot product to prevent the vanishing or exploding of gradients.

[0167] To perform natural language generation using the natural language generation module in this embodiment, a sequence-to-sequence model and a Transformer-based language model are required. A language model is usually trained by maximizing the conditional probability of predicting words:

[0168]

[0169] P(w1, w2,......, w T T): It represents the probability of the entire word sequence (or sentence).

[0170] It represents the product from 1 to T.

[0171] P(w t | w1, w2,......, w t-1 t-1): It represents the conditional probability of the t-th word given the previous t - 1 words.

[0172] The sequence-to-sequence model is used to convert the image recognition result into text or speech output. Its goal is to map the input sequence to a fixed-length context vector through the encoder and then generate the target sequence through the decoder.

[0173] h t = f encoder (x t , h t-1 )

[0174] h t : It represents the hidden state at time step t.

[0175] f encoder : It represents the encoder function, which is a non-linear function used to combine the current input and the hidden state of the previous time step to generate the hidden state of the current time step.

[0176] x t : It represents the input at time step t.

[0177] h t-1 : It represents the hidden state at time step t - 1.

[0178] The decoder then generates the target sequence Y based on the output hidden state of the encoder:

[0179] y t = f decoder (h t )

[0180] y t : represents the output at time step t.

[0181] f decoder : represents the decoder function, which is a non-linear function used to convert the hidden state h at the current time step t into the output y t .

[0182] The intelligent alarm module of this embodiment relies on real-time data streams and a rule-based triggering mechanism. Its triggering conditions can be represented by a logistic regression model:

[0183] P Alarm = σ(W·X + b)

[0184] P Alarm : represents the probability of an alarm, usually a value between 0 and 1.

[0185] σ: represents the sigmoid function, which is a commonly used activation function for mapping the output of a linear combination to the interval (0, 1). The formula for the sigmoid function is

[0186]

[0187] e: represents the base of the natural logarithm, approximately equal to 2.71828.

[0188] W: represents the weight matrix, which is used to perform a linear transformation on the input vector XX.

[0189] X: represents the input vector, which contains the features used to predict the alarm probability.

[0190] b: represents the bias term, which is a constant used to adjust the output of the linear combination.

[0191] In summary, the system for the dome camera to monitor the construction site safety in this embodiment mainly has the following five functions:

[0192] 1. Visual recognition and target detection

[0193] The dome camera collects the high-definition video stream of the construction site in real time and processes the video stream frame by frame through the visual recognition module. Using convolutional neural networks (CNNs) and target detection algorithms (such as YOLO, Faster R-CNN, etc.), the system can automatically identify people, equipment, objects, signs, dangerous areas, etc. in the image and mark the key targets in the image.

[0194] Through the analysis of the target positions and behaviors in the images, the system can identify dangerous behaviors on the construction site, such as not wearing a safety helmet during high-altitude work, abnormal operation of construction equipment, and personnel entering dangerous areas.

[0195] 2. AI Large Model Analysis and Anomaly Detection

[0196] The target data (such as personnel, equipment, dangerous behaviors, etc.) output by the visual recognition module is transmitted to the AI large model analysis module, which conducts comprehensive analysis based on a large-scale pre-trained deep learning model. The AI large model integrates visual information and scene context information, enabling it to more accurately judge whether there are potential safety hazards in the current scene.

[0197] For example, through the training of a large amount of construction site monitoring data, the model can automatically judge whether there is a risk of "falling objects from height" in a specific area, or analyze whether there are "violation operations" (such as not wearing personal protective equipment as required) on the construction site. The model can also learn from historical data to predict potential safety problems and give early warnings before the problems occur.

[0198] 3. Natural Language Generation and Voice Feedback

[0199] When the AI large model analyzes an anomaly or a safety hazard, the system converts the analysis result into clear and concise voice information through the natural language generation module. For example, if it detects that a certain worker is not wearing a safety helmet, the system can give a voice reminder: "Attention, Worker A is not wearing a safety helmet. Please correct it immediately." Voice feedback can convey the monitoring results in a timely manner, improving the response speed and decision-making efficiency of safety supervisors.

[0200] 4. Intelligent Alarm Mechanism

[0201] When the system detects major safety hazards (such as fires, equipment failures, falling accidents, etc.), the intelligent alarm module will automatically trigger an alarm to notify the safety supervision personnel and relevant responsible persons on the construction site. The alarm not only includes voice prompts but also can send warning messages to relevant personnel through methods such as APPs, text messages, and emails, along with key information such as the location and time of the incident, so as to take emergency measures in a timely manner.

[0202] 5. Personalized Settings and Learning Functions

[0203] The glasses support user-defined settings, such as the speech output speed, volume, and the sensitivity of obstacle reminders. In addition, the built-in AI system of the glasses has a self-learning function, which can be personalized and optimized according to the user's habits and environment. For example, through long-term use, the glasses can better identify the routes that the wearer often passes through and provide more accurate and personalized navigation services.

[0204] Thus, by adopting the system of this embodiment, the following are adopted:

[0205] Multi-modal fusion AI analysis: By combining visual recognition and language model technologies, multi-modal fusion processing of image data and text information is achieved. The large AI model can not only identify objects and actions, but also understand the scene context, thus making more accurate safety judgments.

[0206] Real-time warning and voice feedback system: Through real-time image processing and natural language generation technologies, the system can quickly convert the visual recognition results into voice information and feedback it to the safety supervision personnel, greatly improving the response speed to potential safety hazards.

[0207] Intelligent safety prediction: Using deep learning and large AI models to predict construction site safety risks, potential safety hazards can be identified in advance, and preventive measures can be taken in advance to effectively reduce the probability of accidents.

[0208] Adaptive anomaly detection and learning: This system can continuously optimize the large AI model through historical monitoring data, improve the accuracy and recognition ability of anomaly behavior detection, and has an adaptive learning function, which can be dynamically adjusted according to the construction site environment and operation habits.

[0209] Integrated cloud platform management: The cloud platform is adopted to centrally manage all monitoring data and analysis results, which is not only convenient for remote monitoring and management, but also enables data sharing and collaboration, improving the overall situation and systematicness of construction site safety supervision management.

[0210] The above are only the preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the concept of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A system for monitoring construction site safety using a ball camera, characterized in that: include: Dome camera module: installed in key areas of the construction site to collect on-site video image data; Visual recognition module: Analyzes the images collected by the ball camera module through deep learning and image processing algorithms to identify key objects, people, and abnormal events on the construction site; AI large model analysis module: uses pre-trained large-scale AI language and vision models to perform efficient data processing and analysis to identify and predict potential safety hazards; Natural language generation module: converts visual recognition results into understandable voice information and provides real-time feedback to safety inspectors for timely response; Intelligent alarm module: Through real-time analysis of the safety status of the construction site, the alarm mechanism is automatically triggered when dangerous behavior or abnormal conditions are detected.

2. The system for monitoring construction site safety using a ball camera according to claim 1 is characterized by: The specific method of analyzing and identifying the image by the visual recognition module is as follows: first, the image data is convolved, the input image is scanned through a filter, local features are extracted, and then the convolved feature map is nonlinearly mapped through an activation function; finally, the target detection method is used to locate the target in the image and classify it; During classification, candidate regions are generated through the region proposal network, and then the target is classified.

3. The system for monitoring construction site safety using a ball camera according to claim 1 or 2, characterized in that: The AI ​​large model analysis module uses the long short-term memory network and self-attention mechanism in deep learning to perform anomaly detection and behavior prediction.

4. The system for monitoring construction site safety using a ball camera according to claim 3 is characterized by: The long short-term memory network includes a forget gate, an input gate, a candidate memory, an update memory, an output gate and a hidden state update. The forget gate determines which information should be forgotten from the cell state; the input gate determines which new information should be added to the cell state; the candidate memory is used to represent the new information; the output gate determines which information should be output from the cell state to the hidden state; the hidden state is the output of the LSTM and is used to be passed to the next time step.

5. The system for monitoring construction site safety using a ball camera according to claim 4 is characterized in that: The candidate memories specifically include: in, represents the candidate value of the cell state at time step tt, tanh: represents the hyperbolic tangent function, which is a commonly used activation function used to map the output of the linear combination to the (-1,1) interval, W C : The weight matrix representing the candidate values ​​of the cell state, which is used to calculate the hidden state h of the previous time step t-1 and the input x at the current time step t Perform linear transformation, [h t-1 ,x t ]: indicates that the hidden state h of the previous time step is t-1 and the input x at the current time step t Stitched together, b C : A bias term representing the candidate value of the cell state, which is a constant used to adjust the output of the linear combination.

6. The system for monitoring construction site safety using a ball camera according to claim 4 or 5, characterized in that: The updating memory specifically includes: Among them, C t : represents the cell state at time step tt, f t : represents the value of the forget gate at time step tt, which determines which information should be forgotten from the cell state, C t-1 : represents the cell state at time step t-1, i t : represents the value of the input gate at time step t, which determines what new information should be added to the cell state, represents the candidate value of the cell state at time step t, which is obtained through a nonlinear transformation.

7. The system for monitoring construction site safety using a ball camera according to claim 4 or 5, characterized in that: The hidden state update is specifically as follows: h t =o t ·tanh(C t ) Among them, h t : represents the hidden state at time step t, tanh(C t ): indicates cell state C t The hyperbolic tangent function of , which maps the value of the cell state to the interval (-1,1).

8. The system for monitoring construction site safety using a ball camera according to claim 3 is characterized by: The calculation formula of the self-attention mechanism is as follows: Among them, Attention(Q,K,V): represents the output of the attention mechanism, Q: represents the query vector, K: represents the key vector, V: represents the value vector, softmax: represents the softmax function, which maps a vector to a probability distribution, used to convert the calculation result into a probability distribution, QK T : represents the transpose K of the query vector QQ and the key vector K T Matrix multiplication of , represents the dimension d of the key vector K k The square root of is used to scale the dot product to prevent the gradient from vanishing or exploding.

9. The system for monitoring construction site safety using a ball camera according to claim 1 or 2, characterized in that: The natural language generation module uses a sequence-to-sequence model and a Transformer-based language model to achieve language generation. The language model is trained by maximizing the conditional probability of predicting words, as follows: Among them, P(w1,w2,......,w T ): represents the probability of the entire word sequence (or sentence), represents the continuous product from 1 to T, P(w t |w1,w2,......,w t-1 ): represents the conditional probability of the t-th word given the first t-1 words; The sequence-to-sequence model maps the input sequence to a fixed-length context vector through the encoder, and then generates the target sequence through the decoder, as follows: h t =f encoder (x t ,h t-1 ) Among them, h t : represents the hidden state at time step y, f encoder : represents the encoder function, which is a nonlinear function used to combine the current input and the hidden state of the previous time step to generate the hidden state of the current time step, x t : represents the input at time step t, h t-1 : represents the hidden state at time step t-1; the decoder generates the target sequence Y based on the output hidden state of the encoder: y t =f decoder (h t ) Among them, y t : represents the output at time step t, f decoder : represents the decoder function, which is a nonlinear function used to transform the hidden state h of the current time step t Convert to output y t .

10. The system for monitoring construction site safety using a ball camera according to claim 1 or 2, characterized in that: The intelligent alarm module is based on real-time data flow and rule-based triggering mechanism, and its triggering conditions can be represented by a logistic regression model: P Alarm =σ(W·X+b) Among them, P Alarm : represents the probability of alarm, usually a value between 0 and 1, σ: represents the sigmoid function, which is a commonly used activation function used to map the output of the linear combination to the interval (0,1); the formula of the sigmoid function is Among them, e: represents the base of natural logarithm, which is approximately equal to 2.71828, W: represents the weight matrix, which is used to linearly transform the input vector XX, X: represents the input vector, which contains the features used to predict the alarm probability, and b: represents the bias term, which is a constant used to adjust the output of the linear combination.

Images