Construction site safety early warning method based on intelligent eye strategy

Abstract

The invention discloses a construction site safety early warning method based on an intelligent eye strategy, and the method is characterized in that the method comprises the steps: judging to execute a safety early warning or camera control process according to a problem inputted by a user, entering a step 2 if the safety early warning is performed, and entering a step 3 if the camera control is performed. And 2, screening a construction site scene to determine a safety abnormity early warning area, rewriting data by using a Rag algorithm, performing safety abnormity identification, executing camera control by the system, entering the step 3, and outputting an abnormity reason. And step 3, extracting actions and parameters to control a camera to move and recognize a scene, and then receiving a picture amplification target instruction to amplify the picture to determine a security abnormality early warning position. According to the method, construction site safety early warning and camera control requirements can be flexibly processed, the Rag algorithm improves the abnormity identification accuracy, when abnormity may occur, linkage camera control facilitates accurate positioning of the abnormity, and construction site safety management is powerfully guaranteed.

Description

Technical Field

[0001] The present invention relates to a construction site monitoring system, and more particularly to a construction site safety early warning method based on a "smart eye" strategy. Background Art

[0002] Construction site safety is a very important part. Currently, the application of intelligent monitoring systems can reduce the incidence of safety accidents by more than 70%. The existing construction site safety monitoring system is an intelligent system that uses technologies such as the Internet of Things, artificial intelligence, and video surveillance to conduct all-round safety management of the construction site. During the operation of the safety monitoring system, an image recognition system is used to automatically identify images taken by the camera, identify dangerous locations in the captured images, and then issue safety warnings based on the dangerous locations. However, the current image recognition system mainly identifies dangers based on relevant danger features, but it can only implement safety warnings after identifying dangers, and cannot accurately identify dangerous locations. Therefore, the subsequent danger handling is not timely enough and the accuracy of danger judgment is not high. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a construction site safety early warning method based on the "smart eye" strategy that can accurately and effectively identify dangerous locations.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a construction site safety early warning method based on the "smart eye" strategy, comprising the following steps:

[0005] Step 1: Determine whether to perform a security warning or camera control based on the question input by the user. If it is determined to be a security warning, proceed to step 2; if it is determined to be a camera control, proceed to step 3;

[0006] In step 2, the construction site scene is first screened to determine the areas that require safety anomaly warnings. The Rag algorithm is then used to rewrite the data, followed by safety anomaly identification and output, including identification results for "abnormal," "no abnormal," and "possible abnormality." If the output result indicates a possible abnormality, the system executes camera control and enters step 3, and outputs the cause of the abnormality.

[0007] Step three: receive the camera control instructions input by the user, extract the actions and action parameters according to the user instructions, control the movement of the hardware camera to perform scene recognition, and then receive the image zoom target instructions input by the user, zoom in on the image, and determine the location of the safety anomaly warning.

[0008] As a further improvement of the present invention, the specific method of enlarging the image in step three is: first, the target position is determined by the system, and possible location information is output. The possible information locations include [upper left, upper right, lower left, lower right]. If the location of the safety abnormality warning cannot be accurately determined, "there may be an abnormality" is output, and the image is segmented. The segmented image is then sent to the large model for execution again. The above steps are repeated twice for enlargement processing.

[0009] As a further improvement of the present invention, the specific method of image segmentation processing is: find the center point of the image, divide the image into four equal rectangular areas through vertical and horizontal cutting, located at the upper left, lower left, upper right, and lower right respectively, according to the position output by the model, and then send the image at the specified position to the large model again for execution, and repeat twice for magnification processing.

[0010] As a further improvement of the present invention, it also includes using the overall architecture of pre-trained Bert+linear layer, converting it into a multi-classification problem by training the parameters in Bert and linear, and predicting the location step of the final event.

[0011] As a further improvement of the present invention, the specific steps of converting the training parameters in Bert and linear into a multi-classification problem and predicting the location of the final event are as follows:

[0012] Step 1: The multimodal large model combines the image V and the prompt text T to output the event location description information Tdesc;

[0013] In step 2, the generated description Tdesc is encoded through BERT to obtain a new text feature representation T′. In step 3, T′ is input into the BERT model and the text feature Top is obtained after calculation through Self-Attention. In step 4, the calculated feature Top is input into the linear layer for linear transformation:

[0014] Z=W·Top+b

[0015] Where W and b are the weight matrix and bias vector of the linear layer;

[0016] Step 5: Input the linear transformation result Z into the Softmax layer to obtain the probability distribution of each position category:

[0017] P(y|Top)=softmax(Z)

[0018] Step 6: Calculate the loss using cross entropy loss:

[0019]

[0020] The beneficial effects of the present invention: Compared with the image recognition of the construction site safety monitoring system in the prior art, which can only issue a safety warning after identifying the danger, it cannot accurately identify the dangerous position, resulting in the problem of untimely danger handling and low judgment accuracy. The present invention can more accurately determine the safety anomaly warning position through the safety warning and camera control process, improve the accuracy of the dangerous position judgment by means of image magnification and segmentation processing, and use the pre-trained Bert+linear layer architecture to convert it into a multi-classification problem to predict the location of the event, further improve the comprehensiveness and accuracy of the danger judgment, make the danger handling more timely and efficient, effectively reduce the risk of construction site safety accidents, have significant advancements in construction site safety warning, improve the accuracy and efficiency of construction site safety monitoring, and reduce the risk of accidents. Support voice control input, easy operation, and enhance user experience. Intelligent abnormality recognition and control functions increase the intelligence and automation characteristics of the system. DETAILED DESCRIPTION

[0021] The present invention will be further described in detail with reference to the following examples.

[0022] The construction site safety early warning method based on the "Smart Eye" strategy of this embodiment includes three nodes: problem diversion, safety early warning, and smart eye control functions. Specifically, it includes:

[0023] 1. Problem triage: The system determines whether to issue a security warning or camera control processing flow based on the question input by the user.

[0024] 2. Security Abnormal Warning:

[0025] (1) Scene screening: The system screens the construction site scenes and determines the areas that require safety anomaly warnings.

[0026] (2) Rag instruction rewriting: The system uses the Rag algorithm to rewrite data to improve the accuracy of security anomaly identification.

[0027] (3) Security abnormality identification: The system outputs abnormality identification results, including "abnormal", "no abnormality" and "possible abnormality". If there is a possible abnormality, the system executes camera control and outputs the cause of the abnormality.

[0028] (1) User inputs camera control instructions: The system extracts actions and action parameters based on user instructions and controls the movement of the hardware camera for scene recognition.

[0029] (2) The user inputs the target image zoom command: the system determines the target location and outputs possible location information, including [upper left, upper right, lower left, lower right]. If the location of the safety anomaly warning cannot be accurately determined, the system outputs "possible anomaly" and performs image segmentation processing. The segmentation rule is: find the center point of the image, and divide the image into four equal rectangular areas through vertical and horizontal cutting, located at the upper left, lower left, upper right, and lower right respectively. According to the position output by the model, the image at the specified location is sent to the large model again for execution, and the zooming process is repeated twice.

[0030] In the process of the above-mentioned smart eye control, the use of subsequent image magnification target instructions can achieve precise positioning and response: In the monitoring system, it is crucial to quickly and accurately locate the abnormal position. Our method improves the accuracy of the abnormal position through multiple segmentation and model loop processing, reduces the probability of false alarms and missed alarms, and optimizes resources and improves efficiency: traditional methods may need to process the entire image, while our segmentation method significantly reduces the amount of data that needs to be processed, thereby reducing the consumption of system resources and improving processing efficiency. Enhanced system robustness: Through automatic anomaly detection and partitioning processing, this method improves the robustness of the system in complex scenarios and can better cope with changing monitoring environments.

[0031] Finally, to ensure more accurate location output, a large multimodal model is required to describe the event's location in detail based on the image and determine its orientation. Using a pre-trained Bert+Linear architecture, the Bert and Linear parameters are trained to transform the problem into a multi-classification problem, predicting the final event location.

[0032] The multimodal large model combines the image V and the prompt text T to output the event location description information Tdesc: After encoding the generated description Tdesc through BERT, a new text feature representation T′ is obtained:

[0033] Input T′ into the BERT model and calculate the text feature Top through Self-Attention:

[0034] Input the calculated feature Top into the linear layer for linear transformation:

[0035] Z=W·Top+b

[0036] Where W and b are the weight matrix and bias vector of the linear layer.

[0037] Input the linear transformation result Z into the Softmax layer to obtain the probability distribution of each position category: P(y|Top)=softmax(Z)

[0038] The loss value is calculated using cross entropy loss:

[0039]

[0040] In summary, the construction site safety early warning method based on the "Smart Eye" strategy in this embodiment uniquely combines the advantages of image processing and deep learning models within existing intelligent safety monitoring systems. This method not only improves positioning accuracy but also introduces a new processing mechanism to the system, overcoming the limitations of traditional methods in dynamic scenarios. Furthermore, the processing flow of segmentation and cyclic amplification represents a breakthrough in existing technology, providing a new perspective for addressing the complexity and diversity of surveillance images.

[0041] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A construction site safety early warning method based on the "smart eye" strategy, characterized by: The steps include: Step 1: Determine whether to perform a security warning or camera control based on the question input by the user. If it is a security warning, proceed to step 2; if it is a camera control, proceed to step 3; In step 2, the construction site scene is first screened to determine the areas that require safety anomaly warnings. The Rag algorithm is then used to rewrite the data, followed by safety anomaly identification and output. The output includes three recognition results: "abnormal," "no abnormal," and "possible abnormal." If the output result indicates a possible abnormality, the system executes camera control and enters step 3, and outputs the cause of the abnormality. Step three: receive the camera control instructions input by the user, extract the actions and action parameters according to the user instructions, control the movement of the hardware camera to perform scene recognition, and then receive the image zoom target instructions input by the user, zoom in on the image, and determine the location of the safety anomaly warning.

2. The construction site safety early warning method based on the "smart eye" strategy according to claim 1 is characterized by: The specific method of enlarging the image in step three is as follows: first, the system determines the target location and outputs possible location information, including [upper left, upper right, lower left, lower right]. If the location of the safety anomaly warning cannot be accurately determined, "possible anomaly" is output, and the image is segmented. The segmented image is then sent to the large model for execution again. The above steps are repeated twice for enlargement processing.

3. The construction site safety early warning method based on the "smart eye" strategy according to claim 2 is characterized by: The specific method of image segmentation is as follows: find the center point of the image, divide the image into four equal rectangular areas through vertical and horizontal cutting, located at the upper left, lower left, upper right, and lower right respectively, according to the position output by the model, and then send the image at the specified position to the large model for execution again, and repeat twice for magnification processing.

4. The construction site safety early warning method based on the "smart eye" strategy according to any one of claims 1 to 3, characterized in that: It also includes the overall architecture of using pre-trained Bert+linear layers. By training the parameters in Bert and linear, it is converted into a multi-classification problem to predict the location step of the final event.

5. The construction site safety early warning method based on the "smart eye" strategy according to claim 4 is characterized by: The specific steps of converting the parameters in Bert and Linear into a multi-classification problem and predicting the location of the final event are as follows: Step 1: The multimodal large model combines the image V and the prompt text T to output the event location description information Tdesc; Step 2: After encoding the generated description Tdesc through BERT, a new text feature representation T′ is obtained; Step 3: Input T′ into the BERT model and calculate the text feature Top through Self-Attention. Step 4: Input the calculated feature Top into the linear layer for linear transformation: Z=W·Top+b Where W and b are the weight matrix and bias vector of the linear layer; Step 5: Input the linear transformation result Z into the Softmax layer to obtain the probability distribution of each position category: P(y|Top)=softmax(Z) Step 6: Calculate the loss using cross entropy loss:

Images