Method and system for automatically monitoring and early warning site safety risk

Abstract

The invention discloses a site safety risk automatic monitoring and early warning method and system, and relates to the field of site safety management. According to the method for scheduling the camera to obtain the clear image recognition result, a plurality of role recognition models are constructed by using an image recognition technology, different types of site risks in a monitoring picture can be monitored and analyzed in real time, and by using a camera linkage mode, when one camera cannot carry out site risk recognition, the site risk recognition result can be quickly and accurately obtained. And other cameras in the site are called for auxiliary identification, so that the accuracy and credibility of an identification result are ensured. In addition, comprehensive monitoring of safety risks of different sites is realized by further combining the ability of understanding and judging complex and changeable safety risks of a site risk analysis decision model.

Description

Technical Field

[0001] The present application relates to the field of site safety management, and in particular to a method and system for automatic monitoring and early warning of site safety risks. Background Art

[0002] In the current field of site safety risk monitoring, the traditional method mainly relies on a fixed camera monitoring system combined with manual patrols. Although manual patrols can flexibly respond to various emergencies, they are limited by factors such as labor costs, patrol frequency, and subjective judgment, making it difficult to achieve comprehensive and timely risk monitoring. Although fixed camera monitoring systems can provide continuous monitoring images, they lack intelligent analysis and early warning capabilities and can only rely on manual viewing of surveillance videos to identify potential risks, which also has problems such as low efficiency and easy missed detection.

[0003] With the rapid development of computer technology, camera-based monitoring systems have gradually emerged. Such systems can automatically identify and warn of abnormal behaviors in monitoring images through simple image recognition. However, most existing monitoring systems are developed for specific application scenarios and only have simple face recognition, vehicle license plate recognition, hat removal recognition, etc., and lack comprehensive monitoring of complex site environments. Summary of the invention

[0004] The purpose of this application is to provide a method and system for automatic monitoring and early warning of site safety risks, so as to achieve comprehensive monitoring of complex site environments.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] In a first aspect, the present application provides a method for scheduling a camera to obtain clear image recognition results, comprising the following steps.

[0007] The image information captured by each camera in the site during the patrol mission is input into the image recognition model to identify different types of site risks, and obtain an identification result set for the area where each camera is located; the image recognition model includes multiple role recognition models; the role recognition model configures a corresponding risk rule set according to the site type, which is used to identify different types of site risks; the recognition result set includes: the risk type that triggers the rule and its annotation information, and the annotation information is risk description, suspected risk and unrecognizable.

[0008] When there is suspected risk or unrecognizable annotation information in the recognition result set of the area where the first target camera is located, at least one second target camera is dynamically scheduled to identify the area where the first target camera is located according to the location information of the first target camera until the risk description of the target risk type is obtained or all second target cameras have been scheduled; the first target camera is a camera in the venue, and the second target camera is a camera in the venue other than the first target camera that can capture the area where the first target camera is located; the target risk type is a risk type corresponding to the suspected risk or unrecognizable annotation information.

[0009] When all second target cameras have been scheduled but the risk description of the target risk type still cannot be obtained, the risk description of the target risk type is generated according to the labeling information of the target risk type obtained by all second target cameras.

[0010] In a second aspect, the present application provides a method for automatic monitoring and early warning of site safety risks, comprising the following steps.

[0011] The above method of dispatching cameras to obtain clear image recognition results is used to obtain a recognition result set for the area where each camera in the venue is located.

[0012] The identification result set, target parameters and personnel positioning data of each camera area are input into the site risk analysis decision model, and the monitoring and early warning decision is determined in combination with the preset risk scoring rules and risk level thresholds; the target parameters include: equipment parameters and / or environmental parameters.

[0013] Based on the monitoring and early warning decision, the working mode of the camera is dynamically adjusted to continuously track the risk area, and multi-camera collaborative monitoring is called through pan-tilt commands; at the same time, graded early warning information is sent to designated personnel according to the risk level, and evacuation notifications or equipment dispatch instructions are triggered.

[0014] On the third aspect, the present application provides a system for automatic monitoring and early warning of site safety risks, including: a cloud server, a monitoring center and a mobile APP; the mobile APP is deployed on the mobile phones of personnel at the site, and is used for personal information management, emergency assistance and positioning; the cloud server is connected to the monitoring center, the mobile APP and all cameras deployed at the site, and the cloud server is used to execute the above-mentioned method for automatic monitoring and early warning of site safety risks.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects.

[0016] The present application provides a method and system for automatic monitoring and early warning of site safety risks. The method of dispatching cameras in the present application to obtain clear image recognition results uses image recognition technology to build multiple role recognition models, which can monitor and analyze different types of site risks in the monitoring screen in real time, and use the camera linkage method. When one camera cannot identify the site risk, other cameras in the site are called for auxiliary identification, ensuring the accuracy and credibility of the identification results. In addition, the site risk analysis decision model is further combined with the ability to understand and judge complex and changeable safety risks to achieve comprehensive monitoring of safety risks in different sites.

[0017] The system of this application deploys a site risk analysis decision model containing multiple role capabilities in a cloud server, and classifies and identifies risk characteristics of different types of sites through the role corresponding to a specific risk rule set in the image recognition model; the site risk analysis decision model dynamically scores and accumulates the recognition results, and divides the safety level in combination with equipment parameters, environmental parameters and personnel positioning data, and triggers a hierarchical response mechanism: including interrupting the camera's regular patrol mission, calling multiple cameras to collaboratively focus on risk areas, sending warnings or evacuation notices according to risk levels, and linking the monitoring center's screen window to display hierarchical displays for risk tracking. The system also supports users to customize the combined risk rule set in the site risk analysis decision model and the image recognition model to adapt to the monitoring needs of different sites to form a closed-loop management of "identification-assessment-control-warning". BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a method for scheduling a camera to obtain a clear image recognition result is provided in one embodiment of the present application.

[0020] Figure 2 A flowchart of a method for automatic monitoring and early warning of site safety risks provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0023] In an exemplary embodiment, Figure 1 As shown, a method for scheduling a camera to obtain a clear image recognition result is provided, comprising the following steps: Step 101, input the image information captured by each camera in the site during the patrol mission into the image recognition model to identify different types of site risks, and obtain the recognition result set of the area where each camera is located; the image recognition model includes multiple role recognition models; the role recognition model configures the corresponding risk rule set according to the site type, which is used to identify different types of site risks; the recognition result set includes: the risk type that triggers the rule and its annotation information, and the annotation information is risk description, suspected risk and unrecognizable.

[0024] Step 102: When there is suspected risk or unrecognizable annotation information in the identification result set of the area where the first target camera is located, dynamically schedule at least one second target camera to identify the area where the first target camera is located according to the location information of the first target camera until the risk description of the target risk type is obtained or all second target cameras have been scheduled; the first target camera is a camera in the venue, and the second target camera is a camera in the venue other than the first target camera that can capture the area where the first target camera is located; the target risk type is a risk type corresponding to the suspected risk or unrecognizable annotation information.

[0025] Step 103 , when all second target cameras have been scheduled and the risk description of the target risk type still cannot be obtained, a risk description of the target risk type is generated according to the labeling information of the target risk type obtained by all second target cameras.

[0026] In step 103 of the present application, in the process of generating the risk description of the target risk type, a manual assistance method can be used, and the assistance method can view the scene images captured by each camera, or can be viewed on site. The labeling information of the target risk type obtained by the first target camera and each second target camera can also be automatically generated by the system. For example, when there is a preset number of labeling information that is suspected risk, the risk description is determined to be a risk, otherwise it is determined to be a risk that does not exist. The specific method can be set by yourself according to needs, and will not be repeated here.

[0027] In another exemplary embodiment of the present application, in the above step 101, the image recognition model includes many role recognition models, and the role recognition model is a trained role model, for example: a construction site safety patrol role, a factory security patrol role, a personnel safety and security patrol role, and these role models are a rule set formed by various safety risk rules. Different roles have different rules trained for different models, so the corresponding patrol role is selected according to the site type.

[0028] When the above method is applied to a construction site, the construction site safety patrol role is selected, and the image recognition model will make site risk judgments based on the following rules during the inspection process.

[0029] Rule A1: Gathering of people.

[0030] Regulation A2: Fire safety.

[0031] Rule A3: Smoke detection.

[0032] Rule A4: Construction Falls.

[0033] Rule A5: Collapse and burial.

[0034] Rule A6: Falling objects.

[0035] Rule A7: Proper wearing of safety equipment during construction.

[0036] Rule A8: Electrical safety construction risk detection.

[0037] Rule A9: Mechanical injury safety risk detection.

[0038] Rule A10: Personnel evacuation risk detection.

[0039] Rule A11: Problem Solving Test.

[0040] Each rule can be implemented based on a role identification model. In the embodiment of the present application, a new role identification model can be further added according to the new rules. The above 11 rules are an example. In practical applications, they can be increased or decreased according to needs. They will not be elaborated here. The above 11 rules cannot limit the scope of protection of the method of the present application.

[0041] In the embodiment of the present application, before the above step 101, the following steps need to be performed: According to the type of site to which the above method is applied, the inspection role of the site is obtained, such as the above-mentioned construction site safety inspection role.

[0042] The risk judgment rule set contained in the inspection role will be placed in the module of the image recognition model. At this time, the image recognition module has the recognition task of the corresponding rule set.

[0043] In another exemplary embodiment, the role recognition models corresponding to the above-mentioned different rules are obtained by training a deep learning model for image recognition, which is explained below using the flame detection rule as an example. The training method of the role recognition models of other rules is the same as that of the flame recognition rule, and the only difference is the difference in training samples, which will not be elaborated here.

[0044] Use web crawlers to grab images from the Internet, or purchase existing data sets, build a large number of flame data sets with labeled flames, and ensure that the flame data sets include flame images in different environments, angles, and lighting conditions to improve the generalization ability of the model.

[0045] Perform data cleaning on the flame dataset: remove unclear or unqualified images.

[0046] Label the fire dataset: Use tools such as LabelImg to label the images and mark the locations of the fires.

[0047] Enhance the flame dataset: increase the data volume by rotating, scaling, cropping, etc. to improve the robustness of the model.

[0048] The technical developer selects a deep learning model in the development environment, and may select one or more commonly used models. When a model (for example, a convolutional neural network model (CNN)) is selected, the model is directly trained to obtain the above-mentioned role recognition model for flame detection.

[0049] When multiple models are selected, for example, VGG, ResNet, Inception, MobileNet or EfficientNet are selected for flame recognition, the specific training process includes the following steps.

[0050] Divide the dataset: Divide the dataset into training set, validation set, and test set.

[0051] Configure training parameters: set hyperparameters such as learning rate, batch size, and number of iterations.

[0052] Train the model: Use frameworks such as TensorFlow or PyTorch to train the model. Monitor the loss function and accuracy during training, and adjust hyperparameters to optimize model performance.

[0053] Model evaluation: Use the test set to evaluate the model's accuracy, recall, F1 score and other indicators.

[0054] Further optimize the model and tune the model.

[0055] Parameter adjustment: Adjust hyperparameters such as learning rate, regularization coefficient, etc.

[0056] Ensemble learning: Use multiple models for voting or weighted averaging to improve prediction accuracy.

[0057] Transfer learning: Leverage pre-trained models for fine-tuning to speed up training and improve performance.

[0058] The trained role recognition model is packaged into a flame recognition rule function for loading or calling by other applications.

[0059] By combining multiple trained patrol rules, a rule set can be formed to obtain an image recognition model. Such a rule set is included in the site risk analysis decision model. When using the intelligent patrol monitoring center system, users can directly select a role with a set of rules. If it does not meet the needs of on-site patrols, they can select multiple patrol rules according to the actual situation on site and set such a rule set as a custom patrol role. At the same time, set a list of personnel who receive alarm information after each rule is triggered, as well as the risk threshold score after each rule is triggered. If there is no custom score setting, it will be processed according to the system default. After the patrol role is set, it is now a collection of multi-model, multi-recognition rules, and multi-attribute judgment parameters; if all patrol rules are loaded into the image recognition engine, the image recognition engine will have the recognition capabilities corresponding to these rules.

[0060] In an exemplary embodiment, Figure 2 As shown, a method for automatic monitoring and early warning of site safety risks is provided, including the following steps 201 to 203.

[0061] Step 201 , using the above method of scheduling cameras to obtain clear image recognition results, obtain a recognition result set of the area where each camera in the venue is located.

[0062] Step 202, input the identification result set, target parameters and personnel positioning data of each camera area into the site risk analysis decision model, combine the preset risk scoring rules and risk level thresholds to determine the monitoring and early warning decision; the target parameters include: equipment parameters and / or environmental parameters.

[0063] Step 203, based on the monitoring and early warning decision, dynamically adjust the working mode of the camera to continuously track the risk area, and call multi-camera collaborative monitoring through the pan-tilt command; at the same time, send graded early warning information to designated personnel according to the risk level, and trigger evacuation notification or equipment dispatch command.

[0064] By implementing the above steps 201 to 203, comprehensive monitoring of safety risks of complex sites can be achieved.

[0065] In an embodiment of the present application, the above step 201 sets a daily patrol work task for the site camera. The camera will perform daily patrols according to the working mode and time of the daily patrol task. The camera will transmit the captured picture information in the daily patrol in real time to the image recognition model for risk rule identification; if the risk rule is identified, the image recognition model will mark the key trigger content of the image and convert it into a picture, and the picture will be handed over to the site risk analysis decision model for subsequent risk prediction and evaluation.

[0066] In another exemplary embodiment, the above-mentioned site risk analysis decision model performs risk score statistics based on the risk judgment rule set in the role selected for the current site; determines whether to continue to pay attention to the current site risk or to issue a risk alarm based on the statistical results; interrupts the nearby camera patrol mission when necessary, and switches to continuous attention to the risk area to accurately judge the risk results; uses multiple cameras to repeatedly evaluate the risk content of the risk area, so that the risk estimate will be more accurate.

[0067] The inspection roles included in the site risk analysis decision model are composed of multiple risk judgment rule sets. When the image recognition model loads these risk judgment rule sets, it can perform real-time analysis on the currently captured images. When any rule is triggered, it generates a single alarm message and a risk score in combination with the training content. Depending on the level of risk, the inspection role determines whether the alarm message is sent to the designated site personnel or to the site management personnel.

[0068] Exemplarily, when the site monitoring task is patrol, the risk scoring rule in the above step 202 is a patrol rule, and the patrol rule includes rule B1 to rule B25.

[0069] Rule B1: Gathering of people.

[0070] Regulation B2: Fire detection.

[0071] Rule B3: Flame detection.

[0072] Rule B4: Smoke detection.

[0073] Rule B5: Toxic gas detection.

[0074] Rule B6: Environmental risk detection.

[0075] Rule B7: Theft risk detection.

[0076] Rule B8: Natural disaster detection.

[0077] Rule B9: Construction Falls.

[0078] Rule B10: Collapse and burial.

[0079] Rule B11: Falling Objects.

[0080] Rule B12: Safety equipment must be worn.

[0081] Rule B13: Safety protection of construction equipment.

[0082] Regulation B14: Electrical safety risks.

[0083] Regulation B15: Risk of mechanical injury.

[0084] Rule B16: Outsider Risks.

[0085] Rule B17: Abnormal personnel behavior.

[0086] Rule B18: Suspicious Person Risk.

[0087] Regulation B19: Food safety risks.

[0088] Regulation B20: Sports facility risks.

[0089] Rule B21: Theft and robbery security risks.

[0090] Regulation B22: Operational safety risks.

[0091] Rule B23: Personnel evacuation risk detection.

[0092] Rule B24: Problem Solving Test.

[0093] Rule B25: The camera cannot identify the risk.

[0094] The above rules are an example given in this application and can be added or reduced as needed in actual application.

[0095] Different rules contain different risk scores, which can be accumulated. When the accumulated risk scores are too high, the risk level of the site will increase accordingly.

[0096] The site risk analysis decision model is trained in the following way.

[0097] Technical developers select deep learning models in the development environment, mainly large text models (for example: GPT (Generative Pretrained Transformer), BERT (Bidirectional Encoder Representations from Transformers), deepseek-v3 (big data model) and other initial models). These models are particularly suitable for processing long dependencies, large-scale text data, and logical relationships between contexts while being able to load parameter data.

[0098] The initial model is further combined with the image recognition results for text training. Here, logic training is performed according to the patrol requirements of the construction site safety officer role, and two simple role training tasks are given.

[0099] Judgment basis for gathering of people: the recognition result of the screen or image is notified in text form, and the recognition result in text form is loaded into the large text model. The text content obtained at this time is: [More than 5 (including 5) people appear in one screen at the same time]. At this time, the large text model judges the recognition result of the current image as a gathering of people based on such text content, and records the coordinate information of the deployment area of ​​the current camera according to the current trigger rule, and records the risk value of the current area (for example, increase the risk value by 1). When the risk value is low, observe the current judgment for a long time until the risk is resolved. If the risk is not resolved, choose whether to warn the management personnel or upgrade the current risk according to the situation.

[0100] Basis for judging fire safety hazards: The recognition result of the screen or image is notified in text form. An exemplary text content is: [No fire-fighting facilities, fire extinguishers and other equipment are recognized in the current screen]. At this time, the large text model judges the recognition result of the current image as a fire safety hazard based on such text content, records the coordinate information of the deployment area of ​​the current camera, and records the risk value of the current area (exemplarily, the risk value is increased by 6).

[0101] The above-mentioned site risk analysis decision model also includes: an environment and equipment parameter model, a camera operation model and an early warning notification rule model. The environment and equipment parameter model is used to generate control instructions for the execution structure of collecting target parameters when the large language model needs to obtain the target parameters; the camera operation model is used to generate camera control instructions when the large language model needs to control the camera of the site; the early warning notification rule model is used to generate instructions for sending early warning information when the large language model needs to send early warning information to the personnel of the site.

[0102] Among them, the setting of the camera operation model is mainly to facilitate the large text model to make monitoring and early warning decisions and call the pan-tilt operation instructions of the camera. All pan-tilt operation instructions are encapsulated by the developers to form an instruction parameter table; it is set during the role logic training of the large text model. When the risk area score meets the threshold, it is necessary to obtain the surrounding cameras through the camera deployment area, and send pan-tilt operation instructions to these cameras through the instruction parameter table.

[0103] The setting of the environment and equipment parameter model is mainly to facilitate the use of the big text model to speculate on site risks based on some real-time parameters on site, such as: the current wind speed and direction of the site; the location information of personnel positioning, the coordinate set of the electronic fence within the site range; the working parameters of some construction machinery and equipment, etc.

[0104] The setting of the early warning notification rule model is mainly for the big text model to call some early warning notifications in real time according to the triggered rule events. The notification method, the list of personnel to be notified and other related parameter instructions are packaged by the developers to form an instruction parameter table; it is set during the big text model role logic training, and updated according to the accessed personnel information when it is applied to the system later.

[0105] The training process of the large text model is as follows.

[0106] Configure training parameters: set hyperparameters such as learning rate, batch size, and number of iterations.

[0107] Train the model: Use frameworks such as TensorFlow or PyTorch to train the model. Monitor the loss function and accuracy during training, and adjust hyperparameters to optimize model performance.

[0108] Text generation: Testers evaluate the quality of the text generated by the model, such as coherence, grammatical correctness, etc.

[0109] Code generation: Testers evaluate whether the code generated by the model can run correctly and whether the instruction set of the camera gimbal can be called normally.

[0110] Parameter tuning: Developers adjust the model’s parameters, such as learning rate, regularization coefficient, etc., to help establish stable model output.

[0111] Ensemble learning: Use multiple models for voting or weighted averaging to improve prediction accuracy.

[0112] Transfer learning: Leverage pre-trained models for fine-tuning to speed up training and improve performance.

[0113] Export model: Export the trained model to a deployable format such as TensorFlow SavedModel or ONNX.

[0114] Thus, a site risk analysis decision model is obtained, which can control the camera according to the configuration of the site environment parameters, personnel entry parameters, and the site camera list.

[0115] In another exemplary embodiment, in addition to obtaining the risk judgment rules of the role set, the site risk analysis decision model can obtain other real-time parameter information such as environmental parameter information of some equipment, personnel location information of the mobile APP of the personnel on the site, etc., to facilitate the role to make risk inferences. When the risk items are judged according to the environmental parameters, the work tasks of some pan-tilt cameras can be changed.

[0116] In the site risk analysis decision model, the patrol role risk warning standards and the camera coordination call standards are as follows.

[0117] Site safety risk level score definition: 0-2 safe, 2-4 slightly safe, 4-6 moderately safe, 6-8 low risk, 8-10 moderate risk, 11-15 high risk, 15-20 dangerous.

[0118] Safety: All cameras are performing patrol tasks normally.

[0119] Mild safety: The patrol mission of the camera that has captured the risk item is interrupted, and the current risk item is continuously tracked and photographed until the risk is eliminated or the risk object leaves. During this period, a tracking timer is kept. If the risk is not eliminated within 5 minutes, it is upgraded to moderate safety.

[0120] Moderate safety: The patrol mission of the camera that has captured the risk item is interrupted, and the current risk item is continuously tracked and photographed. When the risk object leaves the shooting range of the current patrol camera, the current tracking mission is assigned to nearby cameras, which then complete the tracking and photographing tasks alternately until the risk is eliminated. During this period, a tracking timer is kept. If the risk is not eliminated within 5 minutes, it is upgraded to low risk.

[0121] Risk: Interrupt the patrol mission of the camera that has captured the risk item, coordinate the PTZ cameras in the area, interrupt the patrol mission of the three nearby PTZ cameras, and switch to multi-angle risk identification of the current risk until the risk is eliminated. During this period, the site manager will be given APP and SMS warning reminders. In the multi-screen window of the monitoring center, the monitoring screen of the current situation will flash in a red window, and the tracking time will continue. If the risk is not eliminated within 5 minutes, it will be upgraded to a moderate risk.

[0122] Moderate risk: The patrol mission of the camera that has captured the risk item is interrupted. At the same time, the PTZ cameras in the area are coordinated to interrupt the patrol mission of the three nearby PTZ cameras. The current risk is identified by multi-angle shooting until the potential risk is eliminated. During this period, the site manager is given APP warning reminders, SMS warning reminders, and telephone warning notifications. In the multi-screen window of the monitoring center, the monitoring screen of the current situation flashes in a red window, and the tracking timer continues. If the potential risk is not eliminated within 5 minutes, it will be upgraded to high risk or danger.

[0123] High risk or danger: During this period, all personnel working on the site will receive APP warning reminders and SMS warning reminders, and an evacuation notice will be automatically sent to the mobile APP of the personnel on site. At the same time, the site management personnel will be warned by phone, and the multi-screen window of the monitoring center will be closed, showing only the main screen and three secondary screens of the current situation until the administrator actively resumes the patrol task and marks the site risk as safe.

[0124] In the embodiment of the present application, the early warning notification rule model selects the notification standard according to the list of alarm personnel entered during the use of the system, the basic information and location information of the entrants, and the dangerous situation of the site risk.

[0125] The site risk analysis decision model can provide intervention notifications for simple risk accidents of light safety, medium safety, risk, and medium risk. Such notifications are single-case notifications. For example, if a construction worker at high altitude does not wear safety equipment, the system will issue an early warning notification to the current worker based on the personnel entry positioning information and parameters.

[0126] The current system can issue event alarm notifications for complex risk accidents with high, medium, high and dangerous risks. Such notifications belong to multiple classification notifications. For example, when a fire occurs on site, the cameras around the fire are coordinated to determine the scope of the fire. According to the scope of the fire, the staff around the scope are notified to stay away from the fire area, expand the risk prediction of the event, and coordinate other cameras in the site to perform image retrieval of fire-fighting equipment. When the fire-fighting equipment is retrieved from the screen, the relevant area of ​​the fire-fighting equipment and other information are notified to the site manager, and the manager is reminded to organize personnel to use the fire-fighting equipment in this area to eliminate the risks on the site.

[0127] In an exemplary embodiment, the above-mentioned large text model is used to perform the following steps 301 to 307 .

[0128] Step 301 , according to a preset risk scoring rule, the risk description of the risk type that triggers the rule in the recognition result set of the area where the camera m is located is scored to obtain scoring results of different types of site risks.

[0129] Step 302: determine whether there is a site risk with a score greater than a first risk level threshold in the recognition result set of the area where the camera m is located, and obtain a first determination result.

[0130] Step 303: If the first judgment result is no, the scoring results of different types of site risks are comprehensively calculated to obtain a comprehensive scoring result, and the risk level of the area where the camera m is located is determined according to the comprehensive scoring result.

[0131] Step 304: If the first judgment result is yes, determine whether the site risk with a score result greater than the first risk level threshold is a site risk related to the target parameter, and obtain a second judgment result.

[0132] Step 305: If the second judgment result is yes, the risk level of the area where the camera m is located is determined based on the site risk whose scoring result is greater than the first risk level threshold and the target parameter.

[0133] Step 306: If the second judgment result is no, determine the risk level of the area where the camera m is located based on the site risk whose scoring result is greater than the first risk level threshold.

[0134] Step 307: Make a monitoring and early warning decision based on the risk level of the area where the camera m is located.

[0135] The specific monitoring and early warning decisions made for different risk levels are as follows.

[0136] When the risk level is safe, the monitoring and early warning decision made is: perform patrol tasks normally.

[0137] When the risk level is mild safety, the monitoring and early warning decision is made as follows: continuously obtain the identification result set of the area where the camera m is located, and determine whether the risk of the site with a risk level of mild safety has been eliminated within the first preset time period. If so, the risk level is updated to safe; if not, the risk level is updated to moderately safe.

[0138] When the risk level is moderately safe, the monitoring and early warning decision is formulated as follows: continuously obtain the identification result set of the area where all cameras are located in the first preset range centered on camera m, and the first preset range is dynamically adjusted according to the camera deployment density and the risk area; track the risk of the site with a risk level of moderately safe, and determine whether the risk of the site with a risk level of moderately safe is lifted within a second preset time period. If lifted, update the risk level to safe; if not lifted, update the risk level to risk.

[0139] When the risk level is risk, the monitoring and early warning decisions made are: issuing APP early warning reminders and SMS early warning reminders to the site manager, and issuing early warning reminders in the multi-screen window of the monitoring center; controlling all cameras in a second preset range centered on camera m to conduct multi-angle monitoring of site risks with a risk level of risk, and the second preset range is dynamically adjusted according to the camera deployment density and the risk area; determining whether the risk of the site with a risk level of risk is lifted within a third preset time period, and if so, updating the risk level to safe, and if not, updating the risk level to moderate risk.

[0140] When the risk level is medium risk, the monitoring and early warning decisions made are: APP early warning reminders, SMS early warning reminders and telephone early warning notifications are issued to the site managers, and early warning prompts are issued in the multi-screen window of the monitoring center; all cameras in a third preset range centered on camera m are controlled to perform multi-angle monitoring of site risks with a medium risk level, and the third preset range is dynamically adjusted according to the camera deployment density and the risk area; determine whether the risk of the site with a medium risk level is lifted within the fifth preset time period, and if so, update the risk level to safe; if not, update the risk level to high risk.

[0141] When the risk level is high risk or dangerous, the monitoring and early warning decision is made as follows: control all cameras in the fourth preset range centered on camera m to perform multi-angle monitoring of site risks with high risk levels; the fourth preset range is dynamically adjusted according to the camera deployment density and the risk area; APP early warning reminders and SMS early warning reminders are given to all personnel in the site, APP evacuation notifications are sent to all personnel in the site, telephone early warning notifications are given to the site manager, all screen windows except the target screen window of the monitoring center are controlled to be closed, and early warning prompts are given in the target screen window, wherein the target screen window includes screen windows of one or more cameras that can capture site risks with high risk or dangerous risk levels, until the system administrator resets the risk level to safe.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a system for automatically monitoring and warning site safety risks for implementing the method for automatically monitoring and warning site safety risks involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more system embodiments provided below can refer to the limitations on the method above, and will not be repeated here.

[0143] In an exemplary embodiment, a system for automatic monitoring and early warning of site safety risks is provided, including: a cloud server, a monitoring center, and a mobile APP; the mobile APP is deployed on the mobile phones of personnel at the site and is used for personal information management, emergency assistance and positioning; the cloud server is connected to the monitoring center, the mobile APP and all cameras deployed at the site, and the cloud server is used to execute the above-mentioned method for automatic monitoring and early warning of site safety risks.

[0144] In an exemplary embodiment, the various components included in the above system can be divided into hardware devices and software devices.

[0145] The hardware equipment mainly includes the monitoring center and the computer terminal used to deploy the cloud server. The monitoring center includes a display and a hard disk recorder.

[0146] Monitor: A supporting monitoring display device installed in the monitoring center, which displays the monitoring screen by connecting to the camera.

[0147] Hard disk recorder: used to link all surveillance cameras and record and save all surveillance images.

[0148] In the embodiment of the present application, the monitoring center is connected to all cameras on site. The camera is a supporting monitoring image acquisition device. By way of example, in the embodiment of the present application, a 360° camera with a pan / tilt head is selected, with a resolution of more than 5 million and a night infrared camera function, as the main control camera for intelligent risk analysis of the site.

[0149] Among them, software equipment includes cloud servers, mobile APP and large-screen monitoring APP.

[0150] Cloud server: used to store, process, analyze and train various data from smart terminals, cameras, mobile personnel apps, models, etc., providing functions such as personnel positioning, trajectory tracking, data analysis, site risk prediction, and real-time camera relay tracking.

[0151] Mobile APP: Installed on the mobile phones of on-site personnel, it provides functions such as personal information viewing, emergency assistance, and personnel positioning.

[0152] Large-screen monitoring APP: installed on a TV device or display device based on the Android underlying layer, used to view site personnel location information and relevant data on personnel safety emergency management.

[0153] In an exemplary embodiment, the working principle of the above system is as follows.

[0154] All cameras and other related hardware of the site are deployed at the site where intelligent patrols are required; cameras are deployed in important areas, mobile areas such as walking areas, etc. in accordance with the requirements of no-blind-angle shooting.

[0155] Barrier devices are deployed at all entrances and exits of the venue. The barrier devices use smart terminal devices to connect to the cloud server to obtain the information of people entering and leaving the venue and their location coordinates. If there are no conditions to deploy barrier devices, people entering the venue can wear positioning identification bracelets and other devices to send the real-time location information and basic information of the entrants to the cloud server.

[0156] The cameras in the venue take real-time photos of the venue and also transmit the image information to the cloud server. The image recognition model deployed in the cloud server analyzes the images in real time.

[0157] Many site risk analysis decision models with different roles that have been trained are deployed in the cloud server, such as construction site safety patrol role, factory security patrol role, and personnel safety patrol role. These role models are a set of rules formed by various safety risk rules. Different roles have different rules trained for different models, so the corresponding patrol role is selected according to the site type.

[0158] The method and system provided by the embodiments of the present application have the following advantages.

[0159] 1. Improve the efficiency of site safety risk monitoring.

[0160] Automated monitoring: Through image recognition technology, real-time monitoring and analysis of abnormal behaviors in the monitoring screen can be achieved to improve monitoring efficiency.

[0161] Accurate early warning: Based on the preset risk rule set, accurate early warning of potential risks can be provided to reduce missed detections and false alarms.

[0162] 2. Improve risk management capabilities.

[0163] Standardized management: Through the site risk analysis decision model, the site safety risk rules are standardized to ensure consistent risk management levels among different personnel.

[0164] Flexible response to complex environments: The site risk analysis decision model can be customized and trained according to the safety risk monitoring needs and risk points of different sites to adapt to complex and changing site environments.

[0165] 3. Enhance risk response capabilities.

[0166] Real-time response: The site risk analysis decision model can monitor and analyze surveillance images in real time and respond quickly to potential risks.

[0167] Multi-level early warning mechanism: formulate different early warning measures according to the risk level to ensure that risks are controlled in a timely and effective manner.

[0168] 4. Improve monitoring accuracy.

[0169] Efficient identification: The image recognition technology included in the site risk analysis decision model can accurately identify abnormal behaviors in the monitoring images and reduce false alarms and missed detections.

[0170] All-round monitoring: Through the cooperation of PTZ cameras and fixed cameras, all-round monitoring of the site can be achieved.

[0171] 5. Scalability and customizability.

[0172] Customizable rule sets: Users can customize risk rule sets based on site requirements to meet security risk monitoring needs in different scenarios.

[0173] Patrol roles are expandable: The site risk analysis decision model can be trained according to new risk rules and new patrol roles can be added.

[0174] 6. Improve resource utilization.

[0175] Intelligent scheduling: The site risk analysis decision model can intelligently schedule PTZ cameras for risk tracking and shooting based on the risk level and camera distribution.

[0176] Data storage and analysis: Cloud servers can store and process large amounts of data, providing data support and analysis basis for site safety risk monitoring.

[0177] 7. Reduce labor costs.

[0178] Replace manual patrols: The intelligent monitoring system can replace some manual patrol tasks and reduce the number of security personnel.

[0179] Reduce training costs: After being trained, the site risk analysis decision model can continuously and stably perform patrol tasks without the need for frequent personnel training and replacement.

[0180] In order to illustrate the specific implementation of the above-mentioned method and system of the present application, the embodiments of the present application also provide 4 specific examples.

[0181] Example 1: Monitoring and early warning of crowd gatherings and fights.

[0182] In the workers' residential area of ​​a large construction site, after nightfall, the workers finished their hard work for the day and some of them chose to gather together to rest and chat. The camera of the intelligent patrol monitoring system captured this scene and clearly recorded the gathering of people.

[0183] When the number of people exceeds the system's preset threshold (such as more than 5 people) and continues for a period of time, the image recognition model immediately identifies the risk rule of "people gathering" and triggers a risk warning. At this time, the system not only marks the key trigger content, but also converts it into a picture form and hands it over to the site risk analysis decision model for further risk prediction and judgment.

[0184] Through analysis, the system found that some people in the crowd were drinking, and the atmosphere was becoming increasingly tense, with signs of a fight. The model quickly mobilized nearby cameras for multi-angle monitoring and initiated a higher level of risk assessment. As the situation escalated, the system immediately interrupted the regular patrol mission of nearby cameras and turned to continuous attention to the risks of the incident.

[0185] At the same time, the system sent an APP warning reminder and SMS warning to the site manager through the cloud server, detailing the risk area, risk type and possible consequences. The manager responded quickly and sent security personnel to the scene to intervene, effectively preventing the fight from happening.

[0186] Advantageous results of case application: Through the combination of image recognition technology and site risk analysis decision model, the system can monitor and analyze abnormal behaviors in the monitoring screen in real time, and accurately warn of potential risks, effectively reducing labor costs and improving risk management level. In addition, the system's real-time response and multi-level warning mechanism also ensure that risks are controlled in a timely and effective manner.

[0187] Example 2: An open fire monitoring warning was issued in the material storage area of ​​the construction site.

[0188] In the material storage area of ​​the construction site, due to improper operation of workers, an open fire was accidentally caused. The camera of the intelligent patrol monitoring system immediately captured this abnormal situation, identified the risk rule of "fire safety hazard", and triggered the risk alarm.

[0189] As the fire gradually grew, the model quickly coordinated multiple cameras in the surrounding area to observe the fire from multiple angles. The system sent an emergency warning reminder to the site manager through the cloud server, and notified all personnel on site through various means such as APP and SMS.

[0190] To ensure the safe evacuation of personnel, the system linked the cameras in the walking area to monitor the evacuation passage in real time to determine whether there were any obstructions or obstacles that affected the evacuation. Through intelligent analysis, the system found that an evacuation passage was blocked by temporarily stacked materials and immediately issued a warning to the manager that the evacuation passage was blocked.

[0191] In order to further reduce property losses, the system will use the on-site cameras to find suitable fire-fighting facilities and equipment, and notify the manager of the location information of the found fire-fighting equipment, reminding the manager to organize personnel to use fire-fighting equipment to further control the fire on site.

[0192] The manager quickly organized workers to clear the evacuation passages to ensure that people could evacuate quickly, and obtained firefighting equipment to further control the spread of the fire. With the assistance of the intelligent patrol monitoring system, the fire was effectively controlled in time, and people were evacuated in an orderly manner, without causing casualties or major property losses.

[0193] Advantages of case application: The system can not only monitor and analyze abnormal behaviors in the monitoring screen in real time, but also formulate different early warning measures according to the risk level. Through intelligent scheduling and linkage control, the system can ensure that risks are controlled in a timely and effective manner, while improving monitoring accuracy and resource utilization.

[0194] Example 3: Object falling risk monitoring and early warning.

[0195] At a large construction site, the building towers into the sky, and the construction workers are busy carrying out various operations at high altitudes. Due to the busy work, some construction workers randomly scattered their tools nearby, and this scene quickly attracted the attention of the intelligent patrol monitoring system. At this time, the system was collecting the location information of each construction worker in real time and learned that a worker was about to pass by here.

[0196] A worker was walking along the road below the building, preparing to go to another construction area. At the same time, the system combined the strong wind weather conditions of the day, as well as the location information and walking speed of the worker, and conducted a series of complex data analysis and evaluation. The system determined that the tools scattered at high altitudes were likely to be blown down by the strong wind, thereby injuring the workers below.

[0197] Faced with this life safety risk, the system immediately activated the early warning mechanism. It not only sent risk warning information to the construction workers at high altitudes, reminding them to immediately sort out the scattered tools and strengthen safety protection; it also sent precise early warning notifications to the workers passing below, informing them to change their walking routes. Thanks to the system's timely intervention and accurate early warning, this potential accident of falling objects injuring people was successfully avoided.

[0198] Example 4: Monitoring and early warning of safety risks in high-altitude operations.

[0199] At the construction site, in a windy weather, the weather data of the day was connected to the big model in real time, providing the system with accurate environmental parameter information. At the same time, the camera was closely monitoring the situation in the high-altitude working area.

[0200] On the aerial work platform, a worker was carrying out intense technical work. However, due to negligence, he did not wear the safety rope correctly. This safety hazard was quickly captured by the camera. The system used image recognition technology to accurately determine that the worker did not wear the safety rope, and immediately launched the safety risk analysis decision model for further evaluation.

[0201] Combined with the strong winds on that day and the worker's work location information, the system determined that the worker was facing a very high safety risk in construction. Therefore, the system quickly sent a risk warning message to the worker, reminding him to stop working immediately and wear a safety rope correctly. At the same time, the system also notified the on-site management staff of the risk so that they could take further safety measures in a timely manner and criticize and correct the dangerous worker.

[0202] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0203] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for scheduling a camera to obtain a clear image recognition result, characterized in that: The steps include: Inputting image information captured by each camera in the site during the patrol mission into an image recognition model to identify different types of site risks, and obtaining a recognition result set of the area where each camera is located; the image recognition model includes multiple role recognition models; The role identification model configures a corresponding risk rule set according to the site type, which is used to identify different types of site risks; the identification result set includes: the risk type that triggers the rule and its annotation information, and the annotation information is risk description, suspected risk and unidentifiable; When there is suspected risk or unrecognizable annotation information in the identification result set of the area where the first target camera is located, at least one second target camera is dynamically scheduled to identify the area where the first target camera is located according to the location information of the first target camera until the risk description of the target risk type is obtained or all second target cameras have been scheduled; the first target camera is a camera in the venue, and the second target camera is a camera in the venue other than the first target camera that can capture the area where the first target camera is located; the target risk type is a risk type corresponding to the suspected risk or unrecognizable annotation information; When all second target cameras have been scheduled and the risk description of the target risk type still cannot be obtained, the risk description of the target risk type is generated according to the labeling information of the target risk type obtained by all second target cameras.

2. A method for automatic monitoring and early warning of site safety risks, characterized in that: The steps include: Using the method for obtaining clear image recognition results by dispatching cameras as described in claim 1, a recognition result set of the area where each camera in the venue is located is obtained; Input the recognition result set, target parameters and personnel positioning data of each camera area into the site risk analysis decision model, and determine the monitoring and early warning decision in combination with the preset risk scoring rules and risk level thresholds; The target parameters include: equipment parameters and / or environmental parameters; Based on the monitoring and early warning decision, the working mode of the camera is dynamically adjusted to continuously track the risk area, and multi-camera collaborative monitoring is called through pan-tilt commands; at the same time, graded early warning information is sent to designated personnel according to the risk level, and evacuation notifications or equipment dispatch instructions are triggered.

3. The method for automatic monitoring and early warning of site safety risks according to claim 2 is characterized in that: The site risk analysis decision model includes a large text model, which is used to perform the following steps: According to the preset risk scoring rules, the risk descriptions of the risk types that trigger the rules in the recognition result set of the area where the camera m is located are scored to obtain scoring results of different types of site risks; Determine whether there is a site risk with a score greater than a first risk level threshold in the recognition result set of the area where the camera m is located, and obtain a first determination result; If the first judgment result is no, then comprehensively calculate the scoring results of different types of site risks to obtain a comprehensive scoring result, and determine the risk level of the area where the camera m is located according to the comprehensive scoring result; If the first judgment result is yes, determining whether the site risk with a score result greater than the first risk level threshold is a site risk related to the target parameter, and obtaining a second judgment result; If the second judgment result is yes, determining the risk level of the area where the camera m is located based on the site risk whose scoring result is greater than the first risk level threshold and the target parameter; If the second judgment result is no, determining the risk level of the area where the camera m is located based on the site risk whose scoring result is greater than the first risk level threshold; Make monitoring and early warning decisions based on the risk level of the area where camera m is located.

4. The method for automatic monitoring and early warning of site safety risks according to claim 3 is characterized in that: Make monitoring and early warning decisions based on the risk level of the area where camera m is located, including: When the risk level is safe, the monitoring and early warning decision is: perform patrol tasks normally; When the risk level is mild safety, the monitoring and early warning decision is: continuously obtain the identification result set of the area where the camera m is located, determine whether the risk of the site with a risk level of mild safety is eliminated within the first preset time period, if eliminated, update the risk level to safe, if not eliminated, update the risk level to moderate safety; When the risk level is moderately safe, the monitoring and early warning decision is formulated as follows: continuously obtain the identification result set of the area where all cameras are located in the first preset range centered on camera m, and the first preset range is dynamically adjusted according to the camera deployment density and the risk area; track the risk of the site with a risk level of moderately safe, and determine whether the risk of the site with a risk level of moderately safe is lifted within a second preset time period. If lifted, update the risk level to safe; if not lifted, update the risk level to risk.

5. The method for automatic monitoring and early warning of site safety risks according to claim 3 is characterized in that: Make monitoring and early warning decisions based on the risk level of the area where camera m is located, including: When the risk level is risk, the monitoring and early warning decisions made are: issuing APP early warning reminders and SMS early warning reminders to the site manager, and issuing early warning reminders in the multi-screen window of the monitoring center; controlling all cameras in a second preset range centered on camera m to conduct multi-angle monitoring of site risks with a risk level of risk, and the second preset range is dynamically adjusted according to the camera deployment density and the risk area; determining whether the risk of the site with a risk level of risk is lifted within a third preset time period, and if so, updating the risk level to safe, and if not, updating the risk level to moderate risk.

6. The method for automatic monitoring and early warning of site safety risks according to claim 3 is characterized in that: Make monitoring and early warning decisions based on the risk level of the area where camera m is located, including: When the risk level is medium risk, the monitoring and early warning decisions made are: APP early warning reminders, SMS early warning reminders and telephone early warning notifications are issued to the site managers, and early warning prompts are issued in the multi-screen window of the monitoring center; all cameras in a third preset range centered on camera m are controlled to perform multi-angle monitoring of site risks with a medium risk level, and the third preset range is dynamically adjusted according to the camera deployment density and the risk area; determine whether the risk of the site with a medium risk level is lifted within the fifth preset time period, and if so, update the risk level to safe; if not, update the risk level to high risk.

7. The method for automatic monitoring and early warning of site safety risks according to claim 3 is characterized in that: Make monitoring and early warning decisions based on the risk level of the area where camera m is located, including: When the risk level is high risk or dangerous, the monitoring and early warning decision is made as follows: control all cameras in the fourth preset range centered on camera m to perform multi-angle monitoring of site risks with high risk levels; the fourth preset range is dynamically adjusted according to the camera deployment density and the risk area; APP early warning reminders and SMS early warning reminders are given to all personnel in the site, APP evacuation notifications are sent to all personnel in the site, telephone early warning notifications are given to the site manager, all screen windows except the target screen window of the monitoring center are controlled to be closed, and early warning prompts are given in the target screen window, wherein the target screen window includes screen windows of one or more cameras that can capture site risks with high risk or dangerous risk levels, until the system administrator resets the risk level to safe.

8. The method for automatic monitoring and early warning of site safety risks according to claim 7 is characterized in that: When the risk level is high risk or dangerous, the monitoring and early warning decisions made also include: The linked camera monitors the personnel evacuation passage in real time to determine whether there are any obstacles. If there are any obstacles, a warning of blocked evacuation passage will be sent to the management personnel. The cleaning equipment used to clean the evacuation passage in the venue will be retrieved, and the location information of the cleaning equipment will be notified to the management personnel.

9. The method for automatic monitoring and early warning of site safety risks according to claim 3 is characterized in that: The site risk analysis decision model also includes: an environment and equipment parameter model, a camera operation model and an early warning notification rule model; The environment and device parameter model is used to generate a control instruction of an execution structure for acquiring target parameters when the large language model needs to obtain target parameters; The camera operation model is used to generate a pan / tilt command for controlling the camera when the large language model needs to control the camera of the venue; The early warning notification rule model is used to generate an instruction for sending early warning information when the large language model needs to send early warning information to personnel at the site.

10. A system for automatic monitoring and early warning of site safety risks, characterized in that: The site safety risk automatic monitoring and early warning system includes: a cloud server, a monitoring center and a mobile APP; The mobile APP is installed on the mobile phones of the personnel at the venue and is used for personal information management, emergency assistance and positioning; The cloud server is connected to the monitoring center, the mobile APP and all cameras deployed on the site, and the cloud server is used to execute the method for automatic monitoring and early warning of site safety risks described in any one of claims 2-9.

Images