An intelligent safety warning method and system for high-altitude operations

Abstract

The present application discloses an intelligent safety warning method and system for high-altitude operations, the method comprising: obtaining a high-altitude operation image at the current moment, and performing image recognition on the high-altitude operation image; when the monitored subject includes a dynamic subject and there is only one dynamic subject, performing image recognition on a first target sub-area; locating the first current key point of the dynamic subject at the current moment, and determining the first current action state of the dynamic subject; determining the end action state according to the first target sub-area at the current moment, and predicting the action duration from the first current action state to the end action state; determining whether the monitored subject includes a static subject; if so, determining a first safety risk coefficient of the dynamic subject; if the first safety risk coefficient is greater than a first preset value, obtaining the first target sub-areas at the next N moments one by one; predicting the motion trajectory of the dynamic subject, determining the action hazard level of the dynamic subject and displaying it.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image recognition technology, and in particular to an intelligent safety warning method and system for high-altitude operations. Background Art

[0002] Aerial work refers to work activities performed at a certain height above the ground or other horizontal surfaces, and is often used in the fields of construction, power communications, warehousing, etc. Due to the potential risk of falling during aerial work, necessary safety measures need to be taken to ensure the safety of people working at heights.

[0003] In addition to the necessary guarantees for the safety of aerial work through safety equipment (such as safety helmets, safety belts and non-slip shoes), fall protection devices (such as guardrails and safety nets) and work platforms, the aerial work process also needs to be monitored in real time so that accidents can be handled and rescued in a timely manner at any time.

[0004] Traditional safety monitoring methods for aerial work are relatively simple and passive, mainly relying on manual inspections or fixed camera monitoring installed on the work platform, which makes it difficult to achieve all-round, all-time real-time monitoring. Manual inspections are easily affected by subjective factors, and it is impossible to achieve 24-hour uninterrupted monitoring. Although fixed cameras can monitor continuously, they lack intelligent analysis capabilities and cannot actively identify dangerous behaviors. Secondly, the aerial work environment is complex and changeable, with many potential risk factors. It is difficult to fully assess the safety status by relying solely on simple sensors.

[0005] Due to the existence of the above-mentioned problems, traditional high-altitude work safety monitoring methods are unable to comprehensively assess the safety of high-altitude work. Even if high-altitude workers wear safety equipment, other unmonitored risk factors may still cause safety accidents, resulting in low accuracy of safety monitoring.

[0006] Based on the above problems, there is currently no effective solution. Summary of the invention

[0007] The embodiments of the present application provide an intelligent safety warning method and system for high-altitude operations, which are used to comprehensively evaluate the safety of high-altitude operations and improve the accuracy of safety monitoring.

[0008] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0009] In a first aspect, an intelligent safety warning method for high-altitude operations is provided, the method comprising:

[0010] Acquire an aerial work image at the current moment, and perform image recognition on the aerial work image to divide the aerial work image into a plurality of sub-areas according to the aerial work project, each of the sub-areas includes at least one monitoring subject, and the monitoring subject includes a static subject and / or a dynamic subject;

[0011] In a case where the monitored subject in one of the sub-regions includes the dynamic subject and the number of the dynamic subject is one, taking the sub-region as a first target sub-region, and performing image recognition on the first target sub-region to obtain the dynamic subject in the first target sub-region;

[0012] Locating a first current key point of the dynamic subject at a current moment, and determining a first current action state of the dynamic subject according to the first current key point;

[0013] Determine an end action state according to the first target sub-region at the current moment, and predict an action duration from the first current action state to the end action state based on the first current action state and the end action state;

[0014] determining whether the monitored subject in the first target sub-area also includes the static subject;

[0015] In a case where the monitoring subject also includes the static subject, determining a first safety risk coefficient of the dynamic subject according to the first current action state and the static subject;

[0016] When the first safety risk coefficient is greater than a first preset value, acquiring the first target sub-areas at the next N moments one by one, and a set of the current moment and the next N moments is less than or equal to the action duration;

[0017] Predicting a motion trajectory of the dynamic subject according to changes in key points of the dynamic subject in the first target sub-region at the current moment and in the first target sub-region at the next N moments;

[0018] According to the motion trajectory of the dynamic subject, the dangerousness level of the action of the dynamic subject is determined and displayed.

[0019] In a possible implementation manner of the first aspect, the method further includes:

[0020] In the case where the monitored subject in one of the sub-areas includes the dynamic subject and there are at least two dynamic subjects, the sub-area is used as a second target sub-area, and preset data of the second target sub-area at the current moment is obtained, wherein the preset data includes a standard number of the dynamic subjects and standard action data of each of the dynamic subjects;

[0021] When the number of the dynamic subjects is consistent with the standard number and the standard action data of each of the dynamic subjects is consistent, locating the second current key point of each of the dynamic subjects at the current moment, and determining the second current action state of each of the dynamic subjects at the current moment according to each of the second current key points;

[0022] determining whether the monitored subject in the second target sub-area also includes the static subject;

[0023] In the case where the monitoring subject also includes the static subject, determining a second safety risk coefficient of each of the dynamic subjects according to the second current key point position of each of the dynamic subjects and the static subject;

[0024] In the event that any of the second safety risk coefficients is greater than a second preset value, a first warning message is issued.

[0025] In another possible implementation manner of the first aspect, the method further includes:

[0026] When the number of the dynamic subjects is consistent with the standard number and the standard action data of each of the dynamic subjects is inconsistent, each of the dynamic subjects is analyzed in parallel to obtain the action data of each of the dynamic subjects in a preset time period, and the end time of the preset time period is the current time;

[0027] Calculating the action deviation between the action data and the corresponding standard action data;

[0028] Creating an interaction matrix and calculating the relative positions and interactions between the dynamic entities in real time, wherein the interaction matrix is ​​used to store the relative positions and interactions in real time;

[0029] Using an individual safety risk index formula, according to the action deviation and the interaction matrix, a third safety risk coefficient of each of the dynamic subjects is calculated;

[0030] In the event that any one of the third safety risk coefficients is greater than a third preset value, a second warning message is issued.

[0031] In another possible implementation manner of the first aspect, locating a first current key point of the dynamic subject at a current moment, and determining a first current action state of the dynamic subject according to the first current key point, includes:

[0032] Using a preset skeleton key point detection algorithm, the key point of the dynamic subject at the current moment is located to obtain the first current key point position of the dynamic subject at the current moment;

[0033] Obtaining the three-dimensional coordinates of each of the first current key points;

[0034] Calculating the relative distance and angle between the first current key points according to the three-dimensional coordinates of each of the first current key points;

[0035] Inputting all the relative distances and the angles into a pre-trained classifier to obtain a probability distribution, wherein the probability distribution is used to characterize the possibility that the current action state of the dynamic subject belongs to a predefined action state;

[0036] The current action state with the highest probability is used as the first current action state.

[0037] In another possible implementation manner of the first aspect, predicting an action duration from the first current action state to the end action state based on the first current action state and the end action state includes:

[0038] Calculating a first vector of the first current action state and a second vector of the end action state, and acquiring subject features of the dynamic subject;

[0039] Inputting the first vector and the second vector into a pre-trained action state space model to obtain action change complexity, wherein the action state space model is used to calculate the Euclidean distance between the first vector and the second vector, and the Euclidean distance is used to characterize the action change complexity;

[0040] The subject feature, the first vector, the second vector, and the action change complexity are input into a pre-trained deep learning model to obtain the action duration.

[0041] In another possible implementation manner of the first aspect, determining the first security risk coefficient of the dynamic subject according to the first current action state and the static subject includes:

[0042] Determining the postural stability of the first current motion state using a biomechanical model;

[0043] Acquire the current height of the dynamic subject, and determine the risk probability of the static subject to the dynamic subject according to the current height and the static subject;

[0044] A first safety risk coefficient of the dynamic body is determined according to the posture stability and the risk probability.

[0045] In another possible implementation manner of the first aspect, determining the risk probability of the static subject to the dynamic subject according to the current height and the static subject includes:

[0046] Mapping the dynamic subject and the static subject into a preset three-dimensional coordinate system, and calculating a first distance and a height difference between the dynamic subject and each of the static subjects;

[0047] A preset risk function is used to calculate the risk probability of the static subject to the dynamic subject according to the first distance and the height difference.

[0048] In another possible implementation manner of the first aspect, determining a first safety risk coefficient of the dynamic subject according to the posture stability and the risk probability includes:

[0049] The first safety risk coefficient of the dynamic body is obtained by performing a weighted summation on the posture stability and the risk probability.

[0050] In a second aspect, the present application provides an electronic device, including:

[0051] a memory configured to store instructions; and

[0052] The processor is configured to call the instructions from the memory and implement the above-mentioned intelligent safety warning method applied to high-altitude operations when executing the instructions.

[0053] In a third aspect, the present application provides an intelligent safety warning system for high-altitude operations, comprising:

[0054] Camera;

[0055] An electronic device is connected to the camera.

[0056] Through the above technical solution, by obtaining the current high-altitude operation image and performing intelligent analysis, all-round and real-time monitoring can be achieved, overcoming the limitations of traditional methods that rely on manual inspections or fixed cameras; by locating key points, determining the action state, and predicting the duration of the action, the behavior of high-altitude workers can be dynamically analyzed. In addition, by calculating the safety risk coefficient and predicting the action hazard level based on the motion trajectory of the dynamic subject, a comprehensive assessment of the safety of high-altitude operations can be achieved, and the impact of static subjects on dynamic subjects can be considered, reflecting a comprehensive consideration of the complex and changeable high-altitude operation environment. In summary, it provides more comprehensive, accurate, and real-time safety monitoring, which can actively identify potential risks and effectively improve the safety of high-altitude operations and the accuracy of safety monitoring.

[0057] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a flow chart of an intelligent safety warning method for high-altitude operations provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of an aerial work image after image recognition provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of the structure of an intelligent safety warning system for high-altitude operations provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0062] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0063] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0064] Figure 1 The following schematically shows a flow chart of an intelligent safety warning method for high-altitude operations according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides an intelligent safety warning method for high-altitude operations, which may include the following steps.

[0065] S110, acquiring an aerial work image at the current moment, and performing image recognition on the aerial work image, so as to divide the aerial work image into a plurality of sub-areas according to the aerial work project, each sub-area including at least one monitoring subject, and the monitoring subject including a static subject and / or a dynamic subject;

[0066] S120: When the monitored subject in one of the sub-regions includes a dynamic subject and the number of the dynamic subject is one, the sub-region is used as a first target sub-region, and image recognition is performed on the first target sub-region to obtain the dynamic subject in the first target sub-region;

[0067] S130, locating a first current key point of the dynamic subject at the current moment, and determining a first current action state of the dynamic subject according to the first current key point;

[0068] S140, determining an end action state according to the first target sub-region at the current moment, and predicting an action duration from the first current action state to the end action state based on the first current action state and the end action state;

[0069] S150, determining whether the monitored subjects in the first target sub-area also include static subjects;

[0070] S160: when the monitored subject also includes a static subject, determining a first safety risk coefficient of the dynamic subject according to the first current action state and the static subject;

[0071] S170, when the first safety risk coefficient is greater than a first preset value, obtaining first target sub-areas at the next N moments one by one, and the set of the current moment and the next N moments is less than or equal to the action duration;

[0072] S180, predicting a motion trajectory of the dynamic subject according to changes in key points of the dynamic subject in the first target sub-region at the current moment and the first target sub-region at the next N moments;

[0073] S190. Determine and display the dangerousness level of the dynamic subject's action according to the motion trajectory of the dynamic subject.

[0074] First, the high-definition camera installed at the work site (such as aerial work platform, hanging basket) collects the aerial work image at the current moment in real time, and performs image recognition processing on the aerial work image. Specifically, deep learning algorithms (such as YOLO) can be used for target detection and semantic segmentation. The entire aerial work image can be divided into multiple sub-areas according to the pre-defined aerial work project types (such as scaffolding work, tower crane work, etc.). Each sub-area contains at least one monitoring subject, which can be static (such as scaffolding, tower crane and other equipment) or dynamic (such as aerial workers). Figure 2FIG. 4 shows an aerial work image after image recognition provided by an embodiment of the present application, such as Figure 2 As shown, the scenario involved is a hanging basket scenario. In this scenario, dynamic subjects (height workers) and static subjects (anti-fall pads, safety equipment and fences) can be obtained through image recognition, and whether the safety equipment is worn normally can be determined by whether the safety equipment is monitored.

[0075] After completing the sub-region division, further analysis is performed on the sub-region that contains only one dynamic subject (such as a single aerial worker). This sub-region is defined as the first target sub-region, and image recognition is performed on it. Specifically, a preset human posture estimation algorithm can be used to accurately locate and recognize the posture of the dynamic subject. The human posture estimation algorithm can be used to identify key points of the human body, such as the head, shoulders, elbows, wrists, hips, knees, and ankles.

[0076] Through the spatial relationship of key points, the posture and action of the dynamic subject can be accurately described, and the first current action state of the dynamic subject can be obtained. For example, for a worker working on a high-altitude scaffold, it is possible to identify whether he is standing, bending over, or squatting. By focusing on a single dynamic subject, subtle changes in individual behavior can be captured more accurately, thereby improving the accuracy and timeliness of safety warnings.

[0077] Based on the dynamic subject image obtained by S120, the key points of the dynamic subject at the current moment are further located. A skeleton key point detection algorithm can be used to accurately locate 17 or more key points of the human body. Key points include the top of the head, neck, shoulders, elbows, wrists, hips, knees, ankles, etc. Each key point has its corresponding two-dimensional or three-dimensional coordinates. According to the relative position and angle relationship of the key points, the current action state of the dynamic subject can be determined. For example, if it is detected that the height of the hips and knees are close, and the upper body is tilted at a large angle, it can be judged as a bent state; if it is detected that the height of all key points is low and distributed horizontally, it can be judged as a lying state.

[0078] In specific implementation, this can be achieved through rule definition. For example, a rule can be defined: if the hip height is less than 50% of the shoulder height, and the knee angle is greater than 150 degrees, it is determined to be a sitting posture.

[0079] After determining the current action state of the dynamic subject, the duration of the action is predicted. First, according to the characteristics of the first target sub-region at the current moment (such as region type, area, height, etc.) and the predefined operation specifications, the expected end action state in the region is determined.

[0080] For example, if the current area is a work platform on a scaffold, the end action state may be standing or squatting. Then, based on the current action state and the expected end state, a time series prediction model is used to predict the time required to transition from the current state to the end state. The time series prediction model can consider multiple factors, such as the average duration of similar action sequences in historical data, the stability of the current action, the difficulty of the operation, etc. For example, if the current state is bending over and the end state is standing, the time series prediction model can predict that this process will take 5-10 seconds based on past data.

[0081] After determining the current action state and predicted action duration of the dynamic subject, confirm whether the first target sub-region also contains a static subject. This step can be achieved in the following ways: 1. Use a semantic segmentation algorithm to finely segment the sub-region and classify each pixel in the image as a dynamic subject, a static subject, or a background. 2. Use a target detection algorithm (such as YOLO) to identify all objects in the sub-region and classify them as dynamic or static. 3. By comparing multiple consecutive frames of images, identify the moving part (dynamic subject) and the immobile part (static subject or background).

[0082] For example, for a scaffolding work area, the scaffolding structure (static subject) and workers (dynamic subject) may be detected. The presence of the static subject may affect the safety status of the dynamic subject. For example, when workers are close to the edge of the scaffolding or large equipment, the safety risk increases.

[0083] When it is confirmed that there is a static subject in the first target sub-area, the first safety risk coefficient of the dynamic subject can be determined based on the first current action state of the dynamic subject and the characteristics of the static subject. Specifically, it can be achieved in the following ways: 1. Extract the characteristics of the dynamic subject (such as posture, position, speed) and the characteristics of the static subject (such as type, position, size). 2. Based on predefined risk assessment criteria, construct a risk matrix to map different first current action states and static subject combinations to different risk levels. 3. Use a machine learning model (such as a random forest or a neural network) to calculate the risk coefficient. The machine learning model input includes the action state feature vector and the static subject feature vector, and the output is a risk coefficient between 0 and 1.

[0084] The calculation formula is as follows:

[0085] R = f(A, S, E);

[0086] Among them, R is the risk factor, A is the action state feature, S is the static subject feature, and E is the environmental factor (such as weather, light, etc.).

[0087] For example, if a worker is detected bending over the edge of a scaffold with unsecured tools nearby, a higher risk factor (such as 0.8) may be calculated. By calculating and updating the risk factor in real time, the sensitivity of safety monitoring can be dynamically adjusted to provide more timely warnings in high-risk situations.

[0088] When the calculated first safety risk coefficient is greater than the first preset value, the first target sub-areas at the next N moments are acquired one by one.

[0089] Specifically, the steps may be to set a first preset value, such as 0.7, based on historical data. And determine the time window to be monitored based on the previously predicted action duration T. For example, if T is 10 seconds, it may be set to monitor the next 5 seconds (N=5). Within the determined time window, continuously acquire an image sequence of the first target sub-area at a higher frequency (such as 10 frames per second).

[0090] The changes in the key points of the dynamic subject in the first target sub-region may include: 1. Using an optical flow algorithm to track the position changes of the key points of the dynamic subject in consecutive frames. 2. Using algorithms such as the least squares method or Kalman filtering to fit the position sequence of the key points to obtain a smooth motion trajectory. 3. Based on the known trajectory data, using a time series prediction model (such as LSTM) to predict the positions of N future time points.

[0091] For example, for a worker moving on a scaffold, his movement path in the next 5 seconds can be predicted, including the possible locations and speed changes. The above trajectory prediction takes into account not only the historical positions but also the current speed and acceleration, so it can more accurately predict nonlinear motion.

[0092] By accurately predicting the motion trajectory of dynamic entities, it is possible to identify in advance movement patterns that may lead to danger, such as sudden acceleration, abnormal steering, or approaching dangerous areas. This provides foresight for the safety warning system, enabling it to respond before the actual danger occurs, greatly improving the ability to prevent accidents.

[0093] Based on the predicted dynamic subject motion trajectory, the action danger level is determined and displayed. The specific implementation method is as follows:

[0094] 1. Define dangerous areas in the working environment in advance, such as the edge of the aerial work platform, the range of equipment movement, etc.

[0095] 2. Compare the predicted motion trajectory with the predefined danger zone, and calculate indicators such as the minimum distance between the trajectory and the danger zone, the probability of entering the danger zone, etc.

[0096] 3. Evaluate velocity and acceleration changes in the trajectory to identify abnormally fast movements or sudden stops.

[0097] 4. Calculate the composite risk score using weighted summation.

[0098] 5. Map the risk scores to predefined levels, such as low, medium, high, and very high.

[0099] 6. Visual display: Use color coding (such as green, yellow, orange, and red) to display the hazard level in real time on the monitoring interface, and may include visualization of the predicted trajectory.

[0100] This embodiment can achieve all-round and real-time monitoring by acquiring the current aerial work image and performing intelligent analysis, overcoming the limitations of traditional methods that rely on manual inspections or fixed cameras; by locating key points, determining the action state, and predicting the action duration, the behavior of aerial workers can be dynamically analyzed. In addition, by calculating the safety risk coefficient and predicting the action hazard level based on the motion trajectory of the dynamic subject, a comprehensive assessment of the safety of aerial work can be achieved, and the impact of static subjects on dynamic subjects can be taken into account, reflecting a comprehensive consideration of the complex and changeable aerial work environment. In summary, a more comprehensive, accurate, and real-time safety monitoring is provided, which can actively identify potential risks and effectively improve the safety of aerial work and the accuracy of safety monitoring.

[0101] In one implementation of this embodiment, the following steps are also included:

[0102] S210: When the monitored subjects in one of the sub-areas include a dynamic subject and there are at least two dynamic subjects, the sub-area is used as a second target sub-area, and preset data of the second target sub-area at the current moment is obtained, where the preset data includes a standard number of dynamic subjects and standard motion data of each dynamic subject;

[0103] S220, when the number of dynamic subjects is consistent with the standard number and the standard action data of each dynamic subject is consistent, locating the second current key point of each dynamic subject at the current moment, and determining the second current action state of each dynamic subject at the current moment according to each second current key point;

[0104] S230, determining whether the monitored subjects in the second target sub-area also include static subjects;

[0105] S240: when the monitored subject also includes a static subject, determining a second safety risk coefficient of each dynamic subject according to a second current key point position of each dynamic subject and the static subject;

[0106] S250: When any second safety risk coefficient is greater than a second preset value, a first warning message is issued.

[0107] When the monitored subjects in a certain sub-area include at least two dynamic subjects, the sub-area is defined as a second target sub-area, and the preset data of the second target sub-area at the current moment is obtained. The preset data contains two key information: the standard number of dynamic subjects and the standard action data of each dynamic subject. The preset data is usually pre-defined according to the requirements of a specific scene or task.

[0108] For a two-person collaborative task scenario, the preset data may specify the standard number as 2 and describe in detail the standard action sequence that each dynamic subject should perform. The standard action data may include information such as key point locations, action types, and action durations.

[0109] After obtaining the preset data, the actual number of detected dynamic subjects needs to be compared with the standard number. If there is a discrepancy, it means that the current scene violates the preset rules and may only allow one person to operate. In this case, the system will issue a warning to remind relevant personnel to pay attention to safety or regulatory issues.

[0110] Warnings can be issued in a variety of forms, such as sound alarms, screen prompts, or notifications to managers. For example, a striking red warning box can be displayed on the monitoring interface, with the content "Warning: Multiple people are detected, violating safety regulations." At the same time, a sound alarm can be triggered and a notification message can be pushed to the mobile device of the security supervisor.

[0111] In this embodiment, operation behaviors that do not conform to preset rules can be discovered and prevented in a timely manner, effectively improving the safety and standardization of operations. By comparing the actual situation with the preset standards, potential risk factors can be quickly identified and corresponding intervention measures can be taken.

[0112] When the number of dynamic subjects is consistent with the standard number and the standard action data of each dynamic subject is also consistent, it is necessary to further locate the second current key point of each dynamic subject at the current moment, and determine the second current action state of each dynamic subject based on the second current key point.

[0113] Specifically, first, the image is processed using a deep learning model to extract the second current key point of each dynamic subject. Key points usually include the main joints of the human body such as the head, neck, shoulders, elbows, wrists, hips, knees and ankles. Each key point has its three-dimensional coordinates (x, y, z), where x and y represent the position on the image plane and z represents the depth information.

[0114] After obtaining the key points, when the number of dynamic subjects is consistent with the standard number and the standard action data of each dynamic subject is consistent, it is necessary to determine the second current action state of each dynamic subject according to the second current key point. This can be achieved by comparing the second current key point with the predefined action template. The similarity between the second current key point and the action template can be first calculated, and based on the calculated similarity, it is determined whether the second current action state meets the expected standard action. If the similarity is higher than the preset similarity threshold, the second current action state is considered to be consistent with the standard action.

[0115] After determining the number of dynamic subjects in the second target sub-area, the next step is to determine whether the monitored subjects in the sub-area also include static subjects. This is to comprehensively evaluate the composition of the entire scene, because the presence of static subjects may have a significant impact on the safety of dynamic subjects. Static subjects can be fixed equipment, obstacles, dangerous areas, etc.

[0116] The process of determining the presence or absence of a static subject involves image segmentation and object detection techniques. In specific implementations, deep learning models, such as the MaskR-CNN model, can be used to detect and segment various objects in an image. Not only can dynamic subjects (height workers) be identified, but also various static objects can be identified.

[0117] You can first configure and load the pre-trained MaskR-CNN model. Then, make predictions on the input image to obtain the categories, bounding boxes, and mask information of all detected objects. Finally, by checking whether the detected object categories contain predefined static object categories (such as machinery and equipment, walls, etc.), you can determine whether there are static subjects in the scene. For example, if a large mechanical equipment (static subject) is detected in the scene, you can more accurately evaluate whether the behavior of the dynamic subject (height operator) is safe, whether it is likely to collide with the equipment, etc.

[0118] When it is confirmed that the monitored subject also includes a static subject, it is necessary to determine the second safety risk coefficient of each dynamic subject based on the positional relationship between the second current key point of each dynamic subject and the static subject. This step involves spatial relationship analysis and risk assessment algorithms.

[0119] In one embodiment, first, the spatial information of the static subject is obtained, including the position, size and shape. At the same time, the characteristics of the static subject need to be considered, such as whether it is dangerous (e.g., high-temperature equipment, sharp edges, etc.). Then, for each dynamic subject, the shortest distance between its second current key point and the static subject is calculated. This can be achieved by the following steps:

[0120] 1. Simplify static bodies into geometric shapes such as rectangles, circles, or polygons.

[0121] 2. For each key point of the dynamic body, calculate its shortest distance to the simplified geometry of the static body.

[0122] 3. Take the minimum distance among all the second current key points as the shortest distance between the dynamic body and the static body.

[0123] Next, the second safety risk coefficient is determined based on the calculated shortest distance and other related factors, which may be the speed of the dynamic subject, the type of action, the degree of danger of the static subject, etc. The second safety risk coefficient may be calculated by weighted summation.

[0124] After calculating the second safety risk coefficient of each dynamic subject, the second safety risk coefficient needs to be compared with the second preset value. If the second safety risk coefficient of any dynamic subject exceeds the second preset value, a first warning message is issued.

[0125] In this implementation, when the second safety risk coefficient of any dynamic subject exceeds the second preset value, a first alarm message is issued, which effectively improves the timeliness and pertinence of the early warning, and takes into account the influence of the static subject when calculating the second safety risk coefficient. The working environment can be comprehensively considered, and the adaptability to complex high-altitude work scenes is effectively enhanced. The accuracy and efficiency of safety monitoring are improved, and a more comprehensive and accurate safety guarantee is provided for collaborative work of multiple people.

[0126] In one implementation of this embodiment, the following steps are also included:

[0127] S310, when the number of dynamic subjects is consistent with the standard number and the standard action data of each dynamic subject is inconsistent, each dynamic subject is analyzed in parallel to obtain the action data of each dynamic subject in a preset time period, and the end time of the preset time period is the current time;

[0128] S320, calculating the action deviation between the action data and the corresponding standard action data;

[0129] S330, creating an interaction matrix, and calculating the relative positions and interactions between dynamic subjects in real time, wherein the interaction matrix is ​​used to store the relative positions and interactions in real time;

[0130] S340, using the individual safety risk index formula, according to the action deviation and the interaction matrix, calculating the third safety risk coefficient of each dynamic subject;

[0131] S350: When any third safety risk coefficient is greater than a third preset value, a second warning message is issued.

[0132] When the number of dynamic subjects is consistent with the standard number but the standard action data of each dynamic subject is inconsistent, each dynamic subject needs to be analyzed in parallel to obtain the action data of each dynamic subject within a preset time period.

[0133] Specifically, the end time of the preset time period is the current time. In the specific implementation, it is necessary to first determine the length of the preset time period, for example, it can be set to 30 seconds, 1 minute or 5 minutes, etc. The specific length can be adjusted according to the actual application scenario. Then, the motion trajectory, speed, acceleration and other action information of each dynamic subject in the preset time period are collected in real time through sensors, cameras and other equipment. In the case where the dynamic subject is a high-altitude worker, the detailed actions such as walking posture and arm swing can be collected as action data; in the case where the dynamic subject is a vehicle, its steering, acceleration, braking and other operations can be collected as action data.

[0134] After the collected motion data is preprocessed by filtering and denoising, the motion deviation between the motion data and the corresponding standard motion data is calculated to evaluate the abnormal degree of dynamic subject behavior.

[0135] Specifically, for each dynamic subject, there is a set of corresponding standard motion data. When calculating the deviation, the concept of vector distance is used. Both the motion data and the standard motion data are represented as multi-dimensional vectors, and each dimension represents an action parameter (such as position, velocity, acceleration, etc.). Then the Euclidean distance between the two vectors is calculated to obtain a scalar value as the deviation. The specific formula is as follows:

[0136] ;

[0137] Wherein x1, x2, ..., xn represent the parameters of the actual motion data, x1', x2', ..., xn' represent the corresponding standard motion data parameters, and P is the degree of deviation.

[0138] The calculated deviation can be further normalized to the range of 0-1 for subsequent processing. The normalization formula is:

[0139] Normalized deviation = (original deviation - minimum deviation) / (maximum deviation - minimum deviation).

[0140] After that, the interaction matrix is ​​created and the relative positions and interactions between dynamic agents are calculated in real time. The interaction matrix is ​​an N×N square matrix, where N is the number of dynamic agents. Each element aij in the matrix represents the interaction information between the i-th agent and the j-th agent. In the specific implementation, first initialize an empty N×N matrix. Then, for each pair of agents (i, j), calculate the relative position vector between them:

[0141] Relative position vector = position of subject j - position of subject i;

[0142] This vector contains distance and direction information. The distance can be obtained by calculating the magnitude of the vector:

[0143] ;

[0144] Direction can be expressed as a unit vector:

[0145] Direction = (relative position vector) / distance;

[0146] Next, calculate the interaction force. A simplified gravitational model can be used: interaction force = ;

[0147] Where G is a constant, mi and mj are the importance or influence of the two entities respectively.

[0148] The calculated relative position and interaction information is stored in the matrix elements aij. Since the interaction is symmetric, only the upper triangular matrix needs to be calculated, and the lower triangular matrix can be directly taken as the transpose of the upper triangular matrix. The diagonal elements aii are set to 0, indicating that the subject has no interaction with itself.

[0149] The interaction matrix needs to be updated in real time, and the update frequency can be set according to actual needs, such as once per second. Each time it is updated, the relative positions and interactions between all pairs of subjects are recalculated and the corresponding elements in the matrix are updated.

[0150] The third safety risk coefficient of each dynamic subject is calculated using the individual safety risk index formula based on the action deviation and interaction matrix. The individual safety risk index formula comprehensively considers the abnormal degree of the dynamic subject's own behavior and the interaction with other subjects. The formula is as follows:

[0151] The third safety risk factor = w1 × normalized deviation + w2 × interaction risk;

[0152] Where w1 and w2 are weight coefficients, satisfying w1+w2=1. The normalized deviation degree comes from the calculation result of step S320, reflecting the degree of deviation between the subject behavior and the standard behavior. The interaction risk degree needs to be calculated based on the interaction matrix created in step S330.

[0153] The calculation method of interaction risk is as follows:

[0154] 1. For subject i, traverse all elements in the i-th row of the interaction matrix (except aii).

[0155] 2. For each non-zero element aij, calculate a local risk value:

[0156] Local risk = distance factor × force factor × direction factor; where distance factor = 1 / (distance + 1), indicating that the closer the distance, the greater the risk; force factor = min(force / threshold, 1), indicating that the greater the force, the greater the risk, but there is an upper limit; direction factor = cosθ, where θ is the angle between relative velocity and relative position, indicating that the risk of head-on encounter is greater.

[0157] 3. Sum all local risk values ​​to get the total interaction risk.

[0158] 4. Normalize the total interaction risk to the range of 0-1.

[0159] Finally, the normalized deviation and interaction risk are substituted into the individual safety risk index formula to obtain the third safety risk coefficient. The third safety risk coefficient comprehensively reflects the abnormal degree of the subject's own behavior and the interaction risk with the surrounding environment, providing a quantitative basis for subsequent alarm judgment.

[0160] In the case where any third safety risk factor is greater than the third preset value, a second alarm message is issued. When any third safety risk factor is detected to exceed the third preset value, the alarm mechanism is immediately triggered. The content of the alarm message may include the alarm time: a timestamp accurate to the second; the alarm object: which dynamic subject or subjects triggered the alarm; the risk level: the risk factor can be divided into multiple levels, such as low, medium, high, and extremely high; the risk type: whether it is caused by abnormal behavior of the user or by interactive risk; the specific situation description: a brief description of the specific reasons for the increase in risk; the recommended measures: operational suggestions for reducing the risk are given.

[0161] The methods of sending alarm information may include: 1. real-time pop-up window on the control center display screen; 2. sending text messages or instant messaging messages to relevant personnel; 3. triggering sound and light alarm devices; 4. automatic recording in the system log.

[0162] This implementation method first obtains the action data of each subject through parallel analysis, and then quantifies the degree of abnormality of the subject's behavior by calculating the action deviation. The creation of an interaction matrix captures the complex relationship between subjects. On this basis, the individual security risk index formula is adopted to comprehensively consider the interaction between its own behavior and the environment, and a more comprehensive risk assessment is obtained. Finally, by setting reasonable thresholds and timely alarm mechanisms, early warning of potential risks is achieved. It can not only identify the abnormal behavior of a single subject, but also discover the potential risks caused by the interaction of multiple subjects, effectively improving the accuracy and timeliness of security monitoring.

[0163] In one implementation of this embodiment, locating a first current key point of a dynamic subject at a current moment, and determining a first current action state of the dynamic subject according to the first current key point, includes the following steps:

[0164] S410, using a preset skeleton key point detection algorithm to locate the key points of the dynamic subject at the current moment, and obtain the first current key point position of the dynamic subject at the current moment;

[0165] S420, obtaining the three-dimensional coordinates of each first current key point;

[0166] S430, calculating the relative distance and angle between the first current key points according to the three-dimensional coordinates of each first current key point;

[0167] S440, inputting all relative distances and angles into a pre-trained classifier to obtain a probability distribution, where the probability distribution is used to characterize the possibility that the current action state of the dynamic subject belongs to a predefined action state;

[0168] S450: The current action state with the highest probability is taken as the first current action state.

[0169] First, a preset skeleton key point detection algorithm is used to locate the key points of the dynamic subject at the current moment. The skeleton key point detection algorithm is usually implemented based on a deep learning model. The skeleton key point detection algorithm can accurately locate the key points of the human body in an image or video frame, such as the head, neck, shoulders, elbows, wrists, hips, knees, and ankles.

[0170] In the specific implementation, it is first necessary to obtain the image or video frame at the current moment, which can be captured in real time by a camera or extracted from a pre-recorded video. Then, the image is input into a pre-trained neural network model. This model usually contains convolutional layers, pooling layers, and fully connected layers, which can automatically extract image features and output the position estimate of each key point. In order to improve accuracy, multi-stage or multi-branch network structures are often used, such as a combination of top-down and bottom-up methods. The output of the model is a heat map, which indicates the probability that each pixel belongs to a certain key point. By post-processing the heat map, such as non-maximum suppression, the precise coordinates of each key point can be obtained.

[0171] For multi-person scenes, a clustering algorithm is also needed to group the key points belonging to the same person. Finally, for each detected dynamic subject, a set of key point coordinates representing its skeletal structure can be obtained, that is, the first current key point position.

[0172] The three-dimensional coordinates of each first current key point are obtained. Specifically, the method for obtaining the three-dimensional coordinates can use a depth camera, which can directly output a depth image. The depth information is aligned with the RGB image to obtain the three-dimensional coordinates of each pixel.

[0173] After obtaining the depth information, the image coordinate system needs to be converted to the world coordinate system. That is, the camera calibration process requires knowing the camera's intrinsic parameters (such as focal length, principal point coordinates) and extrinsic parameters (the position and orientation of the camera in the world coordinate system). The conversion formula is:

[0174] ;

[0175] Where (X, Y, Z) are world coordinates, (x, y) are image coordinates, Z is the depth, and M is the camera projection matrix.

[0176] According to the three-dimensional coordinates of each first current key point, calculate the relative distance and angle between the first current key points. First, calculate the relative distance. For any two key points A (x1, y1, z1) and B (x2, y2, z2), you can first calculate the Euclidean distance between them. In practical applications, the distances between some specific key point pairs are usually calculated, such as the left wrist to the left shoulder, the right ankle to the right hip, etc. The above distances are normalized to eliminate the influence of different human body sizes. For example, they can be divided by the torso length (the distance from the neck to the hip). Secondly, calculate the angle. Angle calculation usually involves three key points. For example, the elbow angle requires the use of three points: shoulder, elbow and wrist. The calculation steps are as follows:

[0177] 1. Construct two vectors: v1 = shoulder to elbow, v2 = wrist to elbow; 2. Calculate the vector dot product: dot = v1 × v2 = v 1x ×v 2x +v 1y ×v 2y +v 1z ×v 2z ; 3. Calculate the vector modulus: , the same for |v2|; 4. Apply the arccosine function: angle = arccos(dot / (|v1|×|v2|)).

[0178] In this embodiment, the angles that need to be calculated include the neck angle, shoulder angle, elbow angle, hip angle, knee angle, etc.

[0179] By calculating the relative distance and angle, a set of feature vectors describing the current posture of the dynamic subject can be obtained. The feature vector usually contains dozens of elements, each of which represents a distance or angle value. It not only reduces the data dimension, but also has rotation and translation invariance, that is, it is not affected by the position and orientation of the dynamic subject in space, thereby improving the accuracy and robustness of subsequent classification.

[0180] All relative distances and angles are input into a pre-trained classifier to obtain a probability distribution, which is used to characterize the possibility that the current action state of the dynamic subject belongs to a predefined action state. The pre-trained classifier can be a deep neural network.

[0181] First, you need to prepare training data. This includes collecting a large amount of labeled action data, where each sample contains the distance and angle features calculated in the previous step, as well as the corresponding action label. The predefined action states may include standing, walking, running, jumping, squatting, falling, etc.

[0182] Then, taking a deep neural network as an example, the following structure can be designed: 1. Input layer: the number of nodes is equal to the dimension of the feature vector; 2. Hidden layer: multiple fully connected layers can be used, and each layer uses the ReLU activation function; 3. Output layer: the number of nodes is equal to the number of predefined action states, and the softmax activation function is used.

[0183] The model training process includes forward propagation, loss calculation, back propagation, and parameter update. After training, the model performance is evaluated using the test set. Common evaluation indicators include accuracy, precision, recall, and F1 score.

[0184] In practical applications, the currently calculated distance and angle features are input into the trained model. The output of the model is a probability distribution, which indicates the probability that the current action belongs to each predefined state. For example, for five predefined states, the output may be:

[0185] [Stand: 0.05, Walk: 0.15, Run: 0.75, Jump: 0.03, Crouch: 0.02];

[0186] The probability distribution not only gives the most likely action state, but also provides the confidence information of the model in the judgment. For example, in the process of going from walking to running, there may be a situation where the probabilities of the two states are close.

[0187] In this way, complex three-dimensional posture information can be converted into action state probabilities, so that the current action state with the highest probability is used as the first current action state. For example, for the obtained probability distribution [standing: 0.05, walking: 0.15, running: 0.75, jumping: 0.03, squatting: 0.02], running is selected as the first current action state.

[0188] This implementation starts with the detection of skeletal key points, obtains three-dimensional coordinate information, calculates the relative distance and angle between key points, and then uses a pre-trained classifier to obtain the probability distribution of the action state. Finally, the final action state is determined through an optimization strategy. It can recognize various complex human movements in real time and accurately.

[0189] In one implementation of this embodiment, based on the first current action state and the end action state, predicting the action duration from the first current action state to the end action state includes the following steps:

[0190] S510, calculating a first vector of a first current action state and a second vector of an end action state, and obtaining subject features of a dynamic subject;

[0191] S520, inputting the first vector and the second vector into a pre-trained motion state space model to obtain motion change complexity, wherein the motion state space model is used to calculate the Euclidean distance between the first vector and the second vector, and the Euclidean distance is used to characterize the motion change complexity;

[0192] S530: Input the subject feature, the first vector, the second vector, and the complexity of the action change into a pre-trained deep learning model to obtain the action duration.

[0193] First, the first vector of the first current action state and the second vector of the end action state are calculated. Specifically, the first vector and the second vector can be constructed using joint angles and position coordinates. For example, for human body movements, the three-dimensional coordinates of 17 key points (such as head, shoulders, elbows, wrists, hips, knees, and ankles) can be used to construct a 51-dimensional vector (17×3) to fully capture the spatial configuration of the movement.

[0194] Next, the subject features of the dynamic subject are obtained. Subject features refer to individual attributes that can affect the execution of actions, such as height, weight, age, gender, or specific physiological indicators. These features have an important impact on the duration of actions, because different individuals may have significant time differences when performing the same action. For example, a tall adult and a short child may take very different times to complete the same jumping action.

[0195] Methods for obtaining subject characteristics may include direct measurement (e.g., height, weight), questionnaires (e.g., age, gender), or the use of specialized sensor devices (e.g., electromyography, heart rate monitors, etc.).

[0196] The first vector (representing the current action state) and the second vector (representing the end action state) are input into the pre-trained action state space model to calculate the Euclidean distance between the two vectors, thereby obtaining the complexity of the action change.

[0197] The motion state space model is a mathematical model designed specifically for processing and analyzing motion data. It can represent motion states as points in a multidimensional space, and the transitions between different motion states can be viewed as paths or trajectories in the multidimensional space. By calculating the Euclidean distance between two motion states, the magnitude and complexity of motion changes can be quantified.

[0198] The main features, the first vector (current action state), the second vector (end action state) and the complexity of action change obtained above are used as input into a pre-trained deep learning model to finally obtain the predicted action duration.

[0199] In the application scenario of this embodiment, the task of the deep learning model is to learn how to accurately predict the duration of an action based on given input features. A possible structure of a deep learning model is as follows:

[0200] 1. Input layer: receives all input features, including subject features (such as height, weight, etc.), current action state vector, end action state vector, and action change complexity.

[0201] 2. Hidden layer: It includes 3-5 hidden layers. The number of neurons in each layer can be reduced layer by layer, such as 512-256-128-64.

[0202] 3. Output layer: One neuron that outputs the predicted action duration.

[0203] This implementation achieves accurate prediction of the duration of an action from the current action state to the target action state by combining action state vectorization, action complexity calculation, and deep learning prediction. First, the action state is converted into a high-dimensional vector and individual characteristics are considered. Then, the Euclidean distance is calculated using a pre-trained action state space model to objectively quantify the complexity of the action change. Finally, all features are input into the deep learning model to predict the action duration. This effectively improves the accuracy of the prediction results.

[0204] In one implementation of this embodiment, determining a first security risk coefficient of a dynamic subject according to a first current action state and a static subject includes the following steps:

[0205] S610, determining the posture stability of the first current action state using a biomechanical model;

[0206] S620, obtaining the current height of the dynamic subject, and determining the risk probability of the static subject to the dynamic subject according to the current height and the static subject;

[0207] S630: Determine a first safety risk coefficient of the dynamic subject according to posture stability and risk probability.

[0208] A biomechanical model is a mathematical model used to analyze the movement and posture of the human body or other organisms. It is based on the principles of physics and is used to describe and predict the movement and force of various parts of the body.

[0209] When implementing it, you first need to build a detailed human body model, which usually includes multiple rigid body segments (such as torso, limbs, etc.) and joints connecting these rigid body segments. Each rigid body segment has properties such as its mass, center of mass position, and inertia tensor. The joints define the degrees of freedom of movement between rigid body segments. For example, a simplified human body model may contain 14 rigid body segments (head, torso, upper arm, forearm, hand, thigh, calf and foot, one on each side) and 13 main joints (neck, shoulder, elbow, wrist, hip, knee and ankle).

[0210] Next, it is necessary to obtain the angle and angular velocity data of each joint in the current action state. This data can be obtained through a motion capture system or other sensors. For example, when using an optical motion capture system, reflective markers can be attached to key points on the body, and the 3D positions of these markers can be captured by multiple high-speed cameras, and then the joint angles can be calculated.

[0211] Once you have the human body model and motion data, you can calculate the postural stability index. The postural stability index is the relationship between the center of gravity projection point (CoP) and the support polygon. The support polygon is the area enclosed by all the points that touch the ground (usually the sole of the foot). The calculation steps are as follows:

[0212] 1. Calculate the center of gravity of the entire body based on the mass and center of mass position of each rigid body segment. Assume there are n rigid body segments, and the mass of each rigid body segment is m i , the center of mass position is r i , then the calculation formula of the center of gravity Rc is:

[0213] ;

[0214] 2. Project the center of gravity onto the ground to get the CoP;

[0215] 3. Determine the boundary of the support polygon based on the contact points between the sole of the foot and the ground;

[0216] 4. Use the shortest distance from the CoP to the edge of the supporting polygon as a stability indicator. The larger the distance, the more stable the pose.

[0217] Get the current height of the dynamic subject, and determine the risk probability of the static subject to the dynamic subject based on the current height and the static subject, so as to assess the potential danger that static objects in the environment may cause to the dynamic subject.

[0218] First, the current height of the dynamic subject can be obtained through a preset database, which stores relevant information of each dynamic subject, including height, weight, etc.

[0219] Next, it is necessary to evaluate the risk probability of the static subject to the dynamic subject based on the current height and the static subject, and determine the first safety risk coefficient of the dynamic subject based on the posture stability and the risk probability, where posture stability is a value between 0 and 1, where 0 means extremely unstable and 1 means very stable, which reflects the ability of the dynamic subject to maintain balance. Risk probability is a value between 0 and 1, where 0 means almost no risk and 1 means extremely high risk, which reflects the potential threat of environmental factors to the dynamic subject.

[0220] For the first safety risk factor, the following conditions should be met:

[0221] 1. When posture stability is high and risk probability is low, the safety risk factor should be low.

[0222] 2. When posture stability is low or risk probability is high, the safety risk factor should be higher.

[0223] 3. The effects of postural stability and risk probability on safety risk may not be linear.

[0224] For example: If the first safety risk factor > 0.8, it may be necessary to take protective measures or issue a warning immediately. If 0.5 < first safety risk factor ≤ 0.8, it may be necessary to increase the monitoring frequency or provide auxiliary support. If the first safety risk factor ≤ 0.5, it can be considered that the current state is relatively safe, but vigilance is still required.

[0225] First, this implementation adopts a biomechanical model to accurately calculate posture stability, taking into account the human body structure and physical laws. Secondly, by analyzing the current height and surrounding environment of the dynamic subject, the risk probability of the static subject to the dynamic subject is calculated, and the external environmental factors are fully considered. Finally, the first safety risk coefficient is calculated based on posture stability and risk probability. Not only internal and external factors are taken into account, but also the accuracy of safety monitoring is effectively improved.

[0226] In one implementation of this embodiment, determining the risk probability of the static subject to the dynamic subject according to the current height and the static subject includes the following steps:

[0227] S710, mapping the dynamic subject and the static subject into a preset three-dimensional coordinate system, and calculating a first distance and a height difference between the dynamic subject and each static subject;

[0228] S720: Using a preset risk function, calculate the risk probability of the static subject to the dynamic subject according to the first distance and the height difference.

[0229] First, establish a preset three-dimensional coordinate system, usually the Cartesian coordinate system, where the x-axis and y-axis represent planar positions, and the z-axis represents height. Map the position information of the dynamic object and the static object into this coordinate system, and each object has a unique (x, y, z) coordinate. For the dynamic object, its coordinates may change over time, while the coordinates of the static object remain unchanged.

[0230] After the mapping is completed, it is necessary to calculate the first distance and height difference between the dynamic object and each static object. The first distance can be calculated using the Euclidean distance formula. Suppose the coordinates of the dynamic object are (x1, y1, z1), and the coordinates of a certain static object are (x2, y2, z2), then the planar distance can calculate the first distance through the Euclidean distance calculation formula, and calculate the height difference through the difference in z coordinates.

[0231] According to the calculated first distance and height difference, use the preset risk function to calculate the risk probability of the static object to the dynamic object.

[0232] The risk function R can be: ;

[0233] where d represents the first distance, h represents the height difference, d0 and h0 are the characteristic scales of distance and height, and w1 and w2 are the weight coefficients. Through this risk function, the risk decreases as the distance and height difference increase.

[0234] In specific implementation, thresholds can be introduced to handle extreme cases: if d < dmin or |h| < hmin, then R = 1; if d > dmax and |h| > hmax, then Risk = 0. Among them, dmin, hmin, dmax, and hmax are thresholds determined according to the specific application scenario.

[0235] This embodiment maps the dynamic object and the static object into a three-dimensional coordinate system, calculates the distance and height difference between them, and then uses the preset risk function to evaluate the risk probability, realizing the precise quantification of the risk between the dynamic and static objects in three-dimensional space.

[0236] In one implementation manner of this embodiment, determining the first safety risk coefficient of the dynamic object according to the posture stability and the risk probability includes the following steps:

[0237] S810. Perform a weighted sum of the posture stability and the risk probability to obtain the first safety risk coefficient of the dynamic object.

[0238] The first safety risk factor of the dynamic subject is obtained by weighted summing the posture stability and risk probability. Specifically, posture stability is a value between 0 and 1, where 0 indicates extreme instability and 1 indicates complete stability. Similarly, risk probability is a value between 0 and 1, indicating the possibility of a risk event occurring when the dynamic subject is affected by the static subject.

[0239] Next, it is necessary to assign weights to posture stability and risk probability. In this embodiment, the weight assigned to posture stability is w1, the weight assigned to risk probability is w2, and w1+w2=1.

[0240] The calculation formula of the weighted sum is as follows: First safety risk factor = w1×(1-posture stability)+w2×risk probability.

[0241] The reason for using (1-posture stability) is that the more stable the posture, the lower the risk.

[0242] In order to make the final safety risk factor more intuitive and easy to understand, the calculation results can be mapped to a predefined risk level table. For example: 0-0.2: low risk; 0.2-0.4: medium-low risk; 0.4-0.6: medium risk; 0.6-0.8: medium-high risk; 0.8-1.0: high risk. The risk level can be directly obtained.

[0243] This implementation method combines posture stability and risk probability into a unified first safety risk coefficient through a weighted summation method, thereby achieving a comprehensive quantitative assessment of the safety status of the dynamic subject.

[0244] The present application also provides an electronic device, including:

[0245] a memory configured to store instructions; and

[0246] The processor is configured to call instructions from the memory and implement the above-mentioned intelligent safety warning method applied to high-altitude operations when executing the instructions.

[0247] Reference Figure 3 The embodiment of the present application also provides an intelligent safety warning system for high-altitude operations, including:

[0248] Camera 100;

[0249] The electronic device 200 is connected to the camera 100 .

[0250] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned intelligent safety warning method applied to high-altitude operations.

[0251] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0252] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0253] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0254] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0255] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0256] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0257] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0258] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0259] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An intelligent safety warning method for high-altitude operations, characterized in that: include: Acquire an aerial work image at the current moment, and perform image recognition on the aerial work image to divide the aerial work image into a plurality of sub-areas according to the aerial work project, each of the sub-areas includes at least one monitoring subject, and the monitoring subject includes a static subject and / or a dynamic subject; In the case where the monitored subject in one of the sub-regions includes the dynamic subject and the number of the dynamic subject is one, taking the sub-region as a first target sub-region, and performing image recognition on the first target sub-region to obtain the dynamic subject in the first target sub-region; Locating a first current key point of the dynamic subject at a current moment, and determining a first current action state of the dynamic subject according to the first current key point; Determine an end action state according to the first target sub-region at the current moment, and predict an action duration from the first current action state to the end action state based on the first current action state and the end action state; determining whether the monitored subject in the first target sub-area also includes the static subject; In a case where the monitoring subject also includes the static subject, determining a first safety risk coefficient of the dynamic subject according to the first current action state and the static subject; When the first safety risk coefficient is greater than a first preset value, acquiring the first target sub-areas at the next N moments one by one, and a set of the current moment and the next N moments is less than or equal to the action duration; Predicting a motion trajectory of the dynamic subject according to changes in key points of the dynamic subject in the first target sub-region at the current moment and in the first target sub-region at the next N moments; According to the motion trajectory of the dynamic subject, determining and displaying the action danger level of the dynamic subject; The determining a first safety risk coefficient of the dynamic subject according to the first current action state and the static subject includes: Determining the postural stability of the first current motion state using a biomechanical model; Acquire the current height of the dynamic subject, and determine the risk probability of the static subject to the dynamic subject according to the current height and the static subject; Determining a first safety risk coefficient of the dynamic subject according to the posture stability and the risk probability; The determining, according to the current height and the static subject, a risk probability of the static subject to the dynamic subject comprises: Mapping the dynamic subject and the static subject into a preset three-dimensional coordinate system, and calculating a first distance and a height difference between the dynamic subject and each of the static subjects; Using a preset risk function, and according to the first distance and the height difference, calculating the risk probability of the static subject to the dynamic subject; Wherein, the risk function includes: ； Where R represents the risk probability, d represents the first distance, h represents the height difference, d0 and h0 are the characteristic scales of distance and height, and w1 and w2 are weight coefficients; The determining a first safety risk coefficient of the dynamic subject according to the posture stability and the risk probability comprises: The first safety risk coefficient of the dynamic body is obtained by performing a weighted summation on the posture stability and the risk probability.

2. The method according to claim 1, characterized in that The method further comprises: In the case where the monitored subject in one of the sub-areas includes the dynamic subject and there are at least two dynamic subjects, the sub-area is used as a second target sub-area, and preset data of the second target sub-area at the current moment is obtained, wherein the preset data includes a standard number of the dynamic subjects and standard action data of each of the dynamic subjects; When the number of the dynamic subjects is consistent with the standard number and the standard action data of each of the dynamic subjects is consistent, locating the second current key point of each of the dynamic subjects at the current moment, and determining the second current action state of each of the dynamic subjects at the current moment according to each of the second current key points; determining whether the monitored subject in the second target sub-area also includes the static subject; In the case where the monitoring subject also includes the static subject, determining a second safety risk coefficient of each of the dynamic subjects according to the second current key point position of each of the dynamic subjects and the static subject; In the event that any of the second safety risk coefficients is greater than a second preset value, a first warning message is issued.

3. The method according to claim 2, characterized in that The method further comprises: When the number of the dynamic subjects is consistent with the standard number and the standard action data of each of the dynamic subjects is inconsistent, each of the dynamic subjects is analyzed in parallel to obtain the action data of each of the dynamic subjects in a preset time period, and the end time of the preset time period is the current time; Calculating the action deviation between the action data and the corresponding standard action data; Creating an interaction matrix and calculating the relative positions and interactions between the dynamic entities in real time, wherein the interaction matrix is ​​used to store the relative positions and interactions in real time; Using an individual safety risk index formula, according to the action deviation and the interaction matrix, a third safety risk coefficient of each of the dynamic subjects is calculated; In the event that any one of the third safety risk coefficients is greater than a third preset value, a second warning message is issued.

4. The method according to claim 1, characterized in that: The step of locating a first current key point of the dynamic subject at the current moment, and determining a first current action state of the dynamic subject according to the first current key point, includes: Using a preset skeleton key point detection algorithm, the key point of the dynamic subject at the current moment is located to obtain the first current key point position of the dynamic subject at the current moment; Obtaining the three-dimensional coordinates of each of the first current key points; Calculating the relative distance and angle between the first current key points according to the three-dimensional coordinates of each of the first current key points; Inputting all the relative distances and the angles into a pre-trained classifier to obtain a probability distribution, wherein the probability distribution is used to characterize the possibility that the current action state of the dynamic subject belongs to a predefined action state; The current action state with the highest probability is used as the first current action state.

5. The method according to claim 1, characterized in that The step of predicting the action duration from the first current action state to the end action state based on the first current action state and the end action state includes: Calculating a first vector of the first current action state and a second vector of the end action state, and acquiring subject features of the dynamic subject; Inputting the first vector and the second vector into a pre-trained action state space model to obtain action change complexity, wherein the action state space model is used to calculate the Euclidean distance between the first vector and the second vector, and the Euclidean distance is used to characterize the action change complexity; The subject feature, the first vector, the second vector, and the action change complexity are input into a pre-trained deep learning model to obtain the action duration.

6. An electronic device, applied to the intelligent safety warning method for high-altitude operations as claimed in any one of claims 1 to 5, characterized in that: include: a memory configured to store instructions; as well as The processor is configured to call the instruction from the memory and implement the method for intelligent safety warning applied to aerial work according to any one of claims 1 to 5 when executing the instruction.

7. An intelligent safety warning system for high-altitude operations, characterized in that: include: Camera; The electronic device according to claim 6, wherein the electronic device is connected to the camera.

Images