AI-based construction site personnel behavior analysis method and system

Abstract

The present invention provides an AI-based construction site personnel behavior analysis method and system, which relates to the field of artificial intelligence technology. An artificial intelligence network model is used to perform confidence assessment on sample construction site video clips, and matching samples that match predefined abnormal behavior paths are screened out based on a first confidence level, thereby ensuring the pertinence of the analysis process. By introducing a first weight parameter and a second confidence level, the matching degree of the sample construction site video clips is further refined, so that the third confidence level more accurately reflects the matching of the sample construction site video clips with the predefined abnormal behavior paths. Finally, through the calculation of network performance indicators and the training of the construction site abnormal behavior analysis network, a high-precision target analysis network is generated, which can efficiently and accurately analyze abnormal behaviors in construction site video clips, thereby facilitating safety management and behavior monitoring of construction sites.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI-based construction site personnel behavior analysis method and system. Background Art

[0002] As a highly dynamic and complex working environment, effective management and monitoring of personnel behavior is key to ensuring site safety, improving work efficiency, and preventing potential risks. Traditionally, construction sites rely on manual surveillance videos to observe and analyze personnel behavior, but this approach has many limitations. Manual monitoring is not only time-consuming and laborious, but also limited by personal experience and attention, and it is often difficult to detect and accurately identify abnormal behavior in a timely manner.

[0003] With the advancement of science and technology, especially the popularization of video surveillance technology and the improvement of resolution, construction sites have generated a huge amount of surveillance data. However, how to efficiently use this data to analyze human behavior has become an urgent problem to be solved. On the one hand, due to the complexity of the construction site environment and the diversity of human behavior, the definition and identification of abnormal behavior has become extremely complex. On the other hand, although there is a large amount of surveillance video data, most of it is not labeled, that is, it is unsupervised data, which makes it difficult to directly apply traditional behavior analysis methods based on supervised learning. Summary of the invention

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides an AI-based construction site personnel behavior analysis method, the method comprising:

[0005] Obtaining a sample construction site video clip sequence of a construction site abnormal behavior analysis network, wherein the sample construction site video clip sequence includes a plurality of supervised sample construction site video clips and a plurality of unsupervised sample construction site video clips, wherein the supervised sample construction site video clips refer to surveillance video clips carrying predefined abnormal behavior paths, and the unsupervised sample construction site video clips refer to surveillance video clips not carrying predefined abnormal behavior paths;

[0006] Outputting first confidences corresponding to respective sample construction site video clips in the sample construction site video clip sequence by using an artificial intelligence network model, and extracting surveillance video clips whose first confidences meet first set requirements from the sample construction site video clip sequence as matching sample construction site video clips matching predefined abnormal behavior paths, wherein the first confidences represent confidences that the sample construction site video clips match the predefined abnormal behavior paths;

[0007] Generate a first weight parameter based on the proportion of the amount of the supervised sample construction site video clip in the amount of the matched sample construction site video clip, and output a second confidence corresponding to each of the sample construction site video clips through the construction site abnormal behavior analysis network, wherein the second confidence represents the confidence that the sample construction site video clip is the supervised sample construction site video clip;

[0008] Based on the first weight parameter and the second confidence corresponding to each of the sample construction site video clips, outputting a third confidence corresponding to each of the sample construction site video clips, wherein the third confidence represents the confidence that the sample construction site video clip matches the predefined abnormal behavior path;

[0009] Based on the third confidence level corresponding to each of the sample construction site video clips, a network performance indicator of the construction site abnormal behavior analysis network is calculated and output, and the construction site abnormal behavior analysis network is trained based on the network performance indicator to generate a target construction site abnormal behavior analysis network, so as to analyze whether the construction site video clip matches the predefined abnormal behavior path based on the target construction site abnormal behavior analysis network, and the network performance indicator characterizes the abnormal behavior analysis accuracy of the construction site abnormal behavior analysis network.

[0010] On the other hand, an embodiment of the present invention also provides an AI-based construction site personnel behavior analysis system, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0011] Based on the above aspects, the embodiments of the present application effectively improve the comprehensiveness and accuracy of abnormal behavior analysis by combining supervised samples and unsupervised sample construction site video clips. The confidence of the sample construction site video clips is evaluated using an artificial intelligence network model, and matching samples that match the predefined abnormal behavior path are screened out based on the first confidence level, ensuring the pertinence of the analysis process. By introducing the first weight parameter and the second confidence level, the matching degree of the sample construction site video clips is further refined, so that the third confidence level more accurately reflects the matching of the sample construction site video clips with the predefined abnormal behavior path. Finally, through the calculation of network performance indicators and the training of the construction site abnormal behavior analysis network, a high-precision target analysis network is generated, which can efficiently and accurately analyze abnormal behaviors in construction site video clips, thereby facilitating safety management and behavior monitoring of construction sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the execution flow of the AI-based construction site personnel behavior analysis method provided in an embodiment of the present invention.

[0013] Figure 2 It is a schematic diagram of the hardware architecture of the AI-based construction site personnel behavior analysis system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This is a flow chart of an AI-based construction site personnel behavior analysis method provided by an embodiment of the present invention. The AI-based construction site personnel behavior analysis method is introduced in detail below.

[0015] Step S110, obtaining a sequence of sample construction site video clips of the construction site abnormal behavior analysis network, wherein the sequence of sample construction site video clips includes multiple supervised sample construction site video clips and multiple unsupervised sample construction site video clips, wherein the supervised sample construction site video clips refer to surveillance video clips carrying predefined abnormal behavior paths, and the unsupervised sample construction site video clips refer to surveillance video clips not carrying predefined abnormal behavior paths.

[0016] In this embodiment, in a large construction site, in order to obtain a sample construction site video clip sequence, the server first connects to the monitoring system of the construction site. This monitoring system covers various areas of the construction site, including the construction area, material storage area, and personnel rest area.

[0017] The server obtains video data from the storage device of the monitoring system. For example, in the construction area, the surveillance camera records the workers performing various construction operations, such as concrete pouring, steel bar tying, etc. Some of the video clips are marked as supervised sample construction site video clips. For example, in the predefined abnormal behavior path, it is stipulated that workers entering a dangerous construction area without wearing a helmet is an abnormal behavior. Then, when the surveillance video captures a worker approaching an area where high-rise concrete pouring is in progress without a helmet, this video clip is marked as a supervised sample construction site video clip and is associated with the corresponding abnormal behavior path (entering a dangerous construction area without a helmet).

[0018] At the same time, the server also obtains unsupervised sample construction site video clips. For example, in the material stacking area, the camera records video clips of workers carrying construction materials and arranging the material stacking. These video clips are not associated with specific predefined abnormal behavior paths. They are just records of normal construction activities. These supervised and unsupervised sample video clips together constitute the sample construction site video clip sequence. The server will organize and number these sample construction site video clips for subsequent processing.

[0019] Step S120, using an artificial intelligence network model to output a first confidence level corresponding to each sample construction site video clip in the sample construction site video clip sequence, and extracting surveillance video clips whose first confidence levels meet first set requirements from the sample construction site video clip sequence as matching sample construction site video clips that match predefined abnormal behavior paths, wherein the first confidence level represents the confidence level that the sample construction site video clips match the predefined abnormal behavior paths.

[0020] In this embodiment, the artificial intelligence network model is pre-trained and can analyze the construction site video clips. For each video clip in the sample construction site video clip sequence, it is input into the artificial intelligence network model.

[0021] For example, for a supervised sample construction site video clip, which shows a worker working at height without using a safety rope. The model will analyze this video clip based on the pre-learned features and algorithms. It will consider factors such as the worker's position (at height), action (working without a safety rope), and surrounding environment (such as the height of the construction structure being built, etc.), and then output a first confidence score. Assume that this confidence score is 0.8, which means that the model believes that this video clip has a high probability of matching the predefined abnormal behavior path (working at height without using a safety rope).

[0022] For unsupervised sample construction site video clips, such as videos of workers carrying tools normally on the ground, the model will also analyze it, but because it has no predefined abnormal behavior path association, it may output a lower first confidence, such as 0.1.

[0023] The server sets the first setting requirement, assuming that the first threshold value is 0.6. Then the server will traverse all video clips in the sample construction site video clip sequence and extract those surveillance video clips whose first confidence is greater than 0.6. For example, there are several video clips showing workers not wearing reflective vests in dangerous areas (confidence 0.7), illegally walking within the operating radius of the crane (confidence 0.85), etc. These video clips will be extracted as matching sample construction site video clips that match the predefined abnormal behavior path.

[0024] Step S130, generates a first weight parameter based on the proportion of the amount of the supervised sample construction site video clips in the amount of the matching sample construction site video clips, and outputs a second confidence corresponding to each of the sample construction site video clips through the construction site abnormal behavior analysis network, wherein the second confidence represents the confidence that the sample construction site video clip is the supervised sample construction site video clip.

[0025] In this embodiment, the server first counts the number of supervised sample construction site video segments and the number of matched sample construction site video segments. Assume that in the entire sample construction site video segment sequence, there are 100 supervised sample construction site video segments, and after screening in step S120, there are 80 matched sample construction site video segments, of which 60 are supervised sample construction site video segments.

[0026] Then the ratio of the amount of the supervised sample construction site video clips to the amount of the matching sample construction site video clips is 60 / 80 = 0.75, and this 0.75 is generated by the server as the first weight parameter.

[0027] Next, the server analyzes each sample construction site video clip through the construction site abnormal behavior analysis network to output a second confidence. For example, for the previously mentioned supervised sample construction site video clip of a worker working at height without a safety rope, the construction site abnormal behavior analysis network will consider more features, such as the worker's body posture, movement coherence, and the reactions of surrounding workers. Assume that after analysis, the output second confidence is 0.9, indicating that this video clip has a high confidence that it is a supervised sample construction site video clip (that is, it is indeed a video clip related to the predefined abnormal behavior path).

[0028] For an unsupervised sample construction site video clip, such as a video of workers resting in a rest area, the construction site abnormal behavior analysis network may output a second confidence level of 0.05 after analysis, indicating that this video clip is almost impossible to be a supervised sample construction site video clip.

[0029] Step S140: Based on the first weight parameter and the second confidence corresponding to each of the sample construction site video clips, output a third confidence corresponding to each of the sample construction site video clips, wherein the third confidence represents the confidence that the sample construction site video clip matches the predefined abnormal behavior path.

[0030] In this embodiment, the server divides the second confidence corresponding to each sample construction site video clip by the first weight parameter according to a set calculation method to output a third confidence.

[0031] For example, the video clip of a worker working at height without using a safety rope, the second confidence is 0.9, the first weight parameter is 0.75, and the third confidence is 0.9 / 0.75 = 1.2 (this is just the result obtained by calculation, and reasonable adjustments or restrictions may be made in actual applications, such as limiting it to between 0 and 1). This means that the confidence of this video clip matching the predefined abnormal behavior path has been improved after adjustment, further indicating that this video clip is likely to be a video clip with abnormal behavior (working at height without using a safety rope).

[0032] For the unsupervised sample construction site video clip of workers resting in the rest area, the second confidence is 0.05, the first weight parameter is 0.75, and the third confidence is 0.05 / 0.75 ≈ 0.067, indicating that the confidence that this video clip matches the predefined abnormal behavior path is very low, and it is almost certain that it is a normal video clip with no abnormal behavior.

[0033] Step S150, based on the third confidence level corresponding to each of the sample construction site video clips, calculate and output the network performance index of the construction site abnormal behavior analysis network, and train the construction site abnormal behavior analysis network based on the network performance index to generate a target construction site abnormal behavior analysis network, so as to analyze whether the construction site video clip matches the predefined abnormal behavior path based on the target construction site abnormal behavior analysis network, and the network performance index characterizes the abnormal behavior analysis accuracy of the construction site abnormal behavior analysis network.

[0034] In this embodiment, the network performance index is first calculated based on the third confidence corresponding to each sample construction site video clip. For example, the hit coefficient of the construction site abnormal behavior analysis network is calculated. For the monitoring video clips corresponding to the predefined abnormal behavior paths in multiple supervised sample construction site video clips, such as the previously mentioned video clips of working at heights without using safety ropes, entering dangerous areas without wearing safety helmets, etc., the hit coefficient is calculated according to their third confidence. Assuming that there are 50 such supervised sample construction site video clips, 40 of which have a third confidence higher than a certain threshold (indicating correct identification as abnormal behavior), then the hit coefficient is 40 / 50 = 0.8.

[0035] Then, the server outputs the confidence of the matching sample construction site video segments in the sample construction site video segment sequence based on the third confidence corresponding to each sample construction site video segment. For example, for all matching sample construction site video segments, their average third confidence is calculated, assuming it is 0.6.

[0036] Next, the server calculates the network performance index based on the confidence and hit coefficient of the matching sample construction site video clip in the sample construction site video clip sequence. Assuming a comprehensive calculation method is adopted, the network performance index = hit coefficient × confidence of matching sample construction site video clip = 0.8 × 0.6 = 0.48.

[0037] According to this network performance index, the server trains the abnormal behavior analysis network for construction sites. If the network performance index is low, the server will adjust the parameters of the network, such as adjusting the weights of the neural network, optimizing the parameters of the algorithm, etc. After multiple iterations of training, when the network performance index reaches a satisfactory level, such as reaching above 0.8, the target construction site abnormal behavior analysis network is generated. This target construction site abnormal behavior analysis network can be used to analyze whether new construction site video clips match the predefined abnormal behavior path. For example, when a new video clip is input into this target construction site abnormal behavior analysis network, it can accurately determine whether there are abnormal behaviors such as workers violating regulations or entering dangerous areas without taking protective measures.

[0038] Based on the above steps, the embodiment of the present application effectively improves the comprehensiveness and accuracy of abnormal behavior analysis by combining supervised samples and unsupervised sample construction site video clips. The artificial intelligence network model is used to perform confidence assessment on the sample construction site video clips, and the matching samples that match the predefined abnormal behavior path are screened out based on the first confidence, ensuring the pertinence of the analysis process. By introducing the first weight parameter and the second confidence, the matching degree of the sample construction site video clips is further refined, so that the third confidence more accurately reflects the matching of the sample construction site video clips with the predefined abnormal behavior path. Finally, through the calculation of network performance indicators and the training of the construction site abnormal behavior analysis network, a high-precision target analysis network is generated, which can efficiently and accurately analyze abnormal behaviors in construction site video clips, thereby facilitating safety management and behavior monitoring of construction sites.

[0039] In a possible implementation, step S140 specifically includes:

[0040] Step S1401: divide the second confidence level corresponding to each of the sample construction site video clips by the first weight parameter, and output the third confidence level corresponding to each of the sample construction site video clips.

[0041] In this embodiment, the first weight parameter and the second confidence corresponding to each sample construction site video clip have been obtained. The first weight parameter is obtained based on the ratio of the amount of supervised sample construction site video clips to the amount of matched sample construction site video clips. For example, previously in the entire sample construction site video clip sequence, there were 100 supervised sample construction site video clips, 80 matched sample construction site video clips, of which 60 were supervised sample construction site video clips, and the first weight parameter was 0.75. For the second confidence of each sample construction site video clip, taking the previously mentioned supervised sample construction site video clip of workers working at heights without using safety ropes as an example, its second confidence is 0.9. According to the calculation method of dividing the second confidence by the first weight parameter, the third confidence of this video clip is 0.9 divided by 0.75, which is equal to 1.2. For another example, the unsupervised sample construction site video clip of workers resting in the rest area has a second confidence of 0.05, and the third confidence is 0.05 divided by 0.75, which is approximately 0.067. The server performs such calculations for each sample construction site video clip, and outputs the corresponding third confidence, which can more accurately characterize the confidence that the sample construction site video clip matches the predefined abnormal behavior path.

[0042] In a possible implementation manner, the first setting requirement includes that a first confidence level corresponding to the sample construction site video clip is greater than a first threshold value.

[0043] Step S120 specifically includes:

[0044] Step S1201: extracting, from the sequence of sample construction site video clips, surveillance video clips corresponding to the sample construction site video clips and having a first confidence level greater than a first threshold value, as the matching sample construction site video clips.

[0045] When determining the matching sample construction site video clip that matches the predefined abnormal behavior path, the server is based on the first setting requirement, where the first setting requirement is that the first confidence corresponding to the sample construction site video clip is greater than the first threshold value. The server will traverse each video clip in the sample construction site video clip sequence. Taking the previous surveillance video as an example, assume that the first threshold value is set to 0.6. Among the many video clips, such as a video clip showing a worker not wearing a reflective vest in a dangerous area, its first confidence is 0.7 after analysis by the artificial intelligence network model. Since 0.7 is greater than 0.6, this video clip will be extracted by the server from the sample construction site video clip sequence as a matching sample construction site video clip that matches the predefined abnormal behavior path. Similarly, if the first confidence of a video clip of a worker illegally passing through the operating radius of a crane is 0.85, it also meets the condition of being greater than 0.6 and will also be extracted as a matching sample construction site video clip. However, those video clips with a first confidence less than 0.6, such as workers normally sorting materials in the material storage area, whose first confidence may be 0.1, will not be regarded as matching sample construction site video clips that match the predefined abnormal behavior path, thereby ensuring that the extracted matching sample construction site video clips have a high probability of matching the predefined abnormal behavior path, laying the foundation for subsequent more accurate analysis of abnormal behavior at the construction site.

[0046] In one possible implementation, the sample construction site video clip includes video semantic tracking features and behavioral state transition parameters of the construction site personnel, the video semantic tracking features represent the behavior pattern vectors of the construction site personnel and the behavior pattern vectors of the construction site personnel switching construction behaviors, and the behavior state transition parameters of the construction site personnel represent the behavior state transition vectors of the construction site personnel switching construction behaviors.

[0047] Wherein, the method further comprises:

[0048] Step A110, based on the initial state characteristics of the construction site personnel, the behavior state transition parameters of the construction service node in the ath video surveillance tracking stage, and the first state migration array, output the behavior state transition parameters of the construction site personnel in the ath video surveillance tracking stage, the first state migration array reflects the construction linkage parameters of the construction service node switching the construction behavior of the construction site personnel in the ath video surveillance tracking stage. Wherein, a is a positive integer iterating from 1.

[0049] Step A120, based on the initial state characteristics of the construction service node, the behavior state transfer parameters of the construction service node in the ath video surveillance tracking stage, the behavior state transfer parameters of the construction site personnel in the ath video surveillance tracking stage, the second state migration array and the third state migration array, generate the behavior state transfer parameters of the construction service node in the a+1th video surveillance tracking stage. Among them, the second state migration array reflects the construction linkage parameters of the construction site personnel switching the construction behavior of the construction service node in the ath video surveillance tracking stage, and the third state migration array reflects the synchronization linkage parameters of the construction service node synchronizing the construction site personnel switching the construction behavior to other construction service nodes in the ath video surveillance tracking stage.

[0050] Step A130, when the number of video surveillance tracking stages meets the training convergence requirement, the behavior state transfer parameters of the construction site personnel generated in the last video surveillance tracking stage are loaded into the sample construction site video clip.

[0051] For example, in a possible implementation, step A110 includes:

[0052] The initial behavior pattern vector of the construction site personnel is extracted from the sample construction site video clips, and the initial position coordinates of the construction site personnel are generated by combining the construction site layout map and the location information of the construction service nodes. At the same time, based on the action recognition results of the construction site personnel, the initial action code is generated. Based on the relative position and interaction between the construction site personnel and other construction service nodes, the associated state code is generated. The initial behavior pattern vector, initial position coordinates, initial action code of the construction site personnel and the associated state code with other construction service nodes are integrated into an initial state feature matrix.

[0053] The working status changes, position changes and interaction data with the construction site personnel of the construction service node are collected in real time, and combined with the behavioral state transfer parameters of the a-1th video surveillance tracking stage, the state information of the construction service node is updated to generate the construction service node behavioral state transfer parameter matrix of the ath video surveillance tracking stage.

[0054] By analyzing the interaction rules between the construction service node and the construction site personnel, as well as the impact characteristics of the changes in the working state of the construction service node on the construction behavior of the construction site personnel, a first state transition array is generated. Each element in the first state transition array represents the probability or condition of the construction site personnel switching to another construction behavior under the state of the construction service node.

[0055] The initial state feature matrix of the construction site personnel is subjected to matrix multiplication operation with the first state transition array, and the influence characteristics of the construction service node on the construction behavior are integrated into the state characteristics of the construction site personnel to generate a fused state feature matrix of the construction site personnel.

[0056] By using a Markov chain network, the transition probability of the construction site personnel in different states is calculated according to the fused construction site personnel state feature matrix and the behavior state transition parameter matrix of the construction service node, and a behavior state transition probability matrix is ​​generated.

[0057] According to the current state characteristics and behavior state transition probability matrix of the construction site personnel, the future state of the construction site personnel in the ath video surveillance tracking stage is predicted. By calculating the state transition path with the highest probability, the predicted position, predicted action, predicted posture and predicted association state with the construction service node of the construction site personnel are determined to generate the behavior state transition parameters.

[0058] Step A120 includes:

[0059] Based on the initial state characteristics of the construction service node, the behavior state transfer parameters of the construction service node in the ath video surveillance tracking stage, the behavior state transfer parameters of the construction site personnel in the ath video surveillance tracking stage, the second state migration array and the third state migration array, an empty a+1th video surveillance tracking stage construction service node behavior state transfer parameter matrix is ​​initialized as the output target, wherein the initial state characteristics of the construction service node include the type, location, working status and initial interaction with the construction site personnel of the construction service node, the behavior state transfer parameters of the construction service node in the ath video surveillance tracking stage reflect the state change of the construction service node in the previous video surveillance tracking stage, the behavior state transfer parameters of the construction site personnel in the ath video surveillance tracking stage provide the behavior pattern and state transfer information of the construction site personnel in the previous video surveillance tracking stage, the second state migration array describes the influence of the construction site personnel on the state change of the construction service node, and the third state migration array represents the mechanism and rules of state synchronization between construction service nodes.

[0060] The initial state characteristics of the construction service node are encoded to generate an initial state characteristic vector, which includes all initial state information of the construction service node, such as type code, location coordinates, working state code and initial interaction status code with construction site personnel.

[0061] The behavior state transfer parameter matrix of the construction service node in the ath video surveillance and tracking stage is fused with the initial state feature vector to generate a current state feature vector that integrates historical state information. The behavior state transfer parameters of the construction site personnel in the ath video surveillance and tracking stage are used as input. Combined with the current state feature vector, a predefined impact assessment model is used to assess the impact of the behavior of the construction site personnel on the state of the construction service node, and an impact assessment vector is output. The impact assessment model is constructed based on a neural network and is used to predict the possible impact on the state of the construction service node based on the behavior patterns and state transfer information of the construction site personnel.

[0062] The impact assessment vector is subjected to matrix multiplication operation with the second state transition array, and the influence of the construction site personnel on the state of the construction service node is specifically quantified as a state transition probability or condition, and a state transition influence matrix is ​​generated. The state transition influence matrix reflects the state transition of the construction service node under the influence of the behavior of the construction site personnel.

[0063] The behavior state transfer parameter matrix of the construction service node in the ath video surveillance tracking stage is matrix multiplied with the third state migration array to generate a state synchronization influence matrix, which reflects the impact of state changes of other construction service nodes on the current construction service node state.

[0064] The state transfer influence matrix and the state synchronization influence matrix are integrated to generate the current state transfer probability matrix which integrates the influence of construction site personnel and the state synchronization influence of other construction service nodes.

[0065] Based on the current state transition probability matrix, the possible state of the construction service node in the a+1th video surveillance tracking stage is predicted through a random simulation method to generate a state prediction vector, which reflects the state distribution of the construction service node in the future stage.

[0066] According to the state prediction vector, the behavior state transfer parameter matrix of the construction service node in the a+1th video monitoring and tracking phase is updated, specifically, the state prediction vector is filled into the corresponding state transfer parameter matrix.

[0067] Repeat the above steps until the behavior state transfer parameters of all construction service nodes are updated, and output the final construction service node behavior state transfer parameter matrix of the a+1th video surveillance tracking stage.

[0068] In this embodiment, the video semantic tracking feature is an important data for describing the behavior of construction site personnel. For example, on a large construction site, the video semantic tracking feature received by the server can characterize in detail the behavior pattern vector of the construction site personnel and the behavior pattern vector of their switching construction behavior. Taking concrete pouring as an example, when a worker is pouring concrete, he holds a concrete pouring tool and pours concrete into the mold according to a specific rhythm and action. During this process, his arm swinging speed, direction, body standing posture, and relative position relationship with other workers or construction equipment constitute his behavior pattern vector in the construction behavior of concrete pouring. When the worker switches from concrete pouring to carrying steel bars, he puts down the pouring tool, walks to the steel bar stacking area, bends over to pick up the steel bars, and a series of actions form the behavior pattern vector of his switching construction behavior. The behavior state transition parameters of the construction site personnel characterize the behavior state transition vector of the construction site personnel switching construction behavior. For example, in the process of switching from a stationary state to a walking state and then to bending over to pick up the steel bars, the transition state between each action, the time interval of the conversion, and the related action amplitude changes are all included in the behavior state transition vector.

[0069] The process of outputting the behavioral state transfer parameters of the construction site personnel in the ath video surveillance tracking stage based on the initial state characteristics of the construction site personnel, the behavioral state transfer parameters of the construction service node in the ath video surveillance tracking stage, and the first state migration array is relatively complicated. First, the server needs to determine the initial state characteristics of the construction site personnel. Assuming that the focus is on a construction worker who has just started a day's work, his initial state characteristics include his initial position (for example, near the entrance of the construction site), initial action (maybe standing and checking the day's work task schedule), initial equipment status (for example, the helmet is already worn, but the tools have not yet been taken), etc. These initial state features can be integrated into an initial state feature matrix.

[0070] Then, in the ath video surveillance tracking stage (here it is assumed that a = 1 starts), the behavior state transition parameters of the construction service node also play an important role. For example, the construction service node can be a concrete mixer, and its behavior state transition parameters may include the mixer starting from the stop state, the rotation speed of the mixing barrel gradually increasing from 0 to the normal working speed, and the feed port starting to feed. At the same time, the first state transition array reflects the construction linkage parameters of this construction service node for the construction site personnel to switch the construction behavior. For example, when the concrete mixer starts and starts to work normally, it will generate noise and the surrounding working environment will undergo some changes (such as dust, transportation of concrete raw materials, etc.), which will affect nearby workers. If a worker was originally resting near the mixer, due to the start of the mixer, he may move away from the mixer to other work areas, or start auxiliary work related to concrete (such as assisting in cleaning up debris around the mixer). The server generates the first state transition array by analyzing the interaction rules between the construction service node and the construction site personnel, and the impact characteristics of the working state changes of the construction service node on the construction behavior of the construction site personnel. Each element in this array represents the probability or condition of the construction site personnel switching to another construction behavior under a specific state of the construction service node. For example, when the mixer drum speed reaches a certain speed and the feed inlet is stable, the probability of the worker approaching the mixer for auxiliary operation is 0.3, and the probability of moving away from the mixer to other areas for other work is 0.7.

[0071] The server performs matrix multiplication operation on the initial state feature matrix of the construction site personnel and the first state transition array, integrates the influence characteristics of the construction service node on the construction behavior into the state characteristics of the construction site personnel, and generates a fused state feature matrix of the construction site personnel. Then, using the Markov chain network, according to the fused state feature matrix of the construction site personnel and the behavior state transition parameter matrix of the construction service node, the transition probability of the construction site personnel in different states is calculated, and the behavior state transition probability matrix is ​​generated. For example, according to the current state characteristics, the probability of the worker moving from the current position to other specific positions in the next few minutes is calculated, and the probability of switching from the current rest or waiting state to starting a specific construction operation is calculated. Finally, according to the current state characteristics of the construction site personnel and the behavior state transition probability matrix, the future state of the construction site personnel in the ath video surveillance tracking stage is predicted, and the predicted position, predicted action, predicted posture and predicted association state of the construction site personnel with the construction service node are determined by calculating the state transition path with the highest probability, thereby outputting the behavior state transition parameters of the construction site personnel in the ath video surveillance tracking stage.

[0072] Based on the initial state characteristics of the construction service node, the behavioral state transfer parameters of the construction service node in the ath video surveillance tracking stage, the behavioral state transfer parameters of the construction site personnel in the ath video surveillance tracking stage, the second state transition array and the third state transition array, the process of generating the behavioral state transfer parameters of the construction service node in the a+1th video surveillance tracking stage is equally complex.

[0073] Taking a tower crane as a construction service node as an example, its initial state characteristics include its initial position (located in a specific lifting area of ​​the construction site), initial working state (for example, the boom is in a stationary state and the hook is in a retracted position), and initial interaction with the construction site personnel (there may be a commander waiting nearby to issue a lifting command). In the ath video surveillance tracking stage (assuming that the previous calculations have been completed), the behavior state transition parameters of the tower crane may be the transition of the boom starting to rotate, the hook starting to lower and prepare to lift construction materials, etc. At the same time, the behavior state transition parameters of the construction site personnel in the ath video surveillance tracking stage are also taken into account, such as the gestures of the commander and the position adjustment of the surrounding auxiliary workers.

[0074] The second state transition array reflects the construction linkage parameters of the construction service node switching construction behavior of the construction site personnel in the ath video surveillance tracking stage. For example, when the commander makes a specific gesture (which is a manifestation of the behavioral state transition of the construction site personnel), the rotation direction and speed of the tower crane's boom may be adjusted according to the gesture, and the rules and probabilities of this adjustment are reflected in the second state transition array. The third state transition array reflects the synchronization linkage parameters of the construction service node switching construction behavior to other construction service nodes in synchronization with the construction site personnel in the ath video surveillance tracking stage. Assuming that there are multiple cranes working together on the construction site, when one crane starts hoisting according to the commander's instructions, other cranes may need to adjust their working states (such as changing the height or position of the boom to avoid collision or coordinate work), and the rules and probabilities of this coordinated adjustment are reflected in the third state transition array.

[0075] Based on the above parameters and arrays, the server first initializes an empty a + 1th video surveillance tracking phase construction service node behavior state transfer parameter matrix as the output target. The initial state features of the construction service node are encoded to generate an initial state feature vector, which contains all the initial state information of the tower crane, such as type coding (indicating that it is a tower crane), position coordinates (accurate three-dimensional coordinates), working state coding (such as static, ready to hoist, etc.) and initial interaction coding with site personnel (such as the distance from the commander, whether there are workers under the crane arm, etc.).

[0076] The behavior state transfer parameter matrix of the construction service node in the ath video surveillance tracking stage is fused with the initial state feature vector to generate the current state feature vector that integrates the historical state information. The behavior state transfer parameters of the construction site personnel in the ath video surveillance tracking stage are used as input, combined with the current state feature vector, and the impact of the construction site personnel's behavior on the state of the construction service node is evaluated through a pre-defined impact assessment model (this model is built based on a neural network and is used to predict the possible impact on the state of the construction service node based on the behavior pattern and state transfer information of the construction site personnel) to output the impact assessment vector. For example, the impact assessment results of the commander's gestures on the rotation angle and speed of the tower crane boom are included in this impact assessment vector.

[0077] The impact assessment vector is matrix multiplied with the second state transition array, and the impact of the construction site personnel on the state of the construction service node is specifically quantified as a state transition probability or condition, generating a state transition impact matrix. This matrix reflects the state transition of the construction service node under the influence of the behavior of the construction site personnel. For example, a certain gesture of the commander will cause the tower crane boom to rotate with a certain probability and angle. At the same time, the behavior state transition parameter matrix of the construction service node in the ath video monitoring tracking stage is matrix multiplied with the third state transition array to generate a state synchronization impact matrix, which reflects the impact of the state changes of other construction service nodes on the current state of the construction service node. For example, the change in the boom height of another nearby crane will have a certain probability and condition of affecting the working state of this tower crane.

[0078] The state transition influence matrix and the state synchronization influence matrix are merged to generate the current state transition probability matrix that integrates the influence of construction site personnel and the state synchronization influence of other construction service nodes. Based on this matrix, the possible state of the construction service node in the a + 1th video surveillance tracking stage is predicted by random simulation method to generate a state prediction vector. This vector reflects the state distribution of the construction service node in the future stage, such as the possible position of the boom of the tower crane in the next period of time, the height of the hook, and the progress of the hoisting. Finally, according to the state prediction vector, the behavior state transfer parameter matrix of the construction service node in the a + 1th video surveillance tracking stage is updated, specifically, the state prediction vector is filled into the corresponding state transfer parameter matrix. Repeat the above steps until the behavior state transfer parameters of all construction service nodes are updated, and the final behavior state transfer parameter matrix of the construction service node in the a + 1th video surveillance tracking stage is output.

[0079] During the entire video surveillance tracking process, when the number of video surveillance tracking stages meets the training convergence requirements, the server loads the behavioral state transfer parameters of the construction site personnel generated in the last video surveillance tracking stage into the sample construction site video clip. For example, if the set training convergence requirement is that after 10 video surveillance tracking stages, the changes in various parameters tend to be stable within a certain range. When these 10 stages are completed, the behavioral state transfer parameters of the construction site personnel obtained in the last stage (such as the final action, position, relationship with construction equipment, and other state transfer parameters of the workers after a series of work processes) will be loaded into the sample construction site video clip. This helps to more comprehensively and accurately describe the behavioral state changes of construction site personnel throughout the work process, and provide richer and more accurate data support for the subsequent abnormal behavior analysis of construction sites based on sample construction site video clips.

[0080] In a possible implementation, the method further includes:

[0081] Based on the collaborative monitoring operation data of the construction service nodes, the construction service nodes that do not meet the video monitoring tracking conditions are determined, and the construction service nodes that do not meet the video monitoring tracking conditions do not participate in the video monitoring tracking process of the behavior state transfer parameters of the construction site personnel.

[0082] In this embodiment, the collaborative monitoring operation data covers the working conditions, operating status, and interaction information with other nodes of many construction service nodes on the construction site. For example, in a large construction site, the construction service nodes include various mechanical equipment such as tower cranes, concrete mixers, loaders, etc., as well as some specific work areas such as material storage areas and work areas for specific construction processes.

[0083] The collaborative monitoring operation data contains information such as the working frequency, working hours, and maintenance cycle of each construction service node. Taking a tower crane as an example, its collaborative monitoring operation data shows that its daily working hours are mainly concentrated in a few hours in the morning and afternoon. There is a certain interval between each lifting task, and routine maintenance is performed once a week according to the prescribed maintenance cycle. For some small, temporary construction tools or equipment, such as small manual tool storage areas, their working frequency is very low and may only be used once every few days. Moreover, their working status is relatively independent, which has little impact on the behavioral status transfer of personnel on the entire construction site.

[0084] The server conducts a detailed analysis of these collaborative monitoring operation data to determine the construction service nodes that do not meet the video monitoring tracking conditions. For those construction service nodes with extremely low working frequency, relatively independent working status, and almost no impact on the behavioral state transition of the construction site personnel, they are determined to be construction service nodes that do not meet the video monitoring tracking conditions. For example, a small warehouse for spare materials on the construction site stores some uncommon building materials, and there are few workers around this small warehouse. The frequency of items entering and leaving the warehouse is extremely low. It is determined to be a construction service node that does not meet the video monitoring tracking conditions. Such a construction service node does not participate in the video monitoring tracking process of the behavioral state transition parameters of the construction site personnel, thereby reducing unnecessary data processing and calculation, and improving the efficiency and accuracy of the entire system.

[0085] In a possible implementation, the method further includes:

[0086] Based on the construction relevance of the construction personnel and the number of construction service nodes associated with the construction personnel, the maximum number of associated construction service nodes matching the construction personnel is determined, wherein the number of associated construction service nodes of the construction personnel represents the number of construction service nodes that have a collaborative relationship with the construction personnel.

[0087] Based on the maximum number of associated construction service nodes matched by the construction site personnel, the switching characteristic parameters of the construction service node for switching the construction behavior of the construction site personnel and the switching characteristic parameters of the construction site personnel for switching the construction behavior of the construction service node are obtained.

[0088] The first state transition array is generated based on the switching characteristic parameters of the construction service node for the construction site personnel switching the construction behavior, and the second state transition array is generated based on the switching characteristic parameters of the construction site personnel switching the construction behavior of the construction service node.

[0089] In this embodiment, the relevant data of the construction relevance of the construction site personnel and the number of associated construction service nodes are obtained. The construction relevance reflects the closeness of the working connection between the construction site personnel and different construction service nodes. For example, for a concrete pouring worker, he has a high construction relevance with construction service nodes such as concrete mixers, concrete transport vehicles, and vibrating equipment. Because his main job is to transport the concrete mixed by the mixer to the pouring site by transport vehicles, and then vibrate it with vibrating equipment, the normal operating status of these equipment directly affects his work efficiency and work flow.

[0090] The number of associated construction service nodes clearly indicates the number of construction service nodes that have a collaborative relationship with the site personnel. For example, this concrete pouring worker has a collaborative relationship with the three construction service nodes mentioned above: concrete mixer, concrete transport vehicle, and vibrating equipment, so the number of his associated construction service nodes is 3.

[0091] Based on these data, the server determines the maximum number of associated construction service nodes that the site personnel can match. This process requires comprehensive consideration of factors such as the job functions of the site personnel, the workflow, and the layout and construction arrangements of the entire construction site. For a concrete pouring worker, considering that his workflow mainly revolves around the production, transportation, and pouring of concrete, he may be closely associated with up to 5 construction service nodes, which may include the concrete raw material storage area (for obtaining raw materials such as cement, sand and gravel), concrete mixers, concrete transport vehicles, vibrating equipment, and formwork installation areas (because concrete needs to be poured into the formwork). This 5 is the maximum number of associated construction service nodes he can match.

[0092] Based on the maximum number of associated construction service nodes, the server begins to obtain the switching characteristic parameters of the construction service nodes for the construction site personnel switching construction behaviors and the switching characteristic parameters of the construction site personnel switching construction behaviors for the construction service nodes. Taking the switching characteristic parameters of the concrete mixer for the concrete pouring worker switching construction behaviors as an example, when the concrete mixer fails and stops mixing, the concrete pouring worker may switch from the normal pouring waiting state to the state of assisting the maintenance personnel to check the mixer failure. The relevant parameters in this switching process, such as the triggering condition of the switching (the mixer stops mixing), the time delay of the switching (the time interval from the mixer stopping mixing to the worker arriving at the mixer to assist in the inspection), and the working mode after the switch (from the pouring waiting mode to the assisting maintenance mode, including the actions and focus of attention when assisting maintenance, etc.), constitute the switching characteristic parameters of the construction service node for the construction site personnel switching construction behaviors.

[0093] Similarly, for the switching characteristic parameters of the construction service node switching construction behavior of the construction site personnel, when the concrete pouring worker finds that the concrete pouring speed is too fast and may cause the formwork to overflow, he will send a signal to the operator of the concrete mixer to reduce the mixing speed of the mixer. The relevant parameters in this process, such as the way the worker sends the signal (gestures or shouts, etc.), the timing of the signal (at the moment of discovering that the pouring speed is too fast or with a slight delay), the impact on the construction service node (mixer) (the adjustment range of the mixer mixing speed, etc.), etc., constitute the switching characteristic parameters of the construction service node switching construction behavior of the construction site personnel.

[0094] The first state transition array is generated based on the switching characteristic parameters of the construction service node for the construction site personnel to switch the construction behavior, and the second state transition array is generated based on the switching characteristic parameters of the construction service node for the construction site personnel to switch the construction behavior. The elements in the first state transition array reflect the probability and conditions of the construction site personnel switching the construction behavior under various construction service node state changes. For example, in the first state transition array, when the temperature of the mixing barrel of the concrete mixer is too high, the probability of the worker switching from the normal working state to stopping work and waiting for cooling is 0.8, and the probability of switching to continue working but taking cooling measures is 0.2. The second state transition array reflects the probability and conditions of the construction service node switching the construction behavior under different behavior changes of the construction site personnel. For example, when the worker sends an emergency stop signal, the probability of the concrete mixer stopping mixing is 0.9, and the probability of continuing to mix slowly is 0.1.

[0095] In a possible implementation, the method further includes:

[0096] The first state transition array, the second state transition array and the third state transition array are respectively decomposed to generate a plurality of first state transition array units, a plurality of second state transition array units and a plurality of third state transition array units.

[0097] Among them, the behavioral state transfer sub-parameters of the construction site personnel in the a-th video surveillance tracking stage generated by the multiple first state transition array units are spliced ​​to output the behavioral state transfer parameters of the construction site personnel in the a-th video surveillance tracking stage.

[0098] The behavior state transfer sub-parameters of the construction service node in the a+1th video surveillance tracking stage generated by the multiple second state migration array units and the multiple third state migration array units are spliced ​​to output the behavior state transfer parameters of the construction service node in the a+1th video surveillance tracking stage.

[0099] In this embodiment, the first state migration array, the second state migration array and the third state migration array are then decomposed respectively. Taking the first state migration array as an example, it is assumed that this array is a multidimensional matrix, which contains multiple construction service node states and various associated information of construction site personnel switching construction behaviors. The server decomposes it into multiple first state migration array units according to certain rules. For example, it can be decomposed according to the type of construction service node, the work area or the different properties of the impact on the construction site personnel. If according to the type of construction service node, different first state migration array units are formed for the parts related to the tower crane, the parts related to the concrete mixer and the parts related to the loader.

[0100] Based on these multiple first state transition array units, the behavior state transition sub-parameters of the construction site personnel in the a-th video surveillance tracking stage are generated respectively. Each first state transition array unit contains a specific part of information, and generates the corresponding behavior state transition sub-parameters through specific calculation and analysis methods. For example, for the first state transition array unit related to the tower crane, according to the state information such as the tower crane's boom height, lifting weight, rotation speed, and the relative position of the construction site personnel and the tower crane, work interaction, etc., the behavior state transition sub-parameters of the construction site personnel in this specific situation are generated, such as the movement direction, speed adjustment, and other parameters of the construction site personnel under the tower crane's boom.

[0101] Then, the behavior state transition sub-parameters of the construction site personnel in the ath video surveillance tracking stage generated by the multiple first state transition array units are spliced ​​to output the behavior state transition parameters of the construction site personnel in the ath video surveillance tracking stage. This splicing process is carried out in accordance with the pre-set order and rules to ensure that the various sub-parameters can be accurately combined to form a complete behavior state transition parameter, which fully describes the behavior state transition of the construction site personnel in the ath video surveillance tracking stage.

[0102] Similarly, for the second state migration array and the third state migration array, the server decomposes them into multiple second state migration array units and multiple third state migration array units. Based on these array units, the behavior state transfer sub-parameters of the construction service node in the a+1th video surveillance tracking stage are generated. For example, for the second state migration array unit, according to the influence of different actions of the construction site personnel (such as command gestures, carrying direction, etc.) on the construction service node (such as tower crane, loader, etc.), the behavior state transfer sub-parameters of the construction service node are generated, such as the rotation angle adjustment amount of the tower crane boom, the lifting height adjustment amount of the loader bucket, etc. For the third state migration array unit, according to the synchronous linkage relationship between a construction service node (such as a tower crane) and other construction service nodes (such as other nearby cranes or related transportation equipment), the behavior state transfer sub-parameters of the construction service node are generated, such as the parameters for the tower crane to adjust its own position according to the position of other cranes.

[0103] Finally, the behavior state transfer sub-parameters of the construction service node in the a+1th video surveillance tracking stage generated by the multiple second state transfer array units and the multiple third state transfer array units are spliced ​​to output the behavior state transfer parameters of the construction service node in the a+1th video surveillance tracking stage. This complete behavior state transfer parameter can accurately describe the behavior state transfer of the construction service node in the a+1th video surveillance tracking stage, thereby providing comprehensive and accurate data support for the monitoring, analysis and management of the entire construction site.

[0104] In a possible implementation, step S130 includes:

[0105] Step S131: generating a plurality of sample construction site video clip subsequences based on the sample construction site video clip sequence, wherein the sample construction site video clip subsequences include the same number of supervised sample construction site video clips and unsupervised sample construction site video clips.

[0106] Step S132: outputting, through the construction site abnormal behavior analysis network, initial second confidences corresponding to each of the sample construction site video clips included in the sample construction site video clip subsequence.

[0107] Step S133: outputting the second confidences corresponding to each of the sample construction site video clips based on the average confidence of the initial second confidences corresponding to each of the sample construction site video clips.

[0108] In this embodiment, first, a plurality of sample construction site video clip subsequences are generated based on the sample construction site video clip sequence. For example, the server divides the obtained large number of sample construction site video clip sequences. Assume that the original sample construction site video clip sequence contains 100 supervised sample construction site video clips and 200 unsupervised sample construction site video clips. The server generates sample construction site video clip subsequences according to certain rules, and each subsequence contains the same number of supervised sample construction site video clips and unsupervised sample construction site video clips. For example, each subsequence is set to contain 5 supervised sample construction site video clips and 5 unsupervised sample construction site video clips.

[0109] Then, for each sample construction site video clip in each sample construction site video clip subsequence, the server outputs its respective initial second confidence through the construction site abnormal behavior analysis network. Taking one of the subsequences as an example, for a supervised sample construction site video clip, such as a video clip showing a worker not using protective equipment in a dangerous area, the construction site abnormal behavior analysis network will comprehensively analyze multiple factors in the video. It will consider the worker's location information (determined to be in the dangerous area through the spatial coordinates in the video), action characteristics (no action related to the use of protective equipment), surrounding environment (the identification of the dangerous area and the surrounding construction scene), and time information (such as occurring during the normal construction time period). Based on these factors, the network may output an initial second confidence of 0.8 for this video clip, indicating that the video clip has a high probability of being a supervised sample construction site video clip (i.e., related to the predefined abnormal behavior path).

[0110] For the unsupervised sample construction site video clips in the same subsequence, such as a video clip showing workers carrying construction materials normally in a safe area, the construction site abnormal behavior analysis network will also analyze it. It will detect whether the workers' actions conform to the normal carrying mode (such as the carrying posture, walking route, etc.), whether the stacking and handling of materials follow the regular process, whether there are no abnormal conditions in the surrounding environment (such as no danger signs or abnormal equipment operation status), etc. Based on these analyses, the network may output the initial second confidence of this video clip as 0.05, indicating that the video clip is unlikely to be a supervised sample construction site video clip.

[0111] The server performs such an initial second confidence output operation for each sample construction site video clip in each sample construction site video clip subsequence. Afterwards, the server outputs the second confidence corresponding to each sample construction site video clip based on the average confidence of the initial second confidence corresponding to each sample construction site video clip. For example, for a certain sample construction site video clip, it may exist in multiple sample construction site video clip subsequences. In these subsequences, its corresponding initial second confidences are 0.8, 0.75, 0.85, etc. The server calculates the average value of these initial second confidences, and assuming that the average of these values ​​is 0.8, then the second confidence corresponding to this sample construction site video clip is determined to be 0.8. This second confidence can more stably and accurately characterize the confidence of the sample construction site video clip as a supervisory sample construction site video clip.

[0112] In a possible implementation, step S150 includes:

[0113] Step S151, based on the third confidence corresponding to each of the sample construction site video clips, calculate and output the hit coefficient of the construction site abnormal behavior analysis network, wherein the hit coefficient represents the confidence of the monitoring video clips corresponding to the predefined abnormal behavior paths in the multiple supervised sample construction site video clips.

[0114] Step S152: outputting the confidence of the matching sample construction site video clip in the sequence of sample construction site video clips based on the third confidence corresponding to each of the sample construction site video clips.

[0115] Step S153, calculating the network performance index of the construction site abnormal behavior analysis network based on the confidence of the matching sample construction site video clips in the sample construction site video clip sequence and the hit coefficient.

[0116] First, the server calculates and outputs the hit coefficient of the abnormal behavior analysis network of the construction site based on the third confidence level corresponding to each sample construction site video clip. In the entire sample construction site video clip sequence, there are multiple supervised sample construction site video clips, among which the surveillance video clips corresponding to the predefined abnormal behavior path are the focus of attention. For example, among the 100 supervised sample construction site video clips, some are surveillance video clips corresponding to the predefined abnormal behavior path, such as video clips of workers working at heights without wearing safety belts and operating illegally near electrical equipment.

[0117] For the surveillance video clips corresponding to these predefined abnormal behavior paths, the server calculates the hit coefficient based on their respective corresponding third confidences. Assuming there are 30 such video clips, the server will check the third confidences of these video clips. If the third confidences of 25 of the video clips are higher than a certain set threshold (for example, 0.6, indicating that the video clip is considered to be correctly identified as abnormal behavior), then the hit coefficient is 25 / 30 ≈ 0.83. This hit coefficient characterizes the confidence of the surveillance video clips corresponding to the predefined abnormal behavior paths in multiple supervised sample construction site video clips, that is, the ability of the network to correctly identify the surveillance video clips corresponding to the predefined abnormal behavior paths.

[0118] Next, the server outputs the confidence of the matching sample construction site video clips in the sample construction site video clip sequence based on the third confidence corresponding to each sample construction site video clip. The server traverses all matching sample construction site video clips in the sample construction site video clip sequence (these are video clips that are screened out according to the first confidence in the previous step and are considered to be likely to match the predefined abnormal behavior path). For example, for these matching sample construction site video clips, their respective third confidences are 0.7, 0.8, 0.65, etc. The server can output the confidence of the matching sample construction site video clips in the sample construction site video clip sequence by calculating the average of the third confidences of these matching sample construction site video clips. Assuming that the average of these values ​​is 0.72, then this 0.72 represents the confidence of the matching sample construction site video clips in the sample construction site video clip sequence.

[0119] Finally, the network performance index of the abnormal behavior analysis network of the construction site is calculated based on the confidence and hit coefficient of the matching sample construction site video clips in the sample construction site video clip sequence. For example, using a simple calculation method, the network performance index is equal to the hit coefficient multiplied by the confidence of the matching sample construction site video clip. In the previous example, the hit coefficient is 0.83, and the confidence of the matching sample construction site video clip is 0.72, then the network performance index is 0.83×0.72 = 0.5976. This network performance index characterizes the abnormal behavior analysis accuracy of the abnormal behavior analysis network of the construction site. If this index is low, it means that there is room for improvement in the network in analyzing whether the construction site video clip matches the predefined abnormal behavior path. The server may adjust and optimize the abnormal behavior analysis network of the construction site according to this index, such as adjusting the parameters of the network, adding training data or improving the algorithm, so as to improve the performance of the network so that it can more accurately identify abnormal behaviors in the construction site.

[0120] Figure 2 The hardware structure of the AI-based construction site personnel behavior analysis system 100 for implementing the above-mentioned AI-based construction site personnel behavior analysis method provided by an embodiment of the present invention is shown as follows: Figure 2 As shown, the AI-based construction site personnel behavior analysis system 100 may include a processor 110 , a machine-readable storage medium 120 , a bus 130 , and a communication unit 140 .

[0121] The machine-readable storage medium 120 may store data and / or instructions. In some embodiments, the machine-readable storage medium 120 may store data acquired from an external terminal. In some embodiments, the machine-readable storage medium 120 may store data and / or instructions that the AI-based construction site personnel behavior analysis system 100 uses to execute or use to complete the exemplary method described in the present invention.

[0122] During the specific implementation process, one or more processors 110 execute computer executable instructions stored in the machine-readable storage medium 120, so that the processor 110 can execute the AI-based construction site personnel behavior analysis method of the above method embodiment. The processor 110, the machine-readable storage medium 120 and the communication unit 140 are connected via the bus 130, and the processor 110 can be used to control the sending and receiving actions of the communication unit 140.

[0123] The specific implementation process of the processor 110 can refer to the various method embodiments executed by the above-mentioned AI-based construction site personnel behavior analysis system 100. The implementation principles and technical effects are similar, and this embodiment will not be repeated here.

[0124] In addition, an embodiment of the present invention also provides a readable storage medium, in which computer executable instructions are preset. When the processor executes the computer executable instructions, the above-mentioned AI-based construction site personnel behavior analysis method is implemented.

[0125] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, various features are sometimes combined into one embodiment, drawing or description thereof.

Claims

1. A construction site personnel behavior analysis method based on AI, characterized in that: The method comprises: Obtaining a sample construction site video clip sequence of a construction site abnormal behavior analysis network, wherein the sample construction site video clip sequence includes a plurality of supervised sample construction site video clips and a plurality of unsupervised sample construction site video clips, wherein the supervised sample construction site video clips refer to surveillance video clips carrying predefined abnormal behavior paths, and the unsupervised sample construction site video clips refer to surveillance video clips not carrying predefined abnormal behavior paths; Outputting first confidences corresponding to respective sample construction site video clips in the sample construction site video clip sequence by using an artificial intelligence network model, and extracting surveillance video clips whose first confidences meet first set requirements from the sample construction site video clip sequence as matching sample construction site video clips matching predefined abnormal behavior paths, wherein the first confidences represent confidences that the sample construction site video clips match the predefined abnormal behavior paths; Generate a first weight parameter based on the proportion of the amount of the supervised sample construction site video clip in the amount of the matched sample construction site video clip, and output a second confidence corresponding to each of the sample construction site video clips through the construction site abnormal behavior analysis network, wherein the second confidence represents the confidence that the sample construction site video clip is the supervised sample construction site video clip; Based on the first weight parameter and the second confidence corresponding to each of the sample construction site video clips, outputting a third confidence corresponding to each of the sample construction site video clips, wherein the third confidence represents the confidence that the sample construction site video clip matches the predefined abnormal behavior path; Based on the third confidence level corresponding to each of the sample construction site video clips, a network performance indicator of the construction site abnormal behavior analysis network is calculated and output, and the construction site abnormal behavior analysis network is trained based on the network performance indicator to generate a target construction site abnormal behavior analysis network, so as to analyze whether the construction site video clip matches the predefined abnormal behavior path based on the target construction site abnormal behavior analysis network, and the network performance indicator characterizes the abnormal behavior analysis accuracy of the construction site abnormal behavior analysis network.

2. The AI-based construction site personnel behavior analysis method according to claim 1 is characterized in that: The step of outputting a third confidence level corresponding to each of the sample construction site video clips based on the first weight parameter and the second confidence level corresponding to each of the sample construction site video clips comprises: The second confidence scores corresponding to each of the sample construction site video clips are divided by the first weight parameter, and the third confidence scores corresponding to each of the sample construction site video clips are output.

3. The AI-based construction site personnel behavior analysis method according to claim 1 is characterized in that: The first setting requirement includes that a first confidence level corresponding to the sample construction site video clip is greater than a first threshold value; The step of extracting the surveillance video segment whose first confidence meets the first set requirement from the sample construction site video segment sequence as the matching sample construction site video segment that matches the predefined abnormal behavior path includes: A surveillance video segment corresponding to the sample construction site video segment and having a first confidence level greater than a first threshold value is extracted from the sequence of sample construction site video segments as the matching sample construction site video segment.

4. The AI-based construction site personnel behavior analysis method according to claim 1, characterized in that: The sample construction site video clip includes video semantic tracking features and behavior state transition parameters of construction site personnel, the video semantic tracking features represent the behavior pattern vectors of the construction site personnel and the behavior pattern vectors of the construction site personnel switching construction behaviors, and the behavior state transition parameters of the construction site personnel represent the behavior state transition vectors of the construction site personnel switching construction behaviors; Wherein, the method further comprises: Based on the initial state characteristics of the construction site personnel, the behavior state transfer parameters of the construction service node in the a-th video surveillance tracking stage and the first state migration array, the behavior state transfer parameters of the construction site personnel in the a-th video surveillance tracking stage are output, and the first state migration array reflects the construction linkage parameters of the construction service node switching the construction behavior of the construction site personnel in the a-th video surveillance tracking stage; wherein a is a positive integer iterating from 1; Based on the initial state characteristics of the construction service node, the behavior state transfer parameters of the construction service node in the a-th video surveillance tracking stage, the behavior state transfer parameters of the construction site personnel in the a-th video surveillance tracking stage, the second state migration array and the third state migration array, the behavior state transfer parameters of the construction service node in the a+1-th video surveillance tracking stage are generated; wherein the second state migration array reflects the construction linkage parameters of the construction site personnel switching the construction behavior of the construction service node in the a-th video surveillance tracking stage, and the third state migration array reflects the synchronization linkage parameters of the construction service node synchronizing the construction site personnel switching the construction behavior to other construction service nodes in the a-th video surveillance tracking stage; When the number of video surveillance tracking stages meets the training convergence requirement, the behavior state transfer parameters of the construction site personnel generated in the last video surveillance tracking stage are loaded into the sample construction site video clip.

5. The AI-based construction site personnel behavior analysis method according to claim 4 is characterized in that: The method further comprises: Based on the collaborative monitoring operation data of the construction service nodes, the construction service nodes that do not meet the video monitoring tracking conditions are determined, and the construction service nodes that do not meet the video monitoring tracking conditions do not participate in the video monitoring tracking process of the behavior state transfer parameters of the construction site personnel.

6. The AI-based construction site personnel behavior analysis method according to claim 4 is characterized in that: The method further comprises: Based on the construction relevance of the construction site personnel and the number of construction service nodes associated with the construction site personnel, determining the maximum number of associated construction service nodes matched with the construction site personnel; wherein the number of associated construction service nodes of the construction site personnel represents the number of construction service nodes having a collaborative relationship with the construction site personnel; Based on the maximum number of associated construction service nodes matched by the construction site personnel, obtaining switching characteristic parameters of the construction service node for switching the construction behavior of the construction site personnel and switching characteristic parameters of the construction site personnel for switching the construction behavior of the construction service node; The first state transition array is generated based on the switching characteristic parameters of the construction service node for the construction site personnel switching the construction behavior, and the second state transition array is generated based on the switching characteristic parameters of the construction site personnel switching the construction behavior of the construction service node.

7. The AI-based construction site personnel behavior analysis method according to claim 4 is characterized in that: The method further comprises: Decomposing the first state transition array, the second state transition array and the third state transition array respectively to generate a plurality of first state transition array units, a plurality of second state transition array units and a plurality of third state transition array units; Wherein, the behavior state transfer sub-parameters of the construction site personnel in the a-th video monitoring and tracking stage respectively generated by the multiple first state transition array units are spliced ​​to output the behavior state transfer parameters of the construction site personnel in the a-th video monitoring and tracking stage; The behavior state transfer sub-parameters of the construction service node in the a+1th video surveillance tracking stage generated by the multiple second state migration array units and the multiple third state migration array units are spliced ​​to output the behavior state transfer parameters of the construction service node in the a+1th video surveillance tracking stage.

8. The AI-based construction site personnel behavior analysis method according to claim 1, characterized in that: Outputting the second confidence corresponding to each of the sample construction site video clips through the construction site abnormal behavior analysis network includes: Based on the sample construction site video clip sequence, generating a plurality of sample construction site video clip subsequences, wherein the sample construction site video clip subsequences include the same number of supervised sample construction site video clips and unsupervised sample construction site video clips; Outputting, by the construction site abnormal behavior analysis network, initial second confidences corresponding to each of the sample construction site video clips included in the sample construction site video clip subsequence; Based on the average confidence of the initial second confidences corresponding to the respective sample construction site video clips, the second confidences corresponding to the respective sample construction site video clips are output.

9. The AI-based construction site personnel behavior analysis method according to claim 1, characterized in that: The calculating and outputting the network performance index of the abnormal behavior analysis network of the construction site based on the third confidence corresponding to each of the sample construction site video clips includes: Based on the third confidences corresponding to the respective sample construction site video clips, a hit coefficient of the construction site abnormal behavior analysis network is calculated and outputted, wherein the hit coefficient represents the confidence of the surveillance video clips corresponding to the predefined abnormal behavior paths in the plurality of supervised sample construction site video clips; Outputting the confidence of the matching sample construction site video clip in the sequence of sample construction site video clips based on the third confidence corresponding to each of the sample construction site video clips; Based on the confidence of matching the sample construction site video clips in the sample construction site video clip sequence and the hit coefficient, a network performance index of the construction site abnormal behavior analysis network is calculated.

10. An AI-based construction site personnel behavior analysis system, characterized in that: The AI-based construction site personnel behavior analysis system includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the AI-based construction site personnel behavior analysis method described in any one of claims 1 to 9 above.

Images