Construction personnel operation behavior monitoring method and device based on image recognition

Abstract

The invention discloses a method and a device for monitoring operation behaviors of constructors based on image recognition, and relates to the field of construction monitoring. In the method, a first image including a constructor is analyzed to determine a current light condition, a target exposure parameter is calculated according to the current light condition, a target camera is adjusted according to the target exposure parameter, and a video stream is acquired through the adjusted target camera; determining a target constructor in the video stream, and tracking the target constructor in combination with contextual information of the video stream and continuity of front and back image frames; acquiring spatial features of a target constructor in each image frame in the video stream, and acquiring time features of the target constructor between different image frames in the video stream; and comparing the spatial feature and the time feature with a preset feature to determine whether the operation behavior of the target constructor meets the requirement or not. By implementing the technical scheme provided by the invention, the safety management level of constructors can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of construction monitoring, and in particular to a method and device for monitoring the working behavior of construction personnel based on image recognition. Background Art

[0002] With the development of the construction industry and the advancement of technology, it becomes increasingly important to ensure that construction workers comply with safe working standards on complex and ever-changing construction sites.

[0003] Existing safety management measures mainly rely on manual inspection and monitoring. Although these methods can reduce the occurrence of accidents to a certain extent, they still have many limitations and shortcomings in actual operation. Manual monitoring is inefficient, difficult to cover large areas of construction sites, and prone to missed inspections or misjudgments due to fatigue.

[0004] Therefore, a more intelligent and reliable method is urgently needed to solve this problem. Summary of the invention

[0005] The present application provides a method and device for monitoring the working behavior of construction personnel based on image recognition. Intelligent monitoring of the working behavior of construction personnel is realized through image recognition technology, which can not only reduce the burden of manual monitoring and improve monitoring efficiency, but also improve the safety management level of the construction site and reduce the risk of accidents.

[0006] In a first aspect of the present application, a method for monitoring the working behavior of construction personnel based on image recognition is provided, which is applied to a behavior monitoring platform. The method comprises: In response to a target camera capturing a first image including a construction worker, analyzing the first image to determine a current light condition, calculating a target exposure parameter according to the current light condition, adjusting the target camera according to the target exposure parameter, and acquiring a video stream through the adjusted target camera; Determine a target construction worker in the video stream, and track the target construction worker in combination with context information of the video stream and continuity of previous and next image frames; Acquire spatial features of the target construction worker in each image frame in the video stream, and acquire temporal features of the target construction worker between different image frames in the video stream, wherein the spatial features include clothing, actions, and facial expressions, and the temporal features include motion trajectories; The spatial feature and the temporal feature are compared with preset features to determine whether the working behavior of the target construction personnel meets the requirements.

[0007] Optionally, comparing the spatial feature and the temporal feature with preset features to determine whether the work behavior of the target construction personnel meets the requirements includes: Acquire voice and ambient sound of a target area, and analyze the voice to determine a first keyword, wherein the target area is an area where the target construction personnel are located; Constructing a spatiotemporal correlation map based on the first keyword and the image frames in the video stream to determine a first probability of the violation occurring, and adjusting the first probability in combination with the ambient sound to obtain a second probability; When the second probability is greater than a threshold, it is prompted that the working behavior of the target construction worker is abnormal.

[0008] Optionally, constructing a spatiotemporal association graph according to the first keyword and the image frames in the video stream to determine a first probability of the occurrence of the violation includes: Acquire a keyword set of construction specifications or safety requirements from the first keywords; Performing time stamp matching on the keyword set and the image frame in the video stream to establish a time association between a second keyword in the keyword set and the image frame; Compare the behavior of the target construction worker in the image frame with the corresponding second keyword in the keyword set to construct a three-dimensional association map of behavior-time-space; Based on the three-dimensional association map, the degree of deviation of the target construction personnel from the standard behavior within a preset time period is calculated to determine the first probability of the violation occurring.

[0009] Optionally, adjusting the first probability in combination with the ambient sound to obtain a second probability includes: Performing spectrum analysis and voiceprint recognition on the ambient sound, extracting characteristic sounds related to construction activities, comparing the characteristic sounds with a preset sound feature library, and identifying abnormal sound events; Determining an adjustment factor of the probability according to the frequency of occurrence, duration, and synchronization of the abnormal sound event with the behavior of the target construction personnel; The first probability is adjusted according to the adjustment factor to obtain a second probability.

[0010] Optionally, the tracking of the target construction worker in combination with the context information of the video stream and the continuity of previous and next frames includes: Identify and locate the target construction worker in the initial image frame of the video stream and generate a tracking frame; In subsequent image frames of the video stream, the target construction worker is tracked according to the tracking frame position of the previous image frame, combined with first context information and the continuity of the appearance features of the target construction worker in the previous and next frames, wherein the first context information includes background conditions in the image frame and the relative position of the tracking frame in the image frame; If the target construction worker cannot be tracked in multiple consecutive image frames, target re-detection is performed based on the second context information to re-lock the target construction worker. The second context information includes the movement path prediction of the target construction worker and the correlation with surrounding construction workers or objects.

[0011] Optionally, the acquiring of the spatial features of the target construction worker in each image frame in the video stream, and the acquiring of the temporal features of the target construction worker between different image frames in the video stream includes: Extracting features of the target construction workers in the image frame to identify and quantify wearing features, including the wearing of helmets, the color and logo of work clothes, and whether personal protective equipment is complete; Analyzing the motion characteristics of the target construction worker by a posture estimation algorithm, wherein the motion characteristics include body posture, hand gestures, and body tilt angles; Using facial expression recognition technology to identify the facial expression of the target construction worker, and determining the change of emotional state according to the facial expression; The motion trajectory of the target construction worker in the continuous image frames is analyzed, and the displacement, velocity and acceleration of the target construction worker in the three-dimensional space are calculated according to the motion trajectory.

[0012] Optionally, comparing the spatial feature and the temporal feature with preset features to determine whether the work behavior of the target construction personnel meets the requirements includes: Determine the behavior classification result of the target construction personnel according to the spatial characteristics and the temporal characteristics; Comparing the behavior classification result with a preset unsafe operation behavior standard library, wherein the preset unsafe operation behavior standard library includes operation behavior features that do not meet safety specifications and corresponding classification labels; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is higher than or equal to a preset threshold, it is determined that the operation behavior of the target construction worker does not meet the requirements; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is lower than the preset threshold, it is determined that the operation behavior of the target construction personnel meets the requirements.

[0013] In a second aspect of the present application, a monitoring system for construction workers' work behavior based on image recognition is provided, comprising an image module, a tracking module, a feature module and an execution module, wherein: An image module is configured to analyze a first image including a construction worker in response to a target camera capturing the first image, determine a current light condition, calculate a target exposure parameter according to the current light condition, adjust the target camera according to the target exposure parameter, and obtain a video stream through the adjusted target camera; A tracking module configured to determine a target construction worker in the video stream, and track the target construction worker in combination with context information of the video stream and continuity of previous and next image frames; A feature module configured to obtain spatial features of the target construction worker in each image frame in the video stream, and to obtain temporal features of the target construction worker between different image frames in the video stream, wherein the spatial features include clothing, actions and facial expressions, and the temporal features include motion trajectories; The execution module is configured to compare the spatial feature and the temporal feature with preset features to determine whether the working behavior of the target construction personnel meets the requirements.

[0014] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Analyze the current light conditions through the images collected by the target camera, and calculate the target exposure parameters accordingly to adjust the camera. This light adaptive adjustment can ensure that clear and stable video streams can be obtained in different lighting environments, providing reliable basic data for subsequent behavior analysis; 2. By combining the contextual information of the video stream and the continuity of the previous and next image frames, it is possible to accurately track the target construction workers. This helps to continuously and stably focus on the behavior of specific construction workers in complex construction site environments, and improve the accuracy and continuity of behavior monitoring; 3. Not only the spatial features of the target construction workers (such as clothing, movements and facial expressions) are extracted, but also the temporal features (such as movement trajectory). This comprehensive feature extraction can provide a deeper understanding of the construction workers' behavior patterns and provide richer information for behavior analysis; 4. By comparing the extracted spatial and temporal features with the preset features, it is possible to efficiently determine whether the target construction personnel's work behavior meets the requirements. This helps to promptly discover and correct irregular behaviors and improve the safety and efficiency of the construction site. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a method for monitoring the working behavior of construction personnel based on image recognition disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of a monitoring system for construction personnel's working behavior based on image recognition disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0018] Explanation of the reference numerals: 201, image module; 202, tracking module; 203, feature module; 204, execution module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0019] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0021] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0022] This embodiment discloses a method for monitoring the working behavior of construction personnel based on image recognition, which is applied to a behavior monitoring platform. Figure 1 is a flow chart of a method for monitoring construction personnel's work behavior based on image recognition disclosed in an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: S110, in response to a target camera capturing a first image including a construction worker, analyzing the first image to determine a current light condition, calculating a target exposure parameter according to the current light condition, adjusting the target camera according to the target exposure parameter, and acquiring a video stream through the adjusted target camera; A target camera is a camera used to monitor a specific area, such as a construction site. It captures images at preset intervals. The first image is the first image captured by the target camera that contains construction personnel. This image serves as the starting point for subsequent analysis. The first image is analyzed using an image processing algorithm or a deep learning model. By analyzing parameters such as image brightness, contrast, color saturation, and possible shadows and reflections, the system is able to infer the current light conditions. This includes determining whether it is daytime, nighttime, cloudy, sunny, or other specific lighting conditions. Exposure parameters include shutter speed, aperture size, and sensitivity, which together determine the brightness of the image captured by the camera. Based on the analyzed light conditions, the system calculates the optimal exposure parameter combination to ensure that the image is neither overexposed nor overly dark while maintaining sufficient detail and contrast. The target camera is adjusted according to the calculated target exposure parameters. This process is usually real-time or near real-time to ensure that the camera can quickly adapt to changes in light conditions. After the target camera is adjusted, it starts capturing images with the new exposure parameters and outputs these images in the form of a continuous video stream. Since the camera has been optimized according to the current light conditions, the output video stream will be improved in brightness, contrast and clarity, thereby improving the monitoring effect.

[0023] S120, determining a target construction worker in the video stream, and tracking the target construction worker in combination with context information of the video stream and continuity of previous and next image frames; The initial image of the target construction worker is searched and identified in the video stream. This is usually achieved through image matching, face recognition or other biometric technologies. When the initial image of the target construction worker is found, the system records its location and other relevant features. Contextual information includes other information related to the target construction worker in the video stream, such as the background environment, the location and movement of other construction workers, the status of construction equipment, etc. This information helps the system understand the behavior and location of the target construction worker more accurately. The video stream consists of a series of continuous image frames. These image frames are continuous in time, so there is a certain correlation and similarity between them. The system can use this continuity to predict the location and movement of the target construction worker in the next frame. When the target construction worker is determined, the system uses one or more tracking algorithms to continuously track its location and movement in the video stream. These algorithms may include Kalman filtering, optical flow method, particle filtering, etc. During the tracking process, the system updates the location and feature information of the target construction worker in real time. This helps the system to continuously and accurately track the target construction worker and maintain tracking continuity even when he / she moves or changes his / her movements. If the target construction worker suddenly disappears or behaves abnormally (such as leaving the designated area, staying still for a long time, etc.), the system can trigger an alarm or notify relevant personnel.

[0024] Optionally, the tracking of the target construction worker in combination with the context information of the video stream and the continuity of previous and next frames includes: Identify and locate the target construction worker in the initial image frame of the video stream and generate a tracking frame; In subsequent image frames of the video stream, the target construction worker is tracked according to the tracking frame position of the previous image frame, combined with first context information and continuity of appearance features of the target construction worker in previous and next frames, wherein the first context information includes background conditions in the image frame and a relative position of the tracking frame in the image frame; If the target construction worker cannot be tracked in multiple consecutive image frames, target re-detection is performed based on the second context information to re-lock the target construction worker. The second context information includes the movement path prediction of the target construction worker and the correlation with surrounding construction workers or objects.

[0025] The initial image frame is the first image frame in the video stream that starts tracking the target construction worker. The system uses image recognition algorithms (such as face detection, object detection, etc.) to identify and locate the target construction worker in the initial image frame. This usually involves matching and analyzing the appearance features of the target construction worker (such as color, shape, texture, etc.). When the target construction worker is identified and located, the system generates a tracking frame around it. This tracking frame is used to mark the position of the target construction worker in the image frame and provide a reference for the subsequent tracking process. The position of the tracking frame in the previous image frame is an important reference point in the tracking process, which provides the position information of the target construction worker in the previous frame. The first context information includes the background conditions in the image frame and the relative position of the tracking frame in the image frame. The background conditions may include light, shadows, occlusions, etc., all of which may affect the appearance features of the target construction worker. The relative position refers to the position of the tracking frame relative to the center or edge of the image frame, which helps the system understand the spatial distribution of the target construction worker in the image frame. The continuity of the appearance features refers to the stability and consistency of the appearance features of the target construction worker between the previous and next frames. The system tracks the target worker's position by comparing the similarity of their appearance features (such as color, shape, texture, etc.) in the previous and next frames. Failure to track for multiple consecutive image frames: This may be due to reasons such as the target worker being blocked, moving out of the field of view, or undergoing drastic changes. Second context information: This includes the prediction of the target worker's movement path, the relevance of surrounding workers or objects, etc. The movement path prediction estimates the possible location of the target worker based on the movement trajectory of the target worker in the previous frame. The relevance of surrounding workers or objects refers to the spatial or temporal relationship between the target worker and other workers or objects, which may help the system find it again after losing the target. When the system fails to track the target worker for multiple consecutive frames, it uses the second context information to re-detect the target. This usually involves searching for the appearance features of the target worker in a larger range or more complex background, and using more advanced matching algorithms to identify it.

[0026] By identifying and locating the target construction worker in the initial image frame of the video stream and generating a tracking frame, a benchmark is provided for tracking in subsequent frames. In subsequent image frames, tracking is not only based on the tracking frame position of the previous frame, but also combined with the first context information (such as background conditions and the relative position of the tracking frame in the image frame) and the continuity of the appearance features of the target construction worker, which helps to maintain the accuracy of tracking in complex or changing scenes. When the target construction worker cannot be tracked in multiple consecutive image frames, the method can re-detect the target based on the second context information (such as the prediction of the target construction worker's movement path, the association of surrounding construction workers or objects) to re-lock the target. This strategy increases the robustness of tracking, especially when the target is blocked, out of sight, or in violent motion. By combining context information and the continuity of previous and next frames for tracking, the information in the video stream can be more effectively utilized, reducing unnecessary calculations and resource consumption. The application of the target re-detection strategy also avoids unnecessary searches when the target cannot be tracked for a long time, improving the overall efficiency.

[0027] S130, obtaining spatial features of the target construction worker in each image frame in the video stream, and obtaining temporal features of the target construction worker between different image frames in the video stream, wherein the spatial features include clothing, actions, and facial expressions, and the temporal features include motion trajectories; Spatial features refer to the features displayed by the target construction workers in a single image frame, which mainly reflect the appearance and behavior of the construction workers. The system analyzes the target construction workers' clothing in each image frame, including the color and style of the clothing, whether they wear safety equipment such as helmets and reflective vests. This information helps to identify the identity, responsibilities and compliance with safety regulations of the construction workers. The system captures the action features of the target construction workers, such as gestures, body postures, walking or working postures, etc. These action features can reflect the activity status, work efficiency and potential safety hazards of the construction workers. By analyzing the facial expressions of the target construction workers, the system can judge their emotional state, such as whether they are tired, nervous or focused. This is of great significance for evaluating the psychological state of the construction workers and preventing safety accidents caused by emotional problems. Temporal features refer to the features displayed by the target construction workers between consecutive image frames, which mainly reflect the movement trajectory and behavior pattern of the construction workers. The system tracks the movement trajectory of the target construction workers in the video stream, including the direction, speed, path, etc. of their movement. This information helps to understand the scope of the construction workers' activities, movement patterns and whether they follow the predetermined construction process. By analyzing the changes in the target construction workers’ behaviors between consecutive image frames, the system can identify their behavior patterns, such as periodic activities, repetitive actions, etc. These behavior patterns help predict the future behavior of construction workers, so that safety measures can be taken in advance.

[0028] Optionally, the acquiring of the spatial features of the target construction worker in each image frame in the video stream, and the acquiring of the temporal features of the target construction worker between different image frames in the video stream includes: Extracting features of the target construction workers in the image frame to identify and quantify wearing features, including the wearing of helmets, the color and logo of work clothes, and whether personal protective equipment is complete; Analyzing the motion characteristics of the target construction worker by a posture estimation algorithm, wherein the motion characteristics include body posture, hand gestures, and body tilt angles; Using facial expression recognition technology to identify the facial expression of the target construction worker, and determining the change of emotional state according to the facial expression; The motion trajectory of the target construction worker in the continuous image frames is analyzed, and the displacement, velocity and acceleration of the target construction worker in the three-dimensional space are calculated according to the motion trajectory.

[0029] The system uses image recognition technology to check whether the target construction worker is wearing a helmet, and may further determine whether the helmet is worn correctly (such as whether it is fastened tightly, whether it blocks the view, etc.). The system recognizes the color and possible identification of the work clothes (such as company name, position identification, etc.), which helps to distinguish construction workers with different identities or responsibilities. In addition to the helmet, the system also checks whether other personal protective equipment, such as safety ropes, safety shoes, goggles, reflective vests, etc., are complete to ensure the safety of the construction workers. The system uses posture estimation algorithms (such as deep learning models) to analyze the body posture, gestures, and body tilt angles of the target construction workers. These features can reflect the activity status and work efficiency of the construction workers. Based on the results of posture estimation, the system can further identify specific actions, such as carrying heavy objects and using tools, which is of great significance for evaluating whether the construction workers' behavior complies with safety regulations. The system applies facial expression recognition technology to capture and analyze the facial expressions of the target construction workers. This usually involves using deep learning models to identify facial expression features, such as smiling, frowning, closing eyes, etc. Based on the results of facial expression recognition, the system can infer the emotional state of the construction workers, such as happiness, fatigue, tension, etc. This helps to detect emotional problems of construction workers in a timely manner and take appropriate measures. The system constructs the motion trajectory of the target construction worker by analyzing the position changes of the target construction worker in consecutive image frames. This usually involves using image registration and tracking algorithms to track the movement of the target construction worker in the video stream. Based on the analysis of the motion trajectory, the system can calculate the displacement, velocity and acceleration of the target construction worker in three-dimensional space. These parameters help to evaluate the movement patterns and work efficiency of construction workers and may reveal potential safety hazards.

[0030] By identifying the wearing characteristics of safety helmets, the color and logo of work clothes, and whether personal protective equipment is complete, the system can accurately determine whether the construction workers have complied with safety regulations and dress requirements, thereby improving the accuracy of monitoring. Using the posture estimation algorithm, the system can analyze the construction workers' body posture, gestures, body tilt angles and other motion characteristics in real time, which is of great significance for identifying unsafe behaviors and improving work efficiency. Through facial expression recognition technology, the system can judge the emotional state of construction workers, such as fatigue, tension or pleasure, which helps to timely discover potential safety hazards and improve the intelligent level of monitoring. By analyzing the motion trajectory of construction workers in continuous image frames, the system can calculate the motion parameters of construction workers in three-dimensional space, such as displacement, velocity and acceleration, which has a significant effect on monitoring the range of activities of construction workers, preventing collision accidents, and optimizing construction processes. Combining multi-dimensional information such as wearing characteristics, motion characteristics, facial expressions and motion trajectories, the system can comprehensively evaluate the behavior of construction workers, timely discover and warn potential safety risks, thereby effectively enhancing the safety management of construction sites. By analyzing the movement trajectories and action characteristics of construction workers, the system can identify the behavior patterns of construction workers, such as periodic activities, repetitive actions, etc., which helps optimize the construction process, reduce unnecessary waiting and idle time, and improve work efficiency. The system can visualize the extracted feature data, such as generating construction worker activity heat maps, behavior pattern analysis charts, etc., to provide intuitive and comprehensive data support for construction management personnel, helping to make more scientific decisions.

[0031] S140: Compare the spatial feature and the temporal feature with preset features to determine whether the working behavior of the target construction personnel meets the requirements.

[0032] Spatial features include the expected wearing features of the target construction workers (such as the correct wearing of hard hats, the legal color and logo of work clothes, and the integrity of personal protective equipment), action features (such as correct operating posture, avoiding dangerous actions), and facial expressions (such as staying focused and avoiding signs of fatigue). Temporal features involve the time rules that the target construction workers should follow during the operation, such as the expected movement trajectory (such as moving along the predetermined route), speed control (such as avoiding too fast or too slow operations), and acceleration changes (such as smooth and safe action execution). The system continuously extracts the spatial and temporal features of the target construction workers from the video stream. The extracted features are compared one by one with the preset features. This may involve complex algorithms, such as machine learning models, which are trained to identify and quantify the differences between features and preset standards. In order to determine whether the behavior meets the requirements, one or more thresholds need to be set for each feature. For example, for the wearing of hard hats, a clear boundary between "wearing" and "not wearing" can be set; for action features, a scoring range of "correct" and "incorrect" can be set. Based on the results of feature comparison, the system can determine whether the behavior of the target construction workers is compliant. If most or key features are within the preset range, the behavior is considered to be in compliance with the requirements; otherwise, further review or intervention may be required. When it is found that the behavior does not meet the requirements, the system can immediately send warnings or prompts to construction personnel or managers so that the wrong behavior can be corrected in time or preventive measures can be taken. As the operating environment and requirements change, the preset features may need to be updated or adjusted regularly to ensure the accuracy and effectiveness of the assessment. The comparison algorithm also needs to be continuously optimized to improve recognition accuracy and response speed and reduce false positives and negatives.

[0033] Optionally, comparing the spatial feature and the temporal feature with preset features to determine whether the work behavior of the target construction personnel meets the requirements includes: Acquire voice and ambient sound of a target area, and analyze the voice to determine a first keyword, wherein the target area is an area where the target construction personnel are located; Constructing a spatiotemporal correlation map based on the first keyword and the image frames in the video stream to determine a first probability of the violation occurring, and adjusting the first probability in combination with the ambient sound to obtain a second probability; When the second probability is greater than a threshold, it is prompted that the working behavior of the target construction worker is abnormal.

[0034] The sound can be used to further verify whether the target construction worker's work behavior meets the requirements. The voice and ambient sound of the target construction worker's area are obtained through a microphone or other audio acquisition equipment. These sound information are of great significance for understanding the on-site situation and identifying potential risks. The collected voice is analyzed to identify key information or keywords. These keywords may be directly related to the construction worker's work behavior, such as specific instructions, warnings or conversation content. The recognition of the first keyword provides an important clue for the subsequent construction of the spatiotemporal association map. Based on the identified first keyword and the image frame information in the video stream, a spatiotemporal association map is constructed. This map not only contains spatial features such as the location and action of the construction worker, but also incorporates the continuity in time and the association between voice information and image frames. In this way, the system can understand the construction worker's work behavior more comprehensively. Based on the spatiotemporal association map, the system can preliminarily determine whether the construction worker's behavior is likely to violate the rules and give a preliminary probability value (first probability). This probability value reflects the possibility of the violation based on the current known information. In addition to voice information, the system also considers the impact of ambient sound on the judgment of violation behavior. For example, noisy ambient sound may mean chaos or potential danger at the construction site; while a quiet environment may indicate a pause in construction activities or orderly progress. The system will further adjust the first probability according to the characteristics of the ambient sound to obtain a more accurate second probability. The calculated second probability is compared with a preset threshold. If the second probability is greater than the threshold, it means that the construction workers' work behavior is abnormal or has potential risks. The system will issue a prompt or warning so that managers or construction workers can take timely measures to intervene or correct it.

[0035] By acquiring and analyzing the speech and ambient sound in the target area, the verbal instructions, communication content, and possible environmental interference of the construction workers during operation can be captured. This fusion of multimodal data enables the system to have a more comprehensive understanding of the actual situation at the construction site. Combined with the image frames in the video stream, the system can construct a spatiotemporal correlation map that can show the dynamic relationship between construction workers, tools, equipment, and the environment, so as to more accurately determine whether there is a violation. The system can preliminarily determine the first probability of the violation based on the first keyword (i.e., key information related to the construction behavior) and the image frames in the video stream. This reflects the system's preliminary understanding and judgment ability of complex construction scenes. Furthermore, the system will adjust the first probability in combination with the ambient sound to obtain a more accurate second probability. This adjustment mechanism takes into account the actual situation at the construction site, such as noise level, interference from other construction activities, etc., thereby improving the accuracy and robustness of the judgment. When the second probability determined by the system is greater than the preset threshold, it will immediately prompt that the target construction worker's operation behavior is abnormal. This instant feedback mechanism helps to correct the violations of construction workers in a timely manner and prevent potential safety hazards. In addition, this instant feedback can also serve as an important tool for construction management and quality control, helping managers better understand the dynamic situation at the construction site and improve overall construction efficiency and quality.

[0036] Optionally, constructing a spatiotemporal association graph according to the first keyword and the image frames in the video stream to determine a first probability of the occurrence of the violation includes: Acquire a keyword set of construction specifications or safety requirements from the first keywords; Performing time stamp matching on the keyword set and the image frame in the video stream to establish a time association between a second keyword in the keyword set and the image frame; Compare the behavior of the target construction worker in the image frame with the corresponding second keyword in the keyword set to construct a three-dimensional association map of behavior-time-space; Based on the three-dimensional association map, the degree of deviation of the target construction personnel from the standard behavior within a preset time period is calculated to determine the first probability of the violation occurring.

[0037] Extract a set of keywords directly related to construction specifications or safety requirements from the first keywords (these keywords are analyzed from the speech in the target area and are usually related to construction activities, instructions or safety requirements). Keywords in the keyword set may include specific construction steps, tool use, safety warnings, etc. Match the keyword set with the image frames in the video stream by time stamp. This means that the system needs to determine the time points at which each keyword appears or should appear in the video stream and align these time points with the timestamps of the image frames. In this way, each second keyword is temporally associated with a specific image frame. With the temporal association, the system can start to build a three-dimensional association map of behavior-time-space. This step involves comparing the behavior of the target construction worker in the image frame with the corresponding second keyword in the keyword set. Specifically, the system needs to analyze the construction worker's actions, position, tools used, and other information in the image frame, and match this information with the construction specifications or safety requirements represented by the second keyword. During the matching process, the system will consider the continuity of the construction worker's behavior, the change of spatial position, and the temporal sequence, so as to build a three-dimensional association map that comprehensively reflects the construction worker's behavior, time and space relationship. In the three-dimensional association map, the time axis represents the time point when the keyword is mentioned, the space axis represents the different locations of the construction site, and the behavior axis represents whether the construction workers have complied with the safety regulations. By marking the actual behavior of the construction workers in this map and comparing it with the standard behavior, the degree of deviation of the construction workers from the standard behavior in a specific time period can be intuitively seen. Based on this three-dimensional association map, the system can calculate the degree of deviation of the target construction workers from the standard behavior in a preset time period. This usually involves quantitative analysis of the information in the map, such as calculating the accuracy of the construction workers' behavior, the rationality of the speed, the compliance of the position, etc. By comparing these quantitative indicators with the preset standard behavior standards, the system can derive a numerical value representing the degree of deviation of the construction workers' behavior. The larger this value is, the greater the deviation of the construction workers' behavior from the standard behavior, and therefore the higher the first probability of the violation. For example, from the first keyword extracted by the speech recognition system, a set of keywords related to the safety regulations for high-altitude operations is screened out, such as "safety belt", "safety net", "high-altitude operation license", "prohibition of working under the influence of alcohol", etc. These keyword sets are matched with the image frames in the video stream by time stamp. For example, when the speech recognition system recognizes the keyword "start high-altitude work", the system records the time point and aligns it with the corresponding image frame in the video stream. The system begins to build a three-dimensional association map of behavior-time-space. The system analyzes the behavior of the target construction personnel in the high-altitude work area, such as whether they wear safety belts, whether they set up safety nets, and whether they hold high-altitude work permits. At the same time, the system also considers the spatial location and temporal sequence of his behavior.For example, if the system detects at a certain point in time that the target construction worker is working at height without wearing a safety belt, the system will record this behavior in the three-dimensional association map and associate it with the keyword "seat belt". Based on the constructed three-dimensional association map, the system calculates the degree of deviation of the target construction worker from the standard behavior within a preset time period. For example, if the system finds that the target construction worker has not worn a safety belt during multiple high-altitude operations, then the degree of deviation of his behavior will be high. Based on this degree of deviation, combined with the preset standard behavior standards, the system calculates the first probability of the violation.

[0038] By matching the keyword set of construction specifications or safety requirements with the image frames in the video stream by time stamp, the system can track and record the behavior of construction workers in real time to ensure that construction activities comply with established safety specifications. With the help of the three-dimensional association map of behavior-time-space, the system can intuitively display the behavior trajectory and status of construction workers during the construction process, which is convenient for managers to promptly discover and correct violations, thereby improving the level of construction safety management. Combining voice recognition and video surveillance technology, the system can capture the behavioral characteristics of construction workers in different time periods and different spatial locations, and compare them with standard behaviors, so as to more accurately determine whether there are violations. The system can process and analyze data in real time, issue alarms or prompts in time, so that managers can respond quickly and take measures to avoid potential safety risks. By analyzing the behavioral data of construction workers, the system can evaluate the efficiency and effectiveness of construction activities, and help managers reasonably allocate construction resources such as personnel, tools and equipment to improve construction efficiency and reduce costs.

[0039] Optionally, adjusting the first probability in combination with the ambient sound to obtain a second probability includes: Performing spectrum analysis and voiceprint recognition on the ambient sound, extracting characteristic sounds related to construction activities, comparing the characteristic sounds with a preset sound feature library, and identifying abnormal sound events; Determining an adjustment factor of the probability according to the frequency of occurrence, duration, and synchronization of the abnormal sound event with the behavior of the target construction personnel; The first probability is adjusted according to the adjustment factor to obtain a second probability.

[0040] The ambient sound of the construction site is collected and spectrally analyzed. Spectral analysis can reveal the distribution of sound at different frequencies and help identify specific sound features. Voiceprint recognition is performed, which is a technology that identifies the source or type of sound by comparing sound features. In the construction scene, voiceprint recognition can be used to identify characteristic sounds related to construction activities, such as the operation sound of mechanical equipment, the conversation sound of construction workers, knocking sounds, etc. The system will compare the identified characteristic sounds with the preset sound feature library. This sound feature library contains a variety of sound features related to construction activities and their association information with the safety of construction behavior. Through comparison, the system can identify which characteristic sounds represent abnormal sound events, such as abnormal operation sounds of mechanical equipment, emergency calls for help from construction workers, etc. When abnormal sound events are identified, the system will determine the probability adjustment factor based on the frequency of occurrence, duration, and synchronization of these events with the behavior of the target construction workers. The determination of the adjustment factor takes into account multiple factors, including the urgency of the abnormal sound events, the degree of impact on construction safety, and their direct correlation with the behavior of the target construction workers. The initial safety assessment probability (i.e., the first probability) is adjusted according to the determined adjustment factor to obtain a more accurate second probability. The second probability reflects a more comprehensive assessment of the safety of construction behavior after considering the characteristic sounds in the ambient sound. For example, when abnormal operating sounds occur, the adjustment factor will be positive, and when the ambient sounds are all normal working sounds, the adjustment factor will be negative.

[0041] Through spectrum analysis and voiceprint recognition technology, the system can accurately extract characteristic sounds related to construction activities and compare them with the preset sound feature library. This comparison can identify abnormal sound events, thereby more accurately evaluating the safety of construction behavior. Ambient sound contains rich information, including the status of construction activities, the emotions of construction workers, and potential safety hazards. Through the analysis of ambient sound, the system can perceive the changes in the construction environment in real time and provide more comprehensive data support for safety assessment. The integration of advanced technologies such as spectrum analysis and voiceprint recognition has promoted the intelligent development of the construction field. In the future, with the continuous advancement of technology and the in-depth promotion of applications, this technology is expected to play an important role in more construction scenarios and provide strong support for intelligent construction. By real-time monitoring and analysis of abnormal sound events in ambient sound, the system can promptly discover and respond to potential safety hazards, thereby reducing the probability of safety accidents. This is of great significance for ensuring the life safety of construction workers and the stable development of enterprises. The system can determine the probability adjustment factor based on information such as the frequency and duration of abnormal sound events and the synchronization with the behavior of target construction workers, and adjust the first probability to obtain the second probability. This quantitative assessment result can provide managers with powerful decision-making support and help them better formulate and implement safety management measures.

[0042] Optionally, comparing the spatial feature and the temporal feature with preset features to determine whether the work behavior of the target construction personnel meets the requirements includes: Determine the behavior classification result of the target construction personnel according to the spatial characteristics and the temporal characteristics; Comparing the behavior classification result with a preset unsafe operation behavior standard library, wherein the preset unsafe operation behavior standard library includes operation behavior features that do not meet safety specifications and corresponding classification labels; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is higher than or equal to a preset threshold, it is determined that the operation behavior of the target construction worker does not meet the requirements; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is lower than the preset threshold, it is determined that the operation behavior of the target construction worker meets the requirements.

[0043] The spatial features (such as location, posture, etc.) and temporal features (such as working time, duration, etc.) of the target construction workers are collected, and the behaviors of the construction workers are classified based on these features using machine learning or deep learning algorithms. This classification result may be one or more labels, representing the specific behavior type of the construction workers. The determined behavior classification results are compared with the preset unsafe operation behavior standard library. This standard library is a database containing the characteristics of operation behaviors that do not meet safety specifications and the corresponding classification labels. Each classification label represents a specific type of unsafe operation behavior, which has been evaluated by experts and is considered to violate safety specifications. During the comparison process, the system calculates the matching degree between the behavior classification results and each classification label in the standard library. The matching degree is usually measured by calculating the similarity or distance between the two, such as using indicators such as cosine similarity and Euclidean distance. The system will set a preset threshold, which represents the minimum matching level that is considered "non-compliant". The setting of this threshold needs to comprehensively consider factors such as safety, false alarm rate and false alarm rate. If the matching degree of the behavior classification result with a certain classification label in the standard library is higher than or equal to the preset threshold, the system will determine that the target construction worker's work behavior does not meet the requirements, that is, there is unsafe work behavior. At this time, the system may generate an alarm message to remind the management staff to pay attention and take corresponding measures. If the matching degree of the behavior classification result with all the classification labels in the standard library is lower than the preset threshold, the system will determine that the target construction worker's work behavior meets the requirements, that is, there is no violation of safety regulations. At this time, the system can continue to monitor or perform other operations. For example, the standard library may contain the classification label "not wearing a safety belt for high-altitude work" and describe the corresponding characteristics in detail, such as the construction worker has no safety belt and the body is at high altitude. If the system detects that the construction worker's behavior matches the classification label "not wearing a safety belt for high-altitude work" in the standard library at a higher than the preset threshold (such as 80%), it is determined that the behavior does not meet the requirements.

[0044] By real-time monitoring and comparing the behavioral characteristics of construction workers with safety regulations, this technical process can timely discover and correct work behaviors that do not meet safety regulations. This helps reduce safety hazards in high-risk areas such as high-altitude operations and construction, and protect the life safety and physical health of construction workers. The preset unsafe work behavior standard library contains a variety of work behavior characteristics that do not meet safety regulations and corresponding classification labels. When the matching degree of the construction worker's behavior classification results and the classification labels in these standard libraries is higher than or equal to the preset threshold, the system will immediately issue an alarm to remind managers and construction workers to pay attention to safety. This instant feedback mechanism helps reduce safety accidents caused by ignoring safety regulations. Through the real-time monitoring and alarm functions of the system, construction workers can always feel the presence of safety supervision, so that they can more consciously abide by safety regulations. This continuous supervision and education helps to enhance the safety awareness of construction workers and form a good atmosphere. It integrates advanced technologies such as spatial features, temporal features and machine learning algorithms, and promotes the intelligent development of the construction field. In the future, with the continuous advancement of technology and the in-depth promotion of applications, this technical process is expected to play an important role in more construction scenarios and provide strong support for intelligent construction.

[0045] This embodiment also discloses a monitoring system for construction workers' working behavior based on image recognition. Figure 2 is a module diagram of a monitoring system for construction personnel's work behavior based on image recognition disclosed in an embodiment of the present application, such as Figure 2 As shown, the system includes an image module 201, a tracking module 202, a feature module 203 and an execution module 204, wherein: An image module 201 is configured to analyze a first image including a construction worker in response to a target camera capturing the first image, determine a current light condition, calculate a target exposure parameter according to the current light condition, adjust the target camera according to the target exposure parameter, and obtain a video stream through the adjusted target camera; A tracking module 202, configured to determine a target construction worker in the video stream, and track the target construction worker in combination with context information of the video stream and continuity of previous and next image frames; The feature module 203 is configured to obtain the spatial features of the target construction worker in each image frame in the video stream, and obtain the temporal features of the target construction worker between different image frames in the video stream, wherein the spatial features include clothing, actions and facial expressions, and the temporal features include motion trajectories; The execution module 204 is configured to compare the spatial feature and the temporal feature with preset features to determine whether the working behavior of the target construction personnel meets the requirements.

[0046] Optionally, the execution module 204 is configured to: Acquire voice and ambient sound of a target area, and analyze the voice to determine a first keyword, wherein the target area is an area where the target construction personnel are located; Constructing a spatiotemporal correlation map based on the first keyword and the image frames in the video stream to determine a first probability of the violation occurring, and adjusting the first probability in combination with the ambient sound to obtain a second probability; When the second probability is greater than a threshold, it is prompted that the working behavior of the target construction worker is abnormal.

[0047] Optionally, the execution module 204 is configured to: Acquire a keyword set of construction specifications or safety requirements from the first keywords; Performing time stamp matching on the keyword set and the image frame in the video stream to establish a time association between a second keyword in the keyword set and the image frame; Compare the behavior of the target construction worker in the image frame with the corresponding second keyword in the keyword set to construct a three-dimensional association map of behavior-time-space; Based on the three-dimensional association map, the degree of deviation of the target construction personnel from the standard behavior within a preset time period is calculated to determine the first probability of the violation occurring.

[0048] Optionally, the execution module 204 is configured to: Performing spectrum analysis and voiceprint recognition on the ambient sound, extracting characteristic sounds related to construction activities, comparing the characteristic sounds with a preset sound feature library, and identifying abnormal sound events; Determining an adjustment factor of the probability according to the frequency of occurrence, duration, and synchronization of the abnormal sound event with the behavior of the target construction personnel; The first probability is adjusted according to the adjustment factor to obtain a second probability.

[0049] Optionally, the tracking module 202 is configured to: Identify and locate the target construction worker in the initial image frame of the video stream and generate a tracking frame; In subsequent image frames of the video stream, the target construction worker is tracked according to the tracking frame position of the previous image frame, combined with first context information and the continuity of the appearance features of the target construction worker in the previous and next frames, wherein the first context information includes background conditions in the image frame and the relative position of the tracking frame in the image frame; If the target construction worker cannot be tracked in multiple consecutive image frames, target re-detection is performed based on the second context information to re-lock the target construction worker. The second context information includes the movement path prediction of the target construction worker and the correlation with surrounding construction workers or objects.

[0050] Optionally, the feature module 203 is configured to: Extracting features of the target construction workers in the image frame to identify and quantify wearing features, including the wearing of helmets, the color and logo of work clothes, and whether personal protective equipment is complete; Analyzing the motion characteristics of the target construction worker by a posture estimation algorithm, wherein the motion characteristics include body posture, hand gestures, and body tilt angles; Using facial expression recognition technology to identify the facial expression of the target construction worker, and determining the change of emotional state according to the facial expression; The motion trajectory of the target construction worker in the continuous image frames is analyzed, and the displacement, velocity and acceleration of the target construction worker in the three-dimensional space are calculated according to the motion trajectory.

[0051] Optionally, the execution module 204 is configured to: Determine the behavior classification result of the target construction personnel according to the spatial characteristics and the temporal characteristics; Comparing the behavior classification result with a preset unsafe operation behavior standard library, wherein the preset unsafe operation behavior standard library includes operation behavior features that do not meet safety specifications and corresponding classification labels; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is higher than or equal to a preset threshold, it is determined that the operation behavior of the target construction worker does not meet the requirements; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is lower than the preset threshold, it is determined that the operation behavior of the target construction worker meets the requirements.

[0052] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0053] This embodiment also discloses an electronic device, referring to Figure 3The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0054] The communication bus 302 is used to realize the connection and communication between these components.

[0055] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0056] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0057] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0058] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be optionally at least one storage device located away from the aforementioned processor 301. As Figure 3 As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a method for monitoring the working behavior of construction personnel based on image recognition.

[0059] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call the application program of the method for monitoring the working behavior of construction personnel based on image recognition stored in the memory 305. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0060] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0061] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0062] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0064] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0065] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory 305 and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0066] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for monitoring construction workers' working behavior based on image recognition, characterized in that: Applied to a behavior monitoring platform, the method comprises: In response to a target camera capturing a first image including a construction worker, analyzing the first image to determine a current light condition, calculating a target exposure parameter according to the current light condition, adjusting the target camera according to the target exposure parameter, and acquiring a video stream through the adjusted target camera; Determine a target construction worker in the video stream, and track the target construction worker in combination with context information of the video stream and continuity of previous and next image frames; Acquire spatial features of the target construction worker in each image frame in the video stream, and acquire temporal features of the target construction worker between different image frames in the video stream, wherein the spatial features include clothing, actions, and facial expressions, and the temporal features include motion trajectories; The spatial feature and the temporal feature are compared with preset features to determine whether the working behavior of the target construction personnel meets the requirements.

2. The method for monitoring construction workers' working behavior based on image recognition according to claim 1 is characterized in that: The step of comparing the spatial feature and the temporal feature with the preset feature to determine whether the operation behavior of the target construction personnel meets the requirements includes: Acquire voice and ambient sound of a target area, and analyze the voice to determine a first keyword, wherein the target area is an area where the target construction personnel are located; Constructing a spatiotemporal correlation map based on the first keyword and the image frames in the video stream to determine a first probability of the violation occurring, and adjusting the first probability in combination with the ambient sound to obtain a second probability; When the second probability is greater than a threshold, it is prompted that the working behavior of the target construction worker is abnormal.

3. The method for monitoring construction workers' working behavior based on image recognition according to claim 2 is characterized in that: The step of constructing a spatiotemporal association graph according to the first keyword and the image frames in the video stream to determine a first probability of a violation occurring includes: Acquire a keyword set of construction specifications or safety requirements from the first keywords; Performing time stamp matching on the keyword set and the image frame in the video stream to establish a time association between a second keyword in the keyword set and the image frame; Compare the behavior of the target construction worker in the image frame with the corresponding second keyword in the keyword set to construct a three-dimensional association map of behavior-time-space; Based on the three-dimensional association map, the degree of deviation of the target construction personnel from the standard behavior within a preset time period is calculated to determine the first probability of the violation occurring.

4. The method for monitoring construction workers' working behavior based on image recognition according to claim 3 is characterized in that: The adjusting the first probability in combination with the ambient sound to obtain a second probability comprises: Performing spectrum analysis and voiceprint recognition on the ambient sound, extracting characteristic sounds related to construction activities, comparing the characteristic sounds with a preset sound feature library, and identifying abnormal sound events; Determining an adjustment factor of the probability according to the frequency of occurrence, duration, and synchronization of the abnormal sound event with the behavior of the target construction personnel; The first probability is adjusted according to the adjustment factor to obtain a second probability.

5. The method for monitoring construction workers' working behavior based on image recognition according to claim 1 is characterized in that: The tracking of the target construction worker by combining the context information of the video stream and the continuity of the previous and next frames includes: Identify and locate the target construction worker in the initial image frame of the video stream and generate a tracking frame; In subsequent image frames of the video stream, the target construction worker is tracked according to the tracking frame position of the previous image frame, combined with first context information and continuity of appearance features of the target construction worker in previous and next frames, wherein the first context information includes background conditions in the image frame and a relative position of the tracking frame in the image frame; If the target construction worker cannot be tracked in multiple consecutive image frames, target re-detection is performed based on the second context information to re-lock the target construction worker. The second context information includes the movement path prediction of the target construction worker and the correlation with surrounding construction workers or objects.

6. The method for monitoring construction workers' working behavior based on image recognition according to claim 1 is characterized in that: The step of obtaining the spatial features of the target construction worker in each image frame in the video stream, and obtaining the temporal features of the target construction worker between different image frames in the video stream includes: Extracting features of the target construction workers in the image frame to identify and quantify wearing features, including the wearing of helmets, the color and logo of work clothes, and whether personal protective equipment is complete; Analyzing the motion characteristics of the target construction worker by a posture estimation algorithm, wherein the motion characteristics include body posture, hand gestures, and body tilt angles; Using facial expression recognition technology to identify the facial expression of the target construction worker, and determining the change of emotional state according to the facial expression; The motion trajectory of the target construction worker in the continuous image frames is analyzed, and the displacement, velocity and acceleration of the target construction worker in the three-dimensional space are calculated according to the motion trajectory.

7. The method for monitoring construction workers' working behavior based on image recognition according to claim 1 is characterized in that: The step of comparing the spatial feature and the temporal feature with the preset feature to determine whether the operation behavior of the target construction personnel meets the requirements includes: Determine the behavior classification result of the target construction personnel according to the spatial characteristics and the temporal characteristics; Comparing the behavior classification result with a preset unsafe operation behavior standard library, wherein the preset unsafe operation behavior standard library includes operation behavior features that do not meet safety specifications and corresponding classification labels; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is higher than or equal to a preset threshold, it is determined that the operation behavior of the target construction worker does not meet the requirements; When the matching degree between the behavior classification result and any classification label in the preset unsafe operation behavior standard library is lower than the preset threshold, it is determined that the operation behavior of the target construction worker meets the requirements.

8. A monitoring system for construction workers' working behavior based on image recognition, characterized in that: It includes image module, tracking module, feature module and execution module, among which: An image module is configured to analyze a first image including a construction worker in response to a target camera capturing the first image, determine a current light condition, calculate a target exposure parameter according to the current light condition, adjust the target camera according to the target exposure parameter, and obtain a video stream through the adjusted target camera; A tracking module configured to determine a target construction worker in the video stream, and track the target construction worker in combination with context information of the video stream and continuity of previous and next image frames; A feature module configured to obtain spatial features of the target construction worker in each image frame in the video stream, and to obtain temporal features of the target construction worker between different image frames in the video stream, wherein the spatial features include clothing, actions and facial expressions, and the temporal features include motion trajectories; The execution module is configured to compare the spatial feature and the temporal feature with preset features to determine whether the working behavior of the target construction personnel meets the requirements.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images