Image-based construction site violation recording method and device

Abstract

The invention provides an image-based construction site violation recording method and device. The method comprises the following steps: acquiring a real-time image of a construction area through a camera, and preprocessing, including denoising, contrast enhancement and size adjustment; using an improved YOLOv5 network to identify hand and head features, positioning a constructor, and extracting position information, state information and action information; based on the extracted information, judging whether the constructors have illegal behaviors or not; if yes, the system controls the camera to capture and store violation images. According to the invention, by optimizing the image acquisition and processing technology and combining with the improved YOLOv5 network, the efficiency and accuracy of violation monitoring on the construction site are significantly improved, and the problems of poor image quality, low recognition precision and low violation judgment accuracy in violation monitoring on the construction site in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of construction site safety management, and in particular to an image-based construction site violation recording method, an image-based construction site violation recording device, a computer-readable storage medium and construction monitoring. Background Art

[0002] Construction site safety management is an important part of protecting the lives of construction workers, avoiding accidents, and ensuring the smooth progress of the project. If violations are not discovered and corrected in a timely manner, serious safety accidents may occur, threatening the lives of construction workers, causing huge property losses and project delays. Traditional construction site safety management methods mainly rely on manual inspections, but this method has many limitations. On the one hand, construction sites are usually wide in scope, with many people and complex operations. Manual inspections are difficult to cover all areas, and the real-time performance is poor, and important information is easy to be missed; on the other hand, manual inspections are inefficient and highly subjective. The judgment standards and experience of different inspectors may vary greatly, resulting in inconsistent inspection results and affecting management effectiveness.

[0003] With the rapid development of computer vision technology, monitoring of construction site violations based on image acquisition and analysis has gradually become possible, providing a more efficient and accurate solution. However, there are still some urgent problems to be solved in the existing technology in terms of image acquisition and violation judgment. For example, the installation position and shooting angle of the camera are often unreasonable, resulting in some key areas that cannot be effectively monitored or the image quality is poor, affecting the accuracy of subsequent personnel and part identification; the image preprocessing technology is not effective, the noise removal is not thorough, and the image contrast is low, which limits the subsequent feature extraction; the accuracy and efficiency of the existing personnel and part recognition algorithms need to be improved urgently, especially when identifying key parts such as the head and hands, the accuracy is low, and it is easy to make misjudgments or missed judgments; the judgment of violations lacks systematicity and accuracy, and the existing methods are difficult to flexibly respond to complex and changing construction scenes, and cannot meet the increasingly stringent safety regulations on construction sites.

[0004] These technical deficiencies limit the comprehensiveness and real-time nature of construction site safety management. There is an urgent need for a more precise and intelligent solution to improve the efficiency and accuracy of violation monitoring. Summary of the invention

[0005] The main purpose of the present application is to provide an image-based construction site violation recording method, an image-based construction site violation recording device, a computer-readable storage medium and construction monitoring, so as to at least solve the problems of poor image quality, low recognition accuracy and low violation judgment accuracy in the prior art of construction site violation monitoring.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, there is provided an image-based construction site violation recording method, comprising: acquiring a real-time image including a preset construction area through a camera to obtain a first target image; preprocessing the first target image to obtain a second target image, wherein the preprocessing includes image denoising, contrast enhancement and image size adjustment; recognizing the second target image based on an improved YOLOv5 network to locate the construction personnel by a bounding box to obtain a third target image, wherein the improved YOLOv5 network is obtained by adding a branch network for recognizing the hand features and head features of the construction personnel for separate extraction on the basis of the YOLOv5 network, and the branch network is connected to the feature extraction layer of the YOLOv5 network; recognizing based on the third target image and the second target image, extracting the location information, state information and action information of the construction personnel, obtaining violation criterion information, and performing violation monitoring according to the violation criterion information to obtain a target monitoring result; when the target monitoring result indicates that there is a violation, controlling the camera to capture a picture to obtain a fourth target image, and storing the fourth target image to the target position.

[0007] Optionally, obtaining a real-time image including construction personnel through a camera to obtain a first target image includes: acquiring an image of the preset construction area through the camera to obtain a fifth target image; and processing the fifth target image through an illumination compensation algorithm to obtain the first target image:

[0008] Among them, I(x, y) is the coordinate value of the fifth target image at the coordinate (x, y), L0 is the preset light intensity value, L(x, y) is the estimated light intensity value of the fifth target image at the coordinate (x, y), and I'(x, y) is the light intensity value of the first target image at the coordinate (x, y).

[0009] Optionally, before obtaining a real-time image including a preset construction area through a camera to obtain a first target image, the method further includes: obtaining device parameters of the camera, and calculating installation position scores of different positions based on the device parameters and the preset construction area: Among them, S j Score the installation location, w i The area weight of the preset construction area, n is the total number of areas in the preset construction area, d ij is the distance between the camera and the center of the preset construction area, θ j is the viewing angle of the camera, r j is the effective shooting radius of the camera, R iThe equivalent radius of the preset construction area; the position corresponding to the maximum value of the installation position score is determined as the target installation position; based on the relative position relationship between the target installation position and the preset construction area, an angle fitness function is calculated to obtain an installation angle score; the angle corresponding to the maximum value of the installation angle score is determined as the target installation posture; the target installation position and the target installation posture are determined as the target installation scheme of the camera.

[0010] Optionally, preprocessing the first target image to obtain a second target image includes: calculating a weight parameter corresponding to the first target image based on the first target image:

[0011] Among them, w d (i,j)

[0012] is the spatial domain weight, w r (i,j) is the range weight, σ d is the spatial domain standard deviation, σ r is placed in the standard deviation, f(i, j) is the pixel value of the pixel point (i, j) in the first target image, and f(x, y) is the pixel value of the central pixel point (x, y) in the first target image; bilateral filtering is performed on the first target image based on the weight parameter to obtain a sixth target image: Wherein, k is the size of the Pubo window, g(i,j) is the pixel value of the pixel point (i, j) in the first target image after filtering; the sixth target image is processed by an adaptive histogram equalization algorithm to obtain a seventh target image: Wherein, s is the pixel value in the seventh target image, r is the pixel value in the sixth target image, H(r) is the grayscale histogram of the sixth target image, L is the total number of grayscale levels of the sixth target image, and c is the total number of pixels in different sub-areas in the sixth target image; the seventh target image is resized by bilinear interpolation to obtain the second target image.

[0013] Optionally, before the second target image is recognized based on the improved YOLOv5 network to perform bounding box positioning on the construction worker to obtain the third target image, the method further includes: acquiring a training image, recognizing the training image through the improved YOLOv5 network to obtain a target recognition result; and calculating a loss function value through a bounding box regression loss function based on the target recognition result:

[0014] Among them, L locis the loss function value, m is the number of people in the image, n is the number of parts (head and hands) of each person, λ coord and λ noobj is the weight coefficient, It is used to characterize whether the jth part of the i-th person has a target, (x i ,y i ,w i ,h i ) are the predicted bounding box coordinates and size, is the real bounding box information, C i is the predicted position confidence, is the true part position confidence; based on the loss function value, the improved YOLOv5 network is optimized until the loss function value is less than a first threshold.

[0015] Optionally, based on the third target image and the second target image, identification is performed, the location information, state information and action information of the construction worker are extracted, and violation criterion information is obtained, and violation monitoring is performed according to the violation criterion information to obtain a target monitoring result, including: based on the third target image and the second target image, identification is performed to extract the head information, hand information and dangerous area information of the construction worker to obtain the location information; based on the head information, the head information is compared with a preset safety helmet template and / or a safety helmet feature library to determine whether the construction worker is wearing a safety helmet, and identification is performed through the hand information and the dangerous area information to determine whether the construction worker's hand is less than a safety distance threshold from the dangerous area to obtain the state information; based on the hand information, the hand movement of the construction worker is identified through a motion recognition algorithm to obtain the action information; based on the position information, the state information and the action information, a judgment function is performed to obtain the target monitoring result:

[0016] Wherein, V=1 indicates that the target monitoring result has a violation, V=0 indicates that the target monitoring result has no violation, H state =0 indicates that the construction worker does not wear a helmet, H equipment = 0 means that the action information has a tendency to approach the dangerous area, H state =1 indicates that the construction worker wears a safety helmet, is the distance from the ii-th hand to the boundary of the danger zone, k is the number of hands (usually 2), D safe is the safety distance threshold.

[0017] Optionally, controlling the camera to capture, obtaining a fourth target image, and storing the fourth target image to a target location includes: obtaining a preset time window, taking the moment when the target monitoring result is determined to be a violation as a reference, performing image acquisition according to a first preset frequency, obtaining the fourth target image, wherein the first preset frequency is greater than a second preset frequency, and the second preset frequency is the frequency at which the camera acquires the first target image; naming the fourth target image based on a preset formula: N=T=l+i+f+”.jpg”; wherein T is a timestamp, L is a location code for characterizing the preset construction area, I is a violation number for characterizing the violation type, and F is a sequence number for distinguishing the order of different frames of images captured continuously in the same violation; storing the named fourth target image to a target location based on a preset data matrix;

[0018] M=[TL IP A]; wherein M is the fourth target image stored in the target position, P is the number of the construction worker who has violated the regulations, and A is the specific description information of the violation I.

[0019] According to another aspect of the present application, an image-based construction site violation recording device is provided, the device comprising: a first acquisition unit, used to acquire a real-time image containing a preset construction area through a camera to obtain a first target image; a first processing unit, used to preprocess the first target image to obtain a second target image, the preprocessing comprising image denoising, contrast enhancement and image size adjustment; a second processing unit, used to identify the second target image based on an improved YOLOv5 network to locate the construction personnel by a bounding box to obtain a third target image, the improved YOLOv5 network being based on the YOLOv5 network , adding a branch network for identifying the hand features and head features of the construction workers for separate extraction, the branch network is connected to the feature extraction layer of the YOLOv5 network; a third processing unit is used to identify based on the third target image and the second target image, extract the location information, state information and action information of the construction workers, obtain violation criterion information, and perform violation monitoring according to the violation criterion information to obtain a target monitoring result; a second acquisition unit is used to control the camera to capture when the target monitoring result is that there is a violation, obtain a fourth target image, and store the fourth target image to the target position.

[0020] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0021] According to another aspect of the present application, a construction monitoring is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the described methods.

[0022] By applying the technical solution of this application, accurate monitoring of violations at construction sites is achieved through improved image acquisition and processing methods. Real-time images of the construction area are obtained through a camera and pre-processed by denoising, contrast enhancement, and size adjustment to improve image quality. Using the improved YOLOv5 network, the recognition branch for the hand and head features of construction workers is specially added, which significantly improves the accuracy of personnel positioning and recognition of key parts. Based on the image recognition results, the location information, status information, and action information of the construction workers are extracted to perform accurate violation judgment and monitoring. When a violation is found, the system will automatically capture and store the violation image, which solves the problems of poor image quality, low recognition accuracy, and low violation judgment accuracy in the prior art of construction site violation monitoring, and improves the efficiency and accuracy of construction site safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A hardware structure block diagram of a mobile terminal for recording violations at a construction site based on an image provided in an embodiment of the present application is shown;

[0024] Figure 2 A schematic diagram of a process of recording construction site violations based on an image provided according to an embodiment of the present application is shown;

[0025] Figure 3 A schematic diagram of a system framework of an image-based construction site violation recording method provided according to an embodiment of the present application is shown;

[0026] Figure 4 A structural block diagram of an image-based construction site violation recording device provided according to an embodiment of the present application is shown.

[0027] The above drawings include the following reference numerals:

[0028] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0033] YOLOv5 (You Only Look Once version 5): A deep learning-based object detection model, the fifth generation of the YOLO series of models. YOLOv5 is widely used in object detection tasks and can simultaneously perform object recognition and positioning in images or videos. Its main advantage is that it can achieve fast and efficient object detection in a forward pass, so it is very suitable for real-time object detection scenarios.

[0034] As introduced in the background technology, the existing technology for monitoring construction site violations has problems such as unreasonable camera angles and positions, poor image preprocessing effects, and insufficient accuracy of recognition algorithms, resulting in poor image quality, limited feature extraction, large errors in the recognition of key parts, and a lack of systematicness and accuracy in violation judgment, making it difficult to cope with complex construction environments and safety regulations. In order to solve the problems of poor image quality, low recognition accuracy, and low violation judgment accuracy in the existing technology for monitoring construction site violations, the embodiments of the present application provide an image-based construction site violation recording method, an image-based construction site violation recording device, a computer-readable storage medium, and construction monitoring.

[0035] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0036] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of an image-based construction site violation recording method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0037] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image-based construction site violation recording method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] In this embodiment, an image-based construction site violation recording method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 2 It is a flowchart of an image-based construction site violation recording method according to an embodiment of the present application. Figure 3 is a schematic diagram of the system framework of the image-based construction site violation recording method according to an embodiment of the present application, such as Figure 2 and Figure 3 As shown, the method comprises the following steps:

[0040] Step S201, obtaining a real-time image including a preset construction area through a camera to obtain a first target image;

[0041] Specifically, in real-time image acquisition, the core principle is to reasonably deploy high-definition cameras in the construction area and capture image data containing the construction site through real-time monitoring. High-definition cameras use the principle of optical imaging to convert the scene of the construction site into a digital image signal. The camera lens captures light and focuses the light on the image sensor. The sensor converts the optical signal into an electrical signal, which then undergoes analog-to-digital conversion to become a digital image that can be processed by the computer. By deploying cameras to acquire images of the construction site in real time, the dynamics of the construction site can be fully and continuously grasped, the real-time and integrity of the data can be ensured, and basic data support can be provided for subsequent image processing and violation identification.

[0042] Step S202, preprocessing the first target image to obtain a second target image, wherein the preprocessing includes image denoising, contrast enhancement, and image size adjustment;

[0043] Specifically, in image preprocessing, operations such as denoising, contrast enhancement and resizing are performed on the image to improve the clarity, accuracy and consistency of the image, providing high-quality image data for subsequent violation judgment.

[0044] Step S203, based on the improved YOLOv5 network, the second target image is recognized to locate the construction worker by a bounding box, and a third target image is obtained, wherein the improved YOLOv5 network is obtained by adding a branch network for separately extracting hand features and head features of the construction worker to the YOLOv5 network, and the branch network is connected to the feature extraction layer of the YOLOv5 network;

[0045] Specifically, the improved YOLOv5 algorithm is used. Its core principle is to transform the target detection task into a regression problem, directly predicting the target's bounding box and category probability at multiple locations and scales in the image. A branch network dedicated to extracting the hand and head features of construction workers is added to the feature extraction layer of YOLOv5, which improves the ability to extract fine-grained features. The improved YOLOv5 network significantly improves the detection accuracy and positioning accuracy of construction workers, especially the ability to recognize detailed features such as hands and heads, laying a solid foundation for the determination of violations.

[0046] Step S204, based on the third target image and the second target image, identification is performed to extract the location information, state information and action information of the construction personnel, to obtain violation criterion information, and violation monitoring is performed according to the violation criterion information to obtain a target monitoring result;

[0047] Specifically, in the judgment of illegal behaviors, the personnel and part information output by the personnel and part recognition module in the image is deeply processed, and mathematical models and logical judgments are used to accurately identify illegal behaviors such as not wearing a helmet and hands in dangerous areas, and automatic judgments are performed in combination with the predefined illegal behavior rule library. Comprehensive monitoring from position to behavior is achieved, and through precise extraction and judgment, illegal behaviors on the construction site can be quickly and accurately identified, improving the efficiency and quality of construction safety management.

[0048] Step S205, when the target monitoring result shows that there is a violation, the camera is controlled to capture a fourth target image, and the fourth target image is stored in the target position.

[0049] Specifically, when a violation is detected, the snapshot mechanism is immediately triggered to obtain a clear image of the moment of violation and accurately record relevant information, including time, location, violation type, etc. By working in collaboration with image acquisition equipment, system clocks, and databases, a complete record of violations is achieved, providing reliable data support for subsequent query, statistics, analysis, and accountability tracing.

[0050] Through this embodiment, the present application provides an image-based construction site violation recording method, which realizes real-time monitoring of the construction site and intelligent identification of violations. First, real-time images are obtained through high-definition cameras, including accurate positioning of cameras, optimization of shooting angles and improvement of lighting conditions, which improves image quality; then, pre-processing technologies such as image denoising, contrast enhancement and size adjustment are used to significantly improve the visual quality of the image and the processing efficiency of subsequent models, and enhance image availability; based on the improved YOLOv5 network, construction personnel are accurately positioned and identified, and fine-grained features are effectively extracted; further, the location information, status information and action information of construction personnel are extracted through behavioral analysis, and efficient judgment of violations is realized; finally, the automatic capture and recording mechanism can capture key moments in time, standardize image management and fully record information, which is conducive to the comprehensive improvement of construction site safety management, reduce the risk of safety accidents, and improve management efficiency and scientific decision-making.

[0051] As a possible implementation method, a real-time image including construction workers is acquired through a camera to obtain a first target image, including:

[0052] Step S301, collecting an image of a preset construction area through a camera to obtain a fifth target image;

[0053] Specifically, high-definition camera equipment is deployed at the construction site to collect images for the pre-planned construction area. The camera adopts a fixed-angle or rotatable pan-tilt configuration to achieve a comprehensive view covering the construction area. The lighting conditions at the construction site are complex and changeable, and there may be problems such as strong light exposure and shadow occlusion, which will affect the clarity and quality of the image, and thus interfere with the subsequent image recognition process. In order to solve the lighting problem, a variety of technical means can be used. For areas with too strong light, an automatic aperture lens is installed to automatically adjust the aperture size according to the light intensity to avoid overexposure of the image; for areas with shadows, fill light equipment (such as LED lights) can be used for local fill light. To ensure image quality, the camera supports dynamic adjustment of exposure time, focal length and shutter speed, and can adapt to complex lighting conditions and diverse environmental changes at the construction site. The original image collected is recorded as the fifth target image as input data for subsequent processing.

[0054] Step S302, processing the fifth target image by using an illumination compensation algorithm to obtain a first target image: Among them, I(x, y) is the coordinate value of the fifth target image at the coordinate (x, y), L0 is the preset light intensity value, L(x, y) is the estimated light intensity value of the fifth target image at the coordinate (x, y), and I'(x, y) is the light intensity value of the first target image at the coordinate (x, y).

[0055] Specifically, an illumination compensation algorithm is introduced into the image acquisition system to compensate each pixel. L(x, y) is the estimated value of the illumination intensity (which can be obtained by analyzing the brightness histogram of the image, etc.). In this way, the impact of uneven illumination on image quality can be reduced to a certain extent, and the overall clarity and contrast of the image can be improved, which is beneficial to the subsequent extraction and analysis of the head and hand features of the person.

[0056] As a possible implementation manner, before obtaining a real-time image including a preset construction area through a camera to obtain a first target image, the method further includes:

[0057] Step S401, obtaining the device parameters of the camera, and calculating the installation position scores of different positions based on the device parameters and the preset construction area: Among them, S j Score the installation position, w i The area weight of the preset construction area, n is the total number of areas in the preset construction area, d ij is the distance between the camera and the center of the preset construction area, θ j is the viewing angle of the camera, r j is the effective shooting radius of the camera, R i Preset the equivalent radius of the construction area;

[0058] Specifically, in the construction area planning stage, the camera equipment parameters are first collected, including the camera's viewing angle, effective shooting radius, resolution, etc. In determining the camera installation location, the construction site is usually more complex, and the risk level and frequency of personnel activities in different areas vary. Therefore, it is necessary to divide the construction site into multiple areas. For example, according to the distribution of construction processes, the degree of equipment concentration, and the direction of personnel flow, it can be divided into high-risk operation areas, densely populated areas, material storage areas, etc. For each area, different weights w are assigned according to its importance. i High-risk work areas (such as high-altitude work areas and large machinery operation areas) may have higher weights because once violations occur in these areas, serious safety accidents are likely to occur; while material storage areas have relatively lower weights.

[0059] Furthermore, let the distance from camera j to the center of region i be d ij , the viewing angle of the camera is θ j , the effective shooting radius of camera j is r j , the equivalent radius of region i is R i (Calculated based on the area of ​​the region, for example, the region can be assumed to be circular and the radius can be inferred from the area formula) In order to determine the best installation position of the camera, the camera installation position score S is introduced. j The calculation formula of this formula is that The calculation part is the angle between the line connecting the camera and the center of the area and the tangent line of the edge of the area (the effective part within the viewing angle range), which is calculated by comparing it with the area weight w i Multiply and sum, then divide by the camera viewing angle θ j , normalized, the obtained S j The value can comprehensively reflect the coverage effect of the camera on each area. Select S j Installing cameras at locations with the highest value can ensure effective monitoring of key areas.

[0060] Step S402, determining the position corresponding to the maximum installation position score as the target installation position;

[0061] Specifically, all the installation location scores calculated in step S401 are sorted, and the location with the highest score is selected as the target installation location. If multiple locations have the same score, they can be further screened with reference to the convenience of camera installation or regional priority. By selecting the location with the highest installation location score, it is ensured that the camera can cover the preset construction area with the best effect, maximizing its monitoring range and accuracy.

[0062] Step S403, based on the relative position relationship between the target installation position and the preset construction area, an angle fitness function is calculated to obtain an installation angle score;

[0063] Specifically, the choice of shooting angle directly affects whether the head and hand movements of personnel can be clearly captured. For construction areas of different heights and positions, the camera's pitch angle and horizontal rotation angle need to be adjusted. When monitoring workers at heights, the camera should be raised appropriately to ensure that it can capture whether the person's head is wearing a helmet; when monitoring ground personnel operating hand tools, the camera's pitch angle should be moderate to avoid hand movements being blocked or deformed due to excessive or small angles.

[0064] Furthermore, in order to quantify the rationality of the shooting angle, an angle fitness function F(α, β) can be defined, where α is the pitch angle and β is the horizontal rotation angle. This function is determined based on the expected position range of the head and hands of the lowest personnel in different construction sites, as well as the optical characteristics of the camera (such as factors such as lens distortion). In a working area at a certain height, when the personnel are in a normal working posture, the expected position range of the head and hands in the image is known. By calculating the deviation between the actual position and the expected position at the current shooting angle, combined with the influence of lens distortion on the position, the value of the angle fitness function is obtained. In the actual installation process, the shooting angle is adjusted to maximize the value of F(α, β), thereby determining the optimal shooting angle.

[0065] Step S404, determining the angle corresponding to the maximum installation angle score as the target installation posture;

[0066] Specifically, the installation angle scores calculated in step S403 are sorted, and the angle with the highest score is selected as the target installation posture. By selecting the best installation angle, the coverage of the camera to the key area is maximized, and the quality and stability of the monitoring image are improved.

[0067] Step S405: determine the target installation position and the target installation posture as the target installation solution of the camera.

[0068] Specifically, the final camera installation plan is determined by combining the target installation position and the target installation angle. The plan can be simulated and verified through 3D modeling and virtual verification tools to ensure that it can meet the monitoring needs of the construction area. By comprehensively analyzing the position and angle information, a complete installation plan is formulated, which significantly improves the coverage and monitoring effect of camera monitoring and reduces the monitoring blind spots or false alarm rates caused by improper installation.

[0069] As a possible implementation manner, preprocessing the first target image to obtain the second target image includes:

[0070] Step S501, calculating a weight parameter corresponding to the first target image based on the first target image:

[0071] Among them, w d (i,j) is the spatial domain weight, w r (i,j) is the range weight, σ d is the spatial domain standard deviation, σ r is placed in the standard deviation, f(i, j) is the pixel value of the pixel point (i, j) in the first target image, and f(x, y) is the pixel value of the central pixel point (x, y) in the first target image;

[0072] Step S502, bilateral filtering is performed on the first target image based on the weight parameter to obtain a sixth target image: Where k is the size of the Pubo window, g(i,j) is the pixel value of the pixel point (i, j) in the first target image after filtering;

[0073] Specifically, in image denoising, the noise in construction site images mainly comes from a variety of factors, such as thermal noise of camera sensors, light interference, electromagnetic radiation of electrical equipment, etc. These noises will cause random fluctuations in image pixel values, resulting in blurred image details and affecting subsequent processing. An improved bilateral filtering algorithm is used for image denoising, and its principle is to determine the weight based on the spatial distance between pixels and the difference in pixel values.

[0074] Furthermore, for each pixel point (x, y) in the image, let its pixel value be f(x, y) and the pixel value after denoising be g(x, y). The bilateral filtering algorithm considers the spatial domain weight w d (i,j) and range weight w r (i, j) to calculate the new value of each pixel. Spatial domain weight w d (i, j) is determined by the distance between the pixel point (i, j) and the center pixel point (x, y). The closer the distance, the greater the weight. It is usually calculated using the Gaussian function. The formula is: where σ d is the standard deviation in the spatial domain, which controls the speed at which the weight decays with distance. r (i,j) is calculated based on the difference of pixel values, and the formula is where σ r is the standard deviation of the range, which determines the sensitivity of the weight to the difference in pixel values. By multiplying these two weights, we get the comprehensive weight w(i,j)=w d (i,j)×w r (i, j), and then calculate the denoised pixel value according to the following formula: Among them, k is the size of the filter window, which determines the neighborhood range involved in the filtering calculation. This algorithm can better retain the edge and detail information of the image while removing noise, because it gives lower weights to areas with large changes in pixel values ​​at the edge, avoiding over-smoothing of the edges.

[0075] Step S503, the sixth target image is processed by an adaptive histogram equalization algorithm to obtain a seventh target image: Where s is the pixel value in the seventh target image, r is the pixel value in the sixth target image, H(r) is the grayscale histogram of the sixth target image, L is the total number of grayscale levels of the sixth target image, and c is the total number of pixels in different sub-regions of the sixth target image;

[0076] Specifically, in contrast enhancement, the construction site image may have low contrast due to uneven lighting, differences in the reflective characteristics of the object surface, etc., that is, the brightness difference between different areas in the image is not obvious, making it difficult to distinguish target objects such as the head and hands of the person from the background. In order to enhance the contrast, an adaptive histogram equalization algorithm is used.

[0077] The basic idea of ​​this algorithm is to transform the histogram of the image to make the distribution of pixel values ​​in the image more uniform, thereby improving the contrast. The specific operation is to divide the image into multiple small areas (such as 8x8 or 16x16 sub-blocks), calculate its grayscale histogram for each sub-block, and calculate a mapping function based on the histogram to remap the pixel values ​​in the sub-block. Let r be the pixel value in the original image, s be the enhanced pixel value, H(r) be the grayscale histogram of the original image, L be the total number of gray levels in the image (usually 256), and n be the total number of pixels in the sub-block. is the cumulative distribution function, then the mapping function is In this way, the pixel values ​​in the darker areas of the image become brighter, and the pixel values ​​in the brighter areas become darker, thereby enhancing the overall contrast of the image, making the difference between the person and the background more obvious, which is beneficial to subsequent feature extraction and recognition.

[0078] Step S504: resize the seventh target image by using a bilinear interpolation method to obtain a second target image.

[0079] Specifically, in image resizing, different image recognition algorithms and models have certain requirements on the size of the input image, and unifying the image size can improve processing efficiency and accuracy. In the process of image resizing, a suitable interpolation algorithm is needed to ensure that the image will not be severely distorted during the scaling process. This method uses a bilinear interpolation algorithm for image resizing.

[0080] Furthermore, the bilinear interpolation algorithm calculates the interpolation result based on the four nearest neighbor image table points around the interpolation point. Assume that the coordinates of the four nearest neighbor image table points in the original image are (i, j), (i+1, j), (i, j+1), (i+1, j+1), and their pixel values ​​are f(i, j), f(i+1, j), f(i, j+1), and f(i+1, j+1). The coordinates of the interpolation point are (x, y). Then, linear interpolation is performed in the horizontal direction to obtain f x (i,j)=(1-α)f(i,j)+αf(i+1,j) and f x (i,j+1)=(1-α)f(i,j+1)+αf(i+1,j+1), where α=xi. Then, in the vertical direction, f x (i,j) and f x (i, j+1) performs linear interpolation to obtain the final interpolation result f(,y) = (1-β)f x (i, j) + βf (i, j + 1), where β = yj. In this way, the image size can be adjusted while maintaining the smoothness and detail information of the image as much as possible, reducing distortion, and making the adjusted image more suitable for subsequent processing and analysis.

[0081] As a possible implementation manner, before the second target image is recognized based on the improved YOLOv5 network to locate the construction worker's bounding box and obtain the third target image, the method further includes:

[0082] Step S601, obtaining a training image, and identifying the training image through an improved YOLOv5 network to obtain a target recognition result;

[0083] Specifically, the YOLOv5 algorithm divides the input image into S×S grids (e.g., S=13 or S=26, etc.), and each grid negative table predicts a certain number (e.g., B=3 or B=5) of bounding boxes and the class probabilities of the objects in these bounding boxes. For each bounding box, the model predicts its center coordinates (x, y) (offset relative to the grid unit), width w and height h (relative to the ratio of the entire image), as well as a confidence score indicating the possibility of containing the object in the bounding box. At the same time, the model also predicts the probability of the target category that may exist in each grid cell (such as the probability of the person category).

[0084] During the inference process, the model selects the predicted bounding boxes according to the preset threshold and removes the bounding boxes with low confidence. Then, the retained bounding boxes are processed by the non-maximum suppression (NMS) algorithm to remove the redundant bounding boxes with high confidence, and finally obtain accurate person and part detection results.

[0085] In order to more accurately identify the head and hands of construction site workers, the YOLOv5 algorithm was improved. First, a branch network for head and hand feature extraction was added to the network structure. This branch network is connected to the feature extraction layer of the original network, sharing some feature information while focusing on learning the unique features of the head and hands.

[0086] During the training process, more labeled data about head and hand postures and movements were introduced to enrich the learning samples of the model. At the same time, the loss function was optimized and the weight of the part positioning loss was increased, so that the model paid more attention to the position accuracy of the head and hands.

[0087] Step S602, based on the target recognition result, the loss function value is calculated by using the bounding box regression loss function:

[0088] Among them, L loc is the loss function value, m is the number of people in the image, n is the number of parts (head and hands) of each person, and λ coord and λ noobj is the weight coefficient, It is used to characterize whether the jth part of the i-th person has a target, (xi ,y i ,w i ,h i ) are the predicted bounding box coordinates and size, is the real bounding box information, C i is the predicted position confidence, is the true part position reliability;

[0089] Specifically, in the improved YOLOv5 network, the bounding box regression loss function is used to measure the difference between the predicted bounding box and the true bounding box. By minimizing this loss function, the parameters of the model are optimized to improve the accuracy of bounding box positioning.

[0090] Furthermore, the first part of the loss function It is used to calculate the difference between the predicted bounding box center coordinates and the true center coordinates, and optimize them through the least squares method to make the center coordinates more accurate. The difference between the predicted bounding box width and height and the actual width and height is calculated in the form of square root to make the loss of small-sized objects relatively larger, thereby improving the positioning accuracy of small objects (such as the head and hands of distant people). Part III It is used to penalize the confidence of bounding boxes that have no objects but are mistakenly predicted as having objects, making the model more cautious in prediction and reducing false positives.

[0091] Step S603: Optimize the improved YOLOv5 network based on the loss function value until the loss function value is less than a first threshold.

[0092] Specifically, the gradient descent method or its variants (such as the Adam optimizer) are used to update the parameters of the network, with the goal of minimizing the loss function value. In each round of training, the learning rate is dynamically adjusted (such as using a cosine annealing learning rate strategy) to accelerate convergence. The training process continues until the loss function value is lower than the preset threshold, or the detection accuracy on the validation set reaches the target indicator, and finally the optimized model parameters are output. Through iterative optimization based on the loss function, it is ensured that the network can learn the optimal parameter configuration from the training data, thereby improving the detection capabilities of construction personnel and their key parts.

[0093] As a possible implementation method, recognition is performed based on the third target image and the second target image, the location information, state information and action information of the construction personnel are extracted, and the violation judgment information is obtained. Violation monitoring is performed based on the violation judgment information to obtain the target monitoring results, including:

[0094] Step S701, performing recognition based on the third target image and the second target image, extracting the construction worker's head information, hand information and dangerous area information, and obtaining position information;

[0095] Specifically, the head and hand features in the third target image are extracted through the improved YOLOv5 network, and combined with the overall environmental features in the second target image, the location and range of the dangerous area are determined by calculating the relative position of the construction personnel and the surrounding objects. Dangerous areas may include high-voltage equipment, hanging objects or other areas with potential risks. The extracted location information includes the specific coordinates of the construction personnel's head and hands and their relative position to the dangerous area. Accurately extracting the location information of the construction personnel and the dangerous area provides a basis for subsequent violations.

[0096] Step S702, comparing the head information with a preset helmet template and / or a helmet feature library to determine whether the construction worker is wearing a helmet, and identifying through hand information and dangerous area information to determine whether the construction worker's hand is less than a safety distance threshold from the dangerous area, and obtaining status information;

[0097] Specifically, for the judgment of not wearing a helmet, the analysis is performed based on the position information of the person's head and the feature model of the helmet. The helmet usually has specific features such as shape, color and texture in the image. The identified head area of ​​the person is matched and compared through the pre-established helmet template or feature library. If no object that matches the characteristics of the helmet is detected in the area where the helmet should be worn on the head, and the shape, wheels and other features of the head do not meet the expected form after the helmet is worn, it is determined as a violation of not wearing a helmet. At the same time, for the judgment of whether the hand is in the dangerous area, first determine the boundary and range of the dangerous area according to the layout of the construction site and the division of the dangerous area. Then, based on the position coordinates of the person's hand, calculate the distance between the hand and the boundary of the dangerous area. If the hand position crosses the boundary of the dangerous area or enters the preset dangerous distance range (the hand is less than 0.5 meters away from the dangerous equipment operation area), it is determined to be a dangerous state and the state information is recorded.

[0098] Step S703, identifying the hand movements of the construction worker through a movement recognition algorithm based on the hand information to obtain movement information;

[0099] Specifically, consider the hand's gestures, such as whether the hand is approaching a dangerous device switch, operating a dangerous tool, etc., and analyze and judge these actions through a motion recognition algorithm (such as a posture estimation model based on deep learning). Common actions include grasping, pointing, or waving, etc. Combined with the dangerous area information, it can further identify whether there is a tendency for dangerous actions, such as approaching or touching the dangerous area.

[0100] Step S704, based on the position information, state information and action information, a judgment function is used to calculate and obtain the target monitoring result:

[0101] Among them, V = 1 indicates that the target monitoring result has a violation, V = 0 indicates that the target monitoring result does not have a violation, H state =0 indicates that the construction workers did not wear safety helmets, H equipment = 0 means the action information has a tendency to approach the dangerous area, H state =1 indicates that the construction workers wear safety helmets, D handi is the distance from the ith hand to the boundary of the danger zone, k is the number of hands (usually 2), D safe is the safety distance threshold.

[0102] Specifically, the violation judgment function V is defined in the establishment of the violation judgment function. Its design purpose is to comprehensively consider multiple factors to accurately judge whether there is a violation. state =0 (not wearing a helmet) or H eruipment (without holding the required safety equipment) are directly judged as violations, because these two situations violate the basic safety regulations. state = 1 (wearing a helmet), further check the hand position. If the sum of the distances from the two hands to the boundary of the danger zone is less than the safety distance threshold D safe , which means that the hand may be in the dangerous area and is also judged as a violation. This design can comprehensively consider different types of violations and improve the accuracy and reliability of judgment.

[0103] Furthermore, in the judgment based on the construction scene and safety regulations, different construction areas and work tasks have different safety requirements at the construction site. In the high-altitude work area, in addition to the requirement to wear a safety helmet, it is also required that the safety belt must be properly fastened; in the electrical equipment operation area, it is strictly forbidden to touch the live parts with your hands unless you wear protective equipment such as insulating gloves. Therefore, the violation judgment module needs to dynamically adjust the judgment rules and parameters based on the specific field information and relevant safety regulations of the construction site.

[0104] By integrating with the management system of the construction site, we can obtain information such as the type of construction area and the ongoing work tasks. Based on this information, we can select the corresponding violation judgment criteria and thresholds. In the high-altitude work area, we can set the safety distance threshold D safe The hand is set smaller to more strictly limit the distance between the hand and the dangerous edge; for the electrical equipment operation area, the focus is on monitoring whether the hand holds insulating tools and the distance from the live equipment, etc. This judgment basis combining the construction site and safety regulations can make the judgment of violations more in line with the actual situation and reduce the occurrence of misjudgment and missed judgment.

[0105] As a possible implementation manner, controlling the camera to capture, obtaining the fourth target image, and storing the fourth target image at the target location includes:

[0106] Step S801, obtaining a preset time window, taking the moment when the target monitoring result is determined to be a violation as a reference, performing image acquisition according to a first preset frequency, and obtaining a fourth target image, wherein the first preset frequency is greater than the second preset frequency, and the second preset frequency is the frequency at which the camera acquires the first target image;

[0107] Specifically, in the snapshot trigger mechanism, when the violation judgment module outputs a violation signal (ie, V=1), the signal is transmitted to the image acquisition device as a snapshot trigger instruction. After receiving the instruction, the image acquisition device immediately stops the current conventional image acquisition mode (such as continuously acquiring images at a certain frame rate) and enters the snapshot mode. In the snapshot mode, the device will continuously capture several frames of images at a higher frame rate (for example, 50fps or higher) to ensure that the clearest moment of the violation can be captured. This is because the violation may be an instantaneous action, such as a person suddenly taking off a helmet or quickly entering a dangerous area with his hands. Increasing the frame rate can increase the probability of obtaining critical moment images.

[0108] At the same time, in order to avoid capturing too many redundant images, a time window (for example, 0.5 seconds) is set during the capture process. During this time window, the device continues to capture images, but only retains the images most relevant to the violation, such as images containing complete personnel and illegal actions, or images closest to the danger zone when the hand enters the danger zone. In this way, it can ensure that valid violation evidence images are obtained while reducing unnecessary data storage.

[0109] Step S802, naming the fourth target image based on a preset formula: N=T=l+i+f+”.jpg”; wherein T is a timestamp, L is a location code used to characterize a preset construction area, I is a violation number used to characterize a violation type, and F is a sequence number used to distinguish the order of different frame images captured continuously in the same violation;

[0110] Specifically, in image naming and storage, the naming of captured images adopts the specific formula N=T=l+i+f+”.jpg”, where T is the timestamp, accurate to the second, recording the specific time when the violation occurred, which is helpful for subsequent sorting and querying of violation events in chronological order. L is the location code, which is determined according to the area division of the construction site and can quickly locate the specific location where the violation occurred. I is the violation number, corresponding to different types of violations (such as 01 for not wearing a safety helmet, 02 for hands entering a dangerous area, etc.), which is convenient for classifying and counting violation types. F is a serial number, which is used to distinguish different frames of images captured continuously in the same violation event, so as to facilitate viewing the detailed process of the violation when needed.

[0111] Step S803, storing the named fourth target image to the target position based on the preset data matrix; M=[T LI PA]; wherein M is the fourth target image stored to the target position, P is the number of the construction worker who has violated the regulations, and A is the specific description information of the violation I.

[0112] Specifically, when storing captured images, a hierarchical directory structure is used. First, a first-level directory is created according to time (for example, by date or month), and then a second-level directory is created under each time directory according to the location code, and then a third-level directory is created under the location directory according to the violation number. The captured images are stored in the corresponding directory according to the naming rules. Such a storage structure is conducive to the rapid retrieval and management of a large number of captured images, and improves the efficiency of data query.

[0113] In the violation information record, the relevant information of the violation is recorded in the form of a data matrix M = [TLIPA]. The meanings of T, L, and I are the same as those in the image naming. P is the personnel number, which is used to uniquely identify the violator. When multiple people violate the rules at the same time, the violations of different personnel can be accurately distinguished. A is the description of the violation, which briefly describes the specific circumstances of the violation, such as "not wearing a safety helmet in the high-altitude working area and the hand is close to the edge guardrail", etc., providing more detailed violation information, which is helpful for the subsequent analysis and handling of the violation.

[0114] These violation information records will be stored in the database, which uses a relational database management system (such as MySQL or Oracle). By creating corresponding data tables, the violation information is stored according to fields to ensure the integrity, consistency and scalability of the data. At the same time, the database supports fast query, statistics and analysis operations on the data. For example, it can query according to time range, location, violation type and other conditions, generate violation statistics reports, and provide data basis for safety management of the construction site.

[0115] The embodiment of the present application also provides a kind of construction site violation recording device based on image. It should be noted that the construction site violation recording device based on image of the embodiment of the present application can be used to execute the construction site violation recording method based on image provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0116] The following is an introduction to the image-based construction site violation recording device provided in an embodiment of the present application.

[0117] Figure 4 is a structural block diagram of an image-based construction site violation recording device according to an embodiment of the present application. Figure 4 As shown, the device includes: a first acquisition unit 10, a first processing unit 20, a second processing unit 30, a third processing unit 40 and a second acquisition unit 50.

[0118] The first acquisition unit 10 is used to acquire a real-time image including a preset construction area through a camera to obtain a first target image;

[0119] Specifically, in real-time image acquisition, the core principle is to reasonably deploy high-definition cameras in the construction area and capture image data containing the construction site through real-time monitoring. It can comprehensively and continuously grasp the dynamics of the construction site, ensure the real-time and integrity of the data, and provide basic data support for subsequent image processing and violation identification.

[0120] A first processing unit 20 is used to preprocess the first target image to obtain a second target image, wherein the preprocessing includes image denoising, contrast enhancement, and image size adjustment;

[0121] Specifically, in image preprocessing, operations such as denoising, contrast enhancement and resizing are performed on the image to improve the clarity, accuracy and consistency of the image, providing high-quality image data for subsequent violation judgment.

[0122] The second processing unit 30 is used to identify the second target image based on the improved YOLOv5 network to locate the construction personnel by a bounding box, and obtain a third target image. The improved YOLOv5 network is obtained by adding a branch network for identifying hand features and head features of the construction personnel for separate extraction on the basis of the YOLOv5 network, and the branch network is connected to the feature extraction layer of the YOLOv5 network;

[0123] Specifically, a branch network dedicated to extracting the hand and head features of construction workers was added to the feature extraction layer of YOLOv5, which improved the ability to extract fine-grained features. The improved YOLOv5 network significantly improved the detection accuracy and positioning accuracy of construction workers, especially the recognition ability of detailed features such as hands and heads, laying a solid foundation for the determination of violations.

[0124] The third processing unit 40 is used to perform recognition based on the third target image and the second target image, extract the location information, state information and action information of the construction personnel, obtain the violation judgment information, and perform violation monitoring according to the violation judgment information to obtain the target monitoring result;

[0125] Specifically, in the judgment of violations, the person and part information output by the person and part recognition module in the image is deeply processed, and mathematical models and logical judgments are used to accurately identify violations such as not wearing a safety helmet and hands being in dangerous areas. Combined with the predefined violation rule library, automated judgment is performed.

[0126] The second acquisition unit 50 is used to control the camera to capture a fourth target image when the target monitoring result shows that there is a violation, and store the fourth target image at the target position.

[0127] Specifically, when a violation is detected, the snapshot mechanism is immediately triggered to obtain a clear image of the moment of violation and accurately record relevant information, including time, location, violation type, etc. By working in collaboration with image acquisition equipment, system clocks, and databases, a complete record of violations is achieved, providing reliable data support for subsequent query, statistics, analysis, and accountability tracing.

[0128] Through this embodiment, the present application provides an image-based construction site violation recording device, which includes: a first acquisition unit, a first processing unit, a second processing unit, a third processing unit and a second acquisition unit. The first acquisition unit is used to acquire a real-time image including a preset construction area through a camera to obtain a first target image; the first processing unit is used to preprocess the first target image to obtain a second target image, and the preprocessing includes image denoising, contrast enhancement and image size adjustment; the second processing unit is used to identify the second target image based on an improved YOLOv5 network to locate the boundary box of the construction personnel to obtain a third target image, and the improved YOLOv5 network is obtained by adding a branch network for identifying the hand features and head features of the construction personnel for separate extraction on the basis of the YOLOv5 network, and the branch network is connected to the feature extraction layer of the YOLOv5 network; the third processing unit is used to identify based on the third target image and the second target image, extract the location information, state information and action information of the construction personnel, obtain the violation criterion information, and perform violation monitoring according to the violation criterion information to obtain the target monitoring result; the second acquisition unit is used to control the camera to capture when the target monitoring result is that there is a violation, obtain the fourth target image, and store the fourth target image to the target position. The device significantly improves the efficiency and accuracy of construction site violation monitoring by optimizing image acquisition and processing technology and combining it with an improved YOLOv5 network, solving the problems of poor image quality, low recognition accuracy, and low violation judgment accuracy in construction site violation monitoring in existing technologies.

[0129] As a possible implementation manner, the first acquisition unit includes: a collection module and a compensation processing module.

[0130] A collection module, used to collect an image of a preset construction area through a camera to obtain a fifth target image;

[0131] The compensation processing module is used to process the fifth target image through an illumination compensation algorithm to obtain the first target image: Among them, I(x, y) is the coordinate value of the fifth target image at the coordinate (x, y), L0 is the preset light intensity value, L(x, y) is the estimated light intensity value of the fifth target image at the coordinate (x, y), and I'(x, y) is the light intensity value of the first target image at the coordinate (x, y).

[0132] As a possible implementation manner, the device also includes: a parameter acquisition unit, a position determination unit, an angle scoring unit, a posture determination unit and a solution determination unit.

[0133] The parameter acquisition unit is used to obtain the device parameters of the camera and calculate the installation location scores of different locations based on the device parameters and the preset construction area: Among them, S j Score for installation position, w i The area weight of the preset construction area, n is the total number of areas in the preset construction area, d ij is the distance between the camera and the center of the preset construction area, θ j is the viewing angle of the camera, r j is the effective shooting radius of the camera, R i Preset the equivalent radius of the construction area;

[0134] A position determination unit, configured to determine a position corresponding to a maximum installation position score as a target installation position;

[0135] An angle scoring unit, used to calculate the angle fitness function based on the relative position relationship between the target installation position and the preset construction area to obtain the installation angle score;

[0136] A posture determination unit, used to determine the angle corresponding to the maximum installation angle score as the target installation posture;

[0137] The solution determination unit is used to determine the target installation position and the target installation posture as the target installation solution of the camera.

[0138] As a possible implementation manner, the first processing unit includes: a weight calculation module, a filtering module, an equalization module and an adjustment module.

[0139] A weight calculation module, used to calculate a weight parameter corresponding to the first target image based on the first target image:

[0140]

[0141] Among them, w d (i,j) is the spatial domain weight, w r (i,j) is the range weight, σ d is the spatial domain standard deviation, σ r is placed in the standard deviation, f(i, j) is the pixel value of the pixel point (i, j) in the first target image, and f(x, y) is the pixel value of the central pixel point (x, y) in the first target image;

[0142] A filtering module is used to perform bilateral filtering on the first target image based on the weight parameter to obtain a sixth target image: Where k is the size of the Pubo window, g(i,j) is the pixel value of the pixel point (i, j) in the first target image after filtering;

[0143] The equalization module is used to process the sixth target image by an adaptive histogram equalization algorithm to obtain a seventh target image: Where s is the pixel value in the seventh target image, r is the pixel value in the sixth target image, H(r) is the grayscale histogram of the sixth target image, L is the total number of grayscale levels of the sixth target image, and c is the total number of pixels in different sub-regions of the sixth target image;

[0144] The adjustment module is used to adjust the image size of the seventh target image by bilinear interpolation to obtain a second target image.

[0145] As a possible implementation, the device further includes: an identification unit, a loss unit and an optimization unit.

[0146] The recognition unit is used to obtain training images, recognize the training images through the improved YOLOv5 network, and obtain target recognition results;

[0147] The loss unit is used to calculate the loss function value based on the target recognition result through the bounding box regression loss function:

[0148] Among them, L loc is the loss function value, m is the number of people in the image, n is the number of parts (head and hands) of each person, and λ coord and λ noobj is the weight coefficient, It is used to characterize whether the jth part of the i-th person has a target, (x i ,y i ,w i ,h i ) are the predicted bounding box coordinates and size, is the real bounding box information, C i is the predicted position confidence, is the true part position reliability;

[0149] The optimization unit is used to optimize the improved YOLOv5 network based on the loss function value until the loss function value is less than a first threshold.

[0150] As a possible implementation manner, the third processing unit includes: a position recognition module, a comparison module, an action recognition module and a monitoring result module.

[0151] A position recognition module is used to perform recognition based on the third target image and the second target image, extract the head information, hand information and dangerous area information of the construction worker, and obtain the position information;

[0152] A comparison module is used to compare the head information with a preset helmet template and / or a helmet feature library to determine whether the construction worker is wearing a helmet, and to identify the hand information and the dangerous area information to determine whether the construction worker's hand is less than a safety distance threshold from the dangerous area to obtain status information;

[0153] The motion recognition module is used to recognize the hand motions of the construction workers through the motion recognition algorithm based on the hand information to obtain the motion information;

[0154] The monitoring result module is used to calculate the target monitoring result based on the position information, state information and action information through the judgment function:

[0155] Among them, V = 1 indicates that the target monitoring result has a violation, V = 0 indicates that the target monitoring result does not have a violation, H state =0 indicates that the construction workers did not wear safety helmets, H equipment = 0 means the action information has a tendency to approach the dangerous area, H state =1 indicates that the construction workers wear safety helmets. is the distance from the ith hand to the boundary of the danger zone, k is the number of hands (usually 2), D safe is the safety distance threshold.

[0156] As a possible implementation manner, the second acquisition unit includes: an image acquisition module, a naming module and a storage module.

[0157] An image acquisition module is used to obtain a preset time window, to determine the time when the target monitoring result is a violation as a reference, to acquire images according to a first preset frequency, to obtain a fourth target image, the first preset frequency is greater than a second preset frequency, and the second preset frequency is a frequency at which the camera acquires the first target image;

[0158] A naming module, used for naming the fourth target image based on a preset formula: N=T=l+i+f+”.jpg”; wherein T is a timestamp, L is a location code used to characterize a preset construction area, I is a violation number used to characterize a violation type, and F is a sequence number used to distinguish the order of different frame images captured continuously in the same violation;

[0159] A storage module is used to store the named fourth target image to the target position based on a preset data matrix; M=[TL IP A]; wherein M is the fourth target image stored to the target position, P is the number of the construction worker with illegal behavior, and A is the specific description information of the illegal behavior I.

[0160] The image-based construction site violation recording device includes a processor and a memory. The first acquisition unit, the first processing unit, the second processing unit, the third processing unit, and the second acquisition unit are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0161] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the communication efficiency can be improved by adjusting the kernel parameters.

[0162] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0163] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the image-based construction site violation recording method.

[0164] An embodiment of the present invention provides a construction monitoring system, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the above-mentioned image-based construction site violation recording method.

[0165] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0166] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0171] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0172] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0173] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0174] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0175] 1) The image-based construction site violation recording method of this application realizes real-time monitoring of the construction site and intelligent identification of violations. First, real-time images are obtained through high-definition cameras, including accurate positioning of the camera, optimization of shooting angles and improvement of lighting conditions, which improves image quality; then, pre-processing techniques such as image denoising, contrast enhancement and size adjustment are used to significantly improve the visual quality of the image and the processing efficiency of the subsequent model, and enhance image availability; based on the improved YOLOv5 network, construction personnel are accurately positioned and identified, and fine-grained features are effectively extracted; further, the location information, status information and action information of the construction personnel are extracted through behavioral analysis, and efficient judgment of violations is realized; finally, the automatic capture and recording mechanism can capture key moments in time, standardize image management and fully record information, which will help to comprehensively improve the safety management of the construction site, reduce the risk of safety accidents, and improve management efficiency and scientific decision-making.

[0176] 2) The image-based construction site violation recording device of the present application includes: a first acquisition unit, a first processing unit, a second processing unit, a third processing unit and a second acquisition unit. The first acquisition unit is used to acquire a real-time image including a preset construction area through a camera to obtain a first target image; the first processing unit is used to preprocess the first target image to obtain a second target image, and the preprocessing includes image denoising, contrast enhancement and image size adjustment; the second processing unit is used to identify the second target image based on an improved YOLOv5 network to locate the boundary box of the construction personnel to obtain a third target image, and the improved YOLOv5 network is obtained by adding a branch network for identifying the hand features and head features of the construction personnel for separate extraction on the basis of the YOLOv5 network, and the branch network is connected to the feature extraction layer of the YOLOv5 network; the third processing unit is used to identify based on the third target image and the second target image, extract the location information, state information and action information of the construction personnel, obtain the violation criterion information, and perform violation monitoring according to the violation criterion information to obtain the target monitoring result; the second acquisition unit is used to control the camera to capture when the target monitoring result is that there is a violation, obtain the fourth target image, and store the fourth target image to the target position. The device significantly improves the efficiency and accuracy of construction site violation monitoring by optimizing image acquisition and processing technology and combining it with an improved YOLOv5 network, solving the problems of poor image quality, low recognition accuracy, and low violation judgment accuracy in construction site violation monitoring in existing technologies.

[0177] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A construction site violation recording method based on an image, characterized in that: include: Acquire a real-time image including a preset construction area through a camera to obtain a first target image; Preprocessing the first target image to obtain a second target image, wherein the preprocessing includes image denoising, contrast enhancement, and image size adjustment; The second target image is recognized based on an improved YOLOv5 network to locate the construction worker by a bounding box, thereby obtaining a third target image, wherein the improved YOLOv5 network is obtained by adding a branch network for separately extracting hand features and head features of the construction worker to the YOLOv5 network, and the branch network is connected to a feature extraction layer of the YOLOv5 network; Based on the third target image and the second target image, identification is performed to extract the location information, state information and action information of the construction personnel to obtain violation criterion information, and violation monitoring is performed according to the violation criterion information to obtain a target monitoring result; When the target monitoring result shows that there is a violation, the camera is controlled to capture a picture to obtain a fourth target image, and the fourth target image is stored in a target position.

2. The method according to claim 1, characterized in that The real-time image including the construction personnel is acquired through the camera to obtain the first target image, including: The camera collects an image of the preset construction area to obtain a fifth target image; The fifth target image is processed by an illumination compensation algorithm to obtain the first target image: Among them, I(x, y) is the coordinate value of the fifth target image at the coordinate (x, y), L0 is the preset light intensity value, L(x, y) is the estimated light intensity value of the fifth target image at the coordinate (x, y), and I'(x, y) is the light intensity value of the first target image at the coordinate (x, y).

3. The method according to claim 1, characterized in that Before obtaining a real-time image including a preset construction area through a camera to obtain a first target image, the method further includes: Obtain device parameters of the camera, and calculate installation location scores of different locations based on the device parameters and the preset construction area: Among them, S j Score the installation location, w i The area weight of the preset construction area, n is the total number of areas in the preset construction area, d ij is the distance between the camera and the center of the preset construction area, θ j is the viewing angle of the camera, r j is the effective shooting radius of the camera, R i The equivalent radius of the preset construction area; Determine the position corresponding to the maximum installation position score as the target installation position; Based on the relative position relationship between the target installation position and the preset construction area, an angle fitness function is calculated to obtain an installation angle score; Determine the angle corresponding to the maximum installation angle score as the target installation posture; The target installation position and the target installation posture are determined as a target installation solution for the camera.

4. The method according to claim 1, characterized in that: Preprocessing the first target image to obtain a second target image includes: Calculate the weight parameter corresponding to the first target image based on the first target image: Among them, w d (i,j) is the spatial domain weight, w r (i,j) is the range weight, σ d is the spatial domain standard deviation, σ r is the standard deviation, f(i, j) is the pixel value of the pixel point (i, j) in the first target image, and f(x, y) is the pixel value of the central pixel point (x, y) in the first target image; Perform bilateral filtering on the first target image based on the weight parameter to obtain a sixth target image: Wherein, k is the size of the Pubo window, and g(i,j) is the pixel value of the pixel point (i, j) in the first target image after filtering; The sixth target image is processed by an adaptive histogram equalization algorithm to obtain a seventh target image: Wherein, s is the pixel value in the seventh target image, r is the pixel value in the sixth target image, H(r) is a grayscale histogram of the sixth target image, L is the total number of grayscale levels of the sixth target image, and c is the total number of pixels in different sub-regions of the sixth target image; The seventh target image is resized by bilinear interpolation to obtain the second target image.

5. The method according to claim 1, characterized in that Before identifying the second target image based on the improved YOLOv5 network to locate the construction worker by a bounding box to obtain the third target image, the method further includes: Acquire a training image, and recognize the training image through the improved YOLOv5 network to obtain a target recognition result; Based on the target recognition result, the loss function value is calculated by the bounding box regression loss function: Among them, L loc is the loss function value, m is the number of people in the image, n is the number of parts (head and hands) of each person, λ coord and λ noobj is the weight coefficient, It is used to characterize whether the jth part of the i-th person has a target, (x i ,y i ,w i ,h i ) are the predicted bounding box coordinates and size, is the real bounding box information, C i is the predicted position confidence, is the true part position reliability; Based on the loss function value, the improved YOLOv5 network is optimized until the loss function value is less than a first threshold.

6. The method according to claim 1, characterized in that Based on the third target image and the second target image, identification is performed, the location information, state information and action information of the construction personnel are extracted, and violation judgment information is obtained. Violation monitoring is performed according to the violation judgment information to obtain target monitoring results, including: Recognize the third target image and the second target image, extract the head information, hand information and dangerous area information of the construction worker, and obtain the position information; Based on the head information, the user is compared with a preset helmet template and / or a helmet feature library to determine whether the construction worker is wearing a helmet, and the user is identified through the hand information and the dangerous area information to determine whether the construction worker's hand is less than a safety distance threshold from the dangerous area to obtain the status information; Based on the hand information, the hand motion of the construction worker is identified by a motion recognition algorithm to obtain the motion information; The target monitoring result is obtained by performing calculations based on the position information, the state information and the action information through a judgment function: Wherein, V=1 indicates that the target monitoring result has a violation, V=0 indicates that the target monitoring result has no violation, H state =0 indicates that the construction worker does not wear a helmet, H equipment = 0 means that the action information has a tendency to approach the dangerous area, H state =1 indicates that the construction worker wears a helmet, D handi is the distance from the ith hand to the boundary of the danger zone, k is the number of hands (usually 2), D safe is the safety distance threshold.

7. The method according to claim 1, characterized in that Controlling the camera to capture, obtaining a fourth target image, and storing the fourth target image in a target position, including: Acquire a preset time window, take the time when the target monitoring result is determined to be a violation as a reference, perform image acquisition according to a first preset frequency, and obtain the fourth target image, wherein the first preset frequency is greater than a second preset frequency, and the second preset frequency is the frequency at which the camera acquires the first target image; The fourth target image is named based on a preset formula: N = T = l + i + f + ".jpg"; Wherein, T is a timestamp, L is a location code used to characterize the preset construction area, I is a violation number used to characterize the type of violation, and F is a sequence number used to distinguish the order of different frame images captured continuously in the same violation; storing the named fourth target image at a target position based on a preset data matrix; M = [TLIPA]; Among them, M is the fourth target image stored in the target position, P is the number of the construction worker who has violated the regulations, and A is the specific description information of the violation I.

8. An image-based construction site violation recording device, characterized in that: The device comprises: A first acquisition unit is used to acquire a real-time image including a preset construction area through a camera to obtain a first target image; A first processing unit, configured to preprocess the first target image to obtain a second target image, wherein the preprocessing includes image denoising, contrast enhancement, and image size adjustment; A second processing unit is used to identify the second target image based on an improved YOLOv5 network to locate the construction worker by a bounding box, thereby obtaining a third target image, wherein the improved YOLOv5 network is obtained by adding a branch network for identifying hand features and head features of the construction worker for separate extraction on the basis of the YOLOv5 network, and the branch network is connected to a feature extraction layer of the YOLOv5 network; A third processing unit is used to perform recognition based on the third target image and the second target image, extract the location information, state information and action information of the construction personnel, obtain violation criterion information, and perform violation monitoring according to the violation criterion information to obtain a target monitoring result; The second acquisition unit is used to control the camera to capture a fourth target image when the target monitoring result shows that there is a violation, and store the fourth target image in a target position.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A construction monitoring system, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of claims 1 to 7.

Images