Hotel engineering construction monitoring method and system based on image recognition

Abstract

The invention relates to the field of image processing, in particular to a hotel engineering construction monitoring method and system based on image recognition. The method comprises the following steps: collecting m continuous frames of hotel project construction images, and extracting key points of each worker in the images; dangerous workers are screened according to the stability degree of the key points, target key points of the dangerous workers are extracted, and the stability degree represents the angle stability of the key points in the m frames of images; and intercepting a local image of a target key point from the mth frame of image, inputting the mth frame of image, the local image of the target key point in the mth frame of image, the stability degree of the target key point and the criticality of the target key point into the trained random forest model, outputting a behavior category of the mth frame of image, and performing early warning when the behavior category is dangerous. The problem that danger identification is inaccurate due to the fact that key points are prone to mutual influence in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more particularly to a hotel construction monitoring method and system based on image recognition. Background Art

[0002] Traditional construction inspection methods typically rely on manual inspections, manual recording, and traditional sensor technology. These methods require regular checks on construction progress, quality, and safety hazards, and rely on on-site measurement instruments (such as levels and laser scanners) to record data. With the rapid development of computer vision and deep learning technologies, image recognition technology is becoming increasingly common in various fields. Using cameras or drones installed at construction sites, systems can capture real-time image data from the construction site and use image recognition technology to conduct safety monitoring and early warning of worker behavior.

[0003] However, the existing technology usually recognizes the worker's posture based on the extracted joint points of the human skeleton. Due to the influence of redundant joint points, there is a disadvantage of misidentifying dangerous behaviors as normal behaviors or misidentifying normal behaviors as dangerous behaviors.

[0004] At present, the patent application document with the publication number "CN114155601A" and the name "A method and system for detecting dangerous behaviors of workers based on vision" discloses human target detection and human posture key point positioning of workers in monitoring pictures, so as to judge whether there are workers in the monitoring pictures, and determine the specific position information and posture key point positioning information of the workers in the image, and perform dangerous behavior analysis based on the position information and posture key point positioning information of the workers, so as to identify whether the current action posture of the workers is a dangerous behavior, thereby realizing accurate and efficient detection and identification of dangerous behaviors of workers.

[0005] The above method performs hazard analysis based on all the key points of the operator's posture, but these key points are prone to affect each other, resulting in inaccurate hazard identification. Summary of the Invention

[0006] In order to solve the problem in the prior art that key points easily affect each other and lead to inaccurate hazard identification, the present invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides a hotel construction monitoring method based on image recognition, comprising: The system collects m consecutive frames of hotel construction images and extracts the key points of each worker in the images. It then screens dangerous workers based on the stability of the key points and extracts the target key points of the dangerous workers, where the stability represents the angular stability of the key points in the m-frame images. It then captures a partial image of the target key points from the m-th frame image, inputs the m-th frame image, the partial image of the target key points in the m-th frame image, the stability of the target key points, and the criticality of the target key points into a trained random forest model, outputs the behavior category of the m-th frame image, and issues an early warning if the behavior category is dangerous. The method of extracting the target key points of dangerous workers is as follows: calculating the stability correlation and displacement correlation of any two key points; calculating the criticality of each key point, and the key point with a criticality greater than a threshold is the target key point, and the criticality and the key point are the target key points. The average values ​​of the stability correlation and displacement correlation with the remaining key points are negatively correlated.

[0008] Through continuous image acquisition and key point extraction, real-time monitoring of worker behavior is possible. The system can quickly identify dangerous workers and analyze their behavior, issuing early warnings in the early stages of dangerous behavior. Using a trained random forest model, the system automatically identifies dangerous behaviors, reducing reliance on manual intervention and subjective judgment, and improving monitoring accuracy and efficiency. By learning from large amounts of data, the model can more accurately classify dangerous behaviors, avoiding the misjudgments and omissions that can occur with traditional monitoring methods. The criticality is calculated by calculating the stability and displacement correlation of key points. The system then evaluates dangerous worker behavior by combining the mth frame image, the local image of the target key point within the mth frame image, the stability of the target key point, and the criticality of the target key point, enabling more accurate identification of dangerous worker behavior. Compared to traditional monitoring methods, this image recognition-based solution can capture subtle changes in workers' movements, increasing sensitivity to potential hazards.

[0009] Preferably, the criticality for: ,in, and The key points are The average value of the stability correlation and displacement correlation with the rest of the key points, and Represents weight.

[0010] Preferably, the method of screening dangerous workers according to the stability of key points is as follows: Calculate the degree of deviation of each key point in each frame image , ,in, is the angle of key point i in the j-th frame image, is the maximum angle of the normal posture of key point i; Calculate the stability of key point i in the mth frame image , ,in, is the standard deviation of the angle change of key point i in the m-frame image, represents the degree of offset of key point i in the b-th frame image, and m represents the total number of frames of the image; When the stability of any key point is less than the threshold, the worker corresponding to the key point is a dangerous worker.

[0011] Introducing the calculation of deviation and angle change standard deviation transforms a worker's behavior from a qualitative description to a quantitative analysis, providing a scientific and measurable basis. This quantitative approach more intuitively reflects the stability of a worker's movements, helping to enhance the credibility of the analysis results.

[0012] Preferably, the calculation of the stability correlation and displacement correlation of any two key points includes: ,in, 、 are the stability correlation and displacement correlation of key point i and key point h respectively, m represents the total number of frames of the image, 、 are the local fluctuation of stability and displacement of key point i and key point h, respectively. 、 are the stability and displacement of key point i in the jth frame, 、 are the stability and displacement of key point h in the jth frame respectively, and norm() is the normalization function.

[0013] Preferably, the local fluctuation of the stability of the key point i and the key point h , local fluctuation of displacement Specifically: ,in, and are the local window width and radius, and are the stability and displacement of key point i in the f frame image, and are the mean values ​​of key point i in the local window, and are the stability and displacement of the key point h in the f frame image, and are the mean values ​​of the key point h in the local window.

[0014] By simultaneously calculating the stability correlation and displacement correlation between key points, the correlation of key points can be comprehensively evaluated from multiple dimensions. Compared with a single indicator, this method can more comprehensively reflect the dynamic relationship between key points and improve the accuracy of dangerous behavior identification. The introduction of local volatility calculation can sensitively capture the dynamic changes of key points in a small range. The setting of the local window focuses on the changing trend of a specific period of time, avoiding the loss of details caused by large-scale averaging, thereby more accurately reflecting the characteristics of key points. In summary, through the analysis of the stability correlation and displacement correlation of key points, more refined and multi-dimensional construction safety monitoring is achieved, which not only improves the accuracy of dangerous behavior identification, but also enhances the robustness, real-time and adaptability of the system.

[0015] Preferably, the calculating of the stability correlation and the displacement correlation of any two key points includes: respectively calculating the stability correlation and the displacement correlation of any two key points using the Pearson correlation coefficient.

[0016] The Pearson correlation coefficient is a standardized statistical metric that accurately measures the linear relationship between any two key points. By analyzing the stability and displacement of key points, it can comprehensively reflect the dynamic changes and collaborative characteristics between key points, improving the accuracy of behavior recognition.

[0017] Preferably, extracting the key points of each worker in the image includes: extracting the key points of each worker in the image using OpenPose.

[0018] OpenPose can effectively distinguish different workers in complex multi-person scenes and extract key points for each worker. This capability is particularly suitable for safety monitoring on construction sites, ensuring that each worker can be accurately located even in crowded or dynamic environments.

[0019] Preferably, the collecting of m consecutive frames of hotel project construction images includes: installing a plurality of high-definition cameras at the hotel project construction site to collect videos of the construction site.

[0020] Preferably, the issuing of an early warning when the behavior category is dangerous includes: issuing an audible and visual early warning, and marking the workers corresponding to the target key points.

[0021] In a second aspect, the present invention also provides a hotel project construction monitoring system based on image recognition, comprising: a memory and a processor, wherein a computer program is stored on the memory, and the processor executes the computer program to implement the above-mentioned hotel project construction monitoring method based on image recognition.

[0022] The beneficial effects of the present invention are: The solution of the present invention can realize real-time monitoring of workers' behavior through continuous image acquisition and key point extraction, and can more accurately identify workers' dangerous behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a hotel construction monitoring method based on image recognition provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a hotel construction monitoring system based on image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0025] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 1 is a flow chart of a hotel construction monitoring method based on image recognition according to an embodiment of the present invention, comprising the following steps: S101: Collect m consecutive frames of hotel construction images and extract key points of each worker in the images.

[0027] During the hotel construction process, in order to monitor all workers on the entire construction site in real time, the acquisition of m consecutive frames of hotel construction images includes: installing multiple high-definition cameras at the hotel construction site to capture videos of the construction site.

[0028] Hotel construction sites typically have numerous workers. To effectively distinguish between workers in complex multi-person scenes and extract key points for each worker, we used OpenPose to extract key points for each worker. These key points typically include at least the head, neck, shoulders, elbows, wrists, palms, hips, knees, ankles, and soles of the feet.

[0029] S102 , screening dangerous workers based on the stability of key points, and extracting target key points of the dangerous workers, wherein the stability represents the angular stability of the key points in the m frames of images.

[0030] Each key point of the human body has a maximum angle in a normal posture. In addition, the angle fluctuation of each key point in consecutive m-frame images can also reflect the stability of the key point.

[0031] Therefore, in some embodiments, the method of screening dangerous workers according to the stability of key points is specifically as follows: calculating the offset degree of each key point in each frame of image , ,in, is the angle of key point i in the j-th frame image, is the maximum angle of the normal posture of key point i; calculate the stability of key point i in the mth frame image ,When the stability of any key point is less than the threshold, the worker corresponding to the key point is a dangerous worker; ,in, is the standard deviation of the angle change of key point i in the m-frame image, It represents the degree of deviation of key point i in the b-th frame image, and m represents the total number of image frames. The larger the key point The faster the angle changes and the greater the fluctuation, the more critical the point The worse the stability; The larger the key point The more serious the deviation within the m frame, the more critical points The worse the stability.

[0032] Since a worker is at risk if even one key point is unstable, a worker is considered dangerous if even one key point has a stability level below a threshold. For example, if the stability level of all key points on a worker is above a threshold of 0.75, the worker's posture is considered stable and poses no risk of danger. If any key point has a stability level below 0.75, the worker is considered dangerous.

[0033] After screening out dangerous workers, further analysis of their behavior is required. To avoid the impact of other irrelevant key points on the analysis results, which may lead to inaccurate analysis results, it is necessary to extract the target key points of dangerous workers.

[0034] The method of extracting the target key points of dangerous workers is specifically as follows: calculating the stability correlation and displacement correlation of any two key points; calculating the criticality of each key point, and the key point with a criticality greater than a threshold is the target key point. for: ,in, and It is the key point The average value of the stability correlation and displacement correlation with the rest of the key points, and Represents the weight. For the same worker, the stability and displacement of each key point are correlated with other key points. When the stability and displacement correlation of a key point with other key points is low, it means that the key point is more likely to have problems and has a higher criticality.

[0035] The present invention proposes two methods for calculating the stability correlation and displacement correlation of any two key points. The first method is: ,in, 、 are the stability correlation and displacement correlation of key point i and key point h respectively, m represents the total number of frames of the image, 、 are the local fluctuation of stability and displacement of key point i and key point h, respectively. 、 are the stability and displacement of key point i in the jth frame, 、 are the stability and displacement of key point h in the jth frame; the local fluctuation of the stability of key points i and key points h , local fluctuation of displacement Specifically: ,in, and are the local window width and radius, and are the stability and displacement of key point i in the f frame image, and are the mean values ​​of key point i in the local window, and are the stability and displacement of the key point h in the f frame image, and are the mean values ​​of the key point h in the local window.

[0036] The second method uses the Pearson correlation coefficient to calculate the stability correlation and displacement correlation of any two key points.

[0037] Specifically, ,in, 、 Represents the stability correlation and displacement correlation of key point i and key point h respectively, m represents the total number of frames of the image, 、 Represents the key point i in the Frame stability, displacement, 、 They represent the key point h in the Frame stability and displacement.

[0038] It should be noted that the key point is generally an area, so the displacement of the key point i in each frame image is actually the displacement of the center or center of mass of the key point i relative to the previous frame image. Specifically, it can be expressed by the Euclidean distance between the center or center of mass coordinates of the key point i in each frame image and the previous frame image, which will not be repeated here.

[0039] S103. Capture a partial image of the target key point from the m-th frame image, input the m-th frame image, the partial image of the target key point in the m-th frame image, the stability of the target key point, and the criticality of the target key point into a trained random forest model, output the behavior category of the m-th frame image, and issue an early warning when the behavior category is dangerous.

[0040] In one embodiment, in order to find dangerous workers as quickly as possible, an audible and visual warning can be issued, and the workers corresponding to the target key points can be marked.

[0041] The above-mentioned hotel construction monitoring method based on image recognition provided by an embodiment of the present invention calculates the criticality through the stability correlation and displacement correlation of key points, and then comprehensively evaluates the dangerous worker behavior by combining the m-th frame image, the local image of the target key point in the m-th frame image, the stability of the target key point, and the criticality of the target key point, eliminating the influence of irrelevant key points. At the same time, the random forest model is used to analyze the behavior category of the m-th frame image, which can more accurately identify the dangerous behavior of workers.

[0042] The present invention also provides a hotel construction monitoring system based on image recognition. Figure 2 As shown, the system includes a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, a hotel construction monitoring method based on image recognition according to the present invention is implemented.

[0043] The system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are known in the art and thus will not be described in detail here.

[0044] In the present invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, the computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store the required information and can be accessed by an application, module, or both. Any such computer storage medium can be part of, accessible to, or connectable to a device. Any application or module described in the present invention can be implemented by computer-readable / executable instructions stored or otherwise retained by such a computer-readable medium.

[0045] In the description of this specification, "multiple" and "several" mean at least two, such as two, three or more, unless otherwise clearly defined.

[0046] While several embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous modifications, variations, and alternatives will occur to those skilled in the art without departing from the concept and spirit of the present invention. It should be understood that various alternatives to the embodiments of the present invention described herein may be employed in practicing the present invention.

Claims

1. A hotel construction monitoring method based on image recognition, characterized in that: include: Collect m consecutive frames of hotel construction images and extract the key points of each worker in the image; Screening dangerous workers based on the stability of key points and extracting target key points of dangerous workers, wherein the stability represents the angular stability of the key points in m frames of images; A partial image of the target key point is captured from the m-th frame image. The m-th frame image, the partial image of the target key point in the m-th frame image, the stability of the target key point, and the criticality of the target key point are input into the trained random forest model. The behavior category of the m-th frame image is output, and an early warning is issued when the behavior category is dangerous. The extraction of target key points of dangerous workers specifically includes: calculating the stability correlation and displacement correlation of any two key points; Calculate the criticality of each key point. The key point with a criticality greater than the threshold is the target key point. The average values ​​of the stability correlation and displacement correlation with the remaining key points are negatively correlated.

2. The hotel construction monitoring method based on image recognition according to claim 1, characterized in that, The criticality for: ,in, and The key points are The average value of the stability correlation and displacement correlation with the rest of the key points, and Represents weight.

3. The hotel construction monitoring method based on image recognition according to claim 1, characterized in that, The screening of dangerous workers based on the stability of key points is as follows: Calculate the degree of deviation of each key point in each frame image , ,in, is the angle of key point i in the j-th frame image, is the maximum angle of the normal posture of key point i; Calculate the stability of key point i in the mth frame image , ,in, is the standard deviation of the angle change of key point i in the m-frame image, represents the degree of offset of key point i in the b-th frame image, and m represents the total number of frames of the image; When the stability of any key point is less than the threshold, the worker corresponding to the key point is a dangerous worker.

4. The hotel construction monitoring method based on image recognition according to claim 3, characterized in that, The calculation of the stability correlation and displacement correlation of any two key points includes: ,in, 、 are the stability correlation and displacement correlation of key point i and key point h respectively, m represents the total number of frames of the image, 、 are the local fluctuation of stability and displacement of key point i and key point h, respectively. 、 are the stability and displacement of key point i in the jth frame, 、 are the stability and displacement of key point h in the jth frame respectively, and norm() is the normalization function.

5. The hotel construction monitoring method based on image recognition according to claim 4 is characterized in that, The local fluctuation of the stability of the key points i and h , local fluctuation of displacement for: ,in, and are the local window width and radius, and are the stability and displacement of key point i in the f frame image, and are the mean values ​​of key point i in the local window, and are the stability and displacement of the key point h in the f frame image, and are the mean values ​​of the key point h in the local window.

6. The hotel construction monitoring method based on image recognition according to claim 1, characterized in that, The calculation of the stability correlation and displacement correlation of any two key points includes: The Pearson correlation coefficient is used to calculate the stability correlation and displacement correlation of any two key points.

7. The hotel construction monitoring method based on image recognition according to claim 1, characterized in that: The extracting key points of each worker in the image includes: extracting the key points of each worker in the image using OpenPose.

8. The hotel construction monitoring method based on image recognition according to claim 1, characterized in that: The method of collecting m consecutive frames of hotel construction images includes: installing multiple high-definition cameras at the hotel construction site to collect videos of the construction site.

9. The hotel construction monitoring method based on image recognition according to claim 1, characterized in that: The warning when the behavior category is dangerous includes: issuing an audio and visual warning, and marking the workers corresponding to the target key points.

10. A hotel construction monitoring system based on image recognition, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the hotel construction monitoring method based on image recognition as described in any one of claims 1 to 9.

Images