Construction site personnel management method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a construction site personnel management method and device, equipment, a medium and a program product, which can be applied to the technical field of artificial intelligence, such as application scenes of computer vision. According to the embodiment of the invention, an on-site picture of a construction site is obtained, and at least one specified safety helmet color is obtained; determining a specified management area in the scene picture; determining a safety helmet identification result corresponding to the specified safety helmet color in the specified management area according to the field picture; and performing personnel management on the specified management area according to the safety helmet identification result and the personnel configuration requirement of the specified management area. According to the method and the device, the specified safety helmet color in the specified management area is recognized based on the site picture of the construction site, so that personnel management is performed according to the safety helmet recognition result, automatic personnel classification and management of the construction site are realized, errors and untimely conditions caused by manual management are avoided, and the construction safety and the construction efficiency are guaranteed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a construction site personnel management method, device, equipment, medium and program product. Background Art

[0002] Construction sites usually use different colored helmets to represent different types of personnel. For example, yellow helmets represent ordinary workers or other construction workers, blue helmets represent technical personnel or personnel engaged in special operations, red helmets represent management or on-site commanders, and white helmets represent project supervisors or managers of the entire project.

[0003] The existing technology usually manages the personnel represented by different colored helmets manually, which is time-consuming and labor-intensive, and is prone to human errors and untimely management, affecting construction safety and efficiency. Summary of the invention

[0004] The embodiments of the present application provide a construction site personnel management method, device, equipment, medium and program product, which can avoid errors and delays caused by human management and ensure construction safety and efficiency.

[0005] An embodiment of the present application provides a construction site personnel management method, including: obtaining a scene picture of the construction site, and obtaining at least one designated safety helmet color; determining a designated management area in the scene picture; determining a safety helmet recognition result corresponding to the designated safety helmet color in the designated management area based on the scene picture; and performing personnel management on the designated management area based on the safety helmet recognition result and the personnel configuration requirements of the designated management area.

[0006] An embodiment of the present application also provides a construction site personnel management device, including: an acquisition unit, used to acquire a scene picture of the construction site, and acquire at least one designated safety helmet color; a determination unit, used to determine a designated management area in the scene picture; an identification unit, used to determine a safety helmet recognition result corresponding to the designated safety helmet color in the designated management area based on the scene picture; and a management unit, used to perform personnel management on the designated management area based on the safety helmet recognition result and the personnel configuration requirements of the designated management area.

[0007] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute the steps in any construction site personnel management method provided in the embodiment of the present application.

[0008] An embodiment of the present application also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any construction site personnel management method provided in the embodiment of the present application.

[0009] An embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of any construction site personnel management method provided in the embodiment of the present application.

[0010] The embodiment of the present application can obtain a live picture of the construction site and obtain at least one designated safety helmet color; determine a designated management area in the live picture; determine a safety helmet recognition result corresponding to the designated safety helmet color in the designated management area based on the live picture; and perform personnel management on the designated management area based on the safety helmet recognition result and the personnel configuration requirements of the designated management area.

[0011] In this application, based on the on-site images of the construction site, the color of the specified safety helmet in the designated management area is identified, so that different types of personnel in the designated management area can be automatically distinguished according to the safety helmet recognition results, and personnel management in the designated management area is carried out based on the safety helmet recognition results, thereby realizing automated personnel classification and management of the construction site, avoiding errors and delays caused by human management, and ensuring construction safety and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1a It is a scene schematic diagram of the construction site personnel management method provided by the embodiment of the present application;

[0014] Figure 1b It is a flowchart of a construction site personnel management method provided by an embodiment of the present application;

[0015] Figure 1c is a schematic diagram of the interface of the parameter setting page provided in the embodiment of the present application;

[0016] Figure 1d It is a schematic diagram of a scene picture of a construction site provided in an embodiment of the present application;

[0017] Figure 1e is a schematic diagram of a color prompt box in a live picture provided by an embodiment of the present application;

[0018] Figure 1f is a schematic diagram of a target detection model provided in an embodiment of the present application;

[0019] Figure 1g is a schematic diagram of a scene including a person not wearing a helmet provided in an embodiment of the present application;

[0020] Figure 1h It is a schematic diagram of a pop-up window warning for not wearing a helmet provided in an embodiment of the present application;

[0021] Figure 1i is a schematic diagram of an interface for setting identification parameters provided in an embodiment of the present application;

[0022] Figure 1j is a schematic diagram of the mode selection interface provided in an embodiment of the present application;

[0023] Figure 1k It is a schematic diagram of the interface of the personnel configuration interface provided in the embodiment of the present application;

[0024] Figure 1l It is a schematic diagram showing a staff shortage alarm prompt in a scene screen provided by an embodiment of the present application;

[0025] Figure 2a It is a workflow diagram of the construction site personnel management system provided by the embodiment of the present application;

[0026] Figure 2b It is a flowchart of a construction site personnel management method provided by an embodiment of the present application;

[0027] Figure 2c It is a flow chart of voice intercom provided in an embodiment of the present application;

[0028] Figure 3 It is a structural schematic diagram of a construction site personnel management device provided in an embodiment of the present application;

[0029] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0031] Embodiments of the present application provide a construction site personnel management method, device, equipment, medium and program product.

[0032] The construction site personnel management device can be integrated into an electronic device, which can be a terminal, a server, etc. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, or a personal computer (PC); the server can be a single server or a server cluster composed of multiple servers.

[0033] In some embodiments, the construction site personnel management device can also be integrated into multiple electronic devices. For example, the construction site personnel management device can be integrated into multiple servers, and the construction site personnel management method of the present application can be implemented by multiple servers.

[0034] In some embodiments, the server may also be implemented in the form of a terminal.

[0035] For example, refer to Figure 1a The construction site personnel management method is implemented by a server, which can obtain a scene picture of the construction site captured by an image / video acquisition device, and obtain at least one designated safety helmet color; determine a designated management area in the scene picture; determine a safety helmet recognition result corresponding to the designated safety helmet color in the designated management area according to the scene picture; and manage personnel in the designated management area according to the safety helmet recognition result and the personnel configuration requirements of the designated management area.

[0036] The following are detailed descriptions. It should be noted that the order of the following embodiments is not intended to limit the preferred order of the embodiments. It is understandable that in the specific implementation of the present application, it involves user-related data such as on-site images, area selection operations, setting operations for specified helmet colors, mode selection operations, and personnel configuration operations. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data must comply with relevant laws, regulations, and standards of relevant countries and regions.

[0037] Artificial Intelligence (AI) is a technology that uses digital computers to simulate human perception of the environment, acquire knowledge, and use knowledge. This technology can enable machines to have functions similar to human perception, reasoning, and decision-making. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation, and other major directions.

[0038] Computer Vision (CV) is a technology that uses computers to replace human eyes to identify, measure and further process target images. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, virtual reality, augmented reality, simultaneous positioning and map construction, autonomous driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition. For example, image processing technologies such as image coloring and image stroke extraction.

[0039] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0040] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0041] In this embodiment, a construction site personnel management method involving artificial intelligence is provided, such as Figure 1b As shown, the specific process of the construction site personnel management method can be as follows:

[0042] 110. Obtain a live picture of the construction site and obtain at least one designated safety helmet color.

[0043] Wherein, the scene picture refers to the image or video of the construction site condition in real time or recorded. These pictures can show the real-time condition of the construction site, such as including equipment, workers, materials, progress, etc. For example, one or more image / video acquisition devices such as cameras can be set at the construction site, and the video form of the construction site can be collected by live broadcasting. The image / video acquisition device can transmit the collected scene picture to the construction site personnel management system in the form of video stream or other formats for processing, and the construction site personnel management method of the embodiment of the present application is realized by the construction site personnel management system. The construction site personnel management system can be deployed on a server and / or a terminal. When the construction site personnel management system is deployed on a server, the client corresponding to the system can be deployed on the terminal, and the client can display the scene picture obtained by the system and other interfaces of the system, and the client is deployed on the server.

[0044] Among them, the designated helmet color refers to the helmet color that is determined according to actual needs or application scenarios and is to be identified at the construction site. In order to facilitate the distinction between different types of work and responsibilities at the construction site, the meanings of helmet colors in different regions may be different. For example, blue helmets usually represent technical or special operations personnel, red helmets represent management or on-site commanders, white helmets represent engineering supervision or managers of the entire project, yellow helmets represent visitors or temporary workers, and so on. The embodiment of the present application sets one or more helmet colors as designated helmet colors according to actual needs and application scenarios, which can adapt to changes and needs at the construction site.

[0045] In some implementations, the designated helmet color can be selected autonomously in the corresponding page of the construction site personnel management system to increase the autonomy and convenience of determining the designated helmet color, and also facilitate real-time modification of the designated helmet color. Specifically, before obtaining the on-site picture of the construction site and obtaining at least one designated helmet color, it also includes:

[0046] In response to the setting operation of the designated hard hat color, at least one designated hard hat color is set for the construction site.

[0047] For example, a parameter setting page is provided in the construction site personnel management system, and the page can be displayed in the client corresponding to the construction site personnel management system. Figure 1cThe parameter setting page shown in the figure includes a designated helmet color selection interface, in which the system administrator can select one or more helmet colors that match the construction site requirements as designated helmet colors. The designated helmet color selection interface includes a variety of common helmet colors such as yellow, red, blue, and white. The system administrator can select a single, multiple, or all helmet colors in the interface. For example, there should be temporary workers and technical workers on the construction site. At this time, the yellow and blue helmet colors corresponding to these two types of workers can be selected as designated helmet colors in the designated helmet color selection interface, so that when performing personnel management, attention can be paid to the personnel corresponding to the designated helmet colors.

[0048] In some embodiments, in order to make the designated helmet color more compatible with the requirements of the construction site, the designated helmet color can be determined according to the personnel configuration requirements of the designated management area. For example, the designated management area should be configured with two types of personnel: temporary workers and technical workers. The construction site personnel management system can use the yellow and blue colors corresponding to the helmets of these two types of personnel as the designated helmet colors.

[0049] 120. Determine the designated management area in the on-site image.

[0050] The designated management area refers to the area in the construction site that is determined according to actual needs or application scenarios and is required to perform personnel management. For example, in actual applications, in order to improve the efficiency and accuracy of personnel management, the construction site can be divided into several management areas based on factors such as project progress, construction stage, and safety risks, and one or more of them can be used as designated management areas.

[0051] In some implementations, the designated management area can be selected autonomously in the on-site image displayed by the construction site personnel management system to increase the autonomy and convenience of determining the designated management area and facilitate real-time modification of the designated management area. Specifically, determining the designated management area in the on-site image includes:

[0052] In response to an area selection operation in the live picture, a designated management area is determined in the live picture.

[0053] Among them, the area selection operation refers to the operation of selecting a designated management area by drawing lines or selecting a rectangle in the live screen using input devices such as a mouse or touch screen.

[0054] For example, after receiving the on-site images sent by the image / video acquisition device, the construction site personnel management system can display the following in its corresponding client: Figure 1d The system administrator can perform area selection operations on the live screen displayed on the client, such as Figure 1dAs shown in the live picture, a selection box is formed by a dotted line drawn with a finger, and the image area in the selection box is the designated management area.

[0055] In some implementations, the system administrator can freely select parameters such as the shape, size, and position of the box selection according to actual needs. For example, you can choose a rectangular box selection, or you can choose to freely draw a polygon to adapt to different on-site conditions. When performing an area box selection operation, the system can also provide real-time feedback and prompts to facilitate the system administrator to complete the operation more accurately. For example, in the process of drawing a selection box, information such as the size and position of the box selection area can be displayed in real time, allowing the system administrator to better understand the operation effect, so that the system administrator can complete the area box selection operation more easily and improve the accuracy and efficiency of management.

[0056] 130. According to the on-site picture, determine the helmet recognition result corresponding to the specified helmet color in the specified management area.

[0057] For example, the helmet recognition result may include the recognition result of the helmet corresponding to the specified helmet color in the specified management area, etc. For example, the specified management area of ​​the scene picture can be cropped to obtain the picture of the specified management area, and then the picture of the specified management area is input into the target detection model, and the target detection model recognizes the helmets of different specified helmet colors in the picture. The target detection model may include but is not limited to FCN (Fully Convolutional Neural Network), SegNet (Deep Convolution Encoder-Decoder Architecture for Image Segmentation), DeepLab (Dilated Convolution Spatial Pyramid Pooling Layer), Mask R-CNN (Region-based Convolutional Neural Network), U-Net (U-Net-based Image Segmentation Network), Gated SCNN (Gated Shape Convolutional Neural Network for Semantic Segmentation) and the like. One or more combinations thereof.

[0058] Since different designated management areas have different and possibly irregular shapes, this will result in non-standard image data. If the image of the designated management area is directly recognized, the accuracy of the recognition result will be low. In addition, multiple designated management areas may be set in the same scene picture. If each designated management area is recognized separately, multiple recognition processes will be required, thereby reducing the recognition efficiency. In some embodiments, in order to improve the accuracy of the recognition results and improve the recognition efficiency, the scene picture may be recognized once, and then the recognition targets in each designated management area may be determined based on the position information of the corresponding recognition target to obtain the safety helmet recognition results corresponding to each designated management area. Specifically, according to the scene picture, the safety helmet recognition results corresponding to the specified safety helmet color in the designated management area are determined, including:

[0059] In the scene picture, determine the identification target, the identification target includes the safety helmet corresponding to the specified safety helmet color;

[0060] Obtain the target position of the identified target in the scene picture;

[0061] The helmet recognition result is determined based on the identification target whose target position is within the designated management area.

[0062] The identified target refers to the helmet corresponding to the specified helmet color identified in the scene. For example, the specified helmet colors include yellow and blue, and the target detection model can be used to identify the helmet 1 and helmet 2 corresponding to yellow, and the helmet 3 and helmet 4 corresponding to blue in the scene.

[0063] The target position refers to the position of the identified target in the scene image, and the position can be represented by the pixel coordinates of the scene image.

[0064] In the embodiment of the present application, the helmet recognition result includes the recognition result of the helmet corresponding to the specified helmet color. Figure 1d The scene shown in the figure includes four persons 1 to 4 wearing helmets. After the scene is input into the target detection model, the target detection model can output target1-yellow, target2-yellow, target3-blue and target4-blue, where target represents the identification target, i.e., the four persons 1 to 4 wearing helmets are identified, and the colors of the helmets 1 to 4 are yellow, yellow, blue and blue. After identifying helmets 1 to 4, the pixel coordinates of target1 target4 in the scene can be obtained, i.e., target1(x1,y1), target2(x2,y2), target3(x3,y3), target4(x4,y4). Then, the pixel coordinates of the designated management area A [(x1,y1),(x2,y2),(x3,y3),...,(xn,yn)] are obtained. If the designated management area A is a rectangle, the x and y values ​​of the pixel coordinates of each helmet can be compared with the upper left corner coordinates and lower right corner coordinates of the rectangular frame respectively. If the designated management area A is a polygon, the ray method or winding number algorithm (crossing number) can be used to determine whether each helmet is within the designated management area. Figure 1dAs shown, target1 to target3 are located in the designated management area A, and the helmet recognition result corresponding to the designated management area A may include helmet 1-yellow, helmet 2-yellow, and helmet 3-blue in the designated management area. For other designated management areas in the scene except designated management area A, the above method can also be used to determine the helmet recognition result corresponding to the corresponding designated management area.

[0065] In some embodiments, after the helmet recognition result is determined, a corresponding mark can be made in the scene image to prompt the user that the recognition has been completed and the recognition result is prompted, so as to facilitate real-time management of the scene image. After the helmet recognition result corresponding to the specified helmet color in the specified management area is determined according to the scene image, it also includes: marking the helmet recognition result in the scene image.

[0066] The marking method of the helmet identification result may include, but not display, marking the helmet corresponding to the specified helmet color with a prompt such as a prompt box, special symbols, icons, etc. For example, a rectangular box or a polygonal prompt box of the recognition result may be drawn in the specified management area of ​​the scene image to frame the area that has been identified as the helmet corresponding to the specified helmet color, and give a corresponding label or color prompt. Figure 1e The schematic diagram of the color prompt box in the scene picture shown can frame the positions of the safety helmets 1 to 3 in the scene picture with corresponding color prompt boxes. In the figure, a blue prompt box is used to frame the area of ​​blue safety helmets in the designated management area, and a yellow prompt box is used to frame the area of ​​yellow safety helmets in the designated management area.

[0067] In some implementations, a candidate area can be determined based on a feature map of the scene to determine an area in the scene where a target may exist, and then a multi-step classification process is performed through binary classification and color classification to obtain a recognition target, so as to accurately identify the recognition target in the scene to avoid errors or omissions. Specifically, in the scene, determining the recognition target includes:

[0068] Extract features from the scene image to obtain a feature map;

[0069] Obtain multiple candidate regions from the feature map;

[0070] Perform binary classification on multiple candidate regions to determine the foreground region among the multiple candidate regions;

[0071] Perform helmet color classification on the foreground area to determine the recognition target.

[0072] For example, Figure 1fThe schematic diagram of the target detection model shown in the figure, the target detection model may include a convolutional network, a region proposal network, an interest pooling network and a fully connected network. For a scene image of any size P×Q, it is first scaled to an image of a fixed size M×N, and then the M×N image is sent to the convolutional network, and the feature map is extracted by convolution processing; multiple candidate boxes (i.e., candidate regions) are generated on the feature map through the region proposal network, and the candidate boxes are subjected to binary classification after cropping and filtering the feature map to determine whether the candidate region is a foreground or background (i.e., an object or not), and the foreground and background usually refer to the object area and the non-object area in the image. The offset between the target candidate box and the real target box is learned by bounding box regression to correct the target candidate box so that the candidate box can better fit the real position and size of the target object; the feature map collected by the interest pooling network and the candidate box corresponding to the foreground are used to extract the feature map blocks of the foreground in the feature map, and the extracted feature map blocks are sent to the fully connected network for classification processing, and the color of the object (i.e., the helmet) corresponding to the candidate box corresponding to the foreground is output, that is, the identification target corresponding to each helmet color is output.

[0073] In some embodiments, the target detection model is trained by training samples corresponding to different helmet colors. For example, before determining the helmet recognition result corresponding to the specified helmet color in the specified management area based on the scene picture, the initial target detection model and the training sample set can be obtained. The training sample set includes training samples corresponding to multiple different helmet colors. The training samples are scene pictures for training. The training samples corresponding to the helmet colors are marked with sample labels of the helmet colors, such as the sample labels of training samples 1 to 1 are yellow, the sample labels of training samples l+1 to m are blue, the sample labels of training samples m+1 to n are white, and so on. The sample labels of the training samples in the training sample set can cover all helmet colors, so as to train the initial target detection model through the training sample set until the model converges to obtain the target detection model, and the target detection model can realize the recognition of any specified helmet color.

[0074] In some embodiments, in order to improve the accuracy of the recognition result and the recognition efficiency, after the scene image is recognized once, the person not wearing a helmet in each designated management area can be determined based on the location information of the corresponding person not wearing a helmet, so as to generate a non-wearing helmet alarm corresponding to each designated management area. Specifically, the recognition target also includes a person not wearing a helmet. After the helmet recognition result corresponding to the designated helmet color in the designated management area is determined based on the scene image, it also includes:

[0075] If the target position of the person not wearing a helmet in the on-site image is within the designated management area, a not wearing helmet alarm is generated.

[0076] The warning for not wearing a helmet may include, but is not limited to, image or video warnings, voice warnings, and text warnings. The warning for not wearing a helmet may include certain prompt information, such as the location, time, number of people, etc., so as to facilitate timely processing and response.

[0077] In the embodiment of the present application, the helmet recognition result includes the recognition result of the helmet corresponding to the specified helmet color and the recognition result of the person not wearing the helmet. Figure 1g The scene shown includes a scene of people who are not wearing helmets. The scene includes four people 1 to 4 wearing helmets and person 5 who is not wearing a helmet. After the scene is input into the target detection model, the colors of the helmets worn by people 1 to 4 can be identified respectively, and it can be identified that person 5 is not wearing a helmet. For example, the target detection model can output target1-yellow, target2-yellow, target3-blue, target4-blue, target5-0, where target1 to target4 correspond to the helmets of people 1 to 4, target5 corresponds to the helmet of person 5, and 0 means not wearing. At this time, the pixel positions of target1 to target5 in the scene can be obtained. If target1 to target3 and target5 are all located in the designated management area A, then the helmet recognition result corresponding to the designated management area A can be that the designated management area includes helmet 1-yellow, helmet 2-yellow, helmet 3-blue, and person 5 (i.e., target5)-not wearing a helmet, as shown in FIG. Figure 1g As shown, the positions of the helmets 1 to 3 in the scene image can be framed with prompt frames of corresponding colors, and the position of the target 5 in the scene image can be framed with a prompt frame for warning.

[0078] In some embodiments, the warning for not wearing a helmet can be in the form of a pop-up window. For example, when there is a person not wearing a helmet in a designated area, a pop-up window may be generated. Figure 1h The pop-up window shown is a warning for not wearing a helmet. This pop-up window can be displayed on the client to warn. The pop-up window can display a screen with designated management areas and personnel not wearing helmets, as well as location (i.e., place) and time information. Among them, personnel not wearing helmets are marked with a prompt box, and the location is the location of the construction site corresponding to the on-site picture. The pop-up window can also display a control for viewing details. Click the control to view the specific situation on the site, and view the distribution of personnel and real-time video, etc.

[0079] In some embodiments, the target detection model can be used to identify the recognition target and the person not wearing a helmet in the scene. The training sample set used to train the target detection model can also include training samples corresponding to the person not wearing a helmet. The training samples of the person not wearing a helmet are marked with the sample label of not wearing a helmet, such as the sample label of training samples n+1~o is not wearing a helmet. In this way, the target detection model trained by the training sample set can simultaneously identify the recognition target and the person not wearing a helmet in the scene.

[0080] For example, after the scene image is input into the target detection model, the target detection model can simultaneously identify the helmets corresponding to the different designated helmet colors and the people who are not wearing helmets in the scene image. And determine the helmet recognition results in each designated management area. If there are people who are not wearing helmets in any designated management area A, then generate the corresponding warning of not wearing helmets for the designated management area A, such as the warning prompt text "There are people who are not wearing helmets in the designated management area A".

[0081] In some implementations, the corresponding recognition result can be determined by the classification result of the scene picture. Specifically, before determining the helmet recognition result corresponding to the specified helmet color in the specified management area according to the scene picture, it also includes:

[0082] Classify and process the scene images to obtain classification results;

[0083] A first recognition model corresponding to the classification result is obtained, where the first recognition model is used to determine the helmet recognition result, and the first recognition model is trained by a training image corresponding to the classification result.

[0084] For example, the image classification model can be trained using the training scene pictures collected at night, so that the trained image classification model can be used to perform binary classification on the scene pictures, that is, distinguish whether the scene pictures are daytime or nighttime. If the classification result is that the scene pictures are daytime, the recognition model corresponding to the daytime recognition mode is used as the first recognition model. If the classification result is that the scene pictures are nighttime, the recognition model corresponding to the nighttime recognition mode is used as the first recognition model. The image classification model may include, but is not limited to, a combination of one or more of AlexNet (deep convolutional neural network), VGG (VGG convolutional network model), ResNet (residual network), etc.

[0085] In some implementations, the corresponding recognition model can be automatically acquired based on the image parameters of the on-site image, so that the recognition model can better adapt to the characteristics of the construction site and improve the recognition accuracy. Specifically, before determining the helmet recognition result corresponding to the specified helmet color in the specified management area according to the on-site image, it also includes:

[0086] Get the picture parameters of the live picture;

[0087] A second recognition model corresponding to the picture parameters is obtained, where the second recognition model is used to determine a helmet recognition result, and the second recognition model is trained using a training image corresponding to the picture parameters.

[0088] The picture parameters refer to characteristic parameters of the live picture. For example, the picture parameters may include at least one of picture grayscale and picture time. Picture grayscale refers to the color depth of the live picture, which can be expressed in brightness value or grayscale value, etc. Picture time refers to the timestamp of the live picture, that is, the shooting time of the picture time.

[0089] For example, the on-site pictures can be used as training samples through training with different picture parameters to obtain multiple recognition models corresponding to different picture parameters. In this way, the construction site personnel management system can call a recognition model corresponding to the picture parameters of the on-site picture as the second recognition model to determine the recognition target in the on-site picture.

[0090] In some implementations, recognition parameters corresponding to different recognition models can be set to automatically select a corresponding recognition model according to the recognition parameters and the picture parameters to improve recognition efficiency. Specifically, obtaining a second recognition model corresponding to the picture parameters includes:

[0091] The recognition model whose recognition parameters match the picture parameters is used as the second recognition model.

[0092] Among them, the recognition parameters refer to parameters preset according to actual needs and scenarios, such as recognition parameters that may include but are not limited to at least one of recognition grayscale corresponding to the grayscale of the screen and recognition time corresponding to the screen time. For example, recognition parameters corresponding to different recognition modes may be preset, such as setting recognition times corresponding to daytime recognition mode and nighttime recognition mode respectively, so that the recognition mode whose corresponding recognition time matches the screen time may be used as the target recognition mode.

[0093] For example, in the client of the construction site personnel management system, there is a Figure 1i The recognition parameter setting interface shown in the figure shows input boxes 1 and 2 for the start time and end time of the recognition time corresponding to the night recognition mode. The user can slide and select corresponding times such as 19:30 and 06:30 in input boxes 1 and 2 respectively, so that the recognition time 19:30 to 06:30 corresponding to the night recognition mode is formed by the time in input boxes 1 and 2. In this way, within the recognition time 19:30 to 06:30, the recognition model corresponding to the night recognition mode is called as the second recognition model, and outside the time 19:30 to 06:30, the recognition model corresponding to the day recognition mode is called as the second recognition model.

[0094] In some implementations, the recognition mode can be selected autonomously in the corresponding page of the construction site personnel management system to call the corresponding recognition model to process the scene, so as to increase the matching degree between the recognition model and the scene and improve the recognition accuracy. Specifically, before determining the helmet recognition result corresponding to the specified helmet color in the specified management area according to the scene picture, it also includes:

[0095] In response to a mode selection operation on the scene picture, determining a target recognition mode from candidate recognition modes, the candidate recognition modes including at least one of a daytime recognition mode and a nighttime recognition mode;

[0096] A third recognition model corresponding to the target recognition pattern is obtained, where the third recognition model is used to determine the helmet recognition result, and the third recognition model is trained by a training image corresponding to the recognition pattern.

[0097] The mode selection operation refers to an operation for determining a target recognition mode. For example, the mode selection operation may include but is not limited to selecting any candidate recognition mode.

[0098] Among them, the recognition model can be a target detection model. For example, the training scene pictures collected during the day can be used as training samples to train the target detection model to obtain a recognition model corresponding to the daytime recognition mode. The training scene pictures collected at night can also be used as training samples to train the target detection model to obtain a recognition model corresponding to the nighttime recognition mode. It can be understood that by using the training scene pictures collected in the daytime / nighttime scenes as training samples, recognition models for different scenes can be trained, and using a model for a single scene can improve the efficiency and accuracy of recognition in that scene.

[0099] For example, in the client of the construction site personnel management system, there is a Figure 1j The mode selection interface shown in the figure shows controls 1 and 2 corresponding to the daytime recognition mode and the nighttime recognition mode, respectively. The system manager can perform a mode selection operation on the mode selection interface, such as touching control 2 corresponding to the nighttime recognition mode through an input device such as a mouse or a touch screen, to use the nighttime recognition mode as the target recognition mode. The construction site personnel management system can call the recognition model corresponding to the nighttime recognition mode as the first recognition model, which is used to determine the recognition target in the site picture.

[0100] 140. Personnel management shall be carried out in the designated management area based on the helmet identification results and the personnel allocation requirements of the designated management area.

[0101] The staffing requirement refers to the number of personnel that need to be deployed in a specified management area. For example, the staffing requirement may include but is not limited to at least one of the total number of personnel, the number of various types of personnel, and the type of personnel.

[0102] Since different colors of hard hats represent different types of personnel on the construction site, the real-time personnel situation on the construction site can be determined through the hard hat recognition results, and personnel can be reasonably allocated and positions adjusted in combination with the personnel allocation requirements of the designated management area to carry out personnel management in the designated management area.

[0103] In some implementations, the real-time staffing of a designated management area can be determined based on the number of hard hats, and personnel management can be performed accurately, in real time, and dynamically based on the matching of the real-time staffing with the staffing requirements of the area to ensure construction safety and efficiency. Specifically, based on the hard hat recognition results and the staffing requirements of the designated management area, personnel management is performed on the designated management area, including:

[0104] Determine the real-time staffing configuration of the designated management area based on the helmet recognition results;

[0105] Matching the real-time staffing configuration of the designated management area with the staffing configuration requirements of the designated management area to obtain a matching result;

[0106] Based on the matching results, personnel management is carried out in the designated management area.

[0107] For example, the helmet recognition result may include the recognition result of the helmet corresponding to the specified helmet color. For example, the helmet recognition result of the designated management area A includes the designated management area A including helmet 1-yellow, helmet 2-yellow, and helmet 3-blue, so that it can be counted and the real-time personnel configuration of the designated management area A can be obtained based on the color of the helmet as 2 people with yellow helmets and 1 person with blue helmet (that is, there are 2 temporary workers and 1 technical worker). If the staffing requirements of the designated management area A are 3 technical workers and 1 person, and 1 person in the field command (that is, the requirements are 3 blue helmets and 1 red helmet), it is obvious that the real-time personnel configuration of the designated management area A does not meet the required personnel configuration. At this time, 2 people with blue helmets and 1 person with red helmet can be dispatched to enter the designated management area A according to the mismatched result. If the real-time personnel configuration of the designated management area A meets the required personnel configuration, no personnel scheduling is performed.

[0108] In some implementations, the personnel configuration requirements of the designated management area can be set autonomously in the corresponding page of the construction site personnel management system to increase the autonomy and convenience of personnel management. Specifically, according to the helmet recognition results and the personnel configuration requirements of the designated management area, before the personnel management of the designated management area is performed, it also includes:

[0109] In response to a staffing operation for a designated management area, a staffing requirement for the designated management area is determined.

[0110] For example, a personnel configuration interface is provided in the construction site personnel management system, and the interface can be displayed in the client corresponding to the construction site personnel management system. Figure 1k The personnel configuration interface shown in the figure may include multiple helmet colors such as yellow, blue, red, white and others. The number of helmets for each color may be input to obtain the number of personnel required for each color. For example, if 3 yellow, 1 blue, 0 red, 0 white and 0 others are input, the personnel configuration requirement is 3 yellow helmets, 1 blue helmet, 0 red helmets, 0 white helmets and 0 others.

[0111] In some embodiments, the real-time staffing of a designated management area does not meet the staffing requirements, which means that the number and type of personnel in the designated management area do not meet the staffing requirements, and there is a lack of a certain type of personnel. The real-time staffing of a designated management area meets the staffing requirements, which means that the number and type of personnel in the designated management area meet or exceed the staffing requirements, and there is no lack of a certain type of personnel. That is, the real-time staffing of a designated management area meets the staffing requirements, including two situations: fully matching the configuration requirements and exceeding the configuration requirements.

[0112] For example, the staffing requirement for the designated management area A is 3 people in blue safety helmets and 1 person in red safety helmets. If the real-time staffing of the designated management area A is 2 people in yellow safety helmets and 1 person in blue safety helmets, there is a shortage of personnel in both the blue safety helmets and red safety helmets. Therefore, the real-time staffing of the designated management area A does not meet the staffing requirement; if the real-time staffing of the designated management area A is 2 people in yellow safety helmets, 3 people in blue safety helmets, and 2 people in red safety helmets, there is no shortage of personnel in both the blue safety helmets and red safety helmets in the staffing requirement. Therefore, the real-time staffing of the designated management area A meets the staffing requirement.

[0113] In some implementations, when the real-time staffing of a designated management area does not meet the staffing requirements, a staff shortage alarm may be generated to remind the user to promptly deploy staff to the area to ensure construction safety and efficiency. Specifically, based on the matching results, the designated management area is managed, including:

[0114] If the matching result is that the real-time staffing of the specified management area does not meet the staffing requirements of the specified management area, a staff shortage alarm is generated.

[0115] For example, the real-time staffing of the designated management area A is 2 people in yellow helmets and 1 person in blue helmet, which is less than its staffing requirement of 3 people in blue helmets and 1 person in red helmet. Figure 1l As shown, a personnel shortage alarm prompt "missing 2 people in blue safety helmets and 1 person in red safety helmet" can be displayed in the on-site picture.

[0116] In some implementations, when the designated management area lacks a designated type of personnel in real-time personnel configuration, a safety alarm can be generated to ensure construction safety. Specifically, according to the matching results, personnel management is performed on the designated management area, including:

[0117] If the matching result is that the real-time personnel configuration of the specified management area lacks personnel of the specified type, a security alarm is generated.

[0118] The designated type of personnel is personnel related to construction safety, and can be set according to the requirements and actual needs related to construction safety. For example, the designated type of personnel can be set to include at least one of a site commander, a safety officer, a site observer, etc.

[0119] For example, the staffing requirements for designated management area A are 1 of 3 technical workers and 1 on-site commander (i.e., 3 people wearing blue helmets and 1 person wearing red helmets), but designated management area A only has 3 people wearing blue helmets. If the designated type of personnel is the on-site commander, it can be determined that the designated management area A lacks 1 designated type of personnel, so the warning message "1 on-site commander is missing, there is a safety hazard" can be displayed on the on-site screen. This allows for timely dispatch of relevant personnel to ensure construction safety.

[0120] In some implementations, when the real-time staffing of a designated management area exceeds the staffing requirements, an overstaffing alarm may be generated to deploy redundant personnel in the designated management area to other areas to ensure construction safety and efficiency. Specifically, based on the matching results, personnel management is performed on the designated management area, including:

[0121] If the matching result is that the real-time staffing of the specified management area exceeds the staffing requirements of the specified management area, an overcrowding alarm is generated.

[0122] Among them, the overcrowding alarm may include but is not limited to image or video alarm, voice warning, and text alarm, etc. The overcrowding alarm may include certain prompt information, such as the alarm location, time, number of people, etc., so as to facilitate timely processing and response.

[0123] For example, the real-time personnel configuration of the designated management area A is 2 people in yellow hard hats, 3 people in blue hard hats, and 2 people in red hard hats, which exceeds its personnel configuration requirements of 3 blue hard hats and 1 red hard hat. Therefore, the 2 yellow hard hats and 1 red hard hat that exceed the personnel configuration requirements can be outlined in a prompt box on the on-site screen.

[0124] In some implementations, when the matching result is that the real-time staffing does not meet the staffing requirements, if some types of personnel in the real-time staffing exceed the requirements in the staffing requirements, when generating a staff shortage alarm, an overstaffing alarm for some types of personnel may be generated. For example, the real-time staffing of the designated management area A is less than its staffing requirements of 3 blue helmets and 1 red helmet, but there are 2 more yellow helmets, so the prompt message "2 blue helmets are missing, 1 red helmet, 2 yellow helmets are overstaffed" may be displayed in the on-site screen.

[0125] In some implementations, personnel can be dispatched from the overcrowded management area to the designated management area based on the personnel dispatch instruction, so as to dispatch personnel between the various management areas at the construction site, reduce the personnel dispatch cost, improve the dispatch efficiency, and ensure the construction safety and efficiency. Specifically, according to the matching results, personnel management is performed on the designated management area, including:

[0126] If the matching result is that the real-time staffing of the designated management area does not meet the staffing requirements of the designated management area, the target management area is searched in the construction site, and the target management area is the management area whose real-time staffing exceeds its corresponding staffing requirements;

[0127] Generate a personnel dispatch instruction, which is used to instruct the dispatch of personnel from the target management area to the designated management area.

[0128] The management area refers to an area where personnel management is to be performed at the construction site. When there are multiple designated management areas, the management area may also include the designated management area and / or other management areas.

[0129] For example, a construction site may include a designated management area A and multiple other management areas C to F. When designated management area A is short of "2 people in blue helmets and 1 person in red helmet", the construction site personnel management system may determine management areas C to F that are overstaffed from management areas C to F, and generate a personnel dispatch instruction in the form of a text message "2 people in blue helmets and 1 person in red helmet need to be dispatched to designated management area A", and send it to the terminals held by the regional persons in charge of management areas C to F, so that the regional persons in charge of management areas C to F can dispatch personnel according to the personnel dispatch instruction.

[0130] In some embodiments, in order to facilitate personnel scheduling, a staffing table may be created, which may record the personnel shortage data and / or overcrowding data of each designated management area, so as to quickly obtain the personnel data of the construction site, accurately, in real time, and dynamically manage personnel, and ensure construction safety and efficiency. For example, the staffing table may record the corresponding data of a designated management area, such as a row corresponding to designated management area A may record "designated management area A; lack of 2 blue helmets, 1 red helmet; overcrowding of 2 yellow helmets".

[0131] In some implementations, personnel dispatch instructions can be automatically matched and generated based on the personnel shortage data and / or overstaffing data of each management area. Specifically, there are multiple designated management areas, and the target management area is a management area whose real-time staffing exceeds its corresponding staffing requirements. The generated personnel dispatch instructions include:

[0132] Match the personnel shortage data of the target designated management area and the overcrowding data of the target management area to obtain a data matching result;

[0133] According to the data matching result, a personnel dispatch instruction is generated, and the target designated management area is the management area where the real-time personnel configuration does not meet its corresponding personnel configuration requirements.

[0134] For example, the construction site personnel management system can call the personnel configuration table to match the personnel shortage data and overcrowding data of each designated management area in the personnel configuration table, determine the target designated management area 1 and the target management area 2 with a high degree of match, such as the overcrowding data meets the shortage data and the shortage data, and generate a personnel dispatching instruction to dispatch personnel from the target management area 2 to the target designated management area 1.

[0135] In some implementations, a confirmation dispatch instruction may be generated based on the dispatchable personnel information fed back by the terminal corresponding to the target management area, so as to dispatch personnel based on the actual situation of the construction site. Specifically, after generating the personnel dispatch instruction, it also includes:

[0136] Send personnel dispatch instructions to the terminal corresponding to the target management area;

[0137] Receiving the reply information to the personnel dispatch instruction sent by the terminal, the reply information includes the dispatchable personnel information;

[0138] Generate confirmation dispatch instructions based on dispatchable personnel information;

[0139] The confirmed dispatch instruction is sent to the terminal so that personnel can be dispatched from the target management area to the designated management area according to the confirmed dispatch instruction.

[0140] The terminal may be a device that can be used to communicate with the construction site personnel management system, such as an intercom, a dispatch phone, a mobile phone, a tablet computer, etc. In some implementations, in order to increase the immediacy of personnel dispatch and improve the efficiency of personnel dispatch, the terminal may be an intercom.

[0141] For example, the construction site personnel management system can determine the management areas C to F with overstaffing from the management areas C to H, and generate a voice-based personnel dispatch instruction "2 blue helmets and 1 red helmet need to be dispatched to the designated management area A", and send it to the intercoms held by the regional managers of the management areas C to F. For example, when the regional manager of the management area C receives the voice "2 blue helmets and 1 red helmet need to be dispatched to the designated management area A", if there are dispatchable personnel in the management area C that he is responsible for, such as 1 red helmet, he can reply the voice "1 red helmet can be dispatched" to the construction site personnel management system through the intercom. The construction site personnel management system can parse the received voice, and according to the parsed data, count whether the dispatchable personnel replied by the management areas C to F meet the shortage of personnel in the designated management area A. If so, it can generate a voice-based confirmed dispatch instruction and send it to the intercom corresponding to the management area to be dispatched in the management areas C to F, so that the regional manager can dispatch the corresponding personnel to the designated management area A based on the received confirmed dispatch instruction.

[0142] In some implementations, when there are many dispatchable personnel, a dispatch instruction can be generated based on the dispatch cost parameters of the target management area to generate the optimal personnel deployment plan and improve the efficiency and rationality of personnel deployment. Specifically, there are multiple target management areas, and a confirmation dispatch instruction is generated based on the dispatchable personnel information, including:

[0143] According to the information of dispatchable personnel, the number of dispatchable personnel is obtained by counting;

[0144] If the number of dispatched personnel exceeds the number of personnel lacking in the designated management area, a dispatchable management area is determined from the target management area according to the dispatch cost parameter of the target management area;

[0145] Generate a confirmation scheduling instruction corresponding to the schedulable management area;

[0146] Send confirmation dispatch instructions to the terminal, including:

[0147] The confirmation scheduling instruction is sent to the terminal corresponding to the schedulable management area.

[0148] Among them, dispatchable personnel refer to personnel who exceed the staffing requirements of the target management area and can be dispatched.

[0149] Among them, the scheduling cost parameter refers to the parameter used to evaluate the cost of scheduling personnel from the target management area. For example, the scheduling cost parameter may include but is not limited to at least one of the area distance, the number of dispatchable personnel, the workload, the urgency of the construction period, etc.

[0150] For example, the construction site personnel management system can obtain the dispatchable personnel information sent by the intercom corresponding to the target management area C~F where there is an overcrowding situation, and summarize and count all the dispatchable personnel information received from the management area C~F to obtain the number of various dispatchable personnel, such as "5 people in blue helmets, 3 people in red helmets, and 1 person in yellow helmet". Obviously, the number of people corresponding to the blue helmets and red helmets exceeds the "2 people in blue helmets and 1 person in red helmet" missing in the designated management area A. At this time, the management areas C~F can be sorted according to the regional distance, and / or the number of dispatchable personnel, and / or the workload, and / or the urgency of the construction period. The regional distance refers to the distance between the target management area and the designated management area A, and the number of regional overcrowding refers to the number of dispatchable personnel in the target management area. The closer the regional distance, the more dispatchable personnel, the smaller the workload, and the lower the urgency of the construction period, the higher the ranking of the target management area. Therefore, the number of dispatchable personnel in the sorted target management areas can be accumulated in sequence from front to back. When the obtained number reaches the number of personnel missing in the designated management area A, such as the first two target management areas C and E after sorting have a total of "2 people in blue safety helmets and 1 person in red safety helmet" dispatchable personnel, which has reached the number of personnel missing in the designated management area A, therefore, these two target management areas can be used as dispatchable management areas, and confirmation dispatch instructions for the dispatchable personnel in these two management areas are generated, such as "dispatch 1 person in a blue safety helmet and 1 person in a red safety helmet from target management area C to designated management area A" and "dispatch 1 person in a blue safety helmet from target management area F to designated management area A", and these two dispatch instructions are sent to the corresponding walkie-talkies of the target management areas C and E respectively, so that the area persons in charge can dispatch the corresponding personnel to the designated management area A based on the received confirmation dispatch instructions.

[0151] The construction site personnel management solution provided by the embodiment of the present application can be applied to personnel management scenarios of various construction sites. For example, personnel management is performed based on the real-time construction site conditions, a scene picture of the construction site is obtained, and at least one designated safety helmet color is obtained; a designated management area is determined in the scene picture; based on the scene picture, a safety helmet recognition result corresponding to the designated safety helmet color in the designated management area is determined; and personnel management is performed in the designated management area based on the safety helmet recognition result and the personnel configuration requirements of the designated management area.

[0152] From the above, it can be seen that the embodiment of the present application identifies the color of a specified safety helmet in a designated management area based on the on-site picture of the construction site, so that different types of personnel in the designated management area can be automatically distinguished according to the safety helmet recognition results, and personnel management is performed in the designated management area based on the safety helmet recognition results, thereby realizing automated personnel classification and management of the construction site, avoiding errors and delays caused by human management, and ensuring construction safety and efficiency.

[0153] The method described in the above embodiment will be further described in detail below.

[0154] In this embodiment, personnel management based on real-time construction site conditions will be taken as an example to describe the method of the embodiment of the present application in detail.

[0155] The construction site personnel management method of the embodiment of the present application can be implemented by a construction site personnel management system, which includes a helmet area selection module, a helmet wearing and color detection module, a personnel distribution safety supervision module, a personnel scheduling control module, a night safety monitoring module, and a daytime and nighttime mode switching module. The workflow of the construction site personnel management system is as follows: Figure 2a As shown. The system can be deployed on a server, and the system interface of the system, such as the on-site picture and parameter setting page, can be displayed through the corresponding client of the system. In the scenario of personnel management based on the real-time construction site conditions, the video acquisition equipment installed at the construction site collects the on-site pictures of the construction site in real time and sends them to the server in the form of video streams.

[0156] like Figure 2b As shown, the specific process of a construction site personnel management method is as follows:

[0157] 210. Obtain the specified helmet color and recognition model set in the parameter setting page displayed on the client.

[0158] For example, a parameter setting page is provided in the construction site personnel management system, and this page can be displayed on the client corresponding to the construction site personnel management system. The parameter setting page includes a designated safety helmet color selection interface, a personnel configuration interface, and a recognition model setting interface (mode selection interface / recognition parameter setting interface). These three interfaces are respectively. The system administrator can call up the parameter setting page, and set the designated safety helmet color and the recognition parameters corresponding to the recognition model in the designated safety helmet color selection interface and the recognition parameter setting interface, respectively. For example, commonly used safety helmet colors such as yellow, red, blue, white, etc. can be displayed in the designated safety helmet color selection interface according to the needs of the construction project, and single, multiple, or all colors can be selected as designated safety helmet colors. Figure 2a As shown, the helmet wearing and color detection module can obtain the specified helmet color set by the system administrator on the client.

[0159] Due to the low illumination at night, poor image quality will have a certain impact on the algorithm detection results. This article adds the construction of a night construction model to improve the recognition of night construction sites and thus improve the accuracy of the algorithm. For example, the recognition model includes the recognition model corresponding to the night recognition mode and the recognition model corresponding to the daytime recognition mode, such as Figure 2a As shown, the embodiment of the present application can switch modes through the daytime and nighttime mode switching module, and the switching process supports manual mode selection (i.e., manual setting) and automatic mode selection (i.e., automatic setting). In the manual selection mode, the system administrator can set the nighttime construction time through the recognition parameter setting interface to monitor the nighttime construction mode of the construction site, such as: from 19:30 pm to 6:30 am, and select the recognition model corresponding to the nighttime recognition mode. In the automatic mode, the construction screen is classified into two categories through the image classification model, and the mode is selected according to the results of the two categories. For example, the image classification model is trained by collecting pictures of the construction site during the day and at night, so that the model can output the results of the pictures taken in "daytime" and "night" after the pictures are input. If the result is judged to be "daytime", the recognition model corresponding to the daytime recognition mode is automatically matched; if the result is judged to be "nighttime" mode, it is automatically switched to the nighttime monitoring mode to match the recognition model corresponding to the nighttime recognition mode.

[0160] 220. Obtain the on-site image of the construction site and display the on-site image on the client.

[0161] For example, the helmet area selection module can obtain the live picture in the form of a video stream captured by a video capture device in real time and display it on the client.

[0162] 230. In response to an area selection operation in the live picture displayed on the client, a designated management area is determined in the live picture.

[0163] For example, Figure 2a As shown, the helmet area selection module can obtain the system administrator's selection of the scene displayed on the client (i.e., area selection) as needed, which can be all selected, a single area selected, or multiple areas selected. The helmet area selection module can obtain the area selection operation performed by the system administrator on the client to control the boundary between the construction area (i.e., the designated management area) and the non-construction area. Figure 2a As shown, after the helmet wearing and color detection module obtains the specified helmet color and the helmet area selection module obtains the area selection result, the helmet wearing detection can be started.

[0164] In actual applications, subsequent steps may be executed in response to the area selection operation, or subsequent steps such as display screen recognition and personnel deployment may be executed in response to a personnel deployment request generated by the client.

[0165] 240. Obtain the staffing requirements for the specified management area set in the parameter setting page displayed on the client.

[0166] The embodiment of the present application supports configuring the number of personnel required in the area. For example, a key area (designated management area) requires three ordinary workers and one special operations personnel to ensure the safety and quality of the work. The customer can configure it on the client side. The algorithm result will match the color with the personnel type and issue an alarm when the number of personnel is insufficient to ensure the safe progress of the project.

[0167] For example, the system administrator can call up the parameter setting page and set the staffing requirements for each designated management area in the staffing interface of the parameter setting page. The staff distribution security supervision module can obtain the staffing requirements set by the system administrator on the client.

[0168] 250. Call the recognition model to determine the helmet recognition result corresponding to the specified helmet color in the specified management area according to the on-site picture.

[0169] For example, the daytime and nighttime mode switching module can call the corresponding recognition model according to the recognition model set by the system administrator on the client. The nighttime safety monitoring module / daytime safety monitoring module corresponding to the recognition model recognizes the scene. For example, the recognition model corresponding to the nighttime recognition mode uses the nighttime safety monitoring module for recognition, and the recognition model corresponding to the daytime recognition mode uses the daytime safety monitoring module for recognition. The helmet recognition results include the recognition results of the helmets corresponding to each specified helmet color and the recognition results of people who are not wearing helmets.

[0170] In actual applications, the recognition result will be matched with the user-set area (the designated management area determined by the helmet area selection module). Only when the helmet is not worn in the construction area will an alarm be issued to reduce unnecessary alarms. When the recognition result is a person not wearing a helmet, the helmet wearing and color detection module can generate an alarm in the form of a pop-up window and display the pop-up window prompt in the lower right corner of the client. Click the pop-up window to view details to view the specific situation on site, personnel distribution and real-time video, etc. Figure 2a As shown, when the system administrator sets it to night recognition mode, the scene can be identified through the recognition model corresponding to the night recognition mode in the night safety detection module. If the system administrator does not set it, the scene will be identified by the recognition model corresponding to the daytime recognition mode in the helmet wearing and color detection module. After the scene is identified by the recognition model, the helmet detection result (i.e., the identification target) can be obtained. For example Figure 2aAs shown, the personnel distribution safety supervision module matches the identification target with the designated management area, determines the identification target located in the designated management area, and screens the color of the helmet to obtain the helmet identification result.

[0171] Conventional helmet recognition only judges whether a helmet is worn. When the color of a person's hair is close to that of the helmet, missed detection may occur. The embodiment of the present application selects the color of the helmet according to the actual situation at the construction site, and trains the model for each color, thereby improving the accuracy of the helmet construction algorithm recognition. The target detection model is used to perform recognition training on helmet images of various colors separately to obtain the results. The target detection model mainly consists of four main contents: Conv layers, Region Proposal Networks, Roi Pooling, and Classification. For an image of arbitrary size P×Q (i.e., the live picture in the video stream), it is first scaled to a fixed size M×N, and then the M×N image is sent to the convolutional layer to extract features to obtain a feature map; the region proposal network is used to generate an Anchor box, which is cropped and filtered for binary classification (foreground or background, that is, an object or not), and the bounding box regression is used to correct the Anchor box to form a more accurate proposal; Roi Pooling extracts proposal feature maps from the collected feature maps and proposals, and sends them to the fully connected layer to determine the target category, and finally the classification probability and bounding box regression are jointly trained to improve the accuracy.

[0172] 260. Determine the real-time personnel configuration of the designated management area based on the helmet recognition results.

[0173] Usually, yellow safety helmets are used on construction sites to represent ordinary workers or other construction personnel; blue safety helmets are used to represent technical personnel or personnel engaged in special operations; red safety helmets are used to represent management personnel or on-site commanders; and white safety helmets are used to represent project supervisors or leaders of the entire project. Therefore, the types of workers can be distinguished by the color of the safety helmets. For example, Figure 2a As shown, the personnel distribution safety supervision module can judge the distribution of personnel in the key area (designated management area) according to the helmet recognition result output by the helmet wearing and color detection module (i.e. the result after filtering the color of the helmet), and perform safety judgment and safety management (safety control) on the designated management area.

[0174] 270. Match the real-time staffing configuration of the designated management area with the staffing configuration requirements of the designated management area, and perform personnel management on the designated management area.

[0175] The personnel dispatching and control module can further dispatch personnel and manage progress (progress control) based on the personnel distribution determined by the personnel distribution safety supervision module. Whether the number of personnel in the designated management area meets the construction requirements, if not, an alarm will be issued. For example, the construction of a designated management area must ensure that there are three workers and one on-site commander, but by judging that there are only two workers in the area and no on-site commander, it is judged that the construction does not meet the safety requirements, and an alarm will be issued for safety reminders.

[0176] The personnel scheduling and control module can also preview the distribution of workers by the color of the helmets, and schedule and optimize the distribution of workers accordingly to improve the efficiency of project construction. For example, the personnel scheduling and control module can display the number of various types of workers on the client based on the recognition results. When there is a shortage of workers in some areas, it can find areas with more workers and schedule them based on the distribution of workers, thereby ensuring the smooth progress of the project.

[0177] During dispatch, dispatchers such as system administrators can observe the overall situation of the construction site, such as the distribution of personnel in each area of ​​the construction site, and can conduct real-time video confirmation of the area of ​​interest. After confirming the distribution of personnel, they can communicate with responding personnel such as dispatchable personnel or regional managers of dispatchable management areas through voice intercom, so as to conduct dispatch and interactive guidance. For example, Figure 2cThe voice intercom process shown in the figure performs signaling interaction between the dispatcher and the responder. In the process, the voice input device / voice input networking system (referred to as "voice stream sender"), such as the dispatcher's handheld intercom, and the SIP server send a notification message to the voice output device / voice output video networking system (referred to as "voice stream receiver"), such as the responder's handheld intercom. After receiving the notification message, the voice stream receiver makes a judgment and processes it. If it can receive the broadcast, it initiates a call request to the voice stream sender to obtain the broadcast media stream. Among them, signaling 1, 2, 3, and 4 are the message processes of voice broadcast notification and voice broadcast response; signaling 5, 12, 13, 14, 15, and 16 are the media stream signaling process in which the SIP server receives the call request from the client and establishes the media stream signaling process between the voice stream receiver and the media server through the B2BUA proxy; signaling 6 to 11 are the media stream signaling process in which the SIP server establishes the media stream signaling process between the media server and the voice stream sender through three-party call control; signaling 17 to 20 are the media stream signaling process in which the SIP server disconnects the media stream signaling process between the voice stream receiver and the media server; and signaling 21 to 24 are the media stream signaling process in which the SIP server disconnects the media server and the voice stream sender.

[0178] As can be seen from the above, the construction site personnel management system of the embodiment of the present application, based on the detection of whether the helmet is worn, distinguishes the color of the helmet according to the on-site situation; then distinguishes the personnel by the color of the helmet, and can judge the category of workers to obtain the personnel allocation in real time, and perform global personnel assignment and global scheduling of projects when necessary. In this way, the accuracy of the construction site helmet recognition can be improved, and the personnel distribution can be judged according to the recognition results, the construction progress can be understood, and then the construction safety of the construction site can be guaranteed, and the construction progress can be better controlled.

[0179] Compared with the conventional helmet detection algorithm, the embodiment of the present application passes all images into a unified model for training to obtain a set of target detection models. However, in actual use, the camera captures clear images during the day when there is sufficient light, but at night, the images captured by the camera are affected by the light and are black and white. If the model obtained under the condition of sufficient light during the day is still used for judgment, the algorithm recognition accuracy will be reduced. However, the embodiment of the present application supports both daytime and nighttime detection methods, which are suitable for nighttime construction scenes. Moreover, for nighttime construction scenes, nighttime images are specially collected for model building to improve the accuracy of nighttime construction. In addition, the embodiment of the present application supports manual and automatic selection methods, and the daytime and nighttime methods can be switched.

[0180] In order to better implement the above method, the embodiment of the present application also provides a construction site personnel management device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0181] For example, in this embodiment, the method of the embodiment of the present application will be described in detail by taking the construction site personnel management device specifically integrated in the server as an example.

[0182] For example, Figure 3 As shown, the construction site personnel management device may include an acquisition unit 310, a determination unit 320, an identification unit 330, and a management unit 340, as follows:

[0183] (I) Acquisition Unit 310

[0184] Used to obtain on-site images of the construction site and obtain at least one specified safety helmet color.

[0185] (ii) Determination unit 320

[0186] Used to identify the designated management area in the live screen.

[0187] In some implementations, the determining unit 320 may be specifically configured to:

[0188] In response to an area selection operation in the live picture, a designated management area is determined in the live picture.

[0189] (III) Identification Unit 330

[0190] It is used to determine the helmet recognition result corresponding to the specified helmet color in the specified management area based on the on-site picture.

[0191] In some implementations, the identification unit 330 may be specifically configured to:

[0192] In the scene picture, determine the identification target, the identification target includes the safety helmet corresponding to the specified safety helmet color;

[0193] Obtain the target position of the identified target in the scene picture;

[0194] The helmet recognition result is determined based on the identification target whose target position is within the designated management area.

[0195] In some implementations, determining and identifying a target in a live scene includes:

[0196] Extract features from the scene image to obtain a feature map;

[0197] Obtain multiple candidate regions from the feature map;

[0198] Perform binary classification on multiple candidate regions to determine the foreground region among the multiple candidate regions;

[0199] Perform helmet color classification on the foreground area to determine the recognition target.

[0200] In some implementations, the identification unit 330 may also be used to:

[0201] Classify and process the scene images to obtain classification results;

[0202] A first recognition model corresponding to the classification result is obtained, where the first recognition model is used to determine the helmet recognition result, and the first recognition model is trained by a training image corresponding to the classification result.

[0203] In some implementations, the identification unit 330 may also be used to:

[0204] Get the picture parameters of the live picture;

[0205] A second recognition model corresponding to the picture parameters is obtained, where the second recognition model is used to determine a helmet recognition result, and the second recognition model is trained using a training image corresponding to the picture parameters.

[0206] (IV) Management Unit 340

[0207] It is used to manage personnel in the designated management area according to the helmet recognition results and the personnel allocation requirements of the designated management area.

[0208] In some implementations, the management unit 340 may be specifically configured to:

[0209] Determine the real-time staffing configuration of the designated management area based on the helmet recognition results;

[0210] Matching the real-time staffing configuration of the designated management area with the staffing configuration requirements of the designated management area to obtain a matching result;

[0211] Based on the matching results, personnel management is carried out in the designated management area.

[0212] In some implementations, personnel management is performed on the designated management area based on the matching results, including:

[0213] If the matching result is that the real-time staffing of the specified management area does not meet the staffing requirements of the specified management area, a staff shortage alarm is generated.

[0214] In some implementations, personnel management is performed on the designated management area based on the matching results, including:

[0215] If the matching result is that the real-time staffing of the designated management area does not meet the staffing requirements of the designated management area, the target management area is searched in the construction site, and the target management area is the management area whose real-time staffing exceeds its corresponding staffing requirements;

[0216] Generate a personnel dispatch instruction, which is used to instruct the dispatch of personnel from the target management area to the designated management area.

[0217] In some implementations, the management unit 340 may also be configured to:

[0218] Send personnel dispatch instructions to the terminal corresponding to the target management area;

[0219] Receiving the reply information to the personnel dispatch instruction sent by the terminal, the reply information includes the dispatchable personnel information;

[0220] Generate confirmation dispatch instructions based on dispatchable personnel information;

[0221] The confirmed dispatch instruction is sent to the terminal so that personnel can be dispatched from the target management area to the designated management area according to the confirmed dispatch instruction.

[0222] In some embodiments, the identification target also includes a person who is not wearing a helmet, and the management unit 340 can also be used to:

[0223] If the target position of the person not wearing a helmet in the on-site image is within the designated management area, a not wearing helmet alarm is generated.

[0224] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0225] Therefore, the embodiment of the present application can identify the color of the specified safety helmet in the designated management area based on the on-site picture of the construction site, so that different types of personnel in the designated management area can be automatically distinguished according to the safety helmet recognition results, and personnel management in the designated management area can be performed based on the safety helmet recognition results, thereby realizing automated personnel classification and management of the construction site, avoiding errors and delays caused by human management, and ensuring construction safety and efficiency.

[0226] The embodiment of the present application also provides an electronic device, which can be a terminal, a server, etc. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc. The server can be a single server or a server cluster composed of multiple servers, etc.

[0227] In some embodiments, the construction site personnel management device can also be integrated into multiple electronic devices. For example, the construction site personnel management device can be integrated into multiple servers, and the construction site personnel management method of the present application can be implemented by multiple servers.

[0228] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 4 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present application, specifically:

[0229] The server may include one or more processing core processors 410, one or more computer-readable storage media memories 420, a power supply 430, an input module 440, and a communication module 450. Those skilled in the art will appreciate that Figure 4 The server structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0230] The processor 410 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 420, and calling data stored in the memory 420. In some embodiments, the processor 410 may include one or more processing cores; in some embodiments, the processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 410.

[0231] The memory 420 can be used to store software programs and modules. The processor 410 executes various functional applications and data processing by running the software programs and modules stored in the memory 420. The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 420 may also include a memory controller to provide the processor 410 with access to the memory 420.

[0232] The server also includes a power supply 430 for supplying power to various components. In some embodiments, the power supply 430 can be logically connected to the processor 410 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 430 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0233] The server may further include an input module 440, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0234] The server may further include a communication module 450. In some embodiments, the communication module 450 may include a wireless module. The server may perform short-range wireless transmission through the wireless module of the communication module 450, thereby providing wireless broadband Internet access to the user. For example, the communication module 450 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0235] Although not shown, the server may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 410 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 420 according to the following instructions, and the processor 410 will run the application programs stored in the memory 420, thereby realizing various functions, as follows:

[0236] Obtain a live picture of the construction site and at least one designated safety helmet color; determine a designated management area in the live picture; determine a safety helmet recognition result corresponding to the designated safety helmet color in the designated management area based on the live picture; perform personnel management on the designated management area based on the safety helmet recognition result and the personnel allocation requirements of the designated management area.

[0237] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0238] From the above, it can be seen that the embodiment of the present application identifies the color of a specified safety helmet in a designated management area based on the on-site picture of the construction site, so that different types of personnel in the designated management area can be automatically distinguished according to the safety helmet recognition results, and personnel management is performed in the designated management area based on the safety helmet recognition results, thereby realizing automated personnel classification and management of the construction site, avoiding errors and delays caused by human management, and ensuring construction safety and efficiency.

[0239] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0240] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the construction site personnel management methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0241] Based on the on-site images of the construction site, the specified safety helmet color in the designated management area is identified. Different types of personnel in the designated management area can be automatically distinguished based on the safety helmet recognition results. Personnel management in the designated management area is carried out based on the safety helmet recognition results, realizing automated personnel classification and management on the construction site, avoiding errors and delays caused by human management, and ensuring construction safety and efficiency.

[0242] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0243] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations provided in the above embodiments.

[0244] Since the instructions stored in the storage medium can execute the steps in any construction site personnel management method provided in the embodiments of the present application, the beneficial effects that can be achieved by any construction site personnel management method provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0245] The above is a detailed introduction to a construction site personnel management method, device, equipment, medium and program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A construction site personnel management method, characterized in that: include: Obtaining a live picture of the construction site and obtaining at least one designated safety helmet color; Determining a designated management area in the live picture; Determine, according to the on-site image, a helmet recognition result corresponding to the designated helmet color in the designated management area; Personnel management is performed on the designated management area according to the helmet recognition result and the personnel configuration requirements of the designated management area.

2. The construction site personnel management method according to claim 1, characterized in that: Determining the designated management area in the on-site picture includes: In response to an area selection operation in the live picture, the designated management area is determined in the live picture.

3. The construction site personnel management method according to claim 1, characterized in that: The step of determining, according to the on-site image, a helmet recognition result corresponding to the designated helmet color in the designated management area includes: In the scene picture, determining a recognition target, the recognition target including a safety helmet corresponding to the designated safety helmet color; Acquire the target position of the identified target in the scene picture; According to the identification target where the target position is located within the designated management area, a helmet identification result is determined.

4. The construction site personnel management method according to claim 3, characterized in that: Determining the identification target in the scene picture includes: Extracting features from the scene image to obtain a feature map; Acquire multiple candidate regions from the feature map; Performing binary classification processing on the multiple candidate regions to determine a foreground region among the multiple candidate regions; The foreground area is subjected to helmet color classification processing to determine the recognition target.

5. The construction site personnel management method according to claim 3, characterized in that: The identification target also includes a person who is not wearing a helmet. After determining the helmet identification result corresponding to the designated helmet color in the designated management area according to the on-site picture, the method further includes: If the target position of the person not wearing a helmet in the on-site image is within the designated management area, a warning of not wearing a helmet is generated.

6. The construction site personnel management method according to claim 1, characterized in that: The performing personnel management on the designated management area according to the helmet recognition result and the personnel configuration requirements of the designated management area includes: Determining the real-time personnel configuration of the designated management area according to the helmet recognition result; Matching the real-time staffing configuration of the designated management area with the staffing configuration requirements of the designated management area to obtain a matching result; According to the matching result, personnel management is performed on the designated management area.

7. The construction site personnel management method according to claim 6, characterized in that: The performing personnel management on the designated management area according to the matching result includes: If the matching result is that the real-time staffing of the designated management area does not meet the staffing requirements of the designated management area, a staff shortage alarm is generated.

8. The construction site personnel management method according to claim 6, characterized in that: The performing personnel management on the designated management area according to the matching result includes: If the matching result is that the real-time staffing of the designated management area does not meet the staffing requirement of the designated management area, searching for a target management area in the construction site, the target management area being a management area where the real-time staffing exceeds the corresponding staffing requirement; A personnel dispatching instruction is generated, where the personnel dispatching instruction is used to instruct to dispatch personnel from the target management area to the designated management area.

9. The construction site personnel management method according to claim 8, characterized in that: After generating the personnel dispatch instruction, the method further includes: Sending the personnel dispatch instruction to the terminal corresponding to the target management area; receiving a reply message to the personnel dispatch instruction from the terminal, wherein the reply message includes information of dispatchable personnel; Generate a confirmation dispatch instruction according to the dispatchable personnel information; The confirmed dispatch instruction is sent to the terminal so as to dispatch personnel from the target management area to the designated management area according to the confirmed dispatch instruction.

10. The construction site personnel management method according to any one of claims 1 to 9, characterized in that: Before determining the helmet recognition result corresponding to the designated helmet color in the designated management area according to the on-site picture, the method further includes: Classify the scene images to obtain classification results; A first recognition model corresponding to the classification result is obtained, where the first recognition model is used to determine the helmet recognition result, and the first recognition model is trained by a training image corresponding to the classification result.

11. The construction site personnel management method according to any one of claims 1 to 9, characterized in that: Before determining the helmet recognition result corresponding to the designated helmet color in the designated management area according to the on-site picture, the method further includes: Acquire picture parameters of the live picture; A second recognition model corresponding to the picture parameters is obtained, where the second recognition model is used to determine the helmet recognition result, and the second recognition model is trained by the training image corresponding to the picture parameters.

12. A construction site personnel management device, characterized in that: include: An acquisition unit, used to acquire a scene image of the construction site and acquire at least one designated safety helmet color; A determination unit, used to determine a designated management area in the scene picture; An identification unit, configured to determine, according to the on-site picture, a helmet identification result corresponding to the designated helmet color in the designated management area; The management unit is used to manage the personnel in the designated management area according to the helmet recognition result and the personnel configuration requirements of the designated management area.

13. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps in the construction site personnel management method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the construction site personnel management method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the construction site personnel management method according to any one of claims 1 to 11 are implemented.

Images