High-altitude operation safety detection method and device and nonvolatile storage medium

Abstract

The invention discloses a high-altitude operation safety detection method and device and a nonvolatile storage medium. The method comprises the steps that a target image is acquired, the target image is an image of a target object collected at a high-altitude operation site, and the target object comprises high-altitude operation personnel; face information in the target image is determined, and whether the target object in the target image has the high-altitude operation qualification or not is judged according to the face information; the classification model is adopted, the safety belt wearing state of the target object in the target image is determined, and the safety belt wearing state comprises a worn state and a non-worn state; when it is judged that the target object does not have the high-altitude operation qualification or the wearing state of the safety belt of the target object is a non-wearing state, alarm information is sent, and the alarm information is used for forbidding the target object from performing high-altitude operation. The technical problems of lack of real-time monitoring and high management cost caused by adopting a manual inspection mode for wearing of the safety belt in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of wireless communication and terminal technology, and in particular, to a method and device for detecting safety of aerial work, and a non-volatile storage medium. Background Art

[0002] High-altitude work is accompanied by high safety risks and has always been the top priority of construction unit safety work. According to the high-altitude work operation specifications, workers must wear safety ropes when working above 1.8 meters above the ground. This measure can effectively ensure the safety of workers. However, despite clear safety regulations, in actual operations, due to insufficient safety awareness of workers or considerations of work efficiency, sometimes safety ropes are not worn or worn incorrectly.

[0003] Traditional supervision methods mainly rely on manual on-site inspections, that is, construction units need to dispatch full-time personnel to conduct regular or irregular safety inspections on high-altitude work sites to check whether the workers comply with safety regulations and wear appropriate protective equipment. This method has obvious limitations and problems. First, manual inspections are difficult to achieve all-weather, all-round real-time monitoring. Often when inspectors arrive, workers temporarily wear safety ropes to cope with inspections. After the inspectors leave, they may remove the safety ropes out of luck or to improve efficiency, resulting in safety hazards. Secondly, frequent manual inspections increase management costs, including labor costs, travel costs, and time costs. This cost increase is particularly significant for high-altitude work sites scattered in different geographical locations.

[0004] In particular, in the field of communication base station maintenance, with the large-scale construction of 5G base stations, the demand for high-altitude operations has become more frequent and the working environment has become more complex. The traditional manual supervision method is not only inefficient, but also difficult to cover all work sites. It cannot effectively prevent safety accidents caused by not wearing safety ropes, which poses a potential threat to the personal safety of workers, family happiness and the normal operation of the enterprise.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a method, device and non-volatile storage medium for high-altitude work safety detection, so as to at least solve the technical problems of lack of real-time monitoring and high management costs caused by the manual inspection of safety belt wearing in the related art.

[0007] According to one aspect of an embodiment of the present application, a method for high-altitude work safety detection is provided, including: acquiring a target image, wherein the target image is an image of a target object collected at a high-altitude work site, and the target object includes: a person working at high altitude; determining facial information in the target image, and judging whether the target object in the target image has the qualifications for high-altitude work based on the facial information; using a classification model to determine the seat belt wearing status of the target object in the target image, wherein the seat belt wearing status includes: worn, not worn; when it is determined that the target object does not have the qualifications for high-altitude work, or the seat belt wearing status of the target object is not worn, sending an alarm message, wherein the alarm message is used to prohibit the target object from performing high-altitude work.

[0008] Optionally, determining the facial information in the target image includes: using a target cascade classifier to construct a feature integral map corresponding to the target image, wherein the feature integral map is used to calculate the pixel sum of any rectangular area in the target image; based on the feature integral map, determining the target features corresponding to each candidate area in the target image, wherein the target features are determined by calculating the difference in pixel values ​​within the candidate area; using a weak classifier in the target cascade classifier to determine whether the candidate area belongs to a face area based on the target features; when all weak classifiers in the target cascade classifier determine that the candidate area belongs to a face area, determining the candidate area as a face area, thereby obtaining the facial information in the target image, wherein the facial information includes: the position and size of the face.

[0009] Optionally, judging whether the target object in the target image has the qualification for high-altitude work based on the facial information includes: using a target convolutional neural network to extract features of the facial area indicated by the facial information in the target image to obtain a first facial feature; determining the cosine similarity between the first facial feature and a second facial feature in a qualification database, wherein the second facial feature corresponding to facial images of staff with high-altitude work qualification certificates at various angles stored in the qualification database, and the high-altitude work qualification certificates include: a climbing certificate and a low-voltage electrician certificate; when there is a second facial feature in the qualification database whose cosine similarity with the first facial feature is greater than a preset similarity threshold, it is judged that the target object in the target image has the qualification for high-altitude work.

[0010] Optionally, the target convolutional neural network is obtained by training a VGG16 convolutional neural network based on a general face dataset, wherein the general face dataset includes facial images of a variety of different target objects, and the VGG16 convolutional neural network includes: multiple convolutional layers, fully connected layers, and pooling layers of different scales, the convolutional layer is used to extract feature maps of the image, the pooling layer is used to reduce the spatial dimension of the feature map, the fully connected layer is used to integrate the features, and introduce nonlinear features through activation functions.

[0011] Optionally, using a classification model to determine the seat belt wearing status of a target object in a target image includes: using a classification model to segment the target image into multiple candidate sub-images; determining a target sub-image corresponding to a seat belt wearing position from the candidate sub-images, and performing feature extraction on the target sub-image to obtain image features; and determining the seat belt wearing status of the target object in the target image based on the image features.

[0012] Optionally, the training step of the classification model includes: obtaining an initial training data set, and preprocessing the images in the initial training data set to obtain a target training data set, wherein the initial training data set includes: images of multiple different target objects at different angles with and without seat belts, and seat belt wearing status labels corresponding to each image; using an initial network model to predict the seat belt wearing status corresponding to the images in the initial training data set, and determining a loss function value based on the seat belt wearing status predicted by the initial network model and the seat belt wearing status labels; using a back propagation algorithm to train model parameters in the initial network model based on the loss function value to obtain a classification model, wherein during the training process, a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate of the model based on the training rounds.

[0013] Optionally, preprocessing the images in the initial training data set includes: randomly adjusting the size of the images in the initial training data set; and / or randomly flipping the images in the initial training data set, or rotating them by a random angle within a preset angle range; and / or adding a preset white noise value to the pixel value of each point in the image in the initial training data set, and binarizing the image after adding the preset white noise value; and / or standardizing the pixel value of each point in the image in the initial training data set.

[0014] According to another aspect of the embodiment of the present application, a high-altitude work safety detection device is also provided, including: an image acquisition module, used to acquire a target image, wherein the target image is an image of a target object collected at a high-altitude work site, and the target object includes: high-altitude workers; a qualification verification module, used to determine the facial information in the target image, and based on the facial information, determine whether the target object in the target image has the qualifications for high-altitude work; a seat belt verification module, used to use a classification model to determine the seat belt wearing status of the target object in the target image, wherein the seat belt wearing status includes: worn, not worn; a monitoring and early warning module, used to send an alarm message when it is determined that the target object does not have the qualifications for high-altitude work, or the seat belt wearing status of the target object is not worn, wherein the alarm message is used to prohibit the target object from performing high-altitude work.

[0015] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above-mentioned high-altitude work safety detection method is executed when the program is running.

[0016] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned high-altitude work safety detection method by running the computer program.

[0017] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, which implements the steps of the above-mentioned high-altitude work safety detection method when executed by a processor.

[0018] In an embodiment of the present application, a target image is acquired, wherein the target image is an image of a target object collected at a high-altitude operation site, and the target object includes: a person working at high altitude; facial information in the target image is determined, and based on the facial information, whether the target object in the target image is qualified for high-altitude operation is judged; a classification model is used to determine the seat belt wearing status of the target object in the target image, wherein the seat belt wearing status includes: worn, not worn; when it is determined that the target object is not qualified for high-altitude operation, or the seat belt wearing status of the target object is not worn, an alarm message is sent, wherein the alarm message is used to prohibit the target object from performing high-altitude operation, thereby achieving the purpose of continuous real-time monitoring of high-altitude operation, thereby solving the technical problems of lack of real-time monitoring and high management costs caused by the manual inspection of seat belt wearing in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 It is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for high-altitude work safety detection provided in an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of a method flow for high-altitude work safety detection provided according to an embodiment of the present application;

[0022] Figure 3 It is a structural schematic diagram of a high-altitude work safety detection system provided according to an embodiment of the present application;

[0023] Figure 4It is a flowchart of a process of detecting a face using a Haar cascade classifier provided according to an embodiment of the present application.

[0024] Figure 5 It is a structural diagram of a VGG16 convolutional neural network provided according to an embodiment of the present application.

[0025] Figure 6 It is a RestNet structure diagram provided according to an embodiment of the present application.

[0026] Figure 7 It is a structural schematic diagram of a high-altitude work safety detection device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] In the related art, the traditional manual inspection method cannot provide continuous real-time monitoring, and the process of manually monitoring the wearing of the safety rope for aerial work is time-consuming and inefficient. Therefore, there are problems of lack of real-time monitoring and high management costs. In order to solve this problem, the embodiments of the present application provide relevant solutions, which are described in detail below.

[0030] According to an embodiment of the present application, a method embodiment of high-altitude work safety detection is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or electronic device) for implementing a high-altitude work safety detection method is shown. Figure 1 As shown, the computer terminal 10 (or electronic device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0032] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or electronic device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the aerial work safety detection method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned aerial work safety detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or electronic device).

[0036] Under the above operating environment, the embodiment of the present application provides a method for detecting safety of aerial work. Figure 2 is a schematic diagram of a method flow for high-altitude work safety detection provided in accordance with an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0037] Step S202, obtaining a target image, wherein the target image is an image of a target object collected at a high-altitude operation site, and the target object includes: a high-altitude operation worker.

[0038] In step S202, the aerial work site is equipped with a camera or mobile device for collecting real-time images of the work site. These devices transmit the collected images to the safety rope wearing recognition system, and recognize these images as target images, which include personnel performing aerial work. The target image can be collected as a continuous video stream or a single frame image, depending on the implementation scenario and requirements.

[0039] Step S204, determining the facial information in the target image, and judging whether the target object in the target image is qualified for high-altitude work based on the facial information.

[0040] In step S204, first, the target image is processed using a Haar cascade classifier to detect and select the face area therein. Then, the detected facial features are extracted and analyzed by a VGG16 convolutional neural network, compared with a pre-stored qualification database, and the cosine similarity is calculated to determine whether the face matches a person with high-altitude work qualifications. If the similarity is lower than a preset threshold (e.g., 0.5), the target object is considered to be a high-altitude work personnel who has been verified for qualifications and is qualified for high-altitude work; otherwise, the target object is considered to be unqualified for high-altitude work.

[0041] Step S206: using a classification model to determine the seat belt wearing status of the target object in the target image, wherein the seat belt wearing status includes: worn and not worn.

[0042] In step S206, after determining that the target object is qualified for high-altitude work, the target image is further analyzed using the ResNet18 convolutional neural network classification model. The model is pre-trained and can distinguish whether the high-altitude workers are wearing safety belts correctly. Specifically, in this embodiment, the safety belt refers to the safety rope. The target image is then input into the model, and the model will output a binary classification result, that is, whether the target object is wearing a safety rope. This step is completed by analyzing the features related to the safety rope in the target image, such as the unique color, texture and shape of the safety rope.

[0043] Step S208, when it is determined that the target object does not have the qualification for high-altitude work, or the target object's safety belt wearing status is not worn, a warning message is sent, wherein the warning message is used to prohibit the target object from performing high-altitude work.

[0044] In step S208, if the target object obtained through the above steps does not have the qualifications for high-altitude work, or the target object does not wear the safety rope correctly, the alarm mechanism will be automatically triggered and an alarm message will be sent to the on-site manager or remote monitoring center. The alarm message details the specific circumstances of the violation, such as "not meeting the qualifications for high-altitude work" or "not wearing a safety rope", and recommends immediate preventive measures, such as prohibiting the target object from performing high-altitude work until it meets the qualification requirements and the safety rope wearing standards. The alarm information can be sent through SMS, phone calls, on-site voice warnings, or system pop-ups, etc., depending on the implementation environment and system configuration.

[0045] Through the above steps, a target image is obtained, wherein the target image is an image of a target object collected at a high-altitude operation site, and the target object includes: a person working at high altitude; facial information in the target image is determined, and based on the facial information, whether the target object in the target image has the qualifications for high-altitude operation is judged; a classification model is used to determine the seat belt wearing status of the target object in the target image, wherein the seat belt wearing status includes: worn, not worn; when it is determined that the target object does not have the qualifications for high-altitude operation, or the seat belt wearing status of the target object is not worn, an alarm message is sent, wherein the alarm message is used to prohibit the target object from performing high-altitude operation, thereby achieving the purpose of continuous real-time monitoring of high-altitude operation.

[0046] The following is a further introduction to the high-altitude work safety detection method in steps S202 to S208 of the embodiment of the present application.

[0047] In some embodiments of the present application, determining the facial information in the target image can be achieved through the following steps: using a target cascade classifier to construct a feature integral map corresponding to the target image, wherein the feature integral map is used to calculate the pixel sum of any rectangular area in the target image; based on the feature integral map, determining the target features corresponding to each candidate area in the target image, wherein the target features are determined by calculating the difference in pixel values ​​within the candidate area; using a weak classifier in the target cascade classifier to determine whether the candidate area belongs to the face area based on the target features; when all weak classifiers in the target cascade classifier determine that the candidate area belongs to the face area, determining the candidate area as a face area, thereby obtaining the facial information in the target image, wherein the facial information includes: the position and size of the face.

[0048] In the above embodiment, first, an integral map is constructed for the target image collected at the aerial work site. The integral map is a data structure used to quickly calculate the pixel sum of any rectangular area in the image, thereby improving the calculation efficiency of the Haar feature. The integral map is constructed based on the pixel value of the image, and the formula can be expressed as:

[0049]

[0050] Among them, integral_image(i,j) represents the value of the integral image at position (x,y), and f(i,j) represents the value of the original image at pixel position (i,j). Through the integral image, the sum of pixels in any rectangular area can be calculated with constant time complexity, which is crucial for subsequent feature extraction and face detection.

[0051] Then, the feature integral map is used to calculate the Haar feature value of each candidate area in the target image. It should be noted that the Haar feature value is an indicator to measure the brightness difference between adjacent areas in the image. By calculating the difference in pixel values ​​within the candidate area, possible facial features can be identified. This process involves multiple types of Haar features, such as edge features, center features, diagonal features, etc. Each feature is quickly calculated and stored through the integral map.

[0052] Secondly, the weak classifier in the target cascade classifier is used to determine whether the candidate region belongs to the face region based on the target features. Specifically, in the target cascade classifier, the pre-trained weak classifier is applied to each candidate region one by one, and the judgment is made based on the extracted Haar feature value. The weak classifier is trained using the Adaboost algorithm and can identify a part of the image region with specific face features. If a candidate region passes the test of the weak classifier, that is, its feature value meets the judgment criteria of the weak classifier, then the region is likely to be part of the face.

[0053] Finally, when all weak classifiers in the target cascade classifier determine that the candidate region belongs to the face region, the candidate region is determined to be a face region. The cascade classifier in the system is composed of multiple weak classifiers to form a strong classifier. These weak classifiers are cascaded in a certain order and logic. Only when all weak classifiers determine that the candidate region is a positive example (i.e., it belongs to the face region), the region is finally confirmed as a face. This process ensures the accuracy and robustness of recognition and reduces the false positive and false negative rates. Once the candidate region is confirmed to be a face, the face information of the region is output, including the location and size of the face. This information will be used for subsequent deep learning models (such as VGG16) for further identity authentication and qualification verification.

[0054] Through the above specific steps, the face information in the target image can be detected efficiently and accurately, laying a solid foundation for the subsequent qualification verification of high-altitude workers and the identification of the wearing status of safety ropes. This process makes full use of the computational advantages of integral graphs and Haar features, as well as the classification ability of the Adaboost algorithm, to achieve the initial and rapid verification of the identity of the personnel at the high-altitude work site, and is an important part of the entire safety rope wearing identification system.

[0055] In some optional embodiments of the present application, judging whether the target object in the target image has the qualification for high-altitude work based on the facial information can be achieved through the following steps: using a target convolutional neural network to extract features of the facial area indicated by the facial information in the target image to obtain a first facial feature; determining the cosine similarity between the first facial feature and the second facial feature in the qualification database, wherein the qualification database stores the second facial feature corresponding to the facial images of the staff with high-altitude work qualification certificates at various angles, and the high-altitude work qualification certificates include: climbing certificates and low-voltage electrician certificates; when there is a second facial feature in the qualification database whose cosine similarity with the first facial feature is greater than a preset similarity threshold, it is judged that the target object in the target image has the qualification for high-altitude work.

[0056] In the above embodiment, first, according to the facial information detected by the Haar cascade classifier, that is, the position and size of the face, the corresponding face area is cut out from the target image. The cut face image is preprocessed, including adjusting the image size to 224x224 pixels to meet the requirements of the subsequent VGG16 convolutional neural network input size. The pre-trained VGG16 convolutional neural network model is then used to extract features from the processed face image. Specifically, the VGG16 model extracts multi-level information including edges, textures, and facial structures through its 16-layer convolutional structure to form a one-dimensional vector representation of facial features, namely, the first facial features.

[0057] Then the cosine similarity between the first facial feature and the second facial feature in the qualification database is determined. The qualification database stores facial images of all angles of the company's staff with high-altitude work qualification certificates and the corresponding second facial features. These features are pre-calculated and stored by the VGG16 convolutional neural network. The VGGFace2 dataset contains more than 2.6 million facial images, covering 2,622 people of different identities, ensuring the breadth and accuracy of model recognition.

[0058] The cosine similarity is calculated for the first facial feature (i.e., the feature of the face in the target image) and the second facial feature in the database (i.e., the facial feature of the known high-altitude worker). Cosine similarity is an indicator to measure the similarity between vectors, and its calculation formula is:

[0059]

[0060] Among them, a i and b i The vectors representing the first and second facial features, respectively, and |a| and |b| represent the moduli of the two vectors. The closer the cosine similarity calculation result is to 1, the more similar the two facial images are.

[0061] When there is a second facial feature in the qualification database whose cosine similarity with the first facial feature is greater than the preset similarity threshold, it is determined that the target object in the target image has the qualification for high-altitude work. Specifically, the preset similarity threshold is set to 0.5, which is used to determine whether the first facial feature extracted from the target image is similar enough to the second facial feature stored in the qualification database to confirm whether the target object has the qualification for high-altitude work. The calculated cosine similarity is compared with the preset threshold. If there is at least one second facial feature whose cosine similarity with the first facial feature is greater than 0.5, then it is determined that the target object in the target image (high-altitude worker) has the qualification for high-altitude work, that is, has a climbing permit and a low-voltage electrician certificate. Conversely, if the cosine similarity of all second facial features with the first facial feature is less than or equal to 0.5, then it is determined that the target object does not have the qualification for high-altitude work.

[0062] The above specific steps use deep learning technology, especially the VGG16 convolutional neural network, to achieve efficient qualification verification of high-altitude workers. This technical solution can not only accurately identify the identity of personnel, but also effectively reduce misjudgments, ensuring that only qualified personnel can perform high-altitude operations, significantly improving the safety and standardization of operations. At the same time, through the calculation of cosine similarity, the system can flexibly adapt to work scenarios in different environments, providing construction units with an intelligent and efficient face recognition and qualification verification solution.

[0063] As some optional embodiments of the present application, the target convolutional neural network is obtained by training the VGG16 convolutional neural network based on a general face dataset, wherein the facial images of various target objects in the general face dataset, the VGG16 convolutional neural network includes: multiple convolutional layers, fully connected layers, and pooling layers of different scales, the convolutional layer is used to extract feature maps of the image, the pooling layer is used to reduce the spatial dimension of the feature map, the fully connected layer is used to integrate the features, and introduce nonlinear features through activation functions.

[0064] In the above embodiment, the VGGFace data set is first loaded, which contains more than 2.6 million facial images, covering 2622 people of different identities, covering a wide range of postures and ages, ensuring that the model can learn more generalized features. The images in the data set are preprocessed, and the image size is adjusted to 224x224 pixels using Pytorch's transform function to ensure that all input images have a uniform size for network processing. At the same time, the image is standardized, that is, the mean is subtracted and divided by the standard deviation to improve the efficiency and effect of model training. The VGGFace data set is divided into a test set and a training set in a ratio of 2:8 to ensure the generalization ability of the model during training.

[0065] Then create a VGG16 convolutional neural network model structure, including 13 convolutional layers, 3 fully connected layers, and 5 pooling layers. The input layer processes RGB images of size 224x224x3, extracts image features through multiple convolutional layers, and each convolutional layer is followed by a ReLU activation function to introduce nonlinear features; the spatial dimension of the feature map is reduced through the pooling layer to reduce the amount of calculation; finally, the fully connected layer integrates the features and outputs the classification results. Use the Pytorch official interface to call the VGG16 convolutional neural network, create a stochastic gradient descent optimizer (SGD), add a cross entropy loss function, and set the initial learning rate to 0.0001. During the training process, the model will continuously update the weight parameters and bias parameters through the back propagation algorithm to optimize the model's ability to recognize facial features. The preprocessed training set images are input into the model, and the model will automatically extract image features and output the prediction results through the fully connected layer. The system uses the cross entropy loss function to evaluate the gap between the model's predicted results and the actual results, and updates the model parameters through the stochastic gradient descent (SGD) optimizer to minimize the loss. During the training process, in order to prevent the model from overfitting, a Dropout layer can be added and the performance of the model on the test set can be regularly evaluated.

[0066] Through the above steps, the VGG16 convolutional neural network is trained based on the general face dataset to form a target convolutional neural network, which can efficiently extract facial features in the target image. This technical solution makes full use of the image processing and feature extraction capabilities of deep learning, providing an intelligent and efficient technical means for the qualification verification of high-altitude workers.

[0067] In some optional embodiments of the present application, a classification model is used to determine the seat belt wearing status of a target object in a target image, which can be achieved through the following steps: using a classification model to segment the target image into multiple candidate sub-images; determining a target sub-image corresponding to the seat belt wearing position from the candidate sub-images, and performing feature extraction on the target sub-image to obtain image features; based on the image features, determining the seat belt wearing status of the target object in the target image.

[0068] In the above embodiment, a pre-trained ResNet18 convolutional neural network is first used as a classification model. The model has an 18-layer network structure, can process RGB image input of 224x224x3 size, and extracts image features through multi-layer convolution and pooling operations. When the model is initialized, the pre-trained weights of the model need to be loaded. These weights are obtained by training on a large amount of image data and can identify the basic features in the image. The target image is pre-processed, including adjusting the image size to 224x224 pixels to meet the model input requirements. In addition, data enhancement techniques such as random flipping, random rotation, and random color jitter can be used to improve the robustness and generalization ability of the model.

[0069] In order to focus on the position where the safety rope is worn, the target image is segmented using the horizontal cutting technique to eliminate redundant information. The number of image segmentations is set to 6, and the image parts where the safety rope appears are retained, which can be parts 3, 4, and 5, as candidate sub-images.

[0070] Then, from multiple candidate sub-images, the target sub-image that most likely corresponds to the location where the safety rope is worn is selected through preliminary recognition and analysis by the model. For example, if the safety rope is usually worn around the waist or legs, the candidate sub-images in these areas will be analyzed first. The determined target sub-image is input into the ResNet18 convolutional neural network model. The model extracts the feature map of the image through the convolution layer, reduces the spatial dimension of the feature map through the pooling layer, and integrates the features through the fully connected layer. In particular, the residual block design of ResNet18 can help the model learn deeper image features and avoid the problem of gradient vanishing when the network depth increases. In the residual block of ResNet18, residual learning is achieved through skip connections, that is, the output of each layer not only depends on the output of the previous layer, but is also directly connected to the input. This enables the model to learn and recognize safety rope features in images more effectively.

[0071] Finally, the image features extracted from the target sub-image are input into the fully connected layer of ResNet18, and the Dropout layer is used to prevent overfitting. Finally, a binary classification result is output through the softmax function, indicating the safety rope wearing status of the target object (worn or not worn). During the model training process, the Adam optimizer and the cosine annealing learning rate scheduler are used to dynamically adjust the learning rate and accelerate the convergence of the model. At the same time, by adding a Dropout layer to the fully connected layer, neurons are randomly dropped with a probability of 0.5 to further prevent the model from overfitting and ensure the accuracy and stability of the model in identifying the safety rope wearing status. Based on the collected high-altitude workers image dataset, it is divided into a training set and a test set in a ratio of 8:2, which contain images of wearing and not wearing safety ropes, respectively. The model outputs the prediction result of the safety rope wearing status based on the image features of the target sub-image. If the model determines that the target object has correctly worn the safety rope, it will output the "worn" status; otherwise, it will output the "not worn" status.

[0072] Through the above specific steps, the ResNet18 convolutional neural network model can effectively identify the safety rope wearing status of the target images collected at the high-altitude operation site, ensuring that only workers who wear the safety rope correctly can perform high-altitude operations, significantly improving the safety and standardization of operations. It can not only monitor the safety measures of high-altitude workers in real time, but also timely discover and deal with potential safety hazards through intelligent alarm mechanisms, providing a modern solution for high-altitude operation safety management.

[0073] In some optional embodiments of the present application, the training of the classification model can be achieved through the following steps: obtaining an initial training data set, and preprocessing the images in the initial training data set to obtain a target training data set, wherein the initial training data set includes: images of multiple different target objects at different angles with and without seat belts, and seat belt wearing status labels corresponding to each image; using an initial network model to predict the seat belt wearing status corresponding to the images in the initial training data set, and determining the loss function value based on the seat belt wearing status predicted by the initial network model and the seat belt wearing status label; using a back propagation algorithm to train the model parameters in the initial network model based on the loss function value to obtain a classification model, wherein during the training process, a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate of the model based on the training rounds.

[0074] In the above embodiment, images of high-altitude workers are collected from within the company, including full-body photos of the front and back sides with and without safety ropes at different angles and under different lighting conditions, to form an initial training data set. Each image is manually labeled with the wearing status of the safety rope, that is, whether the safety rope is worn. The collected image data is preprocessed, including adjusting the image size to 224x224 pixels to meet the ResNet18 input requirements; using horizontal cutting technology to eliminate redundant information and focus on areas where safety ropes may appear; randomly flipping the image horizontally with a probability of 0.5 to increase the diversity of the data set; randomly rotating the image up to 10 degrees to adapt to different shooting angles; applying random erasing operations with a probability of 0.5 to enhance the model's robustness to occlusion.

[0075] Load the pre-trained ResNet18 as the initial network model. This model is trained based on the ImageNet dataset, can process RGB images, and has certain feature extraction capabilities. When loading the model, it is necessary to adjust the output of the last fully connected layer to adapt to the binary classification problem of the safety rope wearing status. Input the preprocessed image into the initial network model, and the model will output a binary classification result, that is, the predicted safety rope wearing status. Calculate the loss function value based on the safety rope wearing status predicted by the model and the manually labeled true label (seat belt wearing status label). Specifically, the cross entropy loss function is used to measure the difference between the predicted result and the true result. The calculation formula of the cross entropy loss function is as follows, M is the number of classification categories, y is i is the sample label, p i is the probability of predicting a positive sample.

[0076]

[0077] The smaller the loss function value, the smaller the difference between the model prediction result and the actual result.

[0078] Then, through the back propagation algorithm, the weight parameters and bias parameters of the model are updated according to the calculated loss function value to reduce the difference between the predicted results and the actual results. During the update process, the Dropout layer is used to prevent overfitting, and a certain proportion of neurons are randomly discarded (for example, the probability is set to 0.5), so that the model relies more extensively on the input data and improves the generalization ability of the model.

[0079] During the training process, in order to avoid the oscillation and instability caused by excessive learning rate and to prevent overfitting to a certain extent, cosine annealing is used to dynamically adjust the learning rate. The formula for cosine annealing is:

[0080] lr=lr max *0.5*(1+cos(epoch / Ttotal *pi))

[0081] Among them, lr represents the learning rate, lr max represents the maximum learning rate, epoch represents the current number of training rounds, T total Represents the total number of training rounds. Through the cosine annealing strategy, the learning rate will be high at the beginning of training and then gradually decrease, which helps the model learn quickly in the early stage of training and avoids shock and instability caused by excessive learning rate in the later stage.

[0082] Finally, the preprocessed images and corresponding labels are input into the model for training. Through continuous iteration, the model will learn how to accurately predict the wearing status of the safety rope in the image. During the training process, the performance of the model on the validation set needs to be evaluated regularly to monitor whether the model is overfitting or underfitting, and the model parameters and training strategies are adjusted according to the evaluation results. The collected image dataset is divided into a training set and a test set in a ratio of 8:2. The training set is used for model learning and parameter updating, and the test set is used to evaluate the generalization ability of the model to ensure that the model can accurately predict the wearing status of the safety rope on unseen data.

[0083] Through the above specific steps, the safety rope wearing status of high-altitude workers can be efficiently and accurately identified, providing strong technical support for ensuring work safety. This technical solution combines key steps such as data preprocessing, deep learning model prediction and parameter optimization, and realizes the automated process from raw image data to safety rope wearing status identification, providing a practical solution for the intelligent monitoring of safety ropes for high-altitude operations.

[0084] As other optional embodiments of the present application, preprocessing of images in the initial training data set can be achieved through the following steps: randomly adjusting the size of the images in the initial training data set; and / or randomly flipping the images in the initial training data set, or rotating them by a random angle within a preset angle range; and / or adding a preset white noise value to the pixel value of each point in the image in the initial training data set, and binarizing the image after adding the preset white noise value; and / or standardizing the pixel value of each point in the image in the initial training data set.

[0085] In the above embodiment, Python's image processing library PIL (Pillow) or OpenCV is first used to randomly resize each image in the initial training data set. The specific operation is to randomly scale the image to within the range of 224x224 pixels while keeping the aspect ratio of the image unchanged, but the specific size can be randomly selected, for example, it can be 224x224, 220x220, 228x228, etc. This method increases the size diversity of the image, enabling the model to better handle inputs of different sizes.

[0086] Then randomly flip or rotate the images in the initial training data set, and use the ImageOps.flip() or cv2.flip() function of PIL or OpenCV to randomly rotate the images to an angle within a preset angle range. For example, the rotation angle range can be set to [-10°, 10°], which can increase the model's ability to recognize the wearing state of the safety rope in different directions.

[0087] Then, at each pixel position in the image, randomly add a white noise value within a range. For example, you can select a random number in the range of [-10,10] and add it to the pixel value. This increases the complexity of the image and simulates the light changes and noise interference that may be encountered in real scenes. Then use the ImageOps.threshold() or cv2.threshold() function to binarize the image with added white noise. Set a threshold (for example, 128) and adjust the pixel values ​​in the image to a binary representation of black (0) or white (255). This helps the model learn more stable features, especially in the recognition of image edges and contours.

[0088] Finally, the pixel values ​​of each point in the image of the initial training data set are standardized. By traversing all images in the data set and using the mean() and std() functions of the numpy library, the pixel mean and standard deviation of all images in the entire training data set are calculated, and then the pixel values ​​of each image are adjusted using the following standardization formula:

[0089]

[0090] Where μ is the image mean, x represents the image matrix, σ represents the standard deviation, and N represents the number of pixels in image x. Standardization can adjust the pixel values ​​of all images to a similar range, which helps model learning and avoids increased training difficulty caused by differences in the range of input data.

[0091] Through the above steps, the diversity and complexity of the initial training data set can be significantly increased, and the generalization ability of the model can be improved. Especially in the application of safety rope wearing recognition, these processing methods can simulate high-altitude working scenes under different environments, angles and lighting conditions, ensuring that the model can adapt to various actual working environments and improve the accuracy and reliability of recognition.

[0092] Figure 3 is a structural diagram of a high-altitude work safety detection system provided according to an embodiment of the present application, such as Figure 3As shown, the system mainly includes a data entry and verification module, a dual-certificate verification module, and a safety rope wearing identification module, wherein the working steps of each module are as follows:

[0093] The working steps of the data entry and qualification verification module are as follows:

[0094] S1, high-altitude workers use the corresponding account to log in to the wearing identification system of the safety rope for high-altitude work tower climbing.

[0095] S2, the high-altitude staff uses the safety module to upload pictures, clicks on the start authentication background, and will automatically jump to the dual-certificate qualification verification module and the safety rope wearing identification module for dual algorithm decision-making, and finally outputs the qualification verification passed or the quality verification failed to determine whether the user has the ability to safely climb the tower.

[0096] The specific working steps of the dual-certificate qualification verification module are as follows:

[0097] S1, collect full-body photos of high-altitude workers with high-altitude climbing certificates and low-voltage electrician certificates within the company to form a qualification database for personnel information matching. The qualification database contains 26 pictures, involving 13 high-altitude workers. Each person collects 2 full-body photos, one with a safety rope and one without a safety rope.

[0098] S2, input the qualification database and upload the picture, load the pre-trained Haar cascade classifier xml file, use the detectMultiScale function provided by OpenCV to detect the image, and output the location and size of the target face.

[0099] Specifically, the Haar cascade classifier determines whether a given image or sub-image matches a known type by analyzing the contrast between adjacent regions in the image, thereby achieving face detection. Figure 4 As shown in the figure. The Haar cascade classifier is based on Haar-like features, uses integral graphs to accelerate calculations, and uses Adaboost to train strong classifiers for face detection. Haar features are a simple image feature that extracts features by calculating the difference in the sum of pixel values ​​in adjacent areas of the image. It can capture specific patterns of faces. The eigenvalue f of the Haar-like feature i The formula is:

[0100] f i = white area pixels - black area pixels

[0101] In order to reduce the computational pressure, the Haar cascade classifier needs to use an integral graph to increase the computation rate of the eigenvalues. The construction formula of the integral graph is:

[0102]

[0103] That is, the integral image ii(i,j) located at image (i,j) is the sum of all pixels f(k,1) in the upper left corner of the image.

[0104] For a fixed feature in the database and uploaded pictures, the feature value of each image is calculated, the calculated feature values ​​are sorted and labeled, and the strong classifier cascade trained by Adaboost is used to obtain the optimal threshold and obtain the optimal weak classifier under the feature, and the strong classifier of the feature is trained. Finally, the Haar cascade classifier is obtained through cascading.

[0105] Specifically, the Haar cascade classifier is pre-trained using the FDDB (Face Detection Data Set and Benchmark) dataset. The FDDB dataset is a facial region dataset that is widely used to train cascade classifiers for face recognition. The dataset contains 5171 facial annotations from 2845 images obtained from Faces in the Wild Database.

[0106] Specifically, the pre-trained Haar cascade classifier uses the trainCascadeClassifier function provided by OpenCV, setting the parameter number of features to 200, the number of classifier layers to 20, the minimum classifier detection rate to 0.009, the maximum false positive rate to 0.5, and the learning rate to 0.05.

[0107] S3, based on the face position and size output by the Haar cascade classifier, the pre-trained VGG16 convolutional neural network model will extract features from the face image and calculate the similarity (cosine similarity) between the output image and the features in the qualification database. The similarity is compared with the preset threshold. If the similarity is less than the preset threshold, it is considered that there is a matching face; if the similarity is greater than the preset threshold, it is considered that there is no matching face. The preset threshold is set to 0.5.

[0108] Specifically, the large-scale general face dataset used by the VGG16 convolutional neural network is the VGGFace dataset. The VGGFace2 dataset contains more than 2.6 million facial images, covering 2,622 people of different identities. The dataset covers a wide range of postures and ages, allowing the model to learn more generalized features. The images of the VGGFace2 dataset are mainly from Google Images. The VGGFace2 dataset has undergone strict data cleaning during the collection process to ensure the high quality of the dataset.

[0109] Specifically, the VGG16 convolutional neural network uses a 16-layer network structure, which includes 13 convolutional layers, 3 fully connected layers, and 5 pooling layers. Figure 5As shown. The image size of the input VGG16 convolutional neural network is 224*224*3. The first and second layers are convolutional layers with 64 3*3 convolution kernels; the third and fourth layers contain 128 3*3 convolution kernels; the fifth to seventh layers each contain 256 3*3 convolution kernels; the eighth to thirteenth layers each contain 512 3*3 convolution kernels; the fourteenth to eighteenth layers are fully connected layers, each of which uses ReLU as the activation function, and finally outputs 1000 prediction results through softmax. These three fully connected layers have 4096, 4096 and 1000 nodes respectively. Before the fully connected layer, the three-dimensional feature map output by the convolution layer is flattened into a one-dimensional vector. In order to meet the needs of binary classification, the fully connected layer of the last layer of the VGG18 convolutional neural network is modified to change the output nodes from 1000 to 2.

[0110] Specifically, the VGGface dataset is used for pre-training the VGG16 convolutional neural network. After loading the dataset, the image size is adjusted through the transform function of Pytorch and the training set and test set are divided into 20:80. The VGG16 convolutional neural network is called through the official Pytorch interface, a stochastic gradient descent optimizer (SGD) is created, a cross entropy loss function is added, and the initial learning rate is set to 0.0001.

[0111] The calculation formula of the cross entropy loss function is as follows, M is the number of classification categories, y i is the sample label, p i is the probability of predicting a positive sample.

[0112]

[0113] The basic formula of stochastic gradient descent is as follows:

[0114]

[0115] Among them, θ t represents the model parameters after the tth iteration, η t represents the learning rate, f i (θ t ) represents the gradient of the i-th sample of the loss function. In each iteration, the stochastic gradient descent method randomly selects a sample and uses the gradient of the sample to update the model parameters.

[0116] Specifically, cosine similarity is the cosine of the angle between two n-dimensional vectors in n-dimensional space. It is equal to the dot product (vector product) of the two vectors divided by the product of the lengths (or sizes) of the two vectors. The formula for cosine similarity is:

[0117]

[0118] The specific working steps of the safety rope wearing identification module are as follows:

[0119] S1. There are many practical cases of computer vision and AI analysis in improving production safety, but there is no official data set to detect whether high-altitude workers are wearing safety ropes. In order to meet the actual needs of the company and ensure the safety of high-altitude workers, 52 full-body photos of the company's internal high-altitude workers without wearing safety ropes were collected to form negative samples, and 57 full-body photos of the company's internal high-altitude workers wearing safety ropes were collected to form positive samples;

[0120] S2, perform image preprocessing: adjust the image size (Resize); define a class for horizontally cutting the image; randomly flip (Flipping), randomly rotate (Rotation), randomly color jitter (ColorJitter), normalize (Normalize) and randomly erase the image (RandomErasing).

[0121] Specifically, horizontal cutting aims to eliminate redundant information and focus on detecting the target safety rope. The number of image segmentation is set to 6, retaining the 3rd, 4th, and 5th parts where the safety rope is located.

[0122] Specifically, the image is resized to 224*224 pixels; the image is randomly flipped horizontally with a probability of 0.5; the image is randomly rotated by up to 10 degrees; and a random erasing operation is applied with a probability of 0.5.

[0123] Specifically, the principle of random color jitter is as follows:

[0124] The image is uniformly quantized to set a fixed threshold, for example, pixels with a grayscale less than 128 are set to black, and pixels with a grayscale greater than or equal to 128 are set to white, but a lot of details are lost. The expression formula is as follows:

[0125]

[0126] Random color jittering is to first superimpose a layer of white noise on the original image and then perform uniform quantization. This will result in a picture with a lot of white noise, but its detail effect is better than that of a simple binary image. The formula is:

[0127]

[0128] Specifically, the formula for standardization is:

[0129]

[0130] Where μ is the image mean, x represents the image matrix, σ represents the standard deviation, N represents the number of pixels in image x;

[0131] S3, passing the sample data outputted in step S2 as the input of the network into the ResNet18 convolutional neural network classification model for training, and finally obtaining a trained safety rope wearing recognition model;

[0132] S4, input the picture uploaded by the data entry and qualification verification module into the trained safety rope wearing recognition model to determine whether the high-altitude workers are wearing safety ropes.

[0133] Specifically, the safety rope wearing recognition model is generated through the following process:

[0134] S41, the total data set of positive and negative samples after image enhancement processing is divided into a training set and a test set in a ratio of 8:2, the training set contains 87 sample images, and the test set contains 22 sample images.

[0135] S42, the test set is trained using the ResNet18 convolutional neural network. During the training process, the Adam optimizer and the cosine annealing learning rate scheduler are used to help stabilize the convergence to obtain the recognition rate of ResNet18 on the test set.

[0136] S43, for all samples obtained in step S41, the above step S42 is continuously repeated, and the weight parameters and bias parameters of each layer in the convolutional neural network are updated and optimized using the back propagation algorithm to obtain an updated convolutional neural network. The network model is continuously iterated and updated until the recognition rate of ResNet18 on the safety rope wearing recognition test set reaches the maximum value, thereby obtaining a trained safety rope wearing recognition model.

[0137] Specifically, ResNet18 uses an 18-layer network structure, each layer has a specific structure and function. Figure 6As shown. ResNet18 includes an input layer: it processes RGB images of size 224x224x3 and feeds them into the network as input tensors; an initial convolutional layer: a convolutional layer using a 7x7 convolution kernel and a stride of 2, with an output channel number of 64. This layer extracts primary visual features by performing convolution operations on the original image; a maximum pooling layer: following the initial convolutional layer is a 3x3 maximum pooling layer with a stride of 2. It reduces the spatial size of the feature map, thereby reducing the amount of computation and increasing the receptive field; a residual block group conv2_x: the first residual block: contains two 3x3 convolutional layers, each followed by batch normalization and ReLU activation functions, keeping the number of channels at 64. The input is directly added to the output of the block through a residual connection to form the final output. Second residual block: Same structure as above, but the number of channels is adjusted from 64 to 64 using a 1x1 convolution kernel before the first convolution layer to keep the dimension consistent; Residual block group conv3_x: First residual block: Before entering the block, the number of channels is increased from 64 to 128 through a 1x1 convolution layer. Then there are two 3x3 convolution layers, batch normalization and ReLU activation function, residual connection. Second residual block: Same structure as the first one, but no need to adjust the number of channels; Residual block group conv4_x: First residual block: Before entering the block, the number of channels is increased from 128 to 256 through a 1x1 convolution layer, while the spatial size is halved. Then there are two 3x3 convolution layers and other standard residual block structures. Second residual block: Same structure as the first one, but the number of channels is no longer adjusted; Residual block group conv5_x: First residual block: Before entering the block, the number of channels is increased from 256 to 512 through a 1x1 convolution layer, while the spatial size is halved again. Next comes the standard residual block structure with two 3x3 convolutional layers. The second residual block: The structure is the same as the first one, but the number of channels is no longer adjusted; Average pooling layer: After all residual blocks are processed, the 7x7 feature map is converted into a one-dimensional vector using a global average pooling layer. This can reduce the data dimension and retain the most important feature information; The fully connected layer maps the above one-dimensional vector to a specific classification label through a fully connected neural network. In order to adapt to the binary classification problem, a Dropout layer is added before the third layer of the ResNet18 fully connected layer to prevent the neural network from overfitting. The probability of the Dropout layer is set to 0.5.

[0138] Specifically, in order to introduce nonlinearity into the neural network and enhance the learning ability of the network, the neuron activation function used in this step is the relu activation function, and its formula is as follows:

[0139]

[0140] k is a fixed constant, k∈[0,1], and is generally taken as 1.

[0141] Specifically, in order to iteratively update the neural network weights and speed up the model convergence, the back propagation algorithm Adam-Optimizer is used as the optimizer in the training process, and the learning rate is set to 0.001.

[0142] Specifically, in order to avoid the oscillation and instability caused by excessive learning rate and to prevent overfitting to a certain extent, cosine annealing is used to dynamically adjust the learning rate during training. The formula for cosine annealing is:

[0143] lr=lr max *0.5*(1+cos(epoch / T total *pi))

[0144] Among them, lr represents the learning rate, lr max represents the maximum learning rate, epoch represents the current number of training rounds, T total Represents the total number of training rounds.

[0145] According to an embodiment of the present application, an embodiment of a high-altitude work safety detection device is also provided. Figure 7 Schematic diagram of a high-altitude work safety detection device provided according to an embodiment of the present application. Figure 7 As shown, the device comprises:

[0146] The image acquisition module 70 is used to acquire a target image, wherein the target image is an image of a target object collected at a high-altitude operation site, and the target object includes: a high-altitude operator;

[0147] The qualification verification module 72 is used to determine the face information in the target image and judge whether the target object in the target image has the qualification for high-altitude operation based on the face information;

[0148] The seat belt verification module 77 is used to determine the seat belt wearing status of the target object in the target image by using a classification model, wherein the seat belt wearing status includes: worn, not worn;

[0149] The monitoring and early warning module 76 is used to send an alarm message when it is determined that the target object does not have the qualification for high-altitude work, or the target object's safety belt wearing status is not worn, wherein the alarm message is used to prohibit the target object from performing high-altitude work.

[0150] Optionally, the qualification verification module 72 is also used to perform the following steps: using a target cascade classifier to construct a feature integral map corresponding to the target image, wherein the feature integral map is used to calculate the pixel sum of any rectangular area in the target image; based on the feature integral map, determining the target features corresponding to each candidate area in the target image, wherein the target features are determined by calculating the difference in pixel values ​​within the candidate area; using a weak classifier in the target cascade classifier to determine whether the candidate area belongs to a face area based on the target features; when all weak classifiers in the target cascade classifier determine that the candidate area belongs to a face area, determining the candidate area as a face area, thereby obtaining face information in the target image, wherein the face information includes: the position and size of the face.

[0151] Optionally, the qualification verification module 72 is also used to perform the following steps: using a target convolutional neural network to extract features of the facial area indicated by the facial information in the target image to obtain a first facial feature; determining the cosine similarity between the first facial feature and a second facial feature in the qualification database, wherein the second facial feature corresponding to facial images of staff with high-altitude work qualification certificates at various angles stored in the qualification database, and the high-altitude work qualification certificates include: a climbing certificate and a low-voltage electrician certificate; when there is a second facial feature in the qualification database whose cosine similarity with the first facial feature is greater than a preset similarity threshold, it is determined that the target object in the target image is qualified for high-altitude work.

[0152] Optionally, the target convolutional neural network is obtained by training a VGG16 convolutional neural network based on a general face dataset, wherein the general face dataset includes facial images of a variety of different target objects, and the VGG16 convolutional neural network includes: multiple convolutional layers, fully connected layers, and pooling layers of different scales, the convolutional layer is used to extract feature maps of the image, the pooling layer is used to reduce the spatial dimension of the feature map, the fully connected layer is used to integrate the features, and introduce nonlinear features through activation functions.

[0153] Optionally, the seat belt verification module 77 is also used to perform the following steps: using a classification model to segment the target image into multiple candidate sub-images; determining a target sub-image corresponding to the seat belt wearing position from the candidate sub-images, and extracting features of the target sub-image to obtain image features; and determining the seat belt wearing status of the target object in the target image based on the image features.

[0154] Optionally, the training step of the classification model includes: obtaining an initial training data set, and preprocessing the images in the initial training data set to obtain a target training data set, wherein the initial training data set includes: images of multiple different target objects at different angles with and without seat belts, and seat belt wearing status labels corresponding to each image; using an initial network model to predict the seat belt wearing status corresponding to the images in the initial training data set, and determining a loss function value based on the seat belt wearing status predicted by the initial network model and the seat belt wearing status labels; using a back propagation algorithm to train model parameters in the initial network model based on the loss function value to obtain a classification model, wherein during the training process, a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate of the model based on the training rounds.

[0155] Optionally, preprocessing the images in the initial training data set includes: randomly adjusting the size of the images in the initial training data set; and / or randomly flipping the images in the initial training data set, or rotating them by a random angle within a preset angle range; and / or adding a preset white noise value to the pixel value of each point in the image in the initial training data set, and binarizing the image after adding the preset white noise value; and / or standardizing the pixel value of each point in the image in the initial training data set.

[0156] It should be noted that the various modules in the above-mentioned high-altitude work safety detection device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0157] It should be noted that the aerial work safety detection device provided in this embodiment can be used to perform Figure 2 The high-altitude work safety detection method shown, therefore, the relevant explanations and instructions on the above-mentioned high-altitude work safety detection method are also applicable to the embodiments of the present application and will not be repeated here.

[0158] An embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following high-altitude work safety detection method by running the computer program: obtaining a target image, wherein the target image is an image of a target object collected at a high-altitude work site, and the target object includes: high-altitude workers; determining facial information in the target image, and judging whether the target object in the target image is qualified for high-altitude work based on the facial information; using a classification model to determine the seat belt wearing status of the target object in the target image, wherein the seat belt wearing status includes: worn, not worn; when it is determined that the target object is not qualified for high-altitude work, or the seat belt wearing status of the target object is not worn, sending an alarm message, wherein the alarm message is used to prohibit the target object from performing high-altitude work.

[0159] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the high-altitude work safety detection method described in each embodiment of the present application.

[0160] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0161] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0163] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0166] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting safety of aerial work, characterized in that: include: Acquire a target image, wherein the target image is an image of a target object collected at a high-altitude operation site, and the target object includes: a high-altitude operation worker; Determine facial information in the target image, and judge whether the target object in the target image has the qualification for high-altitude work based on the facial information; Using a classification model, determining a seat belt wearing state of the target object in the target image, wherein the seat belt wearing state includes: worn, not worn; When it is determined that the target object does not have the qualification for high-altitude work, or the safety belt wearing status of the target object is not being worn, an alarm message is sent, wherein the alarm message is used to prohibit the target object from performing high-altitude work.

2. The high-altitude operation safety detection method according to claim 1 is characterized in that: Determining the face information in the target image includes: Using a target cascade classifier, construct a feature integral map corresponding to the target image, wherein the feature integral map is used to calculate the pixel sum of any rectangular area in the target image; Determining target features corresponding to each candidate region in the target image according to the feature integral map, wherein the target features are determined by calculating the difference in pixel values ​​within the candidate region; Using a weak classifier in the target cascade classifier, judging whether the candidate region belongs to a face region according to the target feature; When all the weak classifiers in the target cascade classifier determine that the candidate region belongs to a face region, the candidate region is determined to be a face region, thereby obtaining face information in the target image, wherein the face information includes: the position and size of the face.

3. The high-altitude operation safety detection method according to claim 2 is characterized in that: Judging whether the target object in the target image has the qualification for high-altitude work based on the face information includes: Using a target convolutional neural network, extracting features of a face region indicated by the face information in the target image to obtain a first face feature; Determine the cosine similarity between the first facial feature and a second facial feature in a qualification database, wherein the second facial feature corresponding to facial images of workers with high-altitude work qualification certificates at various angles stored in the qualification database, and the high-altitude work qualification certificates include: a climbing certificate and a low-voltage electrician certificate; When there is a second facial feature in the qualification database whose cosine similarity with the first facial feature is greater than a preset similarity threshold, it is determined that the target object in the target image has the qualification for high-altitude work.

4. The high-altitude operation safety detection method according to claim 3 is characterized in that: The target convolutional neural network is obtained by training the VGG16 convolutional neural network based on a general face data set, wherein the general face data set includes facial images of multiple target objects, and the VGG16 convolutional neural network includes: multiple convolutional layers, fully connected layers, and pooling layers of different scales, wherein the convolutional layer is used to extract the feature map of the image, the pooling layer is used to reduce the spatial dimension of the feature map, and the fully connected layer is used to integrate the features and introduce nonlinear features through activation functions.

5. The high-altitude operation safety detection method according to claim 1, characterized in that: Using a classification model to determine the seat belt wearing state of the target object in the target image includes: Using the classification model, segmenting the target image into a plurality of candidate sub-images; Determine a target sub-image corresponding to the seat belt wearing position from the candidate sub-images, and perform feature extraction on the target sub-image to obtain image features; The seat belt wearing status of the target object in the target image is determined according to the image feature.

6. The high-altitude operation safety detection method according to claim 5 is characterized in that: The training steps of the classification model include: Acquire an initial training data set, and preprocess the images in the initial training data set to obtain a target training data set, wherein the initial training data set includes: images of multiple target objects at different angles in the states of wearing seat belts and not wearing seat belts, and seat belt wearing state labels corresponding to each of the images; Using an initial network model, predicting the seat belt wearing state corresponding to the image in the initial training data set, and determining a loss function value according to the seat belt wearing state predicted by the initial network model and the seat belt wearing state label; A back propagation algorithm is used to train the model parameters in the initial network model according to the loss function value to obtain the classification model, wherein during the training process, a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate of the model according to the training rounds.

7. The high-altitude operation safety detection method according to claim 6, characterized in that: Preprocessing the images in the initial training data set includes: Randomly adjusting the size of the images in the initial training data set; And / or, randomly flipping the images in the initial training data set, or rotating them by a random angle within a preset angle range; And / or, adding a preset white noise value to the pixel value of each point of the image in the initial training data set, and performing binarization processing on the image after adding the preset white noise value; And / or, the pixel value of each point in the image of the initial training data set is standardized.

8. A high-altitude operation safety detection device, characterized in that: include: An image acquisition module is used to acquire a target image, wherein the target image is an image of a target object collected at a high-altitude operation site, and the target object includes: a high-altitude operator; A qualification verification module is used to determine the facial information in the target image and, based on the facial information, determine whether the target object in the target image has the qualifications for high-altitude work; A seat belt verification module, used to determine the seat belt wearing status of the target object in the target image by using a classification model, wherein the seat belt wearing status includes: worn, not worn; The monitoring and early warning module is used to send an alarm message when it is determined that the target object does not have the qualifications for high-altitude work, or the safety belt wearing status of the target object is not worn, wherein the alarm message is used to prohibit the target object from performing high-altitude work.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program, when running, executes the aerial work safety detection method described in any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the aerial work safety detection method described in any one of claims 1 to 7 by running the computer program.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the aerial work safety detection method described in any one of claims 1 to 7 are implemented.

Images