Elevator installation process monitoring method based on target detection

Abstract

The invention relates to the technical field of big data detection, and discloses an elevator installation process monitoring method based on target detection, which comprises the following steps: acquiring video stream data of an operation platform in real time, and inputting a trained target detection model to obtain a plurality of target detection results; analyzing the plurality of target detection results in the video stream data of the work platform to obtain a plurality of labels; judging whether labels related to the human body exist in the multiple labels or not; if the labels related to the human body exist in the multiple labels, the multiple target detection results are tracked, and whether an abnormal event exists in the installation personnel or not is judged by adopting a preset rule; and if the labels related to the human body do not exist in the multiple labels, returning to obtain the video stream data to detect the next frame. By identifying whether the installation personnel wear safety helmets, fasten safety belts, safety hooks and the like or not, the operation of the installation personnel in the working process is monitored in the whole process, the working quality is improved, the installation process is standardized, and the accident risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of big data detection, and in particular to an elevator installation process monitoring method based on target detection. Background Art

[0002] As residential buildings become taller and taller, elevators have become an indispensable vertical transportation tool. The installation of a lifting elevator in a vertical shaft requires the installation of a temporary work platform in the shaft first. The platform can move up and down and dock in the shaft to facilitate the installation of hall doors, shaft guide rails and various brackets. The installers complete the installation and fixation of a series of elevator components such as hall doors and shaft brackets on a work platform that can move up and down. The whole process is subject to the risk of accidental injuries caused by factors such as human negligence, improper operation, falling objects in the shaft, and failure to wear safety protection as required. Supervisors have been watching over the work site, which consumes too much manpower and material resources, and the supervision efficiency is not high. It is also easy to have poor supervision. The present invention aims to solve this problem. Summary of the invention

[0003] The present invention provides an elevator installation process monitoring method based on target detection. Aiming at the supervision and monitoring problems of the elevator installation process, the method monitors the operation of the installers during the work process by identifying whether the installers wear safety helmets, fasten safety belts, and safety hooks, etc., which helps to improve the work quality, standardize the installation process, and reduce the risk of accidents. When an abnormality is detected, the method promptly reminds the installers through sound and light alarms, and can also send alarm messages to supervisors through a network platform.

[0004] The present invention provides an elevator installation process monitoring method based on target detection, based on a detection system composed of a camera installed on the top of the inner column of the working platform and an edge terminal with a neural network processor, the method comprising:

[0005] Acquire image data of the work platform, and perform target annotation on the image data to obtain a training data set;

[0006] Constructing a target detection model, and using the training data set to train the target detection model to obtain a trained target detection model;

[0007] Acquire the video stream data of the operation platform in real time, and input the trained target detection model to obtain multiple target detection results;

[0008] Analyze multiple target detection results in the monitoring screen of the work platform to obtain multiple labels; wherein the labels include human body, human head, safety helmet, safety hook, safety belt, safety rope, reflective clothing, hands, gloves;

[0009] Determine whether there are human-related tags among multiple tags;

[0010] If there is a human-related tag among the multiple tags, the multiple target detection results are tracked, and a preset rule is used to determine whether there is an abnormal event with the installer.

[0011] Furthermore, if there is a human-related tag among the multiple tags, after the step of tracking multiple target detection results and using a preset rule to determine whether the installer has an abnormal event, it also includes:

[0012] If there is no human body related label among the multiple labels, return to obtain the next frame of the video stream data of the operation platform for detection.

[0013] Furthermore, before the step of analyzing the multiple target detection results in the monitoring screen of the work platform to obtain multiple labels, it also includes:

[0014] The back-end management personnel log in to the edge terminal through a mobile phone or computer browser to view the real-time image of the camera, and draw the hook beam line on the monitoring image of the working platform; wherein, the edge terminal with a neural network processor is installed in the computer room, communicates with the camera wirelessly, and accesses the external Internet through a 4G router.

[0015] Furthermore, in the step of analyzing the multiple target detection results in the monitoring screen of the work platform to obtain multiple labels, each label is represented by preset letters and ID numbers, wherein the letters represent categories, and the digital ID represents the order in which the targets appear. When tracking each target label, the IDs of target labels of the same category are incremented in numerical order. When the target label is lost, the corresponding ID will no longer be used; wherein, the target label is the detected target, and the ID is used to characterize the uniqueness of the target.

[0016] Furthermore, if there is a human-related tag among the multiple tags, the step of tracking multiple target detection results and using a preset rule to determine whether the installer has an abnormal event includes:

[0017] If there are human-related labels among multiple labels, multiple target detection results are tracked;

[0018] Determining whether the installer is wearing a safety helmet according to the head tag and the safety helmet tag, and recording it as a first alarm event;

[0019] Determining whether the installer is wearing gloves according to the hand tag and the glove tag, and recording the event as a second alarm;

[0020] Determining whether the installer uses a safety belt according to the safety hook label, the safety rope label and the safety belt label, so as to record the third alarm event;

[0021] Determining whether the installer is wearing reflective clothing according to the reflective clothing label, and recording it as a fourth alarm event;

[0022] When any of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event is abnormal, alarm processing is performed.

[0023] Furthermore, the step of determining whether the installer is wearing a safety helmet based on the head tag and the safety helmet tag to record it as a first alarm event includes:

[0024] When a head tag exists in the target detection result, a video frame of the head tag and a rectangular frame of the head tag are recorded as a first target comparison frame;

[0025] Determine whether there is a helmet tag in the first target comparison frame, and whether the overlap degree between the rectangular frame of the helmet tag and the rectangular frame of the head tag reaches a first preset threshold;

[0026] If the overlap degree between the rectangular frame of the helmet label and the rectangular frame of the head label reaches a first preset threshold, it is determined that the installer has correctly worn the helmet;

[0027] If the overlap degree between the rectangular frame of the helmet label and the rectangular frame of the head label does not reach a first preset threshold, counting a first time length of continuous frames in which the helmet label does not appear;

[0028] When the first time length does not exceed a first set value, determining that the installer has correctly worn the safety helmet;

[0029] When the first time length exceeds a first set value, it is determined that the installer does not wear the safety helmet correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a first threshold, a first alarm event abnormality is recorded.

[0030] Furthermore, the step of determining whether the installer is wearing gloves according to the hand tag and the glove tag to record it as a second alarm event includes:

[0031] When the target detection result has a hand label, the video frame of the hand label and the rectangular frame of the hand label are recorded as a second target comparison frame;

[0032] Determine whether there is a glove label in the second target comparison frame, and whether the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold;

[0033] If the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold, it is determined that the installer has correctly worn the gloves;

[0034] If the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label does not reach a second preset threshold, counting a second time length of consecutive frames in which the glove label does not appear;

[0035] When the second time length does not exceed a second set value, it is determined that the installer has correctly worn the gloves;

[0036] When the second time length exceeds a second set value, it is determined that the installer does not wear gloves correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a second threshold, a second alarm event abnormality is recorded.

[0037] Furthermore, the step of determining whether the installer uses a safety belt based on the safety hook label, the safety rope label and the safety belt label to record it as a third alarm event includes:

[0038] When a safety hook label exists in the target detection result, determining whether a rectangular frame of the safety hook label intersects with the hook beam line;

[0039] When the safety hook label and the hook beam line intersect, determining whether the safety rope label and the safety hook label intersect;

[0040] If the safety rope tag and the safety hook tag have an intersection, it is determined whether there is a safety belt tag in the human body area;

[0041] If there is a seat belt tag in the human body area, then record the third time length of the frame where the seat belt tag appears in each set time period, and when the third time length does not exceed a third set value, it is determined that the installer does not wear the seat belt correctly;

[0042] When the third time length exceeds a third set value, it is determined that the installer wears the safety belt correctly, and the normal number is recorded. When the normal number does not reach a third threshold, a third alarm event abnormality is recorded.

[0043] Furthermore, the step of determining whether the installer is wearing reflective clothing according to the reflective clothing label to record it as a fourth alarm event includes:

[0044] When a reflective clothing label exists in the target detection result, a fourth time length of the existence of the reflective clothing label is recorded according to the ID of the reflective clothing label;

[0045] When the fourth time length exceeds a fourth set value, it is determined that the installer is wearing the reflective clothing correctly;

[0046] When the fourth time length does not exceed the fourth set value, it is determined that the installer does not wear the reflective clothing correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a fourth threshold, a fourth alarm event abnormality is recorded.

[0047] Furthermore, after the step of performing alarm processing when any of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event is abnormal, it also includes:

[0048] When any alarm event is triggered, the edge terminal sends a voice command corresponding to the alarm event to the camera, and controls the camera's fill light to flash at a set frequency;

[0049] When the sum of the alarm times of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event reaches a preset number, abnormal videos are recorded and stored; wherein the abnormal videos are videos within a set time period before and after the alarm;

[0050] The edge terminal reports the alarm event message and abnormal video to the server through the mobile network, so that the server can determine whether to push the alarm to the supervisor's mobile phone after receiving the message; wherein the message content includes the project number, elevator number, alarm time, alarm picture, and abnormal video.

[0051] The present invention also provides an elevator installation process monitoring device based on target detection, based on a detection system composed of a camera installed on the top of the inner column of the working platform and an edge terminal with a neural network processor, the device comprising:

[0052] A data module is used to obtain image data of the operation platform and mark the image data to obtain a training data set;

[0053] A training module is used to construct a target detection model and train the target detection model using the training data set to obtain a trained target detection model;

[0054] An acquisition module is used to acquire the video stream data of the operation platform in real time and input the trained target detection model to obtain multiple target detection results;

[0055] An analysis module is used to analyze the detection results of multiple targets in the monitoring screen of the work platform to obtain multiple labels; wherein the labels include a human body, a human head, a safety helmet, a safety hook, a safety belt, a safety rope, a reflective vest, a hand, and a glove;

[0056] A judgment module, used to judge whether there is a human body related label among multiple labels;

[0057] A tracking module, used to track multiple target detection results when there is a human-related tag among multiple tags, and to determine whether there is an abnormal event involving the installer using a preset rule;

[0058] Furthermore, it also includes a return module, which is used to return to obtain the next frame of the video stream data of the operation platform for detection when there is no human body related label among the multiple labels.

[0059] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0060] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.

[0061] The beneficial effects of the present invention are:

[0062] The present invention installs a camera on the work platform to monitor various installation work of the installers on the platform. The camera is connected to an edge terminal with a neural network processor. The target detection algorithm is used in the edge terminal to build a target detection model. The target detection model is trained by collecting and annotating data. According to the trained target detection model and the camera video stream data, the human body, head, safety helmet, safety hook, safety belt, safety rope, reflective clothing, hands, and gloves in the work platform are detected. If someone is detected in the installation area, but there is no safety helmet, safety hook, safety belt, reflective clothing, hand, or glove, after delayed confirmation, it is determined whether an alarm is needed according to the preset logic. If necessary, an alarm message is sent, an on-site sound and light alarm is sounded, and event messages and on-site pictures are pushed to the supervisor's mobile phone. In this way, the operation of the installer during the work process is monitored throughout the whole process, which helps to improve the work quality, standardize the installation process, and reduce the risk of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The figure is a flow chart of the elevator installation process monitoring method based on target detection of the present invention.

[0064] Figure 2 Schematic diagram of the training process of the target detection model in the present invention.

[0065] Figure 3 The present invention is a complete flow chart for monitoring the elevator installation process.

[0066] Figure 4 It is a structural schematic diagram of the elevator installation process monitoring system in the present invention.

[0067] Figure 5It is a structural schematic diagram of an elevator installation process monitoring device based on target detection according to the present invention.

[0068] Figure 6 It is a schematic diagram of the internal structure of the computer device in the present invention.

[0069] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0070] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0071] The installation of a vertical elevator requires the installation of a temporary work platform in the hoistway. The platform can move up and down and stop in the hoistway to facilitate the installation of hall doors, hoistway rails and various brackets. This proposal is to install a camera on the work platform to monitor the various installation work of the installers on the platform.

[0072] The overall process of the present invention is as follows: Use target detection algorithms, such as YOLO, to build a target detection model; Collect data, annotate data, and train the target detection model; Detect human bodies, heads, safety helmets, safety hooks, safety belts, safety ropes, reflective clothing, hands, and gloves in the work platform based on the trained target detection model and camera video stream data. If someone is detected in the installation area, but there is no safety helmet, safety hook, safety belt, reflective clothing, hands, or gloves, after delayed confirmation, determine whether an alarm is needed based on the preset logic, and if necessary, send an alarm message, an on-site sound and light alarm, and push event messages and on-site pictures to the supervisor's mobile phone.

[0073] The details are as follows:

[0074] like Figure 1 As shown, the present invention provides an elevator installation process monitoring method based on target detection, which is based on a detection system consisting of a camera installed on the top of the inner column of the working platform and an edge terminal with a neural network processor.

[0075] The method comprises:

[0076] S1. Obtain image data of the work platform, and mark the image data with objects to obtain a training data set;

[0077] The camera is installed on the top of the column in the work platform, and can capture the entire installation work area. Image materials are collected from multiple installation sites and annotated as training data sets. The human body, head, safety helmet, safety hook, safety belt, safety rope, reflective clothing, hand (including wrist and hand area), and glove targets in the picture are annotated. Each target has a different label and is sorted from 0. The general label is a rectangle or a rectangle plus a rotation angle. The labeled data is used to train the target detection model.

[0078] S2. Constructing a target detection model, and using the training data set to train the target detection model to obtain a trained target detection model;

[0079] Obtain a set of accurate target detection results as training targets, input the training data set obtained in step S1 into the target detection model, use a preset target detection algorithm, such as YOLO, to train the target detection model so that it can achieve the detection results of the training targets, and finally obtain a trained target detection model.

[0080] The target detection model is trained and inferred to obtain rectangular frames of various categories of targets such as human heads and human bodies. The data is the pixel coordinates and width and height of the center point. For example, the upper left corner of the image is 0, and the lower right corner is the image width and height.

[0081] S3, acquiring the video stream data of the operation platform in real time, and inputting the trained target detection model to obtain multiple target detection results,

[0082] In one embodiment, step S3 specifically includes:

[0083] S301, the edge terminal with a neural network processor (NPU) is installed in the computer room, communicates with the camera wirelessly, and accesses the external Internet through a 4G router.

[0084] Detection area: Since the camera is installed inside the working platform, and the working platform is in the elevator shaft, when the working platform moves up and down, in some positions, the edge area of ​​the camera screen may capture the elevator lobby. People standing in the elevator lobby are obviously not the objects to be monitored, so the edge area needs to be excluded.

[0085] The back-end manager logs in to the edge terminal through a mobile phone or computer browser. The terminal obtains the camera video stream data, allowing the back-end manager to view the real-time image of the camera and draw the hook beam line, that is, draw the straight lines HL1 and HL2 of the beams on the left and right sides of the work platform. The purpose is to detect whether the installer wears the safety belt correctly. In actual operation, only the coordinates of the two end points of the beam need to be marked. One terminal can manage multiple cameras, that is, multiple work platforms.

[0086] S4, analyzing the detection results of multiple targets in the monitoring screen of the work platform to obtain multiple labels; wherein the labels include a human body, a human head, a safety helmet, a safety hook, a safety belt, a safety rope, a reflective vest, a hand, and a glove;

[0087] The model infers multiple target detection results. When analyzing the multiple target detection results, the data is manually labeled to obtain the label category (human body, head, safety helmet, safety hook, safety belt, safety rope, reflective clothing, hand, glove) to which each detection result belongs. For example, the rectangular box inferred by the model is labeled with a human body label.

[0088] Each tag is represented by preset letters and ID numbers, where letters represent categories and digital IDs represent the order in which targets appear. When tracking each target tag, the IDs of target tags of the same category are incremented in numerical order. When the target tag is lost, the corresponding ID will no longer be used. The target tag is the detected target, and the ID is used to characterize the uniqueness of the target.

[0089] That is, use the trained target detection model to infer the video frame, get the detection results of various labels, filter the targets outside the detection area, there may be two operators in the monitoring area, and the various targets may not be unique. For the convenience of description, letters plus ID numbers (ID numbers are represented in the form of subscripts) are used to represent the targets of the detection results, that is, human body Pi, human head Hi, safety helmet Mi, safety hook Gi, safety belt Di, safety rope Si, reflective clothing Fi, hand Bi, glove Ai, the number i represents the ID of the category label, and the number after it represents each label of each frame, P i1 -P i25 Indicates that the same person appears in 25 frames. Generally, each frame of the video will be detected. There are 25 frames in 1 second, that is, the human body Pi is P i1 -P i25 Furthermore, pixel feature tracking is performed on each target so that the ID of the same target in subsequent frames remains unchanged. If the target is lost (not detected for more than a certain lifetime), the ID will be retained and no longer used. The ID of the target that appears later will be incremented by 1, i.e., P i1 -P i25 It means that human body Pi appears in the 1st to 25th frames, and disappears in the 26th frame, then Pi will be retained. When human body Pi appears again in subsequent frames, it will be represented by other IDs such as Pj and Pk.

[0090] Target tracking: Track the target. For example, the first person is detected in the first frame and assigned ID = i. This person is detected in subsequent frames. Since we tracked him, we know that he is the same person, so his ID is always i. After the second person appears, his ID is j. If the person with ID 1 disappears from the screen (out of bounds or tracking fails) for more than a certain value (settable, such as 50 frames), the value of ID equal to i will no longer be used. Even if the person who appears later is the same person as the first one, his ID will be k. The same applies to other categories. The target tracking algorithms available are DeepSort and ByteTrack.

[0091] S5. Determine whether there are human-related tags among the multiple tags. When the detection results are obtained above, manual correction has been made to remove the people waiting outside the elevator, so the human tags that appear are all human tags of the installer.

[0092] S6. If there is a human-related tag among the multiple tags, multiple target detection results are tracked, and a preset rule is used to determine whether the installer has any abnormal event;

[0093] In one embodiment, step S6 specifically includes:

[0094] S61, if there is a human body-related label among the multiple labels, tracking the multiple target detection results;

[0095] S62, judging whether the installer is wearing a safety helmet according to the head tag and the safety helmet tag, and recording the event as a first alarm;

[0096] In one embodiment, step S62 specifically includes:

[0097] S621: When a head tag exists in the target detection result, a video frame of the head tag and a rectangular frame of the head tag are recorded as a first target comparison frame;

[0098] S622: Determine whether there is a helmet tag in the first target comparison frame, and whether the overlap degree between the rectangular frame of the helmet tag and the rectangular frame of the head tag reaches a first preset threshold;

[0099] S623: If the overlap degree between the rectangular frame of the helmet label and the rectangular frame of the head label reaches a first preset threshold, it is determined that the installer has correctly worn the helmet;

[0100] S624: if the overlap between the rectangular frame of the helmet label and the rectangular frame of the head label does not reach a first preset threshold, counting a first time length of consecutive frames in which the helmet label does not appear;

[0101] S625: When the first time length does not exceed a first set value, determine that the installer has correctly worn a safety helmet;

[0102] S626. When the first time length exceeds a first set value, determine that the installer does not wear the safety helmet correctly, and record the number of abnormalities. When the number of abnormalities reaches a first threshold, record a first alarm event abnormality.

[0103] As described in S621-S626 above, when a person is detected (there is a human-related label in the detection area), there is a head label in the human label area, and the overlap degree (intersection-over-union ratio) between the rectangular frame of the head label and the rectangular frame of the helmet label reaches 80% (the first preset threshold, which can be adjusted as needed), it is considered that the installer is wearing a helmet. If the overlap degree is less than or equal to 80%, it is possible that the installer took off the helmet during certain periods of time (if it is taken off briefly (within the first time length, which can be set to 3 seconds or other values), it is still considered safe, such as combing hair, etc.), or wore the helmet for a short time. At this time, it is unqualified, and it is determined that the installer did not wear the helmet correctly. That is, if the installer does not wear a helmet for 3 seconds (the first time length) or more, the abnormality is recorded once. When the abnormality reaches 5 times (the first threshold), the first alarm event is determined to be abnormal, and the alarm of not wearing a helmet event is triggered.

[0104] S63, judging whether the installer is wearing gloves according to the hand tag and the glove tag, and recording it as a second alarm event;

[0105] In one embodiment, step S63 specifically includes:

[0106] S631, when the target detection result has a hand tag, recording the video frame of the hand tag and the rectangular frame of the hand tag as a second target comparison frame;

[0107] S632, determining whether there is a glove label in the second target comparison frame, and whether the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold;

[0108] S633: If the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold, it is determined that the installer has correctly worn the gloves;

[0109] S634: if the overlap between the rectangular frame of the glove label and the rectangular frame of the hand label does not reach a second preset threshold, counting a second time length of consecutive frames in which the glove label does not appear;

[0110] S635: When the second time length does not exceed a second set value, determine that the installer has correctly worn the gloves;

[0111] S636: When the second time length exceeds a second set value, determine that the installer does not wear gloves correctly, and record the number of abnormalities. When the number of abnormalities reaches a second threshold, record a second alarm event abnormality.

[0112] As described in S631-S636 above, when a person is detected (there is a human-related label in the detection area), there is a hand label in the human label area, and the overlap degree (intersection-over-union ratio) between the rectangular frame of the hand label and the rectangular frame where the glove label appears reaches 80% (the second preset threshold, which can be adjusted as needed), it is considered that the installer is wearing gloves. If the overlap degree is less than or equal to 80%, it is possible that the installer took off the gloves during certain periods of time (if it is taken off briefly (within the second time length, which can be set to 3 seconds or other values), it is still considered safe, such as taking off gloves to tidy up clothes, etc.), or wore gloves for a short time. At this time, it is an unqualified situation, so it is determined that the installer did not wear gloves correctly. That is, if the installer does not wear gloves for 3 seconds (the second time length) or more, the abnormality is recorded once. When the abnormality reaches 5 times (the second threshold), the second alarm event is determined to be abnormal, and the alarm of the event of not wearing gloves is triggered.

[0113] In one embodiment, step S64 specifically includes:

[0114] S64, judging whether the installer uses a safety belt according to the safety hook label, the safety rope label and the safety belt label, and recording it as a third alarm event;

[0115] S641, when there is a safety hook label in the target detection result, determine whether the rectangular frame of the safety hook label intersects with the hook beam line;

[0116] S642, when the safety hook label and the hook beam line intersect, determining whether the safety rope label and the safety hook label intersect;

[0117] S643: If the safety rope tag and the safety hook tag have an intersection, determine whether there is a safety belt tag in the human body area;

[0118] S644, if there is a seat belt tag in the human body area, then record the third time length of the frame in which the seat belt tag appears in each set time period, and when the third time length does not exceed a third set value, determine that the installer does not wear the seat belt correctly;

[0119] S645: When the third time length exceeds a third set value, it is determined that the installer wears the safety belt correctly and the normal number is recorded. When the normal number does not reach a third threshold, a third alarm event abnormality is recorded.

[0120] As described in S641-S645 above, when a person is detected, it is determined whether there is a safety hook tag. If so, it is determined whether the safety hook intersects with the beams HL1 and HL2 (the straight line and the rectangular frame have an intersection). If so, it is determined whether there is a safety rope (a rope connecting the safety hook and the safety belt). If so, and it intersects with the safety hook rectangular frame, the intersection means that they are connected. Then it is determined whether there is a safety belt tag in the human body area. If 90% (the third set value) of the video frames within 5 consecutive seconds (set period) do not detect that the safety belt is normal, then the safety belt is considered normal, and the normal number is recorded. When the normal number does not reach 5 times (the third threshold), it is determined that the third alarm event is abnormal, and the safety belt violation event alarm is triggered. Taking into account the on-site environment and actual conditions, such as during welding operations, there will be smoke and sparks, the accuracy of target detection may decrease, and the safety hook, safety rope, and safety belt may be lost. The third set value can be appropriately increased. In the present invention, it is set that as long as 10% of the video frames are judged to be normal, it is considered normal.

[0121] S65, judging whether the installer is wearing reflective clothing according to the reflective clothing label, and recording it as a fourth alarm event;

[0122] In one embodiment, step S65 specifically includes:

[0123] S651, when there is a reflective clothing label in the target detection result, recording a fourth time length of the reflective clothing label according to the ID of the reflective clothing label;

[0124] S652: When the fourth time length exceeds a fourth set value, it is determined that the installer is wearing the reflective clothing correctly;

[0125] S653: When the fourth time length does not exceed the fourth set value, it is determined that the installer does not wear the reflective clothing correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a fourth threshold, a fourth alarm event abnormality is recorded.

[0126] As described in S651-S653 above, when a person is detected (there is a human body-related tag in the detection area), the fourth time length of the reflective clothing tag is recorded, such as 5 seconds. If it exceeds the fourth set value (4 seconds), it is determined that the installer is wearing the reflective clothing correctly, otherwise it is determined that the reflective clothing is not worn correctly.

[0127] S66. When any of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event is abnormal, an alarm process is performed.

[0128] S67, when any alarm event is triggered, the edge terminal sends a voice command corresponding to the alarm event to the camera, and controls the camera's fill light to flash at a set frequency for prompting;

[0129] When an alarm event is triggered, the terminal sends instructions to the camera according to the preset parameters: such as voice prompts: "During installation operations, please fasten your seat belts and buckle your safety hooks!", etc., such as light alarms, controlling the camera's fill light to flash at a certain frequency.

[0130] S68, when the sum of the alarm times of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event reaches a preset number, record and store abnormal videos; wherein the abnormal videos are videos within a set time period before and after the alarm;

[0131] S69. The edge terminal reports the alarm event message and abnormal video to the server through the mobile network, so that the server can determine whether to push the alarm to the supervisor's mobile phone after receiving the message; wherein the message content includes the project number, elevator number, alarm time, alarm picture, and abnormal video.

[0132] The terminal reports the alarm event message to the server through the mobile network. The message content includes the project number, elevator number, alarm time, 3 alarm pictures, 5 seconds of video before and after the alarm, etc. After receiving the message, the server determines whether to push the alarm to the supervisor's mobile phone.

[0133] S7. If there is no human body related label among the multiple labels, return to obtain the next frame of the video stream data of the operation platform for detection.

[0134] like Figure 5 As shown, the present invention also provides an elevator installation process monitoring device based on target detection, based on a detection system composed of a camera installed on the top of the inner column of the working platform and an edge terminal with a neural network processor, the device includes:

[0135] Data module 1, used to obtain image data of the work platform and mark the image data to obtain a training data set;

[0136] Training module 2, used to build a target detection model, and train the target detection model using the training data set to obtain a trained target detection model;

[0137] An acquisition module 3 is used to acquire the video stream data of the operation platform in real time and input the trained target detection model to obtain multiple target detection results;

[0138] The analysis module 4 is used to analyze the detection results of multiple targets in the monitoring screen of the work platform to obtain multiple labels; wherein the labels include human body, human head, safety helmet, safety hook, safety belt, safety rope, reflective clothing, hand, and gloves;

[0139] A judging module 5, used to judge whether there is a human body related tag among the multiple tags;

[0140] Tracking module 6, used for tracking multiple target detection results when there is a human body-related tag among multiple tags, and using preset rules to determine whether there is an abnormal event of the installer;

[0141] The return module 7 is used to return to obtain the next frame of the video stream data of the operation platform for detection when there is no human body related label in the multiple labels.

[0142] In one embodiment, it further includes:

[0143] A drawing module is used for backstage management personnel to log in to the edge terminal through a mobile phone or computer browser to view the real-time image of the camera, and at the same time draw the hook beam line on the monitoring image of the work platform; wherein, the edge terminal with a neural network processor is installed in the computer room, communicates with the camera wirelessly, and accesses the external Internet through a 4G router.

[0144] In one embodiment, in the analysis module 4, each tag is represented by preset letters and ID numbers, wherein the letters represent the categories and the digital ID represents the order in which the targets appear. When tracking each target tag, the IDs of target tags of the same category are incremented in numerical order. When the target tag is lost, the corresponding ID will no longer be used. The target tag is the detected target and the ID is used to characterize the uniqueness of the target.

[0145] In one embodiment, the tracking module 6 includes:

[0146] An assignment unit, used for tracking multiple target detection results if a human body label exists among the multiple labels;

[0147] A first alarm unit, used for judging whether the installer is wearing a safety helmet according to the head tag and the safety helmet tag, so as to record it as a first alarm event;

[0148] A second alarm unit, used for determining whether the installer is wearing gloves according to the hand tag and the glove tag, so as to record the result as a second alarm event;

[0149] A third alarm unit, used to determine whether the installer uses a safety belt according to the safety hook label, the safety rope label and the safety belt label, so as to record it as a third alarm event;

[0150] A fourth alarm unit, used to determine whether the installer is wearing reflective clothing according to the reflective clothing label, so as to record it as a fourth alarm event;

[0151] The processing unit is used to perform alarm processing when any of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event is abnormal.

[0152] In one embodiment, the first alarm unit comprises:

[0153] A first recording subunit is used for recording a video frame of the head label and a rectangular frame of the head label as a first target comparison frame when a head label exists in the target detection result;

[0154] A first judging subunit is used to judge whether there is a helmet tag in the first target comparison frame, and whether the overlap degree between the rectangular frame of the helmet tag and the rectangular frame of the head tag reaches a first preset threshold;

[0155] A first determining subunit, configured to determine that the installer has correctly worn the safety helmet when the overlap between the rectangular frame of the safety helmet label and the rectangular frame of the head label reaches a first preset threshold;

[0156] A first counting subunit is used for counting a first time length of continuous frames in which the helmet tag does not appear when the overlap degree between the rectangular frame of the helmet tag and the rectangular frame of the head tag does not reach a first preset threshold;

[0157] A second determining subunit, configured to determine that the installer has correctly worn the safety helmet when the first time length does not exceed a first set value;

[0158] The second recording subunit is used to determine that the installer does not wear the safety helmet correctly when the first time length exceeds a first set value, and record the number of abnormalities, and record the first alarm event abnormality when the number of abnormalities reaches a first threshold.

[0159] In one embodiment, the second alarm unit comprises:

[0160] A hand label recording subunit, used for recording the video frame of the hand label and the rectangular frame of the hand label as a second target comparison frame when the target detection result has a hand label;

[0161] A second judgment subunit is used to judge whether there is a glove label in the second target comparison frame, and whether the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold;

[0162] A third determination subunit is used to determine that the installer has correctly worn the gloves when the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold;

[0163] A second statistical subunit is used for counting a second time length of continuous frames in which no glove label appears when the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label does not reach a second preset threshold;

[0164] a fourth determining subunit, configured to determine that the installer has correctly worn the gloves when the second time length does not exceed a second set value;

[0165] The fifth determination subunit is used to determine that the installer does not wear gloves correctly when the second time length exceeds a second set value, and record the number of abnormalities, and record a second alarm event abnormality when the number of abnormalities reaches a second threshold.

[0166] In one embodiment, the third alarm unit comprises:

[0167] An intersection determination subunit, used for determining whether there is an intersection between the rectangular frame of the safety hook label and the hook beam line when there is a safety hook label in the target detection result;

[0168] An intersection judgment subunit, used for judging whether a safety rope tag and the safety hook tag have an intersection when the safety hook tag and the hook beam line have an intersection;

[0169] A safety belt tag judging subunit, used for judging whether there is a safety belt tag in the human body area when the safety rope tag and the safety hook tag intersect;

[0170] A seat belt wearing subunit is used to record a third time length of a frame in which the seat belt label appears within each set time period when a seat belt label is present in the human body area, and when the third time length does not exceed a third set value, it is determined that the installer does not wear the seat belt correctly;

[0171] The third alarm event recording subunit is used to determine that the installer wears the safety belt correctly and record the normal number of times when the third time length exceeds the third set value, and record the third alarm event abnormality when the normal number of times does not reach the third threshold.

[0172] In one embodiment, the fourth alarm unit comprises:

[0173] a reflective clothing label recording subunit, configured to record, when a reflective clothing label exists in the target detection result, a fourth time length of the existence of the reflective clothing label according to the ID of the reflective clothing label;

[0174] A reflective clothing wearing subunit, used for determining that the installer correctly wears the reflective clothing when the fourth time length exceeds a fourth set value;

[0175] The fourth alarm event recording subunit is used to determine that the installer does not wear the reflective clothing correctly when the fourth time length does not exceed the fourth set value, and record the number of abnormalities. When the number of abnormalities reaches a fourth threshold, record the fourth alarm event abnormality.

[0176] In one embodiment, the tracking module 6 further includes:

[0177] A control unit, which is used for, when any alarm event is triggered, the edge terminal sends a voice command corresponding to the alarm event to the camera, and controls the fill light of the camera to flash at a set frequency for prompting;

[0178] A storage unit, used for recording and storing abnormal videos when the sum of the alarm times of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event reaches a preset number; wherein the abnormal videos are videos within a set time period before and after the alarm;

[0179] The reporting unit is used for the edge terminal to report the alarm event message and abnormal video to the server through the mobile network, so that the server can determine whether to push the alarm to the supervisor's mobile phone after receiving the message; wherein the message content includes the project number, elevator number, alarm time, alarm picture, and abnormal video.

[0180] The above modules, units and sub-units are used to execute the corresponding steps in the above-mentioned elevator installation process monitoring method based on target detection. The specific implementation method thereof is described in the above-mentioned method embodiment and will not be repeated here.

[0181] like Figure 6 As shown, the present invention also provides a computer device, which can be a server, and its internal structure can be as shown in Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store all data required for the process of the elevator installation process monitoring method based on target detection. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the elevator installation process monitoring method based on target detection is implemented.

[0182] Those skilled in the art will understand that Figure 6The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0183] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, any one of the above-mentioned elevator installation process monitoring methods based on target detection is implemented.

[0184] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0185] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0186] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for monitoring the elevator installation process based on target detection, characterized in that: Based on a detection system consisting of a camera installed on the top of the inner column of the working platform and an edge terminal with a neural network processor, the method includes: Acquire image data of the work platform, and perform target annotation on the image data to obtain a training data set; Constructing a target detection model, and using the training data set to train the target detection model to obtain a trained target detection model; Acquire the video stream data of the operation platform in real time, and input the trained target detection model to obtain multiple target detection results; Analyze multiple target detection results in the monitoring screen of the work platform to obtain multiple labels; wherein the labels include human body, human head, safety helmet, safety hook, safety belt, safety rope, reflective clothing, hands, gloves; Determine whether there are human-related tags among multiple tags; If there is a human-related tag among the multiple tags, the multiple target detection results are tracked, and a preset rule is used to determine whether there is an abnormal event with the installer.

2. The elevator installation process monitoring method based on target detection according to claim 1 is characterized in that: Before the step of analyzing the multiple target detection results in the monitoring screen of the operation platform to obtain multiple labels, the step further includes: The back-end management personnel log in to the edge terminal through a mobile phone or computer browser to view the real-time image of the camera, and draw the hook beam line on the monitoring image of the working platform; wherein, the edge terminal with a neural network processor is installed in the computer room, communicates with the camera wirelessly, and accesses the external Internet through a 4G router.

3. The elevator installation process monitoring method based on target detection according to claim 1 is characterized in that: In the step of analyzing the multiple target detection results in the monitoring screen of the work platform to obtain multiple labels, each label is represented by preset letters and ID numbers, wherein the letters represent categories, and the digital ID represents the order in which the targets appear. When tracking each target label, the IDs of target labels of the same category are incremented in numerical order. When the target label is lost, the corresponding ID will no longer be used; wherein the target label is the detected target, and the ID is used to characterize the uniqueness of the target.

4. The elevator installation process monitoring method based on target detection according to claim 1 is characterized in that: If there is a human-related tag among the multiple tags, the step of tracking multiple target detection results and using a preset rule to determine whether the installer has an abnormal event includes: If there are human-related labels among multiple labels, multiple target detection results are tracked; Determining whether the installer is wearing a safety helmet according to the head tag and the safety helmet tag, and recording it as a first alarm event; Determining whether the installer is wearing gloves according to the hand tag and the glove tag, and recording the event as a second alarm; Determining whether the installer uses a safety belt according to the safety hook label, the safety rope label and the safety belt label, so as to record the third alarm event; Determining whether the installer is wearing reflective clothing according to the reflective clothing label, and recording it as a fourth alarm event; When any of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event is abnormal, alarm processing is performed.

5. The elevator installation process monitoring method based on target detection according to claim 4 is characterized in that: The step of determining whether the installer is wearing a safety helmet based on the head tag and the safety helmet tag to record the first alarm event includes: When a head tag exists in the target detection result, a video frame of the head tag and a rectangular frame of the head tag are recorded as a first target comparison frame; Determine whether there is a helmet tag in the first target comparison frame, and whether the overlap degree between the rectangular frame of the helmet tag and the rectangular frame of the head tag reaches a first preset threshold; If the overlap degree between the rectangular frame of the helmet label and the rectangular frame of the head label reaches a first preset threshold, it is determined that the installer has correctly worn the helmet; If the overlap degree between the rectangular frame of the helmet label and the rectangular frame of the head label does not reach a first preset threshold, counting a first time length of continuous frames in which the helmet label does not appear; When the first time length does not exceed a first set value, determining that the installer has correctly worn the safety helmet; When the first time length exceeds a first set value, it is determined that the installer does not wear the safety helmet correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a first threshold, a first alarm event abnormality is recorded.

6. The elevator installation process monitoring method based on target detection according to claim 4 is characterized in that: The step of determining whether the installer is wearing gloves according to the hand tag and the glove tag to record the second alarm event includes: When the target detection result has a hand label, the video frame of the hand label and the rectangular frame of the hand label are recorded as a second target comparison frame; Determine whether there is a glove label in the second target comparison frame, and whether the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold; If the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label reaches a second preset threshold, it is determined that the installer has correctly worn the gloves; If the overlap degree between the rectangular frame of the glove label and the rectangular frame of the hand label does not reach a second preset threshold, counting a second time length of consecutive frames in which the glove label does not appear; When the second time length does not exceed a second set value, it is determined that the installer has correctly worn the gloves; When the second time length exceeds a second set value, it is determined that the installer does not wear gloves correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a second threshold, a second alarm event abnormality is recorded.

7. The elevator installation process monitoring method based on target detection according to claim 4 is characterized in that: The step of determining whether the installer uses a safety belt based on the safety hook label, the safety rope label and the safety belt label to record the third alarm event includes: When a safety hook label exists in the target detection result, determining whether a rectangular frame of the safety hook label intersects with the hook beam line; When the safety hook label and the hook beam line intersect, determining whether the safety rope label and the safety hook label intersect; If the safety rope tag and the safety hook tag have an intersection, it is determined whether there is a safety belt tag in the human body area; If there is a seat belt tag in the human body area, then record the third time length of the frame where the seat belt tag appears in each set time period, and when the third time length does not exceed a third set value, it is determined that the installer does not wear the seat belt correctly; When the third time length exceeds a third set value, it is determined that the installer wears the safety belt correctly, and the normal number is recorded. When the normal number does not reach a third threshold, a third alarm event abnormality is recorded.

8. The elevator installation process monitoring method based on target detection according to claim 4 is characterized in that: The step of judging whether the installer is wearing reflective clothing according to the reflective clothing label to record it as a fourth alarm event includes: When a reflective clothing label exists in the target detection result, a fourth time length of the existence of the reflective clothing label is recorded according to the ID of the reflective clothing label; When the fourth time length exceeds a fourth set value, it is determined that the installer is wearing the reflective clothing correctly; When the fourth time length does not exceed the fourth set value, it is determined that the installer does not wear the reflective clothing correctly, and the number of abnormalities is recorded. When the number of abnormalities reaches a fourth threshold, a fourth alarm event abnormality is recorded.

9. The elevator installation process monitoring method based on target detection according to claim 4 is characterized in that: After the step of performing alarm processing when any of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event is abnormal, the method further includes: When any alarm event is triggered, the edge terminal sends a voice command corresponding to the alarm event to the camera, and controls the camera's fill light to flash at a set frequency; When the sum of the alarm times of the first alarm event, the second alarm event, the third alarm event, and the fourth alarm event reaches a preset number, abnormal videos are recorded and stored; wherein the abnormal videos are videos within a set time period before and after the alarm; The edge terminal reports the alarm event message and abnormal video to the server through the mobile network, so that the server can determine whether to push the alarm to the supervisor's mobile phone after receiving the message; wherein the message content includes the project number, elevator number, alarm time, alarm picture, and abnormal video.

10. An elevator installation process monitoring device based on target detection, characterized in that: A detection system based on a camera installed on the top of the inner column of the working platform and an edge terminal with a neural network processor, the device includes: A data module is used to obtain image data of the operation platform and mark the image data to obtain a training data set; A training module is used to construct a target detection model and train the target detection model using the training data set to obtain a trained target detection model; An acquisition module is used to acquire the video stream data of the operation platform in real time and input the trained target detection model to obtain multiple target detection results; An analysis module is used to analyze the detection results of multiple targets in the monitoring screen of the work platform to obtain multiple labels; wherein the labels include a human body, a human head, a safety helmet, a safety hook, a safety belt, a safety rope, a reflective vest, a hand, and a glove; A judgment module, used to judge whether there is a human body related label among multiple labels; The tracking module is used to track multiple target detection results when there are human-related tags among multiple tags, and use preset rules to determine whether there are any abnormal events involving the installer.

Images