CV-based archive room safety operation monitoring method and device and storage medium

Abstract

The application discloses a CV-based archive room safety operation monitoring method and device and a storage medium, and comprises the following steps: an image acquisition device continuously collects image and / or video data of an operator entering an archive room; image is extracted from the image and / or video data as a to-be-identified image according to a set sampling time interval; a pre-trained identification model is used to identify the to-be-identified image; according to the results of scene identification and risk identification and a preset compliance operation, compliance is judged, risk operation is confirmed, an alarm device initiates an alarm in real time, and the operator complies with the operation. The application can efficiently realize real-time monitoring and early warning of safe operation and effectively prevent safety accidents.

Description

Technical Field

[0001] This invention relates to the field of real-time monitoring technology for safe operations, specifically to a method, device, and storage medium for monitoring safe operations in archive storage facilities based on computer vision (CV). Background Technology

[0002] Currently, government agencies, enterprises, and institutions have an increasing demand for archival preservation. In archive storage rooms, for large quantities of paper archives that are not frequently accessed but still require preservation, multiple copies are usually boxed and stored on high-rise heavy-duty shelving. The overall height of high-rise heavy-duty shelving can reach 6-11 meters. When storing and retrieving archives, it is necessary to use a lifting platform. During the operation, there is a risk of accidents such as personnel falling from the lifting platform or being injured by the archive boxes high on the shelves, which seriously endangers the personal safety of employees and the interests of archive management companies.

[0003] To ensure the safety of workers, record management companies typically require them to take full safety precautions throughout the process and issue relevant rules and regulations. These rules and regulations, along with regular inspections, are used to standardize worker behavior, such as periodically checking internal surveillance footage to identify any violations. However, current methods have the following problems:

[0004] (1) Relying on the self-safety awareness and self-discipline of warehouse workers is not enough to effectively prevent safety accidents.

[0005] (2) Spot checks of surveillance videos revealed non-compliant operations, which was inefficient, unable to provide real-time monitoring and early warning, and unable to effectively and timely prevent security risks caused by violations.

[0006] Therefore, we need to develop a CV-based method and device for monitoring safe operations in archive storage facilities, which can efficiently achieve real-time monitoring and early warning of safe operations and effectively prevent the occurrence of safety accidents. Summary of the Invention

[0007] The purpose of this invention is to provide a method, device and storage medium for monitoring the safety of archive storage operations based on CV, so as to solve the problems mentioned in the background art, such as low efficiency of storage operation safety monitoring, inability to monitor and warn in real time.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] According to one aspect of the present invention, a method for monitoring safe operations in an archive storage facility based on colorimetry (CV) is provided, characterized in that the method comprises:

[0010] Image acquisition equipment continuously acquires images and / or video data from personnel entering the archive storage room;

[0011] extracting images from the image and / or video data as to-be-identified images according to a set sampling time interval;

[0012] performing scene identification and risk identification on the to-be-identified images by using a pre-trained identification model;

[0013] performing compliance judgment according to the results of the scene identification and risk identification and preset compliance operations, confirming a risk operation, and initiating an alarm by an alarm device in real time until an operator performs a compliant operation.

[0014] According to another aspect of the present application, a CV-based safe operation monitoring device for an archive room is provided, which comprises:

[0015] an image acquisition system configured to continuously acquire image and / or video data of an operator entering the archive room and extract images from the image and / or video data as to-be-identified images according to a set sampling time interval;

[0016] an image identification system configured to perform scene identification and risk identification on the to-be-identified images by using a pre-trained identification model;

[0017] a risk warning system configured to perform compliance judgment according to the results of the scene identification and risk identification and preset compliance operations, confirm a risk operation, and initiate an alarm until an operator performs a compliant operation.

[0018] According to the foregoing scheme, the training method of the identification model comprises:

[0019] based on the scene of the archive room, sample images of the operator in different states and under different light conditions are acquired from different angles;

[0020] the target objects in the sample images are sequentially labeled, and the labeled sample images are divided into a training set and a verification set;

[0021] a YOLO algorithm is used to identify the target objects in the labeled sample images, and the identification model is trained.

[0022] Preferably, the model is trained for each target object, and in the process of completing the model training, the model verification can be completed according to the set parameters, thereby ensuring the high accuracy of the trained model.

[0023] According to the foregoing scheme, the identification model comprises a scene identification model and a risk identification model; the scene identification model comprises but is not limited to a model-human and a head identification model, a model-archival shelf area identification model, and a model-human identification model on an elevating platform. a model-human and a head identification model, M b model-archival shelf area identification model, M c model-human identification model on an elevating platform, M dModel - whether the lift platform is raised identifies one or more of the models; the risk identification model includes but is not limited to M e Model - safety helmet and whether the safety helmet is worn on the head of the person identifies the model, M f Model - safety belt and whether it is tied to the target position identifies the model, M g Model - whether the lift platform leg is open identifies the model, M h Model - human body posture recognition model; M i Model - distance between the person and the file shelf area identifies the model, M j Model - whether the lift platform moves identifies one or more of the models.

[0024] Based on the foregoing scheme, the method for identifying the above-mentioned scene comprises:

[0025] Call M a Model, M b Model and M c The model identifies the image to be identified;

[0026] If the M a Model identifies a person and the person's head, the M b Model identifies the file shelf area, and the M c Model identifies that the person is not on the lift platform, then the output scene is N1 scene - the worker enters the file shelf area, but does not use the lift platform;

[0027] If the M a Model identifies a person and the person's head, the M b Model identifies the file shelf area, and the M c Model identifies that the person is on the lift platform, then the output scene is N1 scene - the worker enters the file shelf area, and is on the lift platform;

[0028] Call M d Model further identifies the image to be identified, if the M d Model identifies that the lift platform is in a raised state, then the output scene is N2 scene - the worker is on the lift platform in a raised state; otherwise, the output scene is N3 scene - the worker is on the lift platform in a non-raised state.

[0029] Preferably, scene identification is performed first, which can classify the image into a scene, and output the scene classification of each image to the risk identification module, which can avoid the problem that each image needs to be identified by each identification model, and effectively improve the identification efficiency.

[0030] Based on the foregoing scheme, the method for risk identification according to the result of the scene identification is as follows:

[0031] If the output scene of the scene identification is an N1 scene, M e model and M i The model further identifies the image to be identified.

[0032] If the output scene of the scene identification is an N2 scene, M e model, M f model, M g model, M h model and M j The model further identifies the image to be identified.

[0033] If the output scene of the scene identification is an N3 scene, M e model, M f model, M g model, M h The model further identifies the image to be identified.

[0034] Preferably, the scene identification classifies the image into scenes, and the risk identification module calls different combinations of risk identification models according to each scene to identify risks. This avoids the problem of reduced accuracy and efficiency caused by too many target objects in an image.

[0035] Based on the foregoing scheme, the method for risk identification according to the result of the scene identification, the result of the risk identification, and the preset compliance operation to determine the risk operation includes:

[0036] If the output scene of the scene identification is an N1 scene, the M e model does not identify a safety helmet and / or identifies that the safety helmet is not worn on the head of the person, and the M i model identifies that the distance between the person and the archival shelf area is less than a set threshold value, it is determined that the operation is a risk operation;

[0037] If the output scene of the scene identification is an N2 scene, the M e model does not identify a safety helmet and / or identifies that the safety helmet is not worn on the head of the person, or the M f model does not identify a safety belt and / or identifies that the safety belt is not buckled at the target position, or the M g model identifies that the lifting platform support leg is not opened, or the M h model identifies that the difference between the human body and the upright angle exceeds a set threshold value, or the M j model identifies that the lifting platform moves, it is determined that the operation is a risk operation;

[0038] If the output scene identified by the scene recognition is the N3 scene, the M e The model does not identify the safety helmet and / or identifies that the safety helmet is not worn on the head of the person or through the M f The model does not identify the safety belt and / or identifies that the safety belt is not buckled at the target position or through the M g The model identifies that the lifting platform support leg is not opened or through the M h The model identifies that the human body and the upright angle difference exceeds the set threshold, and confirms the risk operation.

[0039] The compliance judgment module can preset the compliance operation under each scene, compare the results output by the risk identification module with the compliance operation under the corresponding scene, and confirm the risk operation if any item in the risk identification result is inconsistent with the preset compliance operation. Once confirmed, a warning signal is immediately sent to the alarm device. The purpose of real-time monitoring and real-time warning can be achieved, the problem that the operation personnel do not comply with the compliance operation throughout the operation when operating for a long time can be avoided, and the entire process does not require manual operation, and the efficiency is higher.

[0040] Based on the foregoing scheme, the image acquisition system includes an image acquisition device, an image storage module, and an image extraction module. The image acquisition device includes a plurality of camera devices, which ensures that there is no dead angle in the monitoring range.

[0041] Preferably, the installation angle of the image acquisition device should be consistent with the identification model training process to improve the efficiency and accuracy of identification. The image acquisition device continuously acquires image and / or video data from various angles. The acquired image and / or video data can be set according to actual needs, and the sampling time interval is set according to the set value. The image is extracted as a to-be-identified image at regular intervals, which can achieve the purpose of real-time monitoring and real-time warning.

[0042] Based on the foregoing scheme, the image recognition system includes an identification model training module, an identification model storage module, a scene recognition module, and a risk identification module. The risk warning system includes a compliance judgment module and an alarm device.

[0043] The embodiment of the application also provides a computer storage medium, which stores computer executable code. The computer executable code can implement the method provided by one or more of the foregoing technical solutions when executed.

[0044] As can be seen from the above technical solutions, the present application has at least the following advantages and positive effects compared with the prior art:

[0045] (1) The application utilizes an image acquisition device to acquire image and / or video data of the operating personnel in the archive warehouse, sets a sampling time interval, and then sequentially completes scene recognition, risk identification and compliance judgment, and once the risk operation is confirmed, an alarm is immediately initiated, the process of risk monitoring does not depend on manual operation, is more efficient, and can timely and effectively prompt and handle risks before accidents occur, thereby avoiding accidents.

[0046] (2) The image acquisition device in the archive warehouse continuously acquires image and / or video data of the operating personnel and their related operations, and periodically extracts images according to the set time interval for scene recognition and risk identification, thereby achieving the purpose of real-time monitoring and real-time early warning of the compliance operation of the operating personnel, avoiding the problem that the operating personnel do not comply with the operation in the whole process during long-time operation, and effectively preventing safety accidents.

[0047] (3) The application first performs scene recognition to classify the images, and the risk identification module calls different risk identification model combinations for risk identification according to each type of output scene. This can avoid the problem of reduced accuracy and efficiency due to too many target objects to be identified in an image, and realize efficient and accurate risk identification.

[0048] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0049] The drawings incorporated into the specification and forming part of the specification, show embodiments consistent with the application, and together with the specification, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art. In the drawings:

[0050] Figure 1 The flowchart of the real-time early warning method for safe operation of the archive warehouse of the application;

[0051] Figure 2 The flowchart of the training identification model method of the application;

[0052] Figure 3 The flowchart of the scene recognition method of the application;

[0053] Figure 4 The flowchart of the risk identification method of the application;

[0054] Figure 5 The flowchart of the method for performing compliance judgment and confirming risk operation of the application;

[0055] Figure 6 Fig. 1 is a schematic diagram of the working process of the present application from a certain angle;

[0056] Figure 7 Fig. 1 is a schematic diagram of the working process of the present application from a certain angle;

[0057] In the drawings, the components represented by the respective reference numerals are as follows:

[0058] 1. image acquisition device, 2. file box, 3. safety helmet, 4. safety belt, 5. lifting platform, 6. supporting leg, 7. file shelf. DETAILED DESCRIPTION

[0059] In order to more clearly illustrate the purpose, technical solutions and advantages of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The example embodiments can be implemented in various forms, and should not be understood as being limited to the examples described herein; on the contrary, these embodiments are provided so that the present application will be more comprehensive and complete, and the ideas of the example embodiments will be fully conveyed to those skilled in the art.

[0060] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a sufficient understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring the aspects of the present application.

[0061] The block diagrams shown in the drawings are only functional entities, and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0062] The flowcharts shown in the drawings are only exemplary illustrations, and do not necessarily include all contents and operations / steps, nor do they necessarily be executed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0063] CV is the abbreviation of computer vision, which is to understand the meaning expressed by images through computer simulation or to perform some rational operations on images, such as target monitoring, segmentation, classification and the like. Target detection involves main models including Fast R-CNN (Fast Region-Convolutional Neural Network) algorithm, YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) and R-FCN (Region-based Fully Convolutional Networks) and the like. YOLO is an object recognition and positioning algorithm based on deep neural network, which can identify the category and position of objects in images only by looking once, and has the biggest feature of fast running speed and can be used in real-time system.

[0064] The application will be described in detail below in combination with specific embodiments:

[0065] Embodiment 1

[0066] As shown in the figure, the embodiment provides a CV-based safe operation monitoring method for archive storeroom, and the specific steps are as follows: Figure 1

[0067] S1: An image acquisition device continuously acquires image and / or video data of an operator entering the archive storeroom;

[0068] In the embodiment, in order to ensure that the acquired image and / or video data are clear and complete, the object to be recognized can be recognized by the recognition model, and preferably, a plurality of image acquisition devices are used to acquire image and / or video data from various angles.

[0069] S2: According to a set sampling time interval, image is extracted from the image and / or video data as a to-be-recognized image;

[0070] ​In the embodiment, preferably, data collection can be performed in two ways, one is continuous image shooting using an image collection device, and the other is video shooting using an image collection device. Further, in the process of continuous monitoring, a sampling time interval can be set, and images are extracted as to-be-identified images according to the time interval. Specifically, if the collected is image data, one image in the images shot in 10 seconds can be extracted (the sampling frequency can be an important parameter in the system, and can be manually set), and one image is extracted every 10 seconds; if the collected is video data, one image can be extracted from the video data in 10 seconds, and one image is extracted every 10 seconds. It should be noted that the time interval can be increased or decreased or adjusted according to the actual needs of monitoring on the basis of the embodiment. Reasonably setting the sampling time interval can not only achieve continuous monitoring, but also reduce the pressure on the identification server, improve the efficiency of image identification, and further improve the timeliness of real-time early warning.

[0071] S3: using a pre-trained identification model to perform scene identification and risk identification on the to-be-identified image;

[0072] The identification model includes a scene identification model and a risk identification model; the scene identification model includes but is not limited to M a Model - human and head identification model, M b Model - file shelf area identification model, M c Model - whether the person is on the lifting platform identification model, M d Model - whether the lifting platform is raised identification model, one or more of the following models: M e Model - safety helmet and whether the safety helmet is worn on the head identification model, M f Model - safety belt and whether it is tied to the target position identification model, M g Model - whether the lifting platform leg is opened identification model, M h Model - human body posture identification model; M i Model - distance between person and file shelf area identification model, M j Model - whether the lifting platform moves identification model, one or more of the following models.

[0073] As Figure 2 shown, the embodiment provides a method for training an identification model, and the specific process is as follows:

[0074] S311: based on the file warehouse scene, sample images of the operating personnel in different states and different light conditions are collected through different angles;

[0075] In the embodiment, a plurality of identification models are required, and a large number of sample images are required for training each identification model. All the sample images are shot based on the file room scene, and the shooting angle is mainly determined by the installation position of the image collection device in the file room. The image collection device at each position needs to shoot a large number of images under different states of the operating personnel. Taking the M e model as an example, the image collection device needs to collect the overhead view and / or left view and / or right view and / or front view and / or back view of the operating personnel wearing a safety helmet and not wearing a safety helmet in the standing, squatting, bending and other states under different light, to ensure that the safety helmet and whether the safety helmet is worn on the head of the person can be accurately identified from any angle and in any state. Further, in order to improve the efficiency and accuracy of image recognition, a uniform color safety helmet (for example, red), a uniform color safety belt (for example, yellow) and a uniform model lifting platform are provided for the operating personnel during model training and actual operation.

[0076] S312: sequentially labeling the target objects in the sample images, and dividing the labeled sample images into a training set and a validation set;

[0077] In the embodiment, preferably, during training of different identification models, the target objects in the sample images include, but are not limited to, one or more of the file shelf area, the file box carried by the human body, the human body on the lifting platform, the lifting platform, the safety belt, the target position, the lifting platform leg, the human body posture, the distance between the human body and the file shelf area, and the lifting movement.

[0078] Preferably, in the training of the same identification model, different states of the same target object are labeled with different colored boxes and digital bodies, and the box is defined as an actual box for matching with a predicted box in the subsequent training process. Further, in order to ensure that the identification model obtained by training has high accuracy, the labeled sample images are divided into a training set and a validation set. In the embodiment, the ratio of the training set to the validation set is 8:2.

[0079] Specifically, taking the M e model as an example, the head of the person wearing a safety helmet and the head of the person not wearing a safety helmet on the sample image are labeled according to the requirements of the training model system. For example, the head of the person wearing a safety helmet is labeled with a red box, and the number 89 (89 represents the number of the target object of the head of the person wearing a safety helmet) is input in the red box. The head of the person not wearing a safety helmet is labeled with a green box, and the number 99 (99 represents the number of the target object of the head of the person not wearing a safety helmet) is input in the green box. If the number of collected sample images is 5000, the number of the training set is 4000, and the number of the validation set is 1000.

[0080] S313: Use the YOLO algorithm to identify target objects in the labeled sample images and complete the training of the recognition model.

[0081] In this embodiment, preferably, a YOLOv5 network model is used to train the sample images, which has superior speed and accuracy. The model includes an input terminal, a backbone network, a Neck network, and a head output terminal connected in sequence. The input terminal preprocesses the images; the backbone network and Neck network sequentially extract and fuse features from the preprocessed images; the head output terminal includes a loss function and non-maximum suppression (NMS) to predict image features and generate predicted bounding boxes. During the training set phase, the predicted bounding boxes are matched with the actual bounding boxes, and during the validation set phase, the predicted bounding boxes are optimized to improve model parameters. Finally, the recognition model training is completed, and the obtained recognition model can be directly used for scene recognition and risk recognition in this embodiment.

[0082] S4: Based on the results of scenario identification and risk identification, as well as the preset compliance operations, a compliance judgment is made, and a risky operation is confirmed. The alarm device will issue an alarm in real time until the operator performs a compliant operation.

[0083] In this embodiment, preferably, scene recognition is performed first to confirm the work scene, and then risk recognition is performed. The recognition result is uploaded to the compliance judgment module for compliance judgment, thereby confirming whether the operator has performed a risky operation. After a risky operation is detected, the alarm device receives the information and initiates an alarm to remind the operator until the operation is compliant.

[0084] In this embodiment, the safety protection status of personnel working in the archive storage room is monitored at set time intervals. While ensuring high efficiency, this achieves the goal of real-time monitoring and early warning of the safety protection status of personnel, which can effectively prevent safety accidents.

[0085] Example 2

[0086] like Figure 3 As shown in the figure, this embodiment provides a method for scene recognition, and the specific steps are as follows:

[0087] S321: Call M a Model, M b Model and M c The model identifies the image to be identified;

[0088] In this embodiment, the work scenario is first identified, and then the operator's actions are further assessed to determine whether they comply with safety regulations. M is then called. a Model, M b Model and M cThe model can basically determine whether the worker enters the file shelf area and whether the worker uses the lifting platform when entering the file shelf area. It should be noted that the embodiment only shows the combination of the basic work scene recognition model, and the combination can be flexibly combined according to the specific situation of the file warehouse and the requirement of safe work in actual application.

[0089] S322: If the M a model recognizes a person and the head of the person, the M b model recognizes that the person is in the file shelf area, and the M c model recognizes that the person is not on the lifting platform, the scene is output as the N1 scene - the worker enters the file shelf area but does not use the lifting platform.

[0090] Specifically, the M a model, the M b model determines that the worker enters the file warehouse and appears in the file shelf area, that is, the risk recognition is triggered. In the embodiment, in order to further subdivide the scene, the M c model determines whether the lifting platform is used, and if the lifting platform is not used, the scene is output to the risk recognition module for risk recognition.

[0091] S323: If the M a model recognizes a person and the head of the person, the M b model recognizes that the person is in the file shelf area, and the M c model recognizes that the person is on the lifting platform, the scene is confirmed as the worker enters the file shelf area and is on the lifting platform.

[0092] In the embodiment, preferably, the M c model determines whether the lifting platform is used, and if the lifting platform is used, the scene still needs to be further subdivided after the worker is confirmed to be on the lifting platform.

[0093] S324: The M d model is called to further recognize the to-be-recognized image, and if the M d model recognizes that the lifting platform is in the raised state, the scene is output as the N2 scene - the worker is on the lifting platform in the raised state; otherwise, the scene is output as the N3 scene - the worker is on the lifting platform in the un-raised state.

[0094] Preferably, after the worker is confirmed to be on the lifting platform, the state of the lifting platform should be further confirmed. Further, in the embodiment, according to the set sampling time interval, the M dThe model can identify the height change of the platform before and after the two times of lifting, if the height is increased, it means that it is in the state of rising, if the height is decreased, it means that it is in the state of falling, and if the height is unchanged, it means that it is in the state of rising or falling.

[0095] Embodiment 3

[0096] As Figure 4 shown, the embodiment provides a risk identification method, and the specific process is as follows:

[0097] S331: If the output scene of the scene identification is N1 scene, M e model and M i model are called to further identify the to-be-identified image.

[0098] S332: If the output scene of the scene identification is N2 scene, M e model, M f model, M g model, M h model and M j model are called to further identify the to-be-identified image.

[0099] S333: If the output scene of the scene identification is N3 scene, M e model, M f model, M g model, M h model are called to further identify the to-be-identified image.

[0100] Preferably, scene identification is performed first, and then risk identification is performed on the basis of the scene identification. The scene identification can classify images into scenes, and output each image scene classification to a risk identification module. The risk identification module calls different risk identification model combinations according to each type of scene to perform risk identification. In this way, the problem of reduced accuracy and efficiency caused by too many to-be-identified target objects in an image can be avoided, and efficient and high-accuracy risk identification can be achieved.

[0101] As Figure 5 shown, the embodiment provides a method for performing compliance judgment and confirming risk operation, and the specific process includes:

[0102] As Figure 6 shown, an image in a work process shot from a certain angle is exemplarily shown, and the embodiment is described in combination with the image.

[0103] S41: If the output scene of the scene identification is N1 scene, M e model does not identify the safety helmet 3 and / or identifies that the safety helmet 3 is not worn on the head of the person, and M iThe model identifies that the distance between the person and the file shelf area is less than a set threshold value, and determines that the operation is risky;

[0104] In this embodiment, in the N1 scenario, preferably, the compliance operation is that the worker must wear a safety helmet 3. If the safety helmet 3 is not worn, and the distance to the file shelf 7 area is less than a set threshold value, such as 1 meter, it is determined that the operation is risky, and a signal is output to the alarm device, and the alarm device initiates an alarm.

[0105] S42: If the output scenario identified by the scenario identification is an N2 scenario, the M e The model does not identify the safety helmet 3 and / or identifies that the safety helmet 3 is not worn on the head of the person or through the M f The model does not identify the safety belt 4 and / or identifies that the safety belt 4 is not buckled at the target position or through the M g The model identifies that the lifting platform 5 support leg 6 is not opened or through the M h The model identifies that the angle difference between the human body and the upright exceeds a set threshold value or through the M j The model identifies that the lifting platform 5 moves, and determines that the operation is risky;

[0106] In this embodiment, in the N2 scenario, preferably, the compliance operation includes:

[0107] The worker must wear a safety helmet 3 throughout the operation;

[0108] As can be seen from S42, the safety belt 4 must be buckled and buckled on the column hole of the file shelf 7. It should be noted that if the lifting platform 5 needs to be moved, the safety belt 4 should be buckled on the railing of the lifting platform 5. After reaching the specified position, the safety belt 4 is buckled again on the column hole of the file shelf 7, and the corresponding risk operation of the compliance operation is defined as a high risk level;

[0109] The support leg 6 of the lifting platform 5 must be opened. If it is not opened, it may cause the platform to tilt due to unbalanced force after being raised.

[0110] When the worker is on the lifting platform in the raised state, the body inclination angle cannot be too large to prevent falling. In this embodiment, the angle difference between the human body and the upright is set not to exceed a set threshold value, such as greater than 20 degrees, and the corresponding risk operation of the compliance operation is defined as a high risk level.

[0111] When the lifting platform 5 is in the raised state, the lifting platform 5 cannot be directly moved. It must be lowered to the initial state before it can be moved. During the lowering and moving of the lifting platform 5, the safety belt 4 should be buckled on the railing of the lifting platform 5. After reaching the specified position and raising the lifting platform 5 to the specified height, the safety belt 4 is buckled again on the column hole of the file shelf 7 to ensure the safety of the worker.

[0112] Specifically, in this scenario, any one of the five risk identification model identification results is inconsistent with the preset compliance operation, that is, it is confirmed as a risk operation, and a signal is output to the alarm device, and the alarm device initiates an alarm.

[0113] S43: If the output scene identified by the scene identification is the N3 scene, the M e The model does not identify the safety helmet and / or identifies that the safety helmet is not worn on the head of the person or through the M f The model does not identify the safety belt and / or identifies that the safety belt is not buckled at the target position or through the M g The model identifies that the lifting platform support leg is not opened or through the M h The model identifies that the human body and the upright angle difference exceeds the set threshold, and it is confirmed as a risk operation.

[0114] In this embodiment, in the N3 scene, the compliance operation preferably includes:

[0115] The work personnel must wear the safety helmet 3 throughout the process;

[0116] As can be seen from S42, the safety belt 4 must be buckled and buckled on the columnar frame hole of the file shelf 7, and the risk operation corresponding to this compliance operation is defined as a high risk level;

[0117] The support leg 6 of the lifting platform 5 must be opened, and if it is not opened, it may cause the platform to tilt due to unbalanced force after being raised;

[0118] When the work personnel is on the lifting platform in the unraised state, the human body inclination angle cannot be too large to prevent falling, and in this embodiment, the human body and the upright angle difference is set not to exceed a set threshold, such as greater than 20 degrees, and the risk operation corresponding to this compliance operation is defined as a high risk level.

[0119] Specifically, in this scenario, any one of the four risk identification model identification results is inconsistent with the preset compliance operation, that is, it is confirmed as a risk operation, and a signal is output to the alarm device, and the alarm device initiates an alarm.

[0120] In this embodiment, the image acquisition device 1 in the file storage room continuously acquires image and / or video data of the work personnel and their related operations, and periodically extracts images according to a set time interval for scene identification and risk identification, which can achieve the purpose of real-time monitoring and real-time early warning, avoid the problem that the work personnel cannot comply with the compliance operation throughout the process when working for a long time, and the entire process does not require manual operation, and the efficiency is high.

[0121] It should be noted that the embodiment only demonstrates the combination of risk identification models in the example scene, and in actual applications, the combination can be flexible according to the specific situation of the archive warehouse and the requirements for safe operation.

[0122] Embodiment 4

[0123] As Figure 7 shown, the embodiment provides a CV-based archive warehouse safety operation monitoring device, which includes an image acquisition system, an image recognition system, and a risk early warning system.

[0124] The image acquisition system is used for continuously acquiring image and / or video data of the operation personnel entering the archive warehouse, and extracting images from the image and / or video data as to-be-identified images according to a set sampling time interval;

[0125] Specifically, the image acquisition system includes an image acquisition device, an image storage module, and an image extraction module. The image acquisition device includes a plurality of camera devices to ensure that there is no dead angle in the monitoring range and continuously acquire image and / or video data from various angles. It should be noted that the installation angle of the image acquisition device should be consistent with that in the identification model training process to improve the efficiency and accuracy of identification. The acquired image and / or video data are transmitted to the image storage module for storage, and the image extraction module can set the sampling time interval according to actual needs. According to the set value, images are extracted from the image storage module as to-be-identified images and transmitted to the scene recognition module at regular intervals.

[0126] The image recognition system uses a pre-trained identification model to perform scene recognition and risk identification on the to-be-identified images.

[0127] Specifically, the image recognition system includes an identification model training module, an identification model storage module, a scene recognition module, and a risk identification module. In the present application, based on the mature neural network algorithm and machine learning algorithm, a customized identification model is automatically generated by a large number of sample images and combined with artificial annotation. The trained identification model is stored in the identification model storage module; the identification model storage module classifies the identification model into scene identification models and risk identification models; the scene recognition module sequentially calls the scene identification models according to the scene recognition process after receiving the to-be-identified images, and transmits the output scene to the risk identification module; the risk identification module calls the risk identification models according to the output scene, and the risk identification module sets the risk identification model combination to be called for each type of scene for risk identification, and then transmits the risk identification result to the compliance judgment module.

[0128] The risk early warning system is used for compliance judgment, risk operation confirmation, alarm initiation, and compliance operation of the operation personnel according to the results of scene recognition and risk identification and the preset compliance operation.

[0129] Further, the risk early warning system comprises a compliance judgment module and an alarm device. In the embodiment, further, the compliance judgment module can preset the compliance operation under each scene according to the output scene of the scene recognition, compare the result output by the risk identification module with the compliance operation under the corresponding scene, and confirm that any item in the risk identification result is inconsistent with the preset compliance operation as a risk operation. Once confirmed, the early warning signal is immediately sent to the alarm device. Preferably, the alarm device comprises a voice alarm and a buzzer. After the alarm device receives the early warning signal, the control switch is automatically opened, the buzzer rings, the voice alarm makes an explicit safety warning and a reminder of safety requirements, and a safety warning notification is sent to the relevant management personnel of the warehouse for reporting. After the warning is heard by the operating personnel, the operating personnel immediately perform the compliance operation. If the operating personnel continuously perform the risk operation, the management personnel should immediately go to the site to check and ensure safety.

[0130] Further, in the embodiment, after the risk operation is identified, the position of the operating personnel needs to be located. Preferably, the image acquisition device has a corresponding area code. Through the image uploading device, the number of the file shelf and the A / B surface can be determined. The image acquisition device shoots a certain file shelf and performs grid processing on it. Combined with the fixed row height and column number value of the file shelf, the boundary of each row and each column is established. According to the boundary of the row and the column, the grid for cutting the image is constructed. The row number and the column number corresponding to each grid are the row number and the column number of the storage location. Combined with the above information, the position of the operating personnel (shelf number + shelf A / B surface + row number + column number) can be determined.

[0131] If it is determined that the risk operation of the operating personnel is not wearing a safety helmet, the voice alarm will warn every 15 seconds: “The operating personnel at No. 2 file B surface 5 row 3 storage location, please wear a safety helmet correctly.” Until the real-time shooting compliance is achieved. When multiple risk operations occur, they are announced in turn according to the occurrence, every 7 seconds, until the real-time shooting compliance is achieved. Further, if a high-risk level risk operation is identified, the high-risk level risk operation is immediately announced, and after being removed, the other risk operations are announced in turn.

[0132] In the exemplary embodiments of the present application, a computer storage medium capable of implementing the above method is also provided. A program product capable of implementing the above method of the present specification is stored thereon. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code for causing the device to perform the steps according to various exemplary embodiments of the present disclosure described in the “Exemplary Method” section of the present specification when the above program product is run on the device.

[0133] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims. It is understood that the application is not limited to the precise structures herein disclosed and illustrated, and that various modifications and changes can be made therein without departing from the scope thereof. The only true scope of the application is indicated by the appended claims.

Claims

1. A method for monitoring safe operations in an archive storage facility based on color spectroscopy (CV), characterized in that, The method includes: Image acquisition equipment continuously acquires images and / or video data from personnel entering the archive storage room; According to the set sampling time interval, images are extracted from the image and / or video data as images to be identified; Using a pre-trained recognition model, scene recognition and risk recognition are performed on the image to be recognized; The identification model includes a scene identification model and a risk identification model; the scene identification model includes, but is not limited to, M. a Model - Human and human head recognition model, M b Model - Filing Shelf Area Recognition Model, M c Model - Whether a person is on the lifting platform to identify the model, M d Model - Whether the lifting platform is raised is identified by one or more of the following risk identification models; the risk identification models include, but are not limited to, M e Model - a helmet and a model for recognizing whether a helmet is worn on a person's head, M f Model - Seatbelt and whether it is fastened at the target location recognition model, M g Model - Lifting platform outriggers open / closed recognition model, M h Model - Human pose recognition model; M i Model - Distance recognition model between people and filing shelf area, M j Model - Whether the lifting platform has moved can be identified by one or more of the following models; The scene recognition method includes: calling M a Model, M b Model and M c The model identifies the image to be identified; If through the M a The model identifies the person and the person's head, through the M b The model identified the area in the archive shelf region via the M c If the model detects that a person is not on the lifting platform, the output scenario is N1 - the worker enters the file shelf area but does not use the lifting platform; If through the M a The model identifies the person and the person's head, through the M b The model identified the area in the archive shelf region via the M c If the model detects a person on the lifting platform, it confirms that the scene is an operator entering the file shelf area and being on the lifting platform; Call M d The model further identifies the image to be identified, if it passes through the M d If the model identifies the lifting platform as raised, the output scenario is N2 - workers are on the raised lifting platform; otherwise, the output scenario is N3 - workers are on the unraised lifting platform. Based on the set sampling time interval, M... d The model can identify the height changes of the lifting platform in two consecutive steps. If the height increases, it indicates that the platform is in the process of rising; if the height decreases, it indicates that the platform is in the process of falling; if the height remains unchanged, it indicates that the platform has completed either the rising or falling phase. The risk identification method involves identifying risks based on the results of the scenario identification, and the specific process is as follows: If the output scene of the scene recognition is scene N1, then call M. e Model and M i The model further identifies the image to be identified; If the output scene of the scene recognition is N2 scene, then call M. e Model, M f Model, M g Model, M h Model and M j The model performs further identification on the image to be identified; If the output scene of the scene recognition is scene N3, then call M. e Model, M f Model, M g Model, M h The model performs further identification on the image to be identified; Based on the results of scenario identification and risk identification, as well as the preset compliance operations, a compliance judgment is made, and if a risky operation is confirmed, an alarm is triggered in real time by the alarm device until the operator performs a compliant operation.

2. The method for monitoring safe operation of an archive storage facility based on CV according to claim 1, characterized in that, The training method for the recognition model includes: Based on the archive storage scenario, sample images of workers under different conditions and lighting were collected from different angles. The target objects in the sample images are labeled sequentially, and the labeled sample images are divided into training set and validation set; The YOLO algorithm is used to identify target objects in labeled sample images, thus completing the training of the recognition model.

3. The method for monitoring safe operation of an archive storage facility based on CV according to claim 1, characterized in that, The method for making compliance judgments and confirming risky operations based on the results of scenario identification and risk identification, as well as preset compliance operations, includes: If the output scene of the scene recognition is scene N1, then through M e The model did not recognize a helmet and / or recognized that a helmet was not being worn on a person's head, and through the M i If the model detects that the distance between a person and the file shelf area is less than a set threshold, it is considered a risky operation. If the output scene of the scene recognition is N2 scene, then through the M e The model did not recognize the helmet and / or recognized that the helmet was not being worn on the person's head or via the M f The model did not detect the seatbelt and / or detected that the seatbelt was not fastened in the target position or via the M. g The model detected that the outriggers of the lifting platform were not deployed or via the M h If the model detects that the difference between the human body and the upright angle exceeds a set threshold, or if the Mj model detects that the lifting platform has moved, it is confirmed as a risky operation. If the output scene of the scene recognition is scene N3, then through the M e The model did not recognize the helmet and / or recognized that the helmet was not being worn on the person's head or via the M f The model did not detect the seatbelt and / or detected that the seatbelt was not fastened in the target position or via the M. g The model detected that the outriggers of the lifting platform were not deployed or via the M h If the model detects that the difference between the human body and the upright angle exceeds a set threshold, it is considered a risky operation.

4. A computer storage medium storing computer-executable code; wherein, when executed, the computer-executable code is capable of implementing the method provided by any one of claims 1-3.

5. A CV-based safety monitoring device for archive storage, characterized in that, include: Image acquisition system: used to continuously acquire images and / or video data of personnel entering the archive storage room, and extract images from the image and / or video data as images to be identified according to a set sampling time interval; Image recognition system: uses a pre-trained recognition model to perform scene recognition and risk identification on the image to be recognized; The identification model includes a scene identification model and a risk identification model; the scene identification model includes, but is not limited to, M. a Model - Human and human head recognition model, M b Model - Filing Shelf Area Recognition Model, M c Model - Whether a person is on the lifting platform to identify the model, M d Model - Whether the lifting platform is raised is identified by one or more of the following risk identification models; the risk identification models include, but are not limited to, M e Model - a helmet and a model for recognizing whether a helmet is worn on a person's head, M f Model - Seatbelt and whether it is fastened at the target location recognition model, M g Model - Lifting platform outriggers open / closed recognition model, M h Model - Human pose recognition model; M i Model - Distance recognition model between people and filing shelf area, M j Model - Whether the lifting platform has moved can be identified by one or more of the following models; The scene recognition method includes: calling M a Model, M b Model and M c The model identifies the image to be identified; If through the M a The model identifies the person and the person's head, through the M b The model identified the area in the archive shelf region via the M c If the model detects that a person is not on the lifting platform, the output scenario is N1 - the worker enters the file shelf area but does not use the lifting platform; If through the M a The model identifies the person and the person's head, through the M b The model identified the area in the archive shelf region via the M c If the model detects a person on the lifting platform, it confirms that the scene is an operator entering the file shelf area and being on the lifting platform; Call M d The model further identifies the image to be identified, if it passes through the M d If the model identifies the lifting platform as raised, the output scenario is N2 - workers are on the raised lifting platform; otherwise, the output scenario is N3 - workers are on the unraised lifting platform. Based on the set sampling time interval, M... d The model can identify the height changes of the lifting platform in two consecutive steps. If the height increases, it indicates that the platform is in the process of rising; if the height decreases, it indicates that the platform is in the process of falling; if the height remains unchanged, it indicates that the platform has completed either the rising or falling phase. The risk identification method involves identifying risks based on the results of the scenario identification, and the specific process is as follows: If the output scene of the scene recognition is scene N1, then call M. e Model and M i The model further identifies the image to be identified; If the output scene of the scene recognition is N2 scene, then call M. e Model, M f Model, M g Model, M h Model and M j The model performs further identification on the image to be identified; If the output scene of the scene recognition is scene N3, then call M. e Model, M f Model, M g Model, M h The model performs further identification on the image to be identified; Risk warning system: It is used to make compliance judgments based on the results of scenario identification and risk identification, as well as preset compliance operations, to confirm risky operations, issue alarms, and continue until the operators comply with the regulations.

6. The CV-based safe operation monitoring device for archive storage as described in claim 5, characterized in that, The image acquisition system includes an image acquisition device, an image storage module, and an image extraction module. The image acquisition device includes multiple camera devices to ensure that there are no blind spots in the monitoring range.

7. A CV-based safety monitoring device for archive storage as described in claim 5, characterized in that, The image recognition system includes a recognition model training module, a recognition model storage module, a scene recognition module, and a risk recognition module; the risk warning system includes a compliance judgment module and an alarm device.

Images