Smart factory safety supervision method under multi-linkage safety management and control mechanism

Abstract

The invention discloses a smart factory safety supervision method under a multi-linkage safety management and control mechanism, and relates to the technical field of smart factory safety supervision, and the method comprises the following steps: arranging a camera in a smart factory production field area, and collecting the monitoring video data of a supervision area through the camera; segmenting the collected monitoring video data into a plurality of monitoring video segments based on the equipment production operation cycle time; collecting an audio signal of each monitoring video segment after segmentation processing, performing differential analysis on the plurality of intercepted monitoring video segments based on the audio signals, and obtaining a key monitoring video segment and feature time information; correcting the selected key monitoring video segment based on the feature time information to extract a key frame image; according to the method, frame extraction is carried out on the key feature monitoring video to acquire the key frame image, acquired information redundancy is remarkably reduced, repetition of a plurality of continuous frames is avoided, the content diversity of the extracted key frame image is enhanced, and the safety supervision accuracy and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart factory safety supervision, and specifically to a smart factory safety supervision method under a multi-linkage safety control mechanism. Background Art

[0002] Smart factories are based on intelligent technology, digital technology, and information technology. Through the integration of physical infrastructure and information infrastructure, they integrate the personnel, machines, equipment, and infrastructure within the factory, and implement real-time management, coordination, and control among multiple systems. On this basis, production is managed in a more sophisticated and dynamic way to achieve a "smart" state, thereby improving the management efficiency and production efficiency of the factory. Factory equipment safety supervision is a key link to ensure factory production safety and improve production efficiency. The main purpose of smart factory safety supervision is to monitor the operating status of equipment in real time, promptly discover and deal with potential safety hazards, and prevent production interruptions and safety accidents caused by equipment failures. Through monitoring, equipment maintenance plans can also be optimized, equipment service life can be extended, and maintenance costs can be reduced.

[0003] For example, the application publication number is CN116758484A, the application publication date is 2023.09.15, and the name is "Safety Supervision System and Method for Chemical Plants". It uses a convolutional neural network model to extract the production operation dynamic features of the production operation monitoring video of the production workshop workers, and combines the classifier to classify the production operation dynamics to achieve monitoring and intelligent analysis of the workers' production operations. In this way, it can help monitors to detect irregular production operations in a timely manner, thereby reducing the occurrence of production safety accidents.

[0004] The shortcoming of the prior art, including the above-mentioned application, is that in the prior art, many continuous frames of the production operation monitoring video collected by the monitoring equipment for safety monitoring of the factory generation operation through smart factory safety supervision are repeated. When extracting images from the production operation monitoring video for analysis, the video is framed at a preset time interval to obtain several framed images of the video, and then feature recognition analysis is performed based on the collected framed images. This will cause information redundancy and significantly increase the amount of computational complexity for subsequent safety analysis and evaluation of the collected information, while affecting the accuracy of smart factory safety supervision. Summary of the invention

[0005] The purpose of the present invention is to provide a smart factory safety supervision method under a multi-linkage safety control mechanism to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a smart factory safety supervision method under a multi-linkage safety control mechanism, comprising the following steps:

[0007] Deploy cameras in the production area of ​​the smart factory to collect surveillance video data of the supervision area;

[0008] The collected monitoring video data is divided into multiple monitoring video segments based on the equipment production operation cycle time;

[0009] Collect the audio signal of each surveillance video segment after segmentation processing, perform differential analysis on the intercepted multiple surveillance video segments based on the audio signal, and obtain key surveillance video segments and characteristic time information;

[0010] The selected key surveillance video segments are corrected based on characteristic time information to extract key frame images, and the extracted key frame images are integrated to obtain a key surveillance video segment analysis image set;

[0011] Analyze each image in the image set of key monitoring video segments for abnormal identification to determine whether there are images with abnormal movements. If there are abnormal movements, an alarm will be issued.

[0012] As a further description of the above technical solution: the equipment production operation cycle time is the time the equipment uses to produce and process a product. Each monitoring video segment includes monitoring video of all operations of the related equipment in processing a product, ensuring the integrity of key information in the cropped and segmented monitoring video segments.

[0013] As a further description of the above technical solution: performing differential analysis on multiple intercepted surveillance video segments based on audio information to obtain key surveillance video segments and characteristic time information specifically includes the following steps:

[0014] Perform audio feature extraction on the audio signal collected from each surveillance video segment to obtain a noise time series diagram corresponding to each surveillance video segment;

[0015] The noise time series diagrams of each surveillance video segment are integrated into the same noise processing coordinate system, where the horizontal axis of the coordinate system represents time and the vertical axis represents noise intensity;

[0016] The characteristic time mark line perpendicular to the X-axis is drawn at the preset interval time tj, and the intersection data information of the vertex and bottom points of the intersection of each characteristic time mark line and the noise timing diagram of the noise processing coordinate system is collected.

[0017] The intersection data information includes:

[0018] T m (Z m1 ,Z m2 ) represents Z m1 Noise Timing Diagram and Z m2 Noise timing diagram storage and T m The intersection points of the characteristic moment marking lines are the apex and the bottom;

[0019] Based on the integration of intersection data information, key monitoring video segments and characteristic time information are obtained.

[0020] As a further description of the above technical solution: the key monitoring video segments and characteristic time information are obtained based on the integration of intersection data information as follows:

[0021] Based on the fact that the intersection data information includes each noise timing diagram, the monitoring video segment corresponding to the noise timing diagram is marked as a key monitoring video segment, and each characteristic moment marking line data corresponding to the noise timing diagram is marked as the characteristic time information of the corresponding key monitoring video segment.

[0022] As a further description of the above technical solution: the selected key surveillance video segment is corrected based on the characteristic time information to extract the key frame image, which specifically includes the following steps:

[0023] Collect each characteristic time information of the key monitoring video segment, identify the time sequence of the characteristic time information and perform a correction process, and obtain the characteristic time information set corresponding to the key monitoring video segment;

[0024] Perform secondary correction processing on the acquired characteristic time information set to obtain the frame extraction time information set corresponding to the key monitoring video segment;

[0025] Based on the frame extraction time information set corresponding to the key monitoring video segment, the key monitoring video segment is subjected to frame extraction processing to obtain a feature image set.

[0026] As a further description of the above technical solution: collecting each characteristic time information of the key monitoring video segment, identifying the time sequence of the characteristic time information and performing a correction process, and obtaining the characteristic time information set corresponding to the key monitoring video segment specifically includes the following steps:

[0027] Traverse and analyze each feature time information in the image set, and calculate the corrected frame extraction time value when there are two adjacent feature time information in the same key monitoring video segment. The calculation method of the corrected frame extraction time value is: T i , T i+1 Represents two adjacent feature time information in the key monitoring video segment, T X Indicates T i , T i+1 Corrected frame extraction time information of two adjacent characteristic time information;

[0028] Integrate and obtain the characteristic time information set corresponding to the key monitoring video segments.

[0029] As a further description of the above technical solution: the secondary correction processing of the acquired characteristic time information set to obtain the frame extraction time information set corresponding to the key monitoring video segment specifically includes the following steps:

[0030] Retrieve a characteristic time information set obtained after a correction process;

[0031] Based on each feature time information in the feature time information set, a feature moment marking line is marked on the noise time series diagram of the corresponding key monitoring video segment;

[0032] Taking the characteristic moment marking line as the reference line, a rectangular correction window is obtained by extending the length F to both sides, where F < tj, tj represents the preset interval time of the characteristic moment marking line;

[0033] When the intersection of the characteristic moment marking line and the noise timing diagram is the vertex, the maximum noise value of the noise timing diagram in the rectangular correction window is calculated, and the characteristic time information corresponding to the maximum noise value is obtained to replace the current characteristic time information and mark it as the frame extraction time information;

[0034] Each feature time information in the feature time information set is processed in turn, and the frame time information is obtained and integrated to obtain the frame time information set corresponding to the key monitoring video segment.

[0035] As a further description of the above technical solution: based on the frame extraction time information set corresponding to the key monitoring video segment, the key monitoring video segment is subjected to frame extraction processing to obtain the feature image set, specifically:

[0036] Retrieving each frame extraction time information in the frame extraction time information set, extracting frames for the key monitoring video segment according to the frame extraction time information, and marking the obtained frame extraction image as a key frame image;

[0037] The key frame images obtained by extracting frames from the key monitoring video segments are integrated to obtain a set of feature images of the key monitoring video segments.

[0038] As a further description of the above technical solution: the key monitoring video segment analysis image set is used to identify abnormalities in each image, and it is determined whether there are images with abnormal movements. If there are abnormal movements, an alarm is issued. Specifically:

[0039] The key surveillance video segments are extracted and analyzed, and each image in the image set is input into the trained motion anomaly recognition model. The anomaly recognition model is used to determine whether there are images with abnormal motion. If so, an alarm is issued.

[0040] As a further description of the above technical solution: the training method of the abnormality recognition model is specifically:

[0041] Build an abnormal motion recognition model based on deep learning algorithm;

[0042] Collect historical data sets, which include N training data sets, where N is a positive integer. The training data sets include feature data and label data. The feature data is a set of images analyzed from surveillance video segments. The label data includes 1 and 0, where 1 indicates abnormal dynamics and 0 indicates no abnormal movements.

[0043] The feature data in each set of training data is used as the input of the abnormal action recognition model. The abnormal action recognition model predicts whether an abnormal action occurs for each set of feature data as the output. The label data corresponding to each set of feature data is used as the prediction target. The action abnormality recognition model is trained with the minimized sum of prediction accuracies as the training target until the sum of prediction accuracies converges and the training is stopped.

[0044] In the above technical scheme, the present invention provides a smart factory safety supervision method under a multi-linkage safety control mechanism, which extracts audio signals from each monitoring video segment, obtains a noise timing diagram, inserts a characteristic moment marking line based on a preset interval time, collects the intersection data information of the vertex and bottom points of the intersection of the characteristic moment marking line and the noise timing diagram of the noise processing coordinate system, and obtains key monitoring video segments and characteristic time information based on the intersection data information, thereby realizing frame extraction and acquisition of key frame images of key feature monitoring videos. Compared with the prior art of directly extracting frames from videos according to preset intervals to obtain frame extraction images for analysis, the collected information redundancy can be significantly reduced, multiple consecutive frames can be avoided from being repeated, and the content diversity of the extracted key frame images can be enhanced, thereby improving the accuracy and security of smart factory safety supervision;

[0045] Secondly, by identifying the timing of feature time information, the two adjacent feature time information located in the same key monitoring video segment can be integrated to extract the frame time information, so that when the key frame images are subsequently collected based on the frame time information, the key frame images corresponding to the abnormal features of the same equipment can be reduced, further reducing information redundancy, reducing the amount of analysis and calculation, and improving the accuracy of safety supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0047] Figure 1 A flowchart of a smart factory safety supervision method under a multi-linkage safety control mechanism is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0049] See also Figure 1 The embodiment of the present invention provides a technical solution: a smart factory safety supervision method under a multi-linkage safety control mechanism, comprising the following steps:

[0050] Deploy cameras in the production area of ​​the smart factory to collect surveillance video data of the supervision area;

[0051] The collected surveillance video data is divided into multiple surveillance video segments based on the equipment production and operation cycle time; the equipment production and operation cycle time is the time used by the equipment to produce and process a product, so that each surveillance video segment obtained by segmenting the collected surveillance video data based on the equipment production and operation cycle time includes the surveillance video of all operations of the relevant equipment processing a product, ensuring the integrity of the key information of the cropped and segmented surveillance video segments. At the same time, each surveillance video segment corresponds to each other in the time series, so as to facilitate the subsequent analysis and processing of each collected surveillance video segment.

[0052] Collect the audio signal of each surveillance video segment after segmentation processing, perform differential analysis on the intercepted multiple surveillance video segments based on the audio signal, and obtain key surveillance video segments and characteristic time information;

[0053] The selected key surveillance video segments are corrected based on characteristic time information to extract key frame images, and the extracted key frame images are integrated to obtain a key surveillance video segment analysis image set;

[0054] Perform abnormal identification on each image in the key surveillance video segment analysis image set to determine whether there are images with abnormal movements. If there are abnormal movements, issue an alarm. Input each image extracted from the key surveillance video segment analysis image set into the trained abnormal movement identification model to determine whether there are images with abnormal movements. If there are images with abnormal movements, issue an alarm.

[0055] In another embodiment provided by the present invention, performing differential analysis on multiple intercepted surveillance video segments based on audio information to obtain key surveillance video segments and characteristic time information specifically includes the following steps:

[0056] The audio features of the audio signals collected from each surveillance video segment are extracted to obtain a noise time series diagram corresponding to each surveillance video segment; the noise time series diagram is a line graph that represents the change of noise intensity in a time sequence;

[0057] The noise time series diagrams of each surveillance video segment are integrated into the same noise processing coordinate system, where the horizontal axis of the coordinate system represents time and the vertical axis represents noise intensity;

[0058] The characteristic time mark line perpendicular to the X-axis is drawn at the preset interval time tj, and the intersection data information of the vertex and bottom points of the intersection of each characteristic time mark line and the noise timing diagram of the noise processing coordinate system is collected.

[0059] The intersection data information includes:

[0060] T m (Z m1 ,Z m2 ) represents Z m1 Noise Timing Diagram and Z m2 Noise timing diagram storage and T m The intersection points of the characteristic moment marking lines are the vertices and the bottom points; among them, the vertices and the bottom points of the intersection points of the characteristic moment marking lines at different times have corresponding same noise time series diagrams, and secondly, the vertex of the intersection point of the characteristic moment marking line and the noise time series diagram of the noise processing coordinate system indicates that the noise intensity of the monitoring video segment corresponding to the vertex of this time node in each monitoring video segment is the largest, and the bottom point of the intersection point of the characteristic moment marking line and the noise time series diagram of the noise processing coordinate system indicates that the noise intensity of the monitoring video segment corresponding to the vertex of this time node in each monitoring video segment is the smallest.

[0061] Based on the integration of intersection data information, key monitoring video segments and characteristic time information are obtained. That is, the present application extracts the audio signal of each monitoring video segment, obtains the noise timing diagram, inserts the characteristic moment marking line based on the preset interval time, collects the intersection data information of the vertex and bottom point of the intersection of the characteristic moment marking line and the noise timing diagram of the noise processing coordinate system, and obtains the key monitoring video segment and characteristic time information based on the intersection data information, thereby realizing the frame extraction and acquisition of key frame images of key characteristic monitoring videos. Compared with the prior art of directly extracting frames of videos according to the preset interval time to obtain the extracted frame images for analysis, it can significantly reduce the redundancy of the collected information, avoid the occurrence of multiple consecutive frames, enhance the content diversity of the extracted key frame images, and thus improve the accuracy and security of smart factory safety supervision.

[0062] In another embodiment provided by the present invention, the key monitoring video segment and characteristic time information are obtained based on the integration of intersection data information as follows:

[0063] Based on the fact that the intersection data information includes each noise timing diagram, the monitoring video segment corresponding to the noise timing diagram is marked as a key monitoring video segment, and each characteristic moment marking line data corresponding to the noise timing diagram is marked as the characteristic time information of the corresponding key monitoring video segment.

[0064] In another embodiment provided by the present invention, the selected key surveillance video segment is corrected based on the characteristic time information to extract the key frame image, which specifically includes the following steps:

[0065] Collect each characteristic time information of the key monitoring video segment, identify the time sequence of the characteristic time information and perform a correction process, and obtain the characteristic time information set corresponding to the key monitoring video segment;

[0066] Perform secondary correction processing on the acquired characteristic time information set to obtain the frame extraction time information set corresponding to the key monitoring video segment;

[0067] Based on the frame extraction time information set corresponding to the key monitoring video segment, the key monitoring video segment is subjected to frame extraction processing to obtain a feature image set.

[0068] In another embodiment provided by the present invention, collecting each characteristic time information of the key monitoring video segment, identifying the time sequence of the characteristic time information and performing a correction process, and obtaining the characteristic time information set corresponding to the key monitoring video segment specifically includes the following steps:

[0069] Traverse and analyze each feature time information in the image set, and calculate the corrected frame extraction time value when there are two adjacent feature time information in the same key monitoring video segment. The calculation method of the corrected frame extraction time value is: T i , T i+1 Represents two adjacent feature time information in the key monitoring video segment, T X Indicates T i , T i+1 Corrected frame extraction time information of two adjacent characteristic time information;

[0070] The characteristic time information set corresponding to the key monitoring video segment is obtained by integration. It should be noted that when two adjacent frames are in the same key monitoring video segment, it is obvious that the device abnormality features corresponding to the two adjacent frames are consistent. This application recognizes the timing of the characteristic time information, integrates the two adjacent characteristic time information located in the same key monitoring video segment to extract the frame extraction time information, and reduces the key frame images corresponding to the abnormal features of the same device when the key frame images are subsequently collected based on the frame extraction time information, further reducing information redundancy, reducing the amount of analysis and calculation, and improving the accuracy of safety supervision.

[0071] In another embodiment provided by the present invention, performing secondary correction processing on the acquired characteristic time information set to obtain the frame extraction time information set corresponding to the key monitoring video segment specifically includes the following steps:

[0072] Retrieve a characteristic time information set obtained after a correction process;

[0073] Based on each feature time information in the feature time information set, a feature moment marking line is marked on the noise time series diagram of the corresponding key monitoring video segment;

[0074] Taking the characteristic moment marking line as the reference line, a rectangular correction window is obtained by extending the length F to both sides, where F < tj, tj represents the preset interval time of the characteristic moment marking line;

[0075] When the intersection of the characteristic moment marking line and the noise timing diagram is the vertex, the maximum noise value of the noise timing diagram in the rectangular correction window is calculated, and the characteristic time information corresponding to the maximum noise value is obtained to replace the current characteristic time information and marked as the frame extraction time information; it should be noted that when the intersection of the characteristic moment marking line and the noise timing diagram is the bottom point, the minimum noise value of the noise timing diagram in the rectangular correction window is calculated at this time, and the characteristic time information corresponding to the minimum noise value is obtained to replace the current characteristic time information and marked as the frame extraction time information;

[0076] Each feature time information in the feature time information set is processed in turn, and the frame time information is obtained and integrated to obtain the frame time information set corresponding to the key monitoring video segment.

[0077] By inserting the characteristic moment marking line, collecting the intersection data information of the vertex and bottom points of the intersection of the characteristic moment marking line and the noise timing diagram of the noise processing coordinate system, and obtaining the key monitoring video segment and the characteristic time information based on the intersection data information, at this time, directly extracting frames through the obtained characteristic time information to collect images will cause some important characteristic information to be missed due to the preset interval time. The present application sets a rectangular correction window to correct the characteristic time information determined by each characteristic moment marking line to obtain the frame extraction time information, so that the extracted key frame image features are more prominent, further improving the accuracy of subsequent judgment whether there is abnormal motion.

[0078] In another embodiment provided by the present invention, the key monitoring video segment is subjected to frame extraction processing based on the frame extraction time information set corresponding to the key monitoring video segment to obtain the feature image set specifically as follows:

[0079] Retrieving each frame extraction time information in the frame extraction time information set, extracting frames for the key monitoring video segment according to the frame extraction time information, and marking the obtained frame extraction image as a key frame image;

[0080] The key frame images obtained by extracting frames from the key monitoring video segments are integrated to obtain a set of feature images of the key monitoring video segments.

[0081] The training method of the anomaly recognition model is specifically as follows:

[0082] Build an abnormal motion recognition model based on deep learning algorithm;

[0083] Collect historical data sets, which include N training data sets, where N is a positive integer. The training data sets include feature data and label data. Feature data is a set of images analyzed from surveillance video segments. Label data includes 1 and 0, where 1 indicates abnormal dynamics and 0 indicates no abnormal actions. The feature data in each set of training data is used as the input of the abnormal motion recognition model. The abnormal motion recognition model predicts whether abnormal actions occur for each set of feature data as output. The label data corresponding to each set of feature data is used as the prediction target. The minimized sum of prediction accuracy is used as the training target to train the abnormal motion recognition model until the sum of prediction accuracy converges. The training is stopped when the abnormal motion recognition model is trained. The training of the abnormal motion recognition model is a prior art.

[0084] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0085] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A smart factory safety supervision method under a multi-link safety control mechanism, characterized in that: The following steps are involved: Deploy cameras in the production area of ​​the smart factory to collect surveillance video data of the supervision area; The collected monitoring video data is divided into multiple monitoring video segments based on the equipment production operation cycle time; Collect the audio signal of each surveillance video segment after segmentation processing, perform differential analysis on the intercepted multiple surveillance video segments based on the audio signal, and obtain key surveillance video segments and characteristic time information; The selected key surveillance video segments are corrected based on characteristic time information to extract key frame images, and the extracted key frame images are integrated to obtain a key surveillance video segment analysis image set; Analyze each image in the image set of key monitoring video segments for abnormal identification to determine whether there are images with abnormal movements. If there are abnormal movements, an alarm will be issued.

2. According to the method of claim 1, the method is characterized in that: The equipment production operation cycle time is the time the equipment uses to produce and process one product. Each monitoring video segment includes monitoring video of all operations of the relevant equipment in processing one product, ensuring the integrity of key information in the cropped and segmented monitoring video segments.

3. According to the method of claim 1, the method is characterized in that: Based on the audio information, the multiple intercepted surveillance video segments are analyzed differentially to obtain the key surveillance video segments and characteristic time information, which specifically includes the following steps: Perform audio feature extraction on the audio signal collected from each surveillance video segment to obtain a noise time series diagram corresponding to each surveillance video segment; The noise time series diagrams of each surveillance video segment are integrated into the same noise processing coordinate system, where the horizontal axis of the coordinate system represents time and the vertical axis represents noise intensity; The characteristic time mark line perpendicular to the X-axis is drawn at the preset interval time tj, and the intersection data information of the vertex and bottom points of the intersection of each characteristic time mark line and the noise timing diagram of the noise processing coordinate system is collected. The intersection data information includes: T m (Z m1 ,Z m2 ) represents Z m1 Noise Timing Diagram and Z m2 Noise timing diagram storage and T m The intersection points of the characteristic moment marking lines are the apex and the bottom; Based on the integration of intersection data information, key monitoring video segments and characteristic time information are obtained.

4. According to the method of claim 3, the smart factory safety supervision method under the multi-linkage safety control mechanism is characterized in that: Based on the integration of intersection data information, key surveillance video segments and characteristic time information are obtained as follows: Based on the fact that the intersection data information includes each noise timing diagram, the monitoring video segment corresponding to the noise timing diagram is marked as a key monitoring video segment, and each characteristic moment marking line data corresponding to the noise timing diagram is marked as the characteristic time information of the corresponding key monitoring video segment.

5. According to the method of claim 4, the smart factory safety supervision method under the multi-link safety control mechanism is characterized in that: Correcting the selected key surveillance video segment based on the characteristic time information to extract the key frame image specifically includes the following steps: Collect each characteristic time information of the key monitoring video segment, identify the time sequence of the characteristic time information and perform a correction process, and obtain the characteristic time information set corresponding to the key monitoring video segment; Perform secondary correction processing on the acquired characteristic time information set to obtain the frame extraction time information set corresponding to the key monitoring video segment; Based on the frame extraction time information set corresponding to the key monitoring video segment, the key monitoring video segment is subjected to frame extraction processing to obtain a feature image set.

6. The method for safety supervision of a smart factory under a multi-linkage safety control mechanism according to claim 5 is characterized in that: Collecting each characteristic time information of the key monitoring video segment, identifying the time sequence of the characteristic time information and performing a correction process, and obtaining the characteristic time information set corresponding to the key monitoring video segment specifically includes the following steps: Traverse and analyze each feature time information in the image set, and calculate the corrected frame extraction time value when there are two adjacent feature time information in the same key monitoring video segment. The calculation method of the corrected frame extraction time value is: T i , T i+1 Represents two adjacent feature time information in the key monitoring video segment, T X Indicates T i , T i+1 Corrected frame extraction time information of two adjacent characteristic time information; Integrate and obtain the characteristic time information set corresponding to the key monitoring video segments.

7. The method for safety supervision of a smart factory under a multi-linkage safety control mechanism according to claim 6 is characterized in that: The second correction process of the acquired characteristic time information set to obtain the frame extraction time information set corresponding to the key monitoring video segment specifically includes the following steps: Retrieve a characteristic time information set obtained after a correction process; Based on each feature time information in the feature time information set, a feature moment marking line is marked on the noise time series diagram of the corresponding key monitoring video segment; Taking the characteristic moment marking line as the reference line, a rectangular correction window is obtained by extending the length F to both sides, where F < tj, tj represents the preset interval time of the characteristic moment marking line; When the intersection of the characteristic moment marking line and the noise timing diagram is the vertex, the maximum noise value of the noise timing diagram in the rectangular correction window is calculated, and the characteristic time information corresponding to the maximum noise value is obtained to replace the current characteristic time information and mark it as the frame extraction time information; Each feature time information in the feature time information set is processed in turn, and the frame time information is obtained and integrated to obtain the frame time information set corresponding to the key monitoring video segment.

8. The method for safety supervision of a smart factory under a multi-linkage safety control mechanism according to claim 7 is characterized in that: Based on the frame extraction time information set corresponding to the key monitoring video segment, the key monitoring video segment is subjected to frame extraction processing to obtain the feature image set, specifically: Retrieving each frame extraction time information in the frame extraction time information set, extracting frames for the key monitoring video segment according to the frame extraction time information, and marking the obtained frame extraction image as a key frame image; The key frame images obtained by extracting frames from the key monitoring video segments are integrated to obtain a set of feature images of the key monitoring video segments.

9. The method for safety supervision of a smart factory under a multi-linkage safety control mechanism according to claim 1 is characterized in that: Analyze the key surveillance video segments and identify the abnormalities of each image in the image set to determine whether there are images with abnormal movements. If there are abnormal movements, an alarm will be issued. Specifically: The key surveillance video segments are extracted and analyzed, and each image in the image set is input into the trained motion anomaly recognition model. The anomaly recognition model is used to determine whether there are images with abnormal motion. If so, an alarm is issued.

10. The method for safety supervision of a smart factory under a multi-linkage safety control mechanism according to claim 1 is characterized in that: The training method of the anomaly recognition model is specifically as follows: Build an abnormal motion recognition model based on deep learning algorithm; Collect historical data sets, which include N training data sets, where N is a positive integer. The training data sets include feature data and label data. The feature data is a set of images analyzed from surveillance video segments. The label data includes 1 and 0, where 1 indicates abnormal dynamics and 0 indicates no abnormal movements. The feature data in each set of training data is used as the input of the abnormal action recognition model. The abnormal action recognition model predicts whether an abnormal action occurs for each set of feature data as the output. The label data corresponding to each set of feature data is used as the prediction target. The action abnormality recognition model is trained with the minimized sum of prediction accuracies as the training target until the sum of prediction accuracies converges and the training is stopped.

Images