Methods and systems for processing factory equipment operation data applied to safety production management

Abstract

This invention provides a method and system for processing factory equipment operation data for safe production management, relating to the field of artificial intelligence technology. In this invention, multiple typical equipment operation information are extracted, including typical equipment operation monitoring videos and typical optimized operation monitoring videos. Based on this typical equipment operation information, an initial video optimization network is updated and adjusted. Through the updated and adjusted initial video optimization network, video optimization operations are performed on the target equipment operation monitoring video to form a target optimized operation monitoring video corresponding to the target equipment operation monitoring video. Through the equipment operation status analysis network formed by the update and adjustment operations, operation status analysis operations are performed on the target optimized operation monitoring video to output operation status analysis data corresponding to the target factory equipment. Based on the above, the reliability of operation data analysis can be improved to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for processing factory equipment operation data for safe production management. Background Technology

[0002] In factory safety management, the initial step typically involves collecting operational data from factory equipment, such as through video surveillance. This data is then analyzed to determine the operational status, for example, using artificial intelligence (AI) technology. AI is a comprehensive discipline encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, machine learning / deep learning, autonomous driving, and intelligent transportation. Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of AI and the fundamental way to endow computers with intelligence; its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0003] However, in the existing technology, the analysis of surveillance videos based on artificial intelligence technology suffers from low reliability. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method and system for processing factory equipment operation data for safe production management, so as to improve the reliability of operation data analysis to a certain extent.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] A method for processing factory equipment operation data for safety production management, the method comprising:

[0007] Extract the operation information of multiple typical devices, including typical device operation monitoring videos and typical optimized operation monitoring videos corresponding to the typical device operation monitoring videos;

[0008] Based on the operating information of the aforementioned typical devices, the initial video optimization network is updated and adjusted;

[0009] By updating and adjusting the initial video optimization network, video optimization operations are performed on the target device operation monitoring video to be processed, so as to form a target optimized operation monitoring video corresponding to the target device operation monitoring video. The number of video frames included in the target optimized operation monitoring video is less than or equal to the number of video frames included in the target device operation monitoring video. The target device operation monitoring video is formed based on video monitoring of the target factory equipment.

[0010] The equipment operation status analysis network, formed through update and adjustment operations, performs operation status analysis on the target optimized operation monitoring video to output operation status analysis data corresponding to the target factory equipment. The operation status analysis data is used to reflect the operational safety level of the target factory equipment and serves as the basis for safety production management.

[0011] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safety production management, the step of updating and adjusting the initial video optimization network based on the multiple typical equipment operation information includes:

[0012] Based on the undetermined video frame cluster and the typical optimized operation monitoring video, the typical video representative data corresponding to the typical optimized operation monitoring video is analyzed. The undetermined video frame cluster includes multiple undetermined video frames, and the typical video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the typical optimized operation monitoring video.

[0013] The initial video optimization network is used to analyze and mine the typical equipment operation monitoring video to output corresponding mining video representative data. The mining video representative data is used to reflect the probability parameter that each of the pending video frames belongs to the mining optimized operation monitoring video corresponding to the typical equipment operation monitoring video.

[0014] Based on the typical video representative data and the mined video representative data, the initial video optimization network is updated and adjusted.

[0015] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safety production management, the initial video optimization network includes a key information mining model, a key information integration model, and a key information restoration model. The step of analyzing and mining the typical equipment operation monitoring video through the initial video optimization network to output corresponding representative data of the mined video includes:

[0016] The key information mining model is used to perform key information mining on the typical equipment operation monitoring video to output the video key information description vector corresponding to each monitoring video segment in the typical equipment operation monitoring video.

[0017] The key information integration model is used to integrate multiple video key information description vectors to output the corresponding integrated key information description vector.

[0018] The key information restoration model is used to perform key information restoration operations based on the integrated key information description vector, so as to output the corresponding representative data of the mining video.

[0019] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safe production management, the mining video representative data includes local mining video representative data corresponding to multiple frame coordinates in the mining optimization operation monitoring video. The local mining video representative data includes mining video representative parameters corresponding to each of the undetermined video frames. The mining video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the mining optimization operation monitoring video.

[0020] The step of performing key information restoration operation based on the integrated key information description vector using the key information restoration model to output corresponding representative data from the mined video includes:

[0021] The key information restoration model is used to restore the integrated key information description vector to output the corresponding local mining video representative data. The undetermined video frame corresponding to the mining video representative parameter with the maximum value in the local mining video representative data is marked as the first optimized video frame.

[0022] Using the key information restoration model, based on the marked optimized video frames, the integrated key information description vector is restored to output the corresponding subsequent local mining video representative data. Additionally, the undetermined video frames corresponding to the mining video representative parameters with the maximum value in the subsequent local mining video representative data are marked as the second optimized video frames. This process is completed when the marked optimized video frames belong to the end video frames. The end video frames are blank video frames used to reflect the end of the mining optimization operation monitoring video.

[0023] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safety production management, the typical video representative data includes local typical video representative data corresponding to multiple frame coordinates in the typical optimized operation monitoring video. The local typical video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the video frame at the corresponding frame coordinate in the typical optimized operation monitoring video. The mining video representative data includes local mining video representative data corresponding to multiple frame coordinates in the mining optimized operation monitoring video. The local mining video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the video frame at the corresponding frame coordinate in the mining optimized operation monitoring video.

[0024] The step of updating and adjusting the initial video optimization network based on the typical video representative data and the mined video representative data includes:

[0025] Based on the difference information between the local mining video representative data and the local typical video representative data corresponding to each frame coordinate, the local error index corresponding to each frame coordinate is analyzed.

[0026] Based on the analyzed local error indices, a corresponding target error index is calculated, and the target error index has a positive correlation with the multiple local error indices.

[0027] Based on the target error index, the initial video optimization network is updated and adjusted.

[0028] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safety production management, the step of performing video optimization operations on the target equipment operation monitoring video to be processed by updating and adjusting the initial video optimization network to form a target optimized operation monitoring video corresponding to the target equipment operation monitoring video includes:

[0029] The initial video optimization network analyzes and mines the target device operation monitoring video to output corresponding video representative data. Based on the video representative data, multiple confirmed video frames belonging to the target optimized operation monitoring video are analyzed from multiple undetermined video frames included in the undetermined video frame cluster. The video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the target optimized operation monitoring video corresponding to the target device operation monitoring video.

[0030] Each confirmed video frame is combined to form a target optimized operation monitoring video corresponding to the target device operation monitoring video.

[0031] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safe production management, the initial video optimization network includes a key information mining model, a key information integration model, and a key information restoration model. The step of analyzing and mining the target equipment operation monitoring video through the initial video optimization network to output corresponding video representative data, and then, based on the video representative data, analyzing multiple confirmed video frames belonging to the target optimized operation monitoring video from multiple undetermined video frames included in the undetermined video frame cluster, includes:

[0032] The key information mining model is used to perform key information mining on the target device operation monitoring video to output the video key information representation vector corresponding to each monitoring video segment in the target device operation monitoring video.

[0033] The key information integration model is used to integrate multiple video key information representation vectors to output the corresponding integrated key information representation vector.

[0034] The key information restoration model is used to perform key information restoration operations based on the integrated key information representation vector to output corresponding video representative data. Based on the video representative data, multiple confirmed video frames belonging to the target optimized operation monitoring video are analyzed from multiple pending video frames.

[0035] The target device operation monitoring video includes partially ending video frames, which reflect the end of the video segment. The step of performing key information mining on the target device operation monitoring video using the key information mining model to output a video key information representation vector corresponding to each monitoring video segment in the target device operation monitoring video includes:

[0036] Using the key information mining model, the system analyzes the locally ending video frames in the monitoring video of the target device. Video frames before the first locally ending video frame are marked as a monitoring video segment, and video frames between every two locally ending video frames are also marked as a monitoring video segment. Furthermore, using the key information mining model, each monitoring video segment undergoes key information mining to output a video key information representation vector corresponding to each monitoring video segment.

[0037] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safe production management, the video representative data includes local video representative data corresponding to multiple frame coordinates in the target optimized operation monitoring video. The local video representative data includes video representative parameters corresponding to each undetermined video frame. The video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the target optimized operation monitoring video.

[0038] The steps of performing key information restoration operations based on the integrated key information representation vector using the key information restoration model to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimized operation monitoring video from multiple undetermined video frames based on the video representative data, include:

[0039] The key information restoration model is used to restore the integrated key information representation vector to output the corresponding preceding local video representative data. The pending video frame corresponding to the video representative parameter with the maximum value in the preceding local video representative data is marked as the first confirmed video frame.

[0040] Using the key information restoration model, based on the marked confirmed video frames, the integrated key information representation vector is restored to output the corresponding subsequent local video representative data. The pending video frames corresponding to the video representative parameters with the maximum value in the subsequent local video representative data are marked as the second confirmed video frames. This process is completed when the marked confirmed video frame is the end video frame. The end video frame is a blank video frame used to reflect the end of the target optimized operation monitoring video.

[0041] In some preferred embodiments, in the above-described factory equipment operation data processing method applied to safe production management, the video representative data includes local video representative data corresponding to multiple frame coordinates in the target optimized operation monitoring video. The local video representative data includes video representative parameters corresponding to each undetermined video frame. The video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the target optimized operation monitoring video.

[0042] The steps of performing key information restoration operations based on the integrated key information representation vector using the key information restoration model to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimized operation monitoring video from multiple undetermined video frames based on the video representative data, include:

[0043] Using the key information restoration model, based on different video frames, the integrated key information representation vector is restored to output multiple video representative data. For each video representative data, the video frame corresponding to the video representative parameter with the maximum value in each local video representative data is marked as a confirmed video frame. The multiple marked confirmed video frames are combined to form a confirmed video frame cluster corresponding to the video representative data.

[0044] The merged video representative parameters corresponding to each confirmed video frame cluster are analyzed, and there is a positive correlation between the merged video representative parameters and the video representative parameters corresponding to multiple confirmed video frames in the confirmed video frame cluster.

[0045] From the multiple confirmed video frame clusters, identify multiple confirmed video frames from the confirmed video frame clusters that have the maximum value of the merged video representative parameter.

[0046] This invention also provides a factory equipment operation data processing system for safe production management, including a processor and a memory. The memory is used to store computer programs, and the processor is used to execute the computer programs to implement the above-described method.

[0047] The factory equipment operation data processing method and system for safe production management provided in this invention can first extract multiple typical equipment operation information, including typical equipment operation monitoring videos and typical optimized operation monitoring videos. Based on the multiple typical equipment operation information, an initial video optimization network is updated and adjusted. Through the updated and adjusted initial video optimization network, video optimization operations are performed on the target equipment operation monitoring video to form a target optimized operation monitoring video corresponding to the target equipment operation monitoring video. Through the equipment operation status analysis network formed by the update and adjustment operation, operation status analysis operations are performed on the target optimized operation monitoring video to output operation status analysis data corresponding to the target factory equipment. Based on the foregoing, since video optimization operations are performed on the target equipment operation monitoring video based on the updated and adjusted initial video optimization network before performing operation status analysis, the resulting target optimized operation monitoring video is more concise to a certain extent. That is, the number of video frames in the target optimized operation monitoring video is less than or equal to the number of video frames in the target equipment operation monitoring video. This makes the basis for operation status analysis more concise, thus improving the reliability of operation data analysis to a certain extent and addressing the problem of low reliability in existing technologies.

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0049] Figure 1 This is a structural block diagram of a factory equipment operation data processing system for safe production management, provided in an embodiment of the present invention.

[0050] Figure 2 This is a flowchart illustrating the steps of a factory equipment operation data processing method for safe production management provided in an embodiment of the present invention.

[0051] Figure 3 This is a schematic diagram of the modules included in the factory equipment operation data processing device for safe production management provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0054] like Figure 1 As shown, this embodiment of the invention provides a factory equipment operation data processing system for safety production management. The factory equipment operation data processing system for safety production management may include a memory and a processor.

[0055] In detail, the memory and the processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, they can be electrically connected via one or more communication buses or signal lines. The memory may store at least one software functional module (computer program) that exists in the form of software or firmware. The processor can be used to execute the executable computer program stored in the memory, thereby implementing the factory equipment operation data processing method for safe production management provided in this embodiment of the invention.

[0056] It is understood that, in some specific implementations, the memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0057] It is understood that, in some specific implementations, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system-on-chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0058] It is understood that, in some specific implementations, the factory equipment operation data processing system applied to safety production management can be a server with data processing capabilities.

[0059] Combination Figure 2 This invention also provides a method for processing factory equipment operation data for safety production management, applicable to the aforementioned factory equipment operation data processing system for safety production management. The method steps defined in the relevant process of the method for processing factory equipment operation data for safety production management can be implemented by the aforementioned factory equipment operation data processing system for safety production management. The following will describe... Figure 2 The specific process shown will be explained in detail.

[0060] Step S110: Extract operating information from multiple typical devices.

[0061] In this embodiment of the invention, the factory equipment operation data processing system applied to safety production management can extract multiple typical equipment operation information. The typical equipment operation information includes typical equipment operation monitoring videos and corresponding typical optimized operation monitoring videos. The typical optimized operation monitoring videos are simplified versions of the typical equipment operation monitoring videos and can be used as corresponding tagged videos. Furthermore, the typical equipment operation monitoring videos can be manually simplified to obtain the typical optimized operation monitoring videos. This manual simplification process includes, but is not limited to, filtering, editing, and replacing video frames.

[0062] Step S120: Update and adjust the initial video optimization network based on the operating information of the multiple typical devices.

[0063] In this embodiment of the invention, the factory equipment operation data processing system applied to safety production management can update and adjust the initial video optimization network based on the multiple typical equipment operation information (including typical equipment operation monitoring videos and corresponding typical optimized operation monitoring videos). Thus, the updated initial video optimization network can learn the mapping relationship between the typical equipment operation monitoring videos and the corresponding typical optimized operation monitoring videos, enabling the updated initial video optimization network to perform video optimization operations on the target equipment operation monitoring video to be processed based on this mapping relationship, thereby obtaining the corresponding target optimized operation monitoring video.

[0064] Step S130: By updating and adjusting the initial video optimization network, video optimization operation is performed on the target device running monitoring video to be processed, so as to form the target optimized running monitoring video corresponding to the target device running monitoring video.

[0065] In this embodiment of the invention, the factory equipment operation data processing system applied to safety production management can perform video optimization operations on the target equipment operation monitoring video to be processed through the updated and adjusted initial video optimization network, thereby forming a target optimized operation monitoring video corresponding to the target equipment operation monitoring video. The number of video frames included in the target optimized operation monitoring video is less than or equal to the number of video frames included in the target equipment operation monitoring video, that is, video frame simplification and optimization are performed to a certain extent. The target equipment operation monitoring video is formed based on video monitoring of the target factory equipment.

[0066] Step S140: The equipment operation status analysis network formed by updating and adjusting the network performs operation status analysis on the target optimized operation monitoring video to output the operation status analysis data corresponding to the target factory equipment.

[0067] In this embodiment of the invention, the factory equipment operation data processing system applied to safety production management can perform operation status analysis on the target optimized operation monitoring video through an equipment operation status analysis network formed by updating and adjusting operations, so as to output operation status analysis data corresponding to the target factory equipment. The operation status analysis data reflects the operational safety level of the target factory equipment and serves as a basis for safety production management; for example, the operational safety level can be used as a condition for triggering safety production alarms. The equipment operation status analysis network can learn the mapping relationship between the corresponding operation monitoring video and the corresponding operation status label, thus possessing the function of operation status analysis.

[0068] Based on the foregoing, before performing the operational status analysis, the target device's operational monitoring video is optimized using the updated and adjusted initial video optimization network. This results in a more concise target optimized operational monitoring video, where the number of video frames is less than or equal to the number of video frames in the target device's operational monitoring video. Consequently, the basis for the operational status analysis is more streamlined. Therefore, this approach can improve the reliability of operational data analysis to some extent and address the issue of low reliability in existing technologies (such as the redundancy in the analysis basis in existing technologies).

[0069] It is understood that, in some specific implementations, step S110 in the above embodiments, namely the step of extracting multiple typical device operating information, may further include the following implementation details:

[0070] Multiple initial typical equipment operation information is extracted. The initial typical equipment operation information includes the typical equipment operation monitoring video and the initial optimized operation monitoring video corresponding to the typical equipment operation monitoring video. The initial optimized operation monitoring video includes at least two video summary description information and video detailed description information corresponding to each video summary description information (i.e., the corresponding monitoring video frame; for example, the video detailed description information corresponding to a video summary description information is obtained by video monitoring one of the at least two local devices of the typical factory equipment).

[0071] For each of the initial typical equipment operation information, based on at least two video summary descriptions in the initial optimized operation monitoring video, the initial optimized operation monitoring video is segmented to form at least two typical optimized operation monitoring videos. Each typical optimized operation monitoring video includes a video detailed description corresponding to a video summary description, that is, the video detailed description is used as a typical optimized operation monitoring video. Furthermore, the typical equipment operation monitoring video and each of the at least two typical optimized operation monitoring videos are merged to form at least two typical equipment operation information.

[0072] It is understood that, in some specific implementations, step S120 in the above embodiments, namely, the step of updating and adjusting the initial video optimization network based on the operating information of the multiple typical devices, may further include the following implementation content:

[0073] Based on the multiple typical device operation information corresponding to the same video summary description information, an initial video optimization network is updated and adjusted to form at least two initial video optimization networks. That is to say, since there are different initial optimized operation monitoring videos that include the same video summary description information, there are also multiple typical device operation information corresponding to the same video summary description information.

[0074] It is understood that, in some specific implementations, step S130 in the above embodiments, namely, the step of performing video optimization operations on the target device running monitoring video to be processed by updating and adjusting the initial video optimization network to form the target optimized running monitoring video corresponding to the target device running monitoring video, further includes the following implementation content:

[0075] Using at least two of the initial video optimization networks, monitoring videos are run based on the target device, and detailed video description information corresponding to the general description information of each video is output.

[0076] The output of at least two detailed video descriptions (corresponding to at least two of the initial video optimization networks) and at least two general video descriptions are merged to form the target optimized operation monitoring video corresponding to the target device operation monitoring video.

[0077] For example, we can obtain initial typical equipment operation information A and initial typical equipment operation information B. Initial typical equipment operation information A includes typical equipment operation monitoring video A1 and initial optimized operation monitoring video A2. Initial optimized operation monitoring video A2 includes detailed video description information corresponding to the video summary description information "local device 1" and detailed video description information corresponding to the video summary description information "local device 2". Then, we can split the initial optimized operation monitoring video A1 according to the video summary description information to obtain typical optimized operation monitoring video a1 and typical optimized operation monitoring video a2. Typical optimized operation monitoring video a1 includes detailed video description information corresponding to the video summary description information "local device 1", and typical optimized operation monitoring video a2 includes detailed video description information corresponding to the video summary description information "local device 2". Then, we can merge typical equipment operation monitoring video A1 and typical optimized operation monitoring video a2 to obtain typical equipment operation information A3, and merge typical equipment operation monitoring video A1 and typical optimized operation monitoring video a2 to obtain typical equipment operation information A4. Additionally, the initial typical equipment operation information B includes typical equipment operation monitoring video B1 and initial optimized operation monitoring video B2. Initial optimized operation monitoring video B2 includes detailed video description information corresponding to the video summary description information "Local Device 1" (which, like the detailed video description information corresponding to the video summary description information "Local Device 1" in the initial optimized operation monitoring video A2, can correspond to different monitoring angles, etc.) and detailed video description information corresponding to the video summary description information "Local Device 2". Then, the initial optimized operation monitoring video B2 can be split according to the video summary description information to obtain typical optimized operation monitoring video b1 and typical optimized operation monitoring video b2. Typical optimized operation monitoring video b1 includes detailed video description information corresponding to the video summary description information "Local Device 1", and typical optimized operation monitoring video b2 includes detailed video description information corresponding to the video summary description information "Local Device 2". Afterwards, typical equipment operation monitoring video B1 and typical optimized operation monitoring video b2 can be combined to obtain typical equipment operation information B3, and typical equipment operation monitoring video B1 and typical optimized operation monitoring video b2 can be combined to obtain typical equipment operation information B4. Among them, typical equipment operation information A3 and typical equipment operation information B3 are typical data corresponding to the video general description information "local device 1". Therefore, the initial video optimization network C1 can be updated and adjusted based on typical equipment operation information A3 and typical equipment operation information B3. The initial video optimization network C1 is used to generate the video detailed description information corresponding to the video general description information "local device 1".Typical equipment operation information A4 and B4 are both typical data corresponding to the video summary description information "Local Device 2". Therefore, based on typical equipment operation information A4 and B4, the initial video optimization network C2 can be updated and adjusted. This initial video optimization network C2 is used to generate the video detail description information corresponding to the video summary description information "Local Device 2". Based on this, the initial video optimization network C1 can generate the video detail description information D1 corresponding to the video summary description information "Local Device 1" based on the target device operation monitoring video, and the initial video optimization network C2 can generate the video detail description information D2 corresponding to the video summary description information "Local Device 2" based on the target device operation monitoring video. Then, the video summary description information "Local Device 1", the video detail description information D1, the video summary description information "Local Device 2", and the video detail description information D2 can be merged to obtain the target optimized operation monitoring video corresponding to the target device operation monitoring video.

[0078] It is understood that, in some specific implementations, step S120 in the above embodiments, namely, the step of updating and adjusting the initial video optimization network based on the operating information of the multiple typical devices, may further include the following implementation content:

[0079] Based on the undetermined video frame cluster and the typical optimized operation monitoring video, the typical video representative data corresponding to the typical optimized operation monitoring video is analyzed. The undetermined video frame cluster includes multiple undetermined video frames. The typical video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the typical optimized operation monitoring video. Each undetermined video frame included in the undetermined video frame cluster can be formed by manual screening, editing and other operations based on historical monitoring video frames.

[0080] The initial video optimization network is used to analyze and mine the typical equipment operation monitoring video to output corresponding mining video representative data. The mining video representative data is used to reflect the probability parameter, i.e. the probability value, of each undetermined video frame belonging to the mining optimized operation monitoring video corresponding to the typical equipment operation monitoring video.

[0081] Based on the differences between the typical video representative data and the mined video representative data, the initial video optimization network is updated and adjusted.

[0082] It is understood that, in some specific implementations, the initial video optimization network may include a key information mining model, a key information integration model, and a key information restoration model. Based on this, the step of analyzing and mining the typical device operation monitoring video through the initial video optimization network to output corresponding representative data of the mined video may further include the implementation content described below:

[0083] The key information mining model is used to perform key information mining operations on the typical equipment operation monitoring video, such as performing convolution operations, to output the video key information description vector corresponding to each monitoring video segment in the typical equipment operation monitoring video.

[0084] The key information integration model integrates multiple video key information description vectors to output a corresponding integrated key information description vector. For example, multiple video key information description vectors can be superimposed or concatenated to obtain an integrated key information description vector. Alternatively, the results of superimposition or concatenation can be further convolved to obtain an integrated key information description vector. Or, pairwise focusing feature analysis can be performed on the multiple video key information description vectors, and then the results of each focusing feature analysis operation can be superimposed or concatenated.

[0085] The key information restoration model is used to perform key information restoration operations based on the integrated key information description vector to output corresponding representative data of the mining video. For example, the integrated key information description vector can be fully connected and activated to obtain the corresponding representative data of the mining video.

[0086] It is understood that, in some specific implementations, the representative data of the mining video may include local representative data of the mining video corresponding to multiple frame coordinates in the mining optimization operation monitoring video. The local representative data of the mining video includes mining video representative parameters corresponding to each of the undetermined video frames. The mining video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the mining optimization operation monitoring video. Based on this, the step of performing key information restoration operation according to the integrated key information description vector through the key information restoration model to output the corresponding representative data of the mining video may further include the implementation content described below:

[0087] The key information restoration model is used to restore the integrated key information description vector to output the corresponding local mining video representative data. The undetermined video frame corresponding to the mining video representative parameter with the maximum value in the local mining video representative data is marked as the first optimized video frame.

[0088] Using the key information restoration model, based on the marked optimized video frames, the integrated key information description vector is used to perform key information restoration operations to output the corresponding subsequent local mining video representative data. Furthermore, the undetermined video frames corresponding to the mining video representative parameters with the maximum value in the subsequent local mining video representative data are marked as the second optimized video frames. This process can be repeated cyclically and is completed when the marked optimized video frame is the end video frame. The end video frame is a blank video frame used to reflect the end of the mining optimization operation monitoring video, indicating that the key information restoration operation has been completed.

[0089] It is understood that, in some specific implementations, the typical video representative data may include local typical video representative data corresponding to multiple frame coordinates in the typical optimized operation monitoring video. This local typical video representative data reflects the probability parameter that each undetermined video frame belongs to the video frame at the corresponding frame coordinate in the typical optimized operation monitoring video. Similarly, the mined video representative data may include local mined video representative data corresponding to multiple frame coordinates in the mined optimized operation monitoring video. This local mined video representative data reflects the probability parameter that each undetermined video frame belongs to the video frame at the corresponding frame coordinate in the mined optimized operation monitoring video. Therefore, the step of updating and adjusting the initial video optimization network based on the typical video representative data and the mined video representative data may further include the following implementation details:

[0090] Based on the difference information between the local mining video representative data and the local typical video representative data corresponding to each frame coordinate, the local error index corresponding to each frame coordinate is analyzed, and the local error index can be positively correlated with the difference information.

[0091] Based on the analyzed local error indices, a corresponding target error index is calculated. The target error index and the local error indices have a positive correlation. For example, the local error indices can be summed or weighted and calculated.

[0092] Based on the target error index, the initial video optimization network is updated and adjusted. For example, based on the need to reduce the target error index, the network parameters of the initial video optimization network can be updated and adjusted until the current target error index is less than a preset value.

[0093] It is understood that, in some specific implementations, step S130 in the above embodiments, namely, the step of performing video optimization operations on the target device running monitoring video to be processed by updating and adjusting the initial video optimization network to form the target optimized running monitoring video corresponding to the target device running monitoring video, may further include the implementation content described below:

[0094] The initial video optimization network analyzes and mines the target device operation monitoring video to output corresponding video representative data. Based on the video representative data, multiple confirmed video frames belonging to the target optimized operation monitoring video are analyzed from multiple undetermined video frames included in the undetermined video frame cluster. The video representative data is used to reflect the probability parameter, i.e., the probability value, of each undetermined video frame belonging to the target optimized operation monitoring video corresponding to the target device operation monitoring video.

[0095] Each of the confirmed video frames is combined to form a target optimized operation monitoring video corresponding to the target device operation monitoring video. In other words, the target optimized operation monitoring video may include each of the confirmed video frames.

[0096] It is understood that, in some specific implementations, the initial video optimization network may include a key information mining model, a key information integration model, and a key information restoration model. Based on this, the step of analyzing and mining the target device's operational monitoring video through the initial video optimization network to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimized operational monitoring video from multiple undetermined video frames included in the undetermined video frame cluster based on the video representative data, further includes the implementation content described below:

[0097] The key information mining model is used to perform key information mining on the target device operation monitoring video to output the video key information representation vector corresponding to each monitoring video segment in the target device operation monitoring video.

[0098] Through the key information integration model, multiple video key information representation vectors mined are integrated to output the corresponding integrated key information representation vector. The vector integration operation can be referred to the aforementioned description. In this case, the representation vector and the aforementioned description vector both refer to the representation of the corresponding key information in the form of vectors.

[0099] The key information restoration model is used to perform key information restoration operations based on the integrated key information representation vector to output corresponding video representative data. Based on the video representative data, multiple confirmed video frames belonging to the target optimized operation monitoring video are analyzed from multiple pending video frames.

[0100] It is understood that, in some specific implementations, the target device operation monitoring video may include partially ending video frames, which are used to reflect the end of the video segment of the monitoring video clip. Based on this, the step of performing key information mining operations on the target device operation monitoring video through the key information mining model to output the video key information representation vector corresponding to each monitoring video segment in the target device operation monitoring video may further include the implementation content described below:

[0101] By using the key information mining model, the partial end video frames in the target device's operation monitoring video are analyzed. The video frames before the first partial end video frame are marked as a monitoring video segment, and the video frames between every two partial end video frames are marked as a monitoring video segment. In other words, the monitoring video segments in the target device's operation monitoring video can be identified first. The partial end video frames can be blank video frames, which can be used as markers for the end or segmentation of video segments.

[0102] The key information mining model is used to perform key information mining operations on each of the surveillance video segments. For example, convolution operations can be performed on the surveillance video segments to output the video key information representation vector corresponding to each surveillance video segment.

[0103] It is understood that, in some specific implementations, the video representative data may include local video representative data corresponding to multiple frame coordinates in the target optimization operation monitoring video. The local video representative data includes video representative parameters corresponding to each undetermined video frame. These video representative parameters reflect the probability that the undetermined video frame belongs to a video frame at the corresponding frame coordinate in the target optimization operation monitoring video. Based on this, the step of performing key information restoration operations using the key information restoration model and the integrated key information representation vector to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimization operation monitoring video from multiple undetermined video frames based on the video representative data, may further include the implementation details described below:

[0104] The key information restoration model is used to restore the integrated key information representation vector to output the corresponding preceding local video representative data. The pending video frame corresponding to the video representative parameter with the maximum value in the preceding local video representative data is marked as the first confirmed video frame. The key information restoration operation may include corresponding fully connected processing and activation processing, as well as other possible processing procedures.

[0105] Using the key information restoration model, based on the marked confirmed video frames, the integrated key information representation vector is restored to output the corresponding subsequent local video representative data. The pending video frames corresponding to the video representative parameters with the maximum value in the subsequent local video representative data are marked as the second confirmed video frames. This process is completed when the marked confirmed video frame is the end video frame. The end video frame is a blank video frame used to reflect the end of the target optimized operation monitoring video.

[0106] It is understood that, in some specific implementations, the video representative data may include local video representative data corresponding to multiple frame coordinates in the target optimization operation monitoring video. The local video representative data may include video representative parameters corresponding to each undetermined video frame. These video representative parameters reflect the probability that the undetermined video frame belongs to a video frame at the corresponding frame coordinate in the target optimization operation monitoring video. Based on this, the step of performing key information restoration operations using the key information restoration model and the integrated key information representation vector to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimization operation monitoring video from multiple undetermined video frames based on the video representative data, may further include the implementation details described below:

[0107] Using the key information restoration model, based on different video frames, the integrated key information representation vector is restored to output multiple video representative data. For each video representative data, the video frame corresponding to the video representative parameter with the maximum value in each local video representative data is marked as a confirmed video frame. The multiple marked confirmed video frames are combined to form a confirmed video frame cluster corresponding to the video representative data.

[0108] The merged video representative parameters corresponding to each confirmed video frame cluster are analyzed. The merged video representative parameters have a positive correlation with the video representative parameters corresponding to multiple confirmed video frames in the confirmed video frame cluster. For example, the average or summation of the video representative parameters corresponding to the multiple confirmed video frames can be performed to obtain the merged video representative parameters.

[0109] From the multiple confirmed video frame clusters, identify multiple confirmed video frames from the confirmed video frame clusters that have the maximum value of the merged video representative parameter.

[0110] It is understood that, in some specific implementations, the steps of using the key information restoration model to perform key information restoration operations on the integrated key information representation vector based on different video frames to output multiple video representative data, and for each video representative data, marking the undetermined video frame corresponding to the video representative parameter with the maximum value in each local video representative data as a confirmed video frame, and combining the marked multiple confirmed video frames to form a confirmed video frame cluster corresponding to the video representative data, may further include the implementation content described below:

[0111] The key information restoration model is used to restore the key information representation vector to output the local video representative data. The first number of video representative parameters with the maximum value in the local video representative data are marked as the first confirmed video frames. The number of video representative data is equal to the first number. The first confirmed video frame can be used as the first video frame.

[0112] For each first confirmed video frame, the key information restoration model is used to perform key information restoration operations on the integrated key information representation vector based on the first confirmed video frame to output the corresponding subsequent local video representative data. The pending video frame corresponding to the video representative parameter with the maximum value in the subsequent local video representative data is marked as the second confirmed video frame. The second confirmed video frame is assigned to the confirmed video frame cluster to which the first confirmed video frame belongs. In this way, the third confirmed video frame, the fifth confirmed video frame, etc., can be marked in sequence. When the marked confirmed video frame is an end video frame, the end video frame is assigned to the confirmed video frame cluster to which the first confirmed video frame belongs. The end video frame is used to reflect the end of the video of the target optimization operation monitoring video. Each first confirmed video frame is assigned to a different confirmed video frame cluster.

[0113] It is understood that, in some specific implementations, the step of performing key information restoration operations on the integrated key information representation vector based on the first confirmed video frame using the key information restoration model to output the corresponding subsequent local video representative data may further include the following implementation details:

[0114] The key information mining model is used to perform key information mining on the first confirmed video frame to output the video frame key information description vector corresponding to the first confirmed video frame.

[0115] The video frame key information description vector and the integrated key information representation vector are combined to output the corresponding first integrated representation vector (for other confirmed video frames, the video frame key information description vector and the integrated key information representation vector corresponding to each confirmed video frame can be combined together to output the corresponding integrated representation vector).

[0116] The key information restoration model is used to perform key information restoration operations based on the first integrated representation vector row to output the corresponding local video representative data.

[0117] Combination Figure 3 This invention also provides a factory equipment operation data processing device for safety production management, which can be applied to the aforementioned factory equipment operation data processing system for safety production management. The factory equipment operation data processing device (a software virtual device) for safety production management may include:

[0118] The typical information extraction module is used to extract the operation information of multiple typical devices, including typical device operation monitoring videos and typical optimized operation monitoring videos corresponding to the typical device operation monitoring videos.

[0119] The network update and adjustment module is used to update and adjust the initial video optimization network based on the operating information of the multiple typical devices.

[0120] The monitoring video optimization module is used to perform video optimization operations on the target device running monitoring video to be processed by updating and adjusting the initial video optimization network, so as to form a target optimized running monitoring video corresponding to the target device running monitoring video. The number of video frames included in the target optimized running monitoring video is less than or equal to the number of video frames included in the target device running monitoring video. The target device running monitoring video is formed based on video monitoring of the target factory equipment.

[0121] The operation status analysis module is used to perform operation status analysis on the target optimized operation monitoring video through the equipment operation status analysis network formed by updating and adjusting operations, so as to output the operation status analysis data corresponding to the target factory equipment. The operation status analysis data is used to reflect the operation safety level of the target factory equipment and serves as the basis for safety production management.

[0122] In summary, the factory equipment operation data processing method and system for safe production management provided by this invention can first extract multiple typical equipment operation information, including typical equipment operation monitoring videos and typical optimized operation monitoring videos. Based on this information, an initial video optimization network is updated and adjusted. Then, through the updated initial video optimization network, video optimization is performed on the target equipment operation monitoring video to form a target optimized operation monitoring video. Finally, through the equipment operation status analysis network formed by the update and adjustment, operation status analysis is performed on the target optimized operation monitoring video to output operation status analysis data corresponding to the target factory equipment. Based on the foregoing, since the target equipment operation monitoring video is optimized based on the updated initial video optimization network before the operation status analysis, the resulting target optimized operation monitoring video is more concise to a certain extent. That is, the number of video frames in the target optimized operation monitoring video is less than or equal to the number of video frames in the target equipment operation monitoring video. This makes the basis for the operation status analysis more concise, thus improving the reliability of operation data analysis to a certain extent and addressing the problem of low reliability in existing technologies.

[0123] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing factory equipment operation data applied to safety production management, characterized in that, The factory equipment operation data processing method includes: Extract the operation information of multiple typical devices, including typical device operation monitoring videos and typical optimized operation monitoring videos corresponding to the typical device operation monitoring videos; Based on the operating information of the aforementioned typical devices, the initial video optimization network is updated and adjusted; By updating and adjusting the initial video optimization network, video optimization operations are performed on the target device operation monitoring video to be processed, so as to form a target optimized operation monitoring video corresponding to the target device operation monitoring video. The number of video frames included in the target optimized operation monitoring video is less than or equal to the number of video frames included in the target device operation monitoring video. The target device operation monitoring video is formed based on video monitoring of the target factory equipment. The target optimized operation monitoring video is analyzed by the equipment operation status analysis network to output the operation status analysis data corresponding to the target factory equipment. The operation status analysis data is used to reflect the operation safety level of the target factory equipment and serves as the basis for safety production management. The step of updating and adjusting the initial video optimization network based on the operating information of the multiple typical devices includes: Based on the undetermined video frame cluster and the typical optimized operation monitoring video, the typical video representative data corresponding to the typical optimized operation monitoring video is analyzed. The undetermined video frame cluster includes multiple undetermined video frames, and the typical video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the typical optimized operation monitoring video. The initial video optimization network is used to analyze and mine the typical equipment operation monitoring video to output corresponding mining video representative data. The mining video representative data is used to reflect the probability parameter that each of the pending video frames belongs to the mining optimized operation monitoring video corresponding to the typical equipment operation monitoring video. Based on the typical video representative data and the mined video representative data, the initial video optimization network is updated and adjusted; The typical video representative data includes local typical video representative data corresponding to multiple frame coordinates in the typical optimized operation monitoring video. The local typical video representative data is used to reflect the probability parameter that each of the undetermined video frames belongs to the video frame at the corresponding frame coordinate in the typical optimized operation monitoring video. The mining video representative data includes local mining video representative data corresponding to multiple frame coordinates in the mining optimized operation monitoring video. The local mining video representative data is used to reflect the probability parameter that each of the undetermined video frames belongs to the video frame at the corresponding frame coordinate in the mining optimized operation monitoring video. The step of updating and adjusting the initial video optimization network based on the typical video representative data and the mined video representative data includes: Based on the difference information between the local mining video representative data and the local typical video representative data corresponding to each frame coordinate, the local error index corresponding to each frame coordinate is analyzed. Based on the analyzed local error indices, a corresponding target error index is calculated, and the target error index has a positive correlation with the multiple local error indices. Based on the target error index, the initial video optimization network is updated and adjusted.

2. The factory equipment operation data processing method for safety production management as described in claim 1, characterized in that, The initial video optimization network includes a key information mining model, a key information integration model, and a key information restoration model. The step of analyzing and mining the typical equipment operation monitoring video through the initial video optimization network to output corresponding representative data from the mined video includes: The key information mining model is used to perform key information mining on the typical equipment operation monitoring video to output the video key information description vector corresponding to each monitoring video segment in the typical equipment operation monitoring video. The key information integration model is used to integrate multiple video key information description vectors to output the corresponding integrated key information description vector. The key information restoration model is used to perform key information restoration operations based on the integrated key information description vector, so as to output the corresponding representative data of the mining video.

3. The factory equipment operation data processing method for safety production management as described in claim 2, characterized in that, The representative data of the mining video includes local representative data of the mining video corresponding to multiple frame coordinates in the mining optimization operation monitoring video. The local representative data of the mining video includes mining video representative parameters corresponding to each of the undetermined video frames. The mining video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the mining optimization operation monitoring video. The step of performing key information restoration operation based on the integrated key information description vector using the key information restoration model to output corresponding representative data from the mined video includes: The key information restoration model is used to restore the integrated key information description vector to output the corresponding local mining video representative data. The undetermined video frame corresponding to the mining video representative parameter with the maximum value in the local mining video representative data is marked as the first optimized video frame. Using the key information restoration model, based on the marked optimized video frames, the integrated key information description vector is restored to output the corresponding subsequent local mining video representative data. Additionally, the undetermined video frames corresponding to the mining video representative parameters with the maximum value in the subsequent local mining video representative data are marked as the second optimized video frames. This process is completed when the marked optimized video frames belong to the end video frames. The end video frames are blank video frames used to reflect the end of the mining optimization operation monitoring video.

4. The factory equipment operation data processing method for safety production management as described in claim 1, characterized in that, The step of performing video optimization operations on the target device running monitoring video through the updated and adjusted initial video optimization network to form the target optimized running monitoring video corresponding to the target device running monitoring video includes: The initial video optimization network analyzes and mines the target device operation monitoring video to output corresponding video representative data. Based on the video representative data, multiple confirmed video frames belonging to the target optimized operation monitoring video are analyzed from multiple undetermined video frames included in the undetermined video frame cluster. The video representative data is used to reflect the probability parameter that each undetermined video frame belongs to the target optimized operation monitoring video corresponding to the target device operation monitoring video. Each confirmed video frame is combined to form a target optimized operation monitoring video corresponding to the target device operation monitoring video.

5. The factory equipment operation data processing method for safety production management as described in claim 4, characterized in that, The initial video optimization network includes a key information mining model, a key information integration model, and a key information restoration model. The steps of analyzing and mining the target device's operational monitoring video through the initial video optimization network to output corresponding representative video data, and analyzing multiple confirmed video frames belonging to the target optimized operational monitoring video from multiple undetermined video frames included in the undetermined video frame cluster based on the representative video data, include: The key information mining model is used to perform key information mining on the target device operation monitoring video to output the video key information representation vector corresponding to each monitoring video segment in the target device operation monitoring video. The key information integration model is used to integrate multiple video key information representation vectors to output the corresponding integrated key information representation vector. The key information restoration model is used to perform key information restoration operations based on the integrated key information representation vector to output corresponding video representative data. Based on the video representative data, multiple confirmed video frames belonging to the target optimized operation monitoring video are analyzed from multiple pending video frames. The target device operation monitoring video includes partially ending video frames, which reflect the end of the video segment. The step of performing key information mining on the target device operation monitoring video using the key information mining model to output a video key information representation vector corresponding to each monitoring video segment in the target device operation monitoring video includes: Using the key information mining model, the system analyzes the locally ending video frames in the monitoring video of the target device. Video frames before the first locally ending video frame are marked as a monitoring video segment, and video frames between every two locally ending video frames are also marked as a monitoring video segment. Furthermore, using the key information mining model, each monitoring video segment undergoes key information mining to output a video key information representation vector corresponding to each monitoring video segment.

6. The factory equipment operation data processing method for safety production management as described in claim 5, characterized in that, The video representative data includes local video representative data corresponding to multiple frame coordinates in the target optimization operation monitoring video. The local video representative data includes video representative parameters corresponding to each undetermined video frame. The video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the target optimization operation monitoring video. The steps of performing key information restoration operations based on the integrated key information representation vector using the key information restoration model to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimized operation monitoring video from multiple undetermined video frames based on the video representative data, include: The key information restoration model is used to restore the integrated key information representation vector to output the corresponding preceding local video representative data. The pending video frame corresponding to the video representative parameter with the maximum value in the preceding local video representative data is marked as the first confirmed video frame. Using the key information restoration model, based on the marked confirmed video frames, the integrated key information representation vector is restored to output the corresponding subsequent local video representative data. The pending video frames corresponding to the video representative parameters with the maximum value in the subsequent local video representative data are marked as the second confirmed video frames. This process is completed when the marked confirmed video frame is the end video frame. The end video frame is a blank video frame used to reflect the end of the target optimized operation monitoring video.

7. The factory equipment operation data processing method for safety production management as described in claim 5, characterized in that, The video representative data includes local video representative data corresponding to multiple frame coordinates in the target optimization operation monitoring video. The local video representative data includes video representative parameters corresponding to each undetermined video frame. The video representative parameters are used to reflect the probability parameters that the undetermined video frame belongs to the video frame at the corresponding frame coordinate in the target optimization operation monitoring video. The steps of performing key information restoration operations based on the integrated key information representation vector using the key information restoration model to output corresponding video representative data, and analyzing multiple confirmed video frames belonging to the target optimized operation monitoring video from multiple undetermined video frames based on the video representative data, include: Using the key information restoration model, based on different video frames, the integrated key information representation vector is restored to output multiple video representative data. For each video representative data, the video frame corresponding to the video representative parameter with the maximum value in each local video representative data is marked as a confirmed video frame. The multiple marked confirmed video frames are combined to form a confirmed video frame cluster corresponding to the video representative data. The merged video representative parameters corresponding to each confirmed video frame cluster are analyzed, and there is a positive correlation between the merged video representative parameters and the video representative parameters corresponding to multiple confirmed video frames in the confirmed video frame cluster. From the multiple confirmed video frame clusters, identify multiple confirmed video frames from the confirmed video frame clusters that have the maximum value of the merged video representative parameter.

8. A factory equipment operation data processing system applied to safety production management, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to implement the method of any one of claims 1-7.

Images