An artificial intelligence-based safety monitoring method and device, a storage medium and an electronic device

Abstract

The application provides an artificial intelligence-based safety monitoring method and device, a storage medium and an electronic device. The method comprises: acquiring monitoring data of a device; the monitoring data comprises image data of the device and a running state parameter sequence of the device within a preset time period; processing the image data through an image recognition module of a pre-trained safety state recognition model to obtain first features of the device; processing the running state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain second features of the device; fusing the first features and the second features through a feature fusion module of the safety state recognition model to obtain fused features of the device; processing the fused features through a classifier of the safety state recognition model to obtain a safety state recognition result of the device, and outputting the safety state recognition result. The method provided in the application can obtain an accurate safety state recognition result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a safety monitoring method and device based on artificial intelligence, a storage medium and an electronic device. BACKGROUND

[0002] With the development of industrial automation and intelligence, safety monitoring of equipment has become an important link to ensure production efficiency and personnel safety. Traditional equipment state monitoring methods mainly rely on manual inspection and experience judgment of technical personnel. This method is not only inefficient, but also prone to missed detection or misjudgment. In recent years, with the rapid development of Internet of Things (IoT), big data and artificial intelligence technology, more and more research and application focuses on how to use advanced technical means to improve the accuracy and reliability of equipment state monitoring.

[0003] Currently, in the field of equipment state monitoring, there are some technical solutions based on sensor data. These solutions usually collect physical parameters such as temperature, pressure, etc. through various sensors deployed on the equipment during equipment operation, and identify the working state of the equipment through data processing and analysis. However, existing methods often have limitations when facing complex working conditions and multi-modal data, for example: single modal data processing: most existing technologies only focus on a certain type of sensor data, ignoring the relevance and complementarity between different modal data, resulting in incomplete monitoring results. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a safety monitoring method and device based on artificial intelligence, which can obtain accurate safety monitoring results. The specific scheme is as follows:

[0005] A safety monitoring method based on artificial intelligence, comprising:

[0006] obtaining monitoring data of the equipment; the monitoring data includes image data of the equipment and a sequence of operating state parameters of the equipment within a preset time period;

[0007] processing the image data through an image recognition module of a pre-trained safety state recognition model to obtain first features of the equipment;

[0008] processing the sequence of operating state parameters through a convolutional long short-term memory network of the safety state recognition model to obtain second features of the equipment;

[0009] fusing the first features and the second features through a feature fusion module of the safety state recognition model to obtain fusion features of the equipment;

[0010] processing the fusion feature through a classifier of the safety state recognition model to obtain a safety state recognition result of the device;

[0011] outputting the safety state recognition result of the device.

[0012] The method described above, optionally, the image recognition module of the safety state recognition model, the image segmentation module, the patch embedding module, the position encoding module and the multi-head self-attention mechanism module, the image recognition module of the pre-trained safety state recognition model processes the image data to obtain the first feature of the device, including:

[0013] The image segmentation module in the image recognition module converts the image data into a patch sequence;

[0014] The patch embedding module of the image recognition module linearly maps each image patch in the patch sequence to obtain an embedding vector of each image patch;

[0015] The position encoding module of the image recognition module encodes the embedding vector of each image patch based on a preset position encoding matrix to obtain a target embedding vector of each image patch;

[0016] The multi-head self-attention mechanism module of the image recognition module processes the target embedding vector of each image patch to obtain the first feature of the device.

[0017] The method described above, optionally, the training process of the safety state recognition model, including:

[0018] obtain a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device, and the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state and a fault state;

[0019] select a target training sample currently used for training from each training sample in the training data set;

[0020] input the target training sample into the initial safety state recognition model to obtain an identification result output by the initial safety state recognition model;

[0021] According to the preset cross-entropy loss function, the identification result and the safety state label, the loss function value of the initial safety state recognition model is calculated;

[0022] updating model parameters of the initial safety state recognition model through the loss function value;

[0023] in a case where the initial safety state recognition model after updating the model parameters does not satisfy a preset training completion condition, returning to the step of selecting the target training sample currently used for training in each training sample in the training data set;

[0024] in a case where the initial safety state recognition model after updating the model parameters satisfies the preset training completion condition, determining the initial safety state recognition model as a trained safety state recognition model.

[0025] The method described above, optionally, the output safety state recognition result of the device includes:

[0026] sending the safety state recognition result of the device to a preset remote monitoring center.

[0027] The method described above, optionally, further includes:

[0028] in a case where the safety state recognition result satisfies a preset shutdown condition, controlling the device to shut down and starting a backup device of the device.

[0029] A safety monitoring device based on artificial intelligence, comprising:

[0030] an acquisition unit configured to acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating state parameters of the device within a preset time period;

[0031] a first execution unit configured to process the image data through an image recognition module of a pre-trained safety state recognition model to obtain first features of the device;

[0032] a second execution unit configured to process the sequence of operating state parameters through a convolutional long short-term memory network of the safety state recognition model to obtain second features of the device;

[0033] a feature fusion unit configured to fuse the first features and the second features through a feature fusion module of the safety state recognition model to obtain fusion features of the device;

[0034] a recognition unit configured to process the fusion features through a classifier of the safety state recognition model to obtain a safety state recognition result of the device;

[0035] an output unit configured to output the safety state recognition result of the device.

[0036] The device, optionally, the image recognition module includes an image segmentation module, a patch embedding module, a position encoding module, and a multi-head self-attention mechanism module, the first execution unit includes:

[0037] The conversion subunit is configured to convert the image data into a patch sequence by using an image segmentation module in the image recognition module.

[0038] The mapping subunit is configured to perform linear mapping on each image patch in the patch sequence by using a patch embedding module in the image recognition module, to obtain an embedding vector of each image patch.

[0039] The encoding subunit is configured to perform position encoding on the embedding vector of each image patch based on a preset position encoding matrix by using a position encoding module in the image recognition module, to obtain a target embedding vector of each image patch.

[0040] The processing subunit is configured to process the target embedding vector of each image patch by using a multi-head self-attention mechanism module in the image recognition module, to obtain the first feature of the device.

[0041] The device, optionally, the first execution unit includes:

[0042] The acquisition subunit is configured to acquire a training data set and an initial safety state recognition model to be trained, the training data set includes a plurality of training samples and a safety state label of each training sample, each training sample includes historical monitoring data of a device, and the safety state label is used to indicate a safety state of the device, the safety state being one of a normal state, an abnormal state, and a fault state.

[0043] The selection subunit is configured to select a target training sample currently used for training from each training sample in the training data set.

[0044] The first execution subunit is configured to input the target training sample into the initial safety state recognition model, to obtain an identification result output by the initial safety state recognition model.

[0045] The calculation subunit is configured to calculate a loss function value of the initial safety state recognition model according to a preset cross-entropy loss function, the identification result, and the safety state label.

[0046] The update subunit is configured to update model parameters of the initial safety state recognition model by using the loss function value.

[0047] the second execution subunit is configured to return to trigger the selection subunit to execute the step of selecting a target training sample from each training sample in the training data set for training in a case where the initial safety state recognition model after updating the model parameter does not satisfy the preset training completion condition;

[0048] the third execution subunit is configured to determine the initial safety state recognition model as a trained safety state recognition model in a case where the initial safety state recognition model after updating the model parameter satisfies the preset training completion condition.

[0049] A storage medium, the storage medium comprising stored instructions, wherein the instructions, when executed, control a device in which the storage medium is located to perform an artificial intelligence-based safety monitoring method as described above.

[0050] An electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform an artificial intelligence-based safety monitoring method as described above.

[0051] Based on the above-mentioned artificial intelligence-based safety monitoring method, device, storage medium and electronic device provided by the present application, the method comprises: acquiring monitoring data of a device; the monitoring data comprises image data of the device and a running state parameter sequence of the device within a preset time period; processing the image data through an image recognition module of a pre-trained safety state recognition model to obtain first features of the device; processing the running state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain second features of the device; fusing the first features and the second features through a feature fusion module of the safety state recognition model to obtain fused features of the device; processing the fused features through a classifier of the safety state recognition model to obtain a safety state recognition result of the device; and outputting the safety state recognition result of the device. Through the method provided by the present application, an accurate safety state recognition result can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0053] Figure 1 A method flowchart of an artificial intelligence-based safety monitoring method provided by the present application;

[0054] Figure 2 A schematic diagram of the structure of a security status identification model provided in this application;

[0055] Figure 3 A flowchart of a process for obtaining a first characteristic of a device provided by the present application;

[0056] Figure 4 A flowchart of the training process of a security status recognition model provided in this application;

[0057] Figure 5 A schematic diagram of the structure of an artificial intelligence-based security monitoring device provided in this application;

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0061] The embodiment of the present application provides a security monitoring method based on artificial intelligence, which can be applied to electronic devices, such as computers, smart phones, tablet devices, smart wearable devices, server clusters or cloud computing centers, etc. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0062] S101: Acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period.

[0063] In the embodiment, the image data of the equipment can be collected by the image collection device in a timing or real-time manner. Then, the running state parameter sequence of the equipment is collected by the sensor arranged on the equipment. The running state parameter sequence of the equipment can include mechanical state parameters, electrical parameters, environmental parameters and the like at each time point in a preset time period. The mechanical state parameters can include at least one of vibration data, device temperature data, pressure data and the like. The electrical parameters can include at least one of current data, voltage data and power data. The environmental parameters can include one or more of environmental temperature data and air pressure data.

[0064] Optionally, the equipment can be various types of equipment, such as power generation equipment, power transmission equipment or production equipment.

[0065] S102: The image data is processed by an image recognition module of a pre-trained safety state recognition model to obtain first features of the equipment.

[0066] In the embodiment, the image recognition module can be a Vision Transformer (ViT) or a ConvNeXt model.

[0067] Optionally, the first features can be image features of the equipment.

[0068] S103: The running state parameter sequence is processed by a convolutional long short-term memory network of the safety state recognition model to obtain second features of the equipment.

[0069] Optionally, the safety state recognition model can include one or more convolutional long short-term memory networks, and each convolutional long short-term memory network can include an input gate, a forget gate, an output gate and a cell state.

[0070] S104: The first features and the second features are fused by a feature fusion module of the safety state recognition model to obtain fusion features of the equipment.

[0071] In the embodiment, the feature fusion module includes a fully connected layer and a shared multi-head attention mechanism unit.

[0072] Optionally, the first features and the second features are respectively mapped to a preset dimension space by the fully connected layer of the feature fusion module to obtain a first feature vector and a second feature vector. The first feature vector and the second feature vector are input to the shared multi-head attention mechanism unit, and the first feature vector and the second feature vector processed by the shared multi-head attention mechanism unit are fused to obtain the fusion features of the equipment.

[0073] S105: Processing the fusion feature through the classifier of the security status recognition model to obtain a security status recognition result of the device.

[0074] In this embodiment, the classifier may be a softmax classifier, and the safety state identification result may include one of a normal state, a slightly abnormal state, a seriously abnormal state, and a fault state.

[0075] S106: Output the security status identification result of the device.

[0076] In this embodiment, the security status identification result may be output and displayed on a preset visual interface, or the security status identification result may be sent to a preset remote monitoring center.

[0077] In one embodiment provided in the present application, the security state recognition model may include an image recognition module, at least one convolutional long short-term memory network, a feature fusion module, and a classifier.

[0078] See also Figure 2 , which is a structural diagram of a safety state identification model provided in an embodiment of the present application. The safety state identification model specifically includes an input layer, an image recognition module, at least one convolutional long short-term memory network, a feature fusion module, and a classifier. The monitoring data of the device can be input into the safety state identification module. The image recognition module in the safety state identification model extracts the image features (first features) of the image data in the monitoring data, and the convolutional long short-term memory network extracts the parameter features (second features) in the operating state parameter sequence. The feature fusion module then fuses the image features and parameter features to obtain multimodal features (fused features). Finally, the classifier outputs the safety state identification result based on the multimodal features.

[0079] The method provided in the embodiments of the present application can obtain accurate security status identification results.

[0080] In one embodiment provided in the present application, based on the above implementation process, optionally, the image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the image recognition module of the pre-trained security state recognition model processes the image data to obtain the first feature of the device, such as Figure 3 Shown, including:

[0081] S301: Convert the image data into a patch sequence through the image segmentation module in the image recognition module.

[0082] In the embodiment, the patch sequence can include a plurality of image patches, and the image patch is an image block.

[0083] S302: Linearly mapping each image patch in the patch sequence by a patch embedding module of the image recognition module to obtain an embedding vector of each image patch.

[0084] In the embodiment, the image patch can be converted into a fixed-length one-dimensional vector, and then the fixed-length one-dimensional vector is mapped to a fixed-dimension embedding space by a preset weight matrix to obtain the embedding vector of the image patch.

[0085] S303: Position encoding each embedding vector of each image patch based on a preset position encoding matrix by a position encoding module of the image recognition module to obtain a target embedding vector of each image patch.

[0086] In the embodiment, the position encoding corresponding to the embedding vector of each image patch can be determined in the position encoding matrix; and the embedding vector of each image patch and the corresponding position encoding are added to obtain the target embedding vector.

[0087] In the embodiment, the position encoding matrix can be a predefined matrix generated by a sine function or a cosine function, and such position encoding has periodicity and can capture the relationship between different positions; the position encoding matrix can also be a learnable parameter matrix, which can be part of the safety state recognition model and learn the best position information representation through training.

[0088] S304: Processing the target embedding vector of each image patch by a multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

[0089] In an embodiment provided in the present application, based on the above implementation process, optionally, the training process of the safety state recognition model includes: Figure 4 as shown, comprising:

[0090] S401: Obtain a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device.

[0091] Optionally, the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state and a fault state; the abnormal state can be a slight abnormal state or a serious abnormal state.

[0092] In the embodiment, the safety state labels can include a normal state label, a slight abnormal state label, a serious abnormal state label, a fault state label, and the like; the normal state label is used to indicate that the device is in a normal state, the slight abnormal state label is used to indicate that the device is in a slight abnormal state, the serious abnormal state label is used to indicate that the device is in a serious abnormal state, and the fault state label is used to indicate that the device is in a fault state.

[0093] S402: selecting a target training sample currently used for training from each training sample in the training data set.

[0094] S403: inputting the target training sample into the initial safety state identification model to obtain an identification result output by the initial safety state identification model.

[0095] S404: calculating a loss function value of the initial safety state identification model according to a preset cross-entropy loss function, the identification result, and the safety state label.

[0096] S405: updating model parameters of the initial safety state identification model through the loss function value.

[0097] S406: detecting whether the initial safety state identification model after updating the model parameters meets a preset training completion condition, if not, returning to execute S402; if yes, executing S407.

[0098] In the embodiment, the training completion condition can be that a training frequency is greater than a preset training frequency threshold, a prediction accuracy of the initial safety state identification model is greater than a preset accuracy threshold, or a loss function converges, and the like.

[0099] S407: determining the initial safety state identification model as a trained safety state identification model.

[0100] In an embodiment provided in the application, based on the above implementation process, optionally, the method further includes:

[0101] In the case that the safety state identification result meets a preset shutdown condition, the device is controlled to shut down.

[0102] Optionally, after the device is controlled to shut down, maintenance prompt information of the device can be output to prompt a user to repair and maintain the device, so that the accident risk can be reduced and the service life of the device can be prolonged.

[0103] Optionally, if there is a standby device of the device, the standby device of the device can be started.

[0104] In this embodiment, the comprehensive safety score of the device can be determined according to the safety state recognition result, and the device can be controlled to shut down in the case that the comprehensive safety score of the device is less than a preset score threshold.

[0105] Specifically, the process of determining the state score of the device according to the safety state recognition result is as follows:

[0106] S(t) = W label × (W1 × P(t) + W2 × A(t)) + W3 × T(t))

[0107] Where S(t) is the comprehensive safety score of the device at time t; W label is a weight adjusted according to the model-predicted safety state recognition result; W1, W2 and W3 are weight coefficients corresponding to the score items, used to adjust the importance of each item; P(t) is the model-predicted probability score; A(t) is the anomaly score of the sensor data; T(t) is the historical trend score.

[0108] In this embodiment, the model-predicted probability score P(t) can be determined according to the maximum probability in the prediction result. For example, if the probability distribution output by the model is [p1, p2, p3, p4], the score can be defined as the maximum probability value minus a small threshold value to avoid too high a score:

[0109] P(t) = max(p1, p2, p3, p4) - ∈

[0110] Where ∈ is a small positive number used to prevent the score from being too high.

[0111] In this embodiment, the anomaly score A(t) of the sensor data can be determined by calculating the deviation of the data point from the normal range. For example, if the normal range of the sensor data is [a, b], the score can be defined as:

[0112]

[0113] Where x t is the reading of the sensor at time t. If the reading is within the normal range, the score is close to 1; if the reading deviates from the normal range, the score decreases.

[0114] In this embodiment, the historical trend score T(t) can be determined by comparing the current reading with the average value in the past period of time. For example, if the average value of the sensor data in the past period of time is μ, the score can be defined as:

[0115]

[0116] where s is the standard deviation of sensor data over a period of time. If the current reading is close to the historical average, the score is close to 1; if it deviates greatly, the score is reduced.

[0117] In the embodiment, in the case that the safety comprehensive score is less than the preset score threshold, the device can be controlled to stop, the device can be prevented from continuing to run in a dangerous state, the possibility of an accident is reduced, the dependence on manual judgment is reduced through automatic scoring, and the safety hidden danger caused by human factors is reduced.

[0118] With Figure 1 Corresponding to the method, the embodiment of the application further provides a safety monitoring device based on artificial intelligence, which is used for monitoring Figure 1 The specific implementation of the method in the embodiment, a structure diagram is as shown in Figure 5 The embodiment specifically comprises the following steps.

[0119] The acquisition unit 501 is configured to acquire monitoring data of the device; the monitoring data comprises image data of the device and a sequence of operation state parameters of the device in a preset time period;

[0120] The first execution unit 502 is configured to process the image data through an image recognition module of a pre-trained safety state recognition model to obtain first features of the device;

[0121] The second execution unit 503 is configured to process the sequence of operation state parameters through a convolutional long short-term memory network of the safety state recognition model to obtain second features of the device;

[0122] The feature fusion unit 504 is configured to fuse the first features and the second features through a feature fusion module of the safety state recognition model to obtain fusion features of the device;

[0123] The recognition unit 505 is configured to process the fusion features through a classifier of the safety state recognition model to obtain a safety state recognition result of the device;

[0124] The output unit 506 is configured to output the safety state recognition result of the device.

[0125] In an embodiment provided in the application, based on the above scheme, optionally, the image recognition module comprises an image segmentation module, a patch embedding module, a position encoding module and a multi-head self-attention mechanism module, and the first execution unit comprises:

[0126] The conversion subunit is configured to convert the image data into a patch sequence through the image segmentation module in the image recognition module;

[0127] mapping sub-unit, configured to perform linear mapping on each image patch in the patch sequence by a patch embedding module of the image recognition module to obtain an embedding vector of each image patch;

[0128] encoding sub-unit, configured to perform position encoding on the embedding vector of each image patch based on a preset position encoding matrix by a position encoding module of the image recognition module to obtain a target embedding vector of each image patch;

[0129] processing sub-unit, configured to perform processing on the target embedding vector of each image patch by a multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

[0130] In an embodiment provided in the application, based on the above scheme, optionally, the first execution unit comprises:

[0131] an acquisition sub-unit configured to acquire a training data set and an initial safety state recognition model to be trained, wherein the training data set comprises a plurality of training samples and a safety state label of each training sample, each training sample comprises historical monitoring data of a device, and the safety state label is used to indicate a safety state of the device, and the safety state is one of a normal state, an abnormal state and a fault state;

[0132] a selection sub-unit configured to select a target training sample currently used for training from each training sample in the training data set;

[0133] a first execution sub-unit configured to input the target training sample into the initial safety state recognition model to obtain a recognition result output by the initial safety state recognition model;

[0134] a calculation sub-unit configured to calculate a loss function value of the initial safety state recognition model according to a preset cross-entropy loss function, the recognition result and the safety state label;

[0135] an updating sub-unit configured to update model parameters of the initial safety state recognition model by using the loss function value;

[0136] a second execution sub-unit configured to, in a case where the initial safety state recognition model after updating the model parameters does not meet a preset training completion condition, return to trigger the selection sub-unit to perform the step of selecting a target training sample currently used for training from each training sample in the training data set;

[0137] The third execution subunit is configured to determine the initial safety state recognition model as a trained safety state recognition model in a case where the initial safety state recognition model after updating the model parameter satisfies a preset training completion condition.

[0138] The specific principles and execution processes of each unit and module in the safety monitoring device based on artificial intelligence disclosed in the embodiments of the present application are the same as those of the safety monitoring method based on artificial intelligence disclosed in the embodiments of the present application, and can be referred to the corresponding parts in the safety monitoring method based on artificial intelligence provided in the embodiments of the present application, which will not be repeated here.

[0139] The embodiments of the present application further provide a storage medium including stored instructions, wherein the instructions, when executed, control a device where the storage medium is located to perform the safety monitoring method based on artificial intelligence.

[0140] The embodiments of the present application further provide an electronic device, a structure diagram of which is shown in Figure 6 The electronic device specifically includes a memory 601 and one or more instructions 602, wherein the one or more instructions 602 are stored in the memory 601 and are configured to be executed by one or more processors 603 to perform the following operations:

[0141] Obtain monitoring data of the device; the monitoring data includes image data of the device and a sequence of running state parameters of the device in a preset time period;

[0142] Process the image data through an image recognition module of a pre-trained safety state recognition model to obtain first features of the device;

[0143] Process the sequence of running state parameters through a convolutional long short-term memory network of the safety state recognition model to obtain second features of the device;

[0144] Fuse the first features and the second features through a feature fusion module of the safety state recognition model to obtain fused features of the device;

[0145] Process the fused features through a classifier of the safety state recognition model to obtain a safety state recognition result of the device;

[0146] Output the safety state recognition result of the device.

[0147] It should be noted that the various embodiments of the present specification are described in progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between embodiments can be mutually referred to. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0148] Finally, it should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0149] For the convenience of description, the above device is described as various units by function respectively. Of course, in the implementation of the present application, the functions of each unit can be implemented in the same or more software and / or hardware.

[0150] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including a number of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0151] The above provides a detailed introduction to the safety monitoring method based on artificial intelligence provided by the present application, and the principle and implementation manner of the present application are described by applying specific examples; the above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed; in view of the above, the content of the present specification should not be understood as a limitation of the present application.

Claims

1. A security monitoring method based on artificial intelligence, characterized in that: include: Acquiring monitoring data of the device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period; The operating status parameter sequence is acquired based on sensors on the device; The operating status parameter sequence includes vibration data, device temperature data, pressure data, current data, voltage data, power data, ambient temperature data and air pressure data at each time point within a preset time period; Processing the image data using an image recognition module of a pre-trained security state recognition model to obtain a first feature of the device; Processing the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device; fusing the first feature and the second feature through a feature fusion module of the security status identification model to obtain a fusion feature of the device; Processing the fusion features through the classifier of the security status identification model to obtain a security status identification result of the device; Outputting a security status identification result of the device; Obtaining a probability score of a security status identification result, an anomaly score and a historical trend score of sensor data collected by the sensor; Calculating the probability score, anomaly score, and historical trend score based on their respective weight coefficients and the weights adjusted based on the security status identification result to obtain a comprehensive security score for the device; When the comprehensive safety score of the equipment is less than a preset score threshold, the equipment is controlled to shut down and a backup device of the equipment is started; The anomaly score is calculated using the following formula: in, is the abnormality score, is the sensor data at the current time t; a is the lower limit of the normal range of sensor data, and b is the upper limit of the normal range of sensor data; The historical trend score is calculated using the following formula: in, is the historical trend score, μ is the average value of the sensor data in the historical time period, and σ is the standard deviation of the sensor data in the historical time period.

2. The method according to claim 1, characterized in that The image recognition module comprises an image segmentation module, a patch embedding module, a position encoding module, and a multi-head self-attention mechanism module. The image recognition module processes the image data using a pre-trained security state recognition model to obtain the first feature of the device, including: Converting the image data into a patch sequence by an image segmentation module in the image recognition module; Performing linear mapping on each image patch in the patch sequence by a patch embedding module of the image recognition module to obtain an embedding vector for each image patch; Performing position encoding on the embedding vector of each image patch based on a preset position encoding matrix by the position encoding module of the image recognition module to obtain a target embedding vector of each image patch; The target embedding vector of each image patch is processed by the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

3. The method according to claim 1, characterized in that The training process of the security status recognition model includes: Obtaining a training data set and an initial safety state recognition model to be trained; the training data set includes multiple training samples and a safety state label for each training sample; each training sample includes historical monitoring data of a device, and the safety state label is used to indicate the safety state of the device, where the safety state is one of a normal state, an abnormal state, and a fault state; Selecting a target training sample currently used for training from each training sample in the training data set; Inputting the target training sample into the initial security state recognition model to obtain a recognition result output by the initial security state recognition model; Calculating a loss function value of the initial safety state recognition model based on a preset cross entropy loss function, the recognition result, and the safety state label; Updating the model parameters of the initial safety state identification model according to the loss function value; If the initial safety state recognition model after the model parameters are updated does not meet the preset training completion condition, returning to the step of selecting a target training sample currently used for training from each training sample in the training data set; When the initial safety state recognition model after the model parameters are updated meets a preset training completion condition, the initial safety state recognition model is determined as a trained safety state recognition model.

4. The method according to claim 1, wherein Outputting the security status identification result of the device includes: The safety status identification result of the device is sent to a preset remote monitoring center.

5. A safety monitoring device based on artificial intelligence, characterized in that: include: An acquisition unit, configured to acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period; the sequence of operating status parameters is acquired based on data collected by sensors on the device; The operating status parameter sequence includes vibration data, device temperature data, pressure data, current data, voltage data, power data, ambient temperature data and air pressure data at each time point within a preset time period; a first execution unit, configured to process the image data using an image recognition module of a pre-trained security state recognition model to obtain a first feature of the device; a second execution unit, configured to process the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device; a feature fusion unit, configured to fuse the first feature and the second feature through a feature fusion module of the security state recognition model to obtain a fusion feature of the device; an identification unit, configured to process the fusion feature through a classifier of the security status identification model to obtain a security status identification result of the device; An output unit, configured to output a security status identification result of the device; The device is further configured to obtain a probability score of a security status identification result, an anomaly score and a historical trend score of sensor data collected by the sensor; and calculate the probability score, anomaly score and historical trend score based on their respective weight coefficients and a weight adjusted based on the security status identification result to obtain a comprehensive security score for the device; When the comprehensive safety score of the equipment is less than a preset score threshold, the equipment is controlled to shut down and a backup device of the equipment is started; The anomaly score is calculated using the following formula: in, is the abnormality score, is the sensor data at the current time t; a is the lower limit of the normal range of sensor data, and b is the upper limit of the normal range of sensor data; The historical trend score is calculated using the following formula: in, is the historical trend score, μ is the average value of the sensor data in the historical time period, and σ is the standard deviation of the sensor data in the historical time period.

6. The device according to claim 5, characterized in that The image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the first execution unit includes: a conversion subunit, configured to convert the image data into a patch sequence through an image segmentation module in the image recognition module; a mapping subunit, configured to perform linear mapping on each image patch in the patch sequence through a patch embedding module of the image recognition module to obtain an embedding vector for each image patch; an encoding subunit, configured to perform position encoding on the embedding vector of each image patch based on a preset position encoding matrix through the position encoding module of the image recognition module, so as to obtain a target embedding vector of each image patch; A processing subunit is used to process the target embedding vector of each image patch through the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

7. The device according to claim 5, characterized in that The first execution unit includes: an acquisition subunit, configured to acquire a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label for each training sample; each training sample includes historical monitoring data of a device, and the safety state label is used to indicate the safety state of the device, where the safety state is one of a normal state, an abnormal state, and a fault state; A selection subunit, configured to select a target training sample currently used for training from among the training samples in the training data set; A first execution subunit is configured to input the target training sample into the initial security state recognition model to obtain a recognition result output by the initial security state recognition model; a calculation subunit, configured to calculate a loss function value of the initial security state recognition model based on a preset cross entropy loss function, the recognition result, and the security state label; An updating subunit, configured to update the model parameters of the initial safety state identification model using the loss function value; A second execution subunit is configured to, when the initial safety state recognition model after updating the model parameters does not meet the preset training completion condition, return to trigger the selection subunit to execute the step of selecting a target training sample currently used for training from each training sample in the training data set; The third execution subunit is configured to determine the initial security state recognition model as a trained security state recognition model when the initial security state recognition model after updating the model parameters meets a preset training completion condition.

8. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the artificial intelligence-based security monitoring method according to any one of claims 1 to 4.

9. An electronic device, characterized in that: The invention comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 4 as an artificial intelligence-based security monitoring method.

Images