Safety monitoring method and device based on artificial intelligence, storage medium and electronic equipment

Abstract

The invention provides a safety monitoring method and device based on artificial intelligence, a storage medium and electronic equipment. The method comprises the steps of obtaining monitoring data of the equipment; the monitoring data comprises image data of the equipment and a running state parameter sequence of the equipment in a preset time period; processing the image data through an image recognition module of a pre-trained security state recognition model to obtain a first feature of the equipment; processing the operation state parameter sequence through a convolutional long-short term memory network of a security state recognition model to obtain a second feature of the equipment; fusing the first feature and the second feature through a feature fusion module of a security state recognition model to obtain a fused feature of the equipment; and processing the fusion features through a classifier of a security state recognition model to obtain a security state recognition result of the equipment, and outputting the security state recognition result. Through the method provided by the embodiment of the invention, an accurate security state identification result can be obtained.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a security monitoring method, device, storage medium and electronic device based on artificial intelligence. Background Art

[0002] With the development of industrial automation and intelligence, equipment safety monitoring has become an important part of ensuring production efficiency and personnel safety. Traditional equipment status monitoring methods mainly rely on manual inspections and technical personnel's experience and judgment. This method is not only inefficient, but also prone to missed detection or misjudgment. In recent years, with the rapid development of the Internet of Things (IoT), big data, and artificial intelligence technologies, more and more research and applications are focused on how to use advanced technical means to improve the accuracy and reliability of equipment status monitoring.

[0003] At present, in the field of equipment status monitoring, there are some technical solutions based on sensor data. These solutions usually collect physical parameters of the equipment during operation, such as temperature and pressure, through various sensors deployed on the equipment, and identify the working status of the equipment through data processing and analysis. However, existing methods often have limitations when facing complex working conditions and multi-modal data, such as: Single modal data processing: Most existing technologies only focus on a specific type of sensor data, ignoring the correlation and complementarity between different modal data, resulting in incomplete monitoring results. Summary of the invention

[0004] The technical problem to be solved by this application is to provide a security monitoring method, device, storage medium and electronic device based on artificial intelligence, which can obtain accurate security monitoring results. The specific solution is as follows:

[0005] A security monitoring method based on artificial intelligence, comprising:

[0006] Acquiring monitoring data of the device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period;

[0007] Processing the image data through an image recognition module of a pre-trained security status recognition model to obtain a first feature of the device;

[0008] Processing the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device;

[0009] fusing the first feature and the second feature through a feature fusion module of the security status recognition model to obtain a fusion feature of the device;

[0010] Processing the fusion feature through the classifier of the security status recognition model to obtain a security status recognition result of the device;

[0011] Output the security status identification result of the device.

[0012] The above method, optionally, the image recognition module, the image segmentation module, the patch embedding module, the position encoding module and the multi-head self-attention mechanism module, the image recognition module of the pre-trained security state recognition model processes the image data to obtain the first feature of the device, including:

[0013] Converting the image data into a patch sequence by an image segmentation module in the image recognition module;

[0014] Performing linear mapping on each image patch in the patch sequence through a patch embedding module of the image recognition module to obtain an embedding vector for each image patch;

[0015] Performing position encoding on the embedding vector of each image patch based on a preset position encoding matrix by the position encoding module of the image recognition module to obtain a target embedding vector of each image patch;

[0016] The target embedding vector of each image patch is processed by the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

[0017] In the above method, optionally, the training process of the safety status recognition model includes:

[0018] Acquire a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device, the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state and a fault state;

[0019] Selecting a target training sample currently used for training from each training sample in the training data set;

[0020] Inputting the target training sample into the initial safety state recognition model to obtain a recognition result output by the initial safety state recognition model;

[0021] Calculate the loss function value of the initial safety state recognition model according to a preset cross entropy loss function, the recognition result and the safety state label;

[0022] Updating the model parameters of the initial safety state identification model according to the loss function value;

[0023] If the initial safety state recognition model after updating the model parameters does not meet the preset training completion condition, return to the step of selecting a target training sample currently used for training from each training sample in the training data set;

[0024] When the initial safety state recognition model after updating the model parameters satisfies a preset training completion condition, the initial safety state recognition model is determined as a trained safety state recognition model.

[0025] In the above method, optionally, the outputting the security status identification result of the device includes:

[0026] The safety status identification result of the device is sent to a preset remote monitoring center.

[0027] The above method may optionally further include:

[0028] When the safety status identification result meets the preset shutdown condition, the device is controlled to shut down and the backup device of the device is started.

[0029] A safety monitoring device based on artificial intelligence, comprising:

[0030] An acquisition unit, used to acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period;

[0031] A first execution unit, configured to process the image data through an image recognition module of a pre-trained security state recognition model to obtain a first feature of the device;

[0032] A second execution unit is used to process the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device;

[0033] a feature fusion unit, configured to fuse the first feature and the second feature through a feature fusion module of the security state recognition model to obtain a fusion feature of the device;

[0034] an identification unit, configured to process the fusion feature through a classifier of the security status identification model to obtain a security status identification result of the device;

[0035] An output unit is used to output the security status identification result of the device.

[0036] The above-mentioned device, optionally, the image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the first execution unit includes:

[0037] a conversion subunit, configured to convert the image data into a patch sequence through an image segmentation module in the image recognition module;

[0038] A mapping subunit, configured to perform linear mapping on each image patch in the patch sequence through a patch embedding module of the image recognition module to obtain an embedding vector of each image patch;

[0039] an encoding subunit, configured to perform position encoding on the embedding vector of each of the image patches based on a preset position encoding matrix through a position encoding module of the image recognition module, so as to obtain a target embedding vector of each of the image patches;

[0040] A processing subunit is used to process the target embedding vector of each image patch through the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

[0041] In the above device, optionally, the first execution unit includes:

[0042] An acquisition subunit is used to acquire a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device, and the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state, and a fault state;

[0043] A selection subunit, used to select a target training sample currently used for training from each training sample in the training data set;

[0044] A first execution subunit is used to input the target training sample into the initial security state recognition model to obtain a recognition result output by the initial security state recognition model;

[0045] A calculation subunit, configured to calculate a loss function value of the initial safety state recognition model according to a preset cross entropy loss function, the recognition result and the safety state label;

[0046] An updating subunit, used for updating the model parameters of the initial safety state identification model according to the loss function value;

[0047] A second execution subunit is used to return to trigger the selection subunit to execute the step of selecting a target training sample currently used for training from each training sample in the training data set when the initial safety state recognition model after updating the model parameters does not meet the preset training completion condition;

[0048] The third execution subunit is used to determine the initial security state recognition model as a trained security state recognition model when the initial security state recognition model after updating the model parameters meets a preset training completion condition.

[0049] A storage medium, comprising stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute an artificial intelligence-based security monitoring method as described above.

[0050] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform an artificial intelligence-based security monitoring method as described above.

[0051] Based on the implementation of the present application, an artificial intelligence-based security monitoring method, device, storage medium and electronic device are provided. The method includes: obtaining monitoring data of the device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period; processing the image data through an image recognition module of a pre-trained security state recognition model to obtain a first feature of the device; processing the operating status parameter sequence through a convolutional long short-term memory network of the security state recognition model to obtain a second feature of the device; fusing the first feature and the second feature through a feature fusion module of the security state recognition model to obtain a fusion feature of the device; processing the fusion feature through a classifier of the security state recognition model to obtain a security state recognition result of the device; and outputting the security state recognition result of the device. The method provided by the embodiment of the present application can obtain accurate security state recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 A method flow chart of a security monitoring method based on artificial intelligence provided in this application;

[0054] Figure 2 A schematic diagram of the structure of a security status identification model provided for this application;

[0055] Figure 3 A flowchart of a process for obtaining a first characteristic of a device provided by the present application;

[0056] Figure 4 A flowchart of a training process of a security status recognition model provided in this application;

[0057] Figure 5 A schematic diagram of the structure of a safety monitoring device based on artificial intelligence provided in this application;

[0058] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0061] The embodiment of the present application provides a security monitoring method based on artificial intelligence, which can be applied to electronic devices, and the electronic devices can be computers, smart phones, tablet devices, smart wearable devices, server clusters or cloud computing centers, etc. The method flow chart of the method is as follows Figure 1 As shown, specifically including:

[0062] S101: Acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period.

[0063] In this embodiment, the image data of the device can be collected in a timed or real-time manner by an image acquisition device. Then, the operating state parameter sequence of the device is collected by a sensor arranged on the device; the operating state parameter sequence of the device may include information such as mechanical state parameters, electrical parameters, and environmental parameters at each time point within a preset time period; the mechanical state parameters may include at least one of vibration data, device temperature data, and pressure data; the electrical parameters may include at least one of current data, voltage data, and power data; and the environmental parameters may include one or more of ambient temperature data and air pressure data.

[0064] Optionally, the device may be various types of equipment, such as power generation equipment, power transmission equipment or production equipment.

[0065] S102: Process the image data through an image recognition module of a pre-trained security status recognition model to obtain a first feature of the device.

[0066] In this embodiment, the image recognition module may be a vision transformer (ViT) or a ConvNeXt model, etc.

[0067] Optionally, the first feature may be an image feature of the device.

[0068] S103: Processing the operating state parameter sequence through the convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device.

[0069] Optionally, the security state recognition model may include one or more convolutional long short-term memory networks, each of which may include an input gate, a forget gate, an output gate, and a unit state.

[0070] S104: The first feature and the second feature are fused by a feature fusion module of the security status recognition model to obtain a fusion feature of the device.

[0071] In this embodiment, the feature fusion module includes a fully connected layer and a shared multi-head attention mechanism unit.

[0072] Optionally, the first feature and the second feature are respectively mapped to a preset dimensional space through the fully connected layer of the feature fusion module to obtain a first feature vector and a second feature vector; the first feature vector and the second feature vector are input into a shared multi-head attention mechanism unit, and the first feature vector and the second feature vector processed by the shared multi-head attention mechanism unit are fused to obtain a fusion feature of the device.

[0073] S105: Processing the fusion feature through the classifier of the security status recognition model to obtain a security status recognition result of the device.

[0074] In this embodiment, the classifier may be a softmax classifier, and the safety state identification result may include one of a normal state, a slightly abnormal state, a seriously abnormal state, and a fault state.

[0075] S106: Output the security status identification result of the device.

[0076] In this embodiment, the security status identification result may be output and displayed on a preset visual interface, or the security status identification result may be sent to a preset remote monitoring center.

[0077] In one embodiment provided in the present application, the safety status recognition model may include an image recognition module, at least one convolutional long short-term memory network, a feature fusion module and a classifier.

[0078] See also Figure 2 , which is a structural diagram of a safety state recognition model provided in an embodiment of the present application, and the safety state recognition model specifically includes an input layer, an image recognition module, at least one convolutional long short-term memory network, a feature fusion module, and a classifier. The monitoring data of the device can be input into the safety state recognition module, and the image recognition module in the safety state recognition model extracts the image features (first features) of the image data in the monitoring data, and the convolutional long short-term memory network extracts the parameter features (second features) in the operating state parameter sequence, and then the feature fusion module fuses the image features and parameter features to obtain multimodal features (fused features); finally, the classifier outputs the safety state recognition result based on the multimodal features.

[0079] Through the method provided in the embodiment of the present application, accurate security status identification results can be obtained.

[0080] In one embodiment provided in the present application, based on the above implementation process, optionally, the image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the image recognition module of the pre-trained security state recognition model processes the image data to obtain the first feature of the device, such as Figure 3 As shown, including:

[0081] S301: Convert the image data into a patch sequence through an image segmentation module in the image recognition module.

[0082] In this embodiment, the patch sequence may include a plurality of image patches, where the image patches are image blocks; the image segmentation module may segment the image data into a plurality of image patches of preset sizes.

[0083] S302: Linearly map each image patch in the patch sequence through the patch embedding module of the image recognition module to obtain an embedding vector of each image patch.

[0084] In this embodiment, the image patch may be converted into a one-dimensional vector of a fixed length, and then the one-dimensional vector of the fixed length may be mapped to an embedding space of a fixed dimension through a preset weight matrix to obtain an embedding vector of the image patch.

[0085] S303: Position encoding is performed on the embedding vector of each of the image patches based on a preset position encoding matrix by the position encoding module of the image recognition module to obtain a target embedding vector of each of the image patches.

[0086] In this embodiment, the position code corresponding to the embedding vector of each image patch can be determined in the position code matrix; the embedding vector of each image patch is added to the corresponding position code to obtain the target embedding vector.

[0087] In this embodiment, the position encoding matrix can be a predefined matrix generated by a sine function or a cosine function. Such position encoding is periodic and can capture the relationship between different positions. The position encoding matrix can also be a learnable parameter matrix, which can be used as part of a safe state recognition model to learn the optimal position information representation through training.

[0088] S304: Process the target embedding vector of each image patch through the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

[0089] In an embodiment provided in the present application, based on the above implementation process, optionally, the training process of the security state recognition model is as follows: Figure 4 As shown, including:

[0090] S401: Acquire a training data set and an initial safety status recognition model to be trained; the training data set includes a plurality of training samples and a safety status label of each training sample; each training sample includes historical monitoring data of a device.

[0091] Optionally, the safety status label is used to indicate the safety status of the device, where the safety status is one of a normal state, an abnormal state and a fault state; the abnormal state may be a slight abnormal state or a serious abnormal state.

[0092] In this embodiment, the safety status label may include a normal status label, a slight abnormal status label, a serious abnormal status label and a fault status label, etc.; the normal status label is used to indicate that the device is in a normal state, the slight abnormal status label is used to indicate that the device is in a slight abnormal state, the serious abnormal status label is used to indicate that the device is in a serious abnormal state, and the fault status label is used to indicate that the device is in a fault state.

[0093] S402: Selecting a target training sample currently used for training from each training sample in the training data set.

[0094] S403: Input the target training sample into the initial safety state recognition model to obtain a recognition result output by the initial safety state recognition model.

[0095] S404: Calculate the loss function value of the initial safety state recognition model according to a preset cross entropy loss function, the recognition result and the safety state label.

[0096] S405: updating the model parameters of the initial safety state identification model according to the loss function value.

[0097] S406: Detect whether the initial safety state recognition model after updating the model parameters meets the preset training completion condition. If not, return to execute S402; if so, execute S407.

[0098] In this embodiment, the training completion condition may be that the number of training times is greater than a preset training number threshold, the prediction accuracy of the initial safety state recognition model is greater than a preset accuracy threshold, or the loss function converges, etc.

[0099] S407: Determine the initial safety state recognition model as a trained safety state recognition model.

[0100] In an embodiment provided in the present application, based on the above implementation process, optionally, it further includes:

[0101] When the safety status identification result meets the preset shutdown condition, the equipment is controlled to shut down.

[0102] Optionally, after the control device is shut down, maintenance reminder information for the device can be output to prompt the user to inspect and maintain the equipment, thereby reducing the risk of accidents and extending the service life of the equipment.

[0103] Optionally, if there is a backup device for the device, the backup device for the device may be started.

[0104] In this embodiment, the comprehensive safety score of the device can be determined according to the safety status identification result, and when the comprehensive safety score of the device is less than a preset score threshold, the device is controlled to shut down.

[0105] Specifically, the process of determining the status score of the device according to the security status identification result is as follows:

[0106] S(t)=W label ×(W1×P(t)+W2×A(t))+W3×T(t))

[0107] Where S(t) is the comprehensive safety score of the device at time t; W label is the weight adjusted according to the safety status identification result predicted by the model; W1, W2 and W3 are the weight coefficients of the corresponding score items, which are used to adjust the importance of each item; P(t) is the probability score predicted by the model; A(t) is the anomaly score of the sensor data; T(t) is the historical trend score.

[0108] In this embodiment, the probability score P(t) predicted by the model can be determined based on the maximum probability in the prediction results. For example, if the probability distribution of the model output is [p1, p2, p3, p4], the score can be defined as the maximum probability value minus a small threshold to avoid too high a score:

[0109] P(t)=max(p1, p2, p3, p4)-∈

[0110] Where ∈ is a small positive number used to prevent the score from being too high.

[0111] In this embodiment, the abnormal score A(t) of the sensor data can be determined by calculating the deviation of the data point from the normal range. For example, if the normal range of the sensor data is [a, b], the score can be defined as:

[0112]

[0113] where x t is the sensor reading at time t. If the reading is within the normal range, the score is close to 1; if the reading deviates from the normal range, the score decreases.

[0114] In this embodiment, the historical trend score T(t) can be determined by comparing the current reading with the average value over the past period of time. For example, if the average value of the sensor data over the past period of time is μ, the score can be defined as:

[0115]

[0116] Where σ is the standard deviation of the sensor data over the past period of time. If the current reading is close to the historical average, the score is close to 1; if the deviation is large, the score is reduced.

[0117] In this embodiment, when the comprehensive safety score is less than the preset score threshold, the equipment can be controlled to shut down, which can prevent the equipment from continuing to operate in a dangerous state and reduce the possibility of accidents. Automated scoring reduces reliance on manual judgment and reduces safety hazards caused by human factors.

[0118] and Figure 1 Corresponding to the method described above, the embodiment of the present application also provides a safety monitoring device based on artificial intelligence for Figure 1 The specific implementation of the method is shown in the following diagram: Figure 5 As shown, specifically including:

[0119] An acquisition unit 501 is used to acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period;

[0120] A first execution unit 502 is used to process the image data through an image recognition module of a pre-trained security state recognition model to obtain a first feature of the device;

[0121] A second execution unit 503 is used to process the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device;

[0122] A feature fusion unit 504, configured to fuse the first feature and the second feature through a feature fusion module of the security state recognition model to obtain a fusion feature of the device;

[0123] An identification unit 505, configured to process the fusion feature through a classifier of the security status identification model to obtain a security status identification result of the device;

[0124] The output unit 506 is used to output the security status identification result of the device.

[0125] In an embodiment provided in the present application, based on the above solution, optionally, the image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the first execution unit includes:

[0126] a conversion subunit, configured to convert the image data into a patch sequence through an image segmentation module in the image recognition module;

[0127] A mapping subunit, configured to perform linear mapping on each image patch in the patch sequence through a patch embedding module of the image recognition module to obtain an embedding vector of each image patch;

[0128] an encoding subunit, configured to perform position encoding on the embedding vector of each of the image patches based on a preset position encoding matrix through a position encoding module of the image recognition module, so as to obtain a target embedding vector of each of the image patches;

[0129] A processing subunit is used to process the target embedding vector of each image patch through the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

[0130] In an embodiment provided in the present application, based on the above solution, optionally, the first execution unit includes:

[0131] An acquisition subunit is used to acquire a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device, and the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state, and a fault state;

[0132] A selection subunit, used to select a target training sample currently used for training from each training sample in the training data set;

[0133] A first execution subunit is used to input the target training sample into the initial security state recognition model to obtain a recognition result output by the initial security state recognition model;

[0134] A calculation subunit, configured to calculate a loss function value of the initial safety state recognition model according to a preset cross entropy loss function, the recognition result and the safety state label;

[0135] An updating subunit, used for updating the model parameters of the initial safety state identification model according to the loss function value;

[0136] A second execution subunit is used to return to trigger the selection subunit to execute the step of selecting a target training sample currently used for training from each training sample in the training data set when the initial safety state recognition model after updating the model parameters does not meet the preset training completion condition;

[0137] The third execution subunit is used to determine the initial security state recognition model as a trained security state recognition model when the initial security state recognition model after updating the model parameters meets a preset training completion condition.

[0138] The specific principles and execution processes of each unit and module in the artificial intelligence-based security monitoring device disclosed in the above-mentioned embodiment of the present application are the same as the artificial intelligence-based security monitoring method disclosed in the above-mentioned embodiment of the present application. Please refer to the corresponding parts of the artificial intelligence-based security monitoring method provided in the above-mentioned embodiment of the present application, and no further details will be given here.

[0139] An embodiment of the present application also provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned artificial intelligence-based security monitoring method.

[0140] The present application also provides an electronic device, whose structural diagram is as follows: Figure 6 As shown, it specifically includes a memory 601 and one or more instructions 602, wherein the one or more instructions 602 are stored in the memory 601 and are configured to be executed by one or more processors 603 to perform the following operations:

[0141] Acquiring monitoring data of the device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period;

[0142] Processing the image data through an image recognition module of a pre-trained security status recognition model to obtain a first feature of the device;

[0143] Processing the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device;

[0144] fusing the first feature and the second feature through a feature fusion module of the security status recognition model to obtain a fusion feature of the device;

[0145] Processing the fusion feature through the classifier of the security status recognition model to obtain a security status recognition result of the device;

[0146] Output the security status identification result of the device.

[0147] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0148] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0149] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0150] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0151] The above is a detailed introduction to an artificial intelligence-based security monitoring method provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A security monitoring method based on artificial intelligence, characterized in that: include: Acquiring monitoring data of the device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period; Processing the image data through an image recognition module of a pre-trained security status recognition model to obtain a first feature of the device; Processing the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device; fusing the first feature and the second feature through a feature fusion module of the security status recognition model to obtain a fusion feature of the device; Processing the fusion feature through the classifier of the security status recognition model to obtain a security status recognition result of the device; Output the security status identification result of the device.

2. The method according to claim 1, characterized in that The image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the image recognition module of the pre-trained security state recognition model processes the image data to obtain the first feature of the device, including: Converting the image data into a patch sequence by an image segmentation module in the image recognition module; Performing linear mapping on each image patch in the patch sequence through a patch embedding module of the image recognition module to obtain an embedding vector for each image patch; Performing position encoding on the embedding vector of each image patch based on a preset position encoding matrix by the position encoding module of the image recognition module to obtain a target embedding vector of each image patch; The target embedding vector of each image patch is processed by the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

3. The method according to claim 1, characterized in that The training process of the safety status recognition model includes: Acquire a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device, the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state and a fault state; Selecting a target training sample currently used for training from each training sample in the training data set; Inputting the target training sample into the initial safety state recognition model to obtain a recognition result output by the initial safety state recognition model; Calculate the loss function value of the initial safety state recognition model according to a preset cross entropy loss function, the recognition result and the safety state label; Updating the model parameters of the initial safety state identification model according to the loss function value; If the initial safety state recognition model after updating the model parameters does not meet the preset training completion condition, return to the step of selecting a target training sample currently used for training from each training sample in the training data set; When the initial safety state recognition model after updating the model parameters satisfies a preset training completion condition, the initial safety state recognition model is determined as a trained safety state recognition model.

4. The method according to claim 1, characterized in that The outputting the security status identification result of the device includes: The safety status identification result of the device is sent to a preset remote monitoring center.

5. The method according to claim 1, characterized in that Also includes: When the safety status identification result meets the preset shutdown condition, the device is controlled to shut down and the backup device of the device is started.

6. A safety monitoring device based on artificial intelligence, characterized in that: include: An acquisition unit, used to acquire monitoring data of a device; the monitoring data includes image data of the device and a sequence of operating status parameters of the device within a preset time period; A first execution unit, configured to process the image data through an image recognition module of a pre-trained security state recognition model to obtain a first feature of the device; A second execution unit is used to process the operating state parameter sequence through a convolutional long short-term memory network of the safety state recognition model to obtain a second feature of the device; a feature fusion unit, configured to fuse the first feature and the second feature through a feature fusion module of the security state recognition model to obtain a fusion feature of the device; an identification unit, configured to process the fusion feature through a classifier of the security status identification model to obtain a security status identification result of the device; An output unit is used to output the security status identification result of the device.

7. The device according to claim 6, characterized in that The image recognition module, image segmentation module, patch embedding module, position encoding module and multi-head self-attention mechanism module, the first execution unit includes: a conversion subunit, configured to convert the image data into a patch sequence through an image segmentation module in the image recognition module; A mapping subunit, configured to perform linear mapping on each image patch in the patch sequence through a patch embedding module of the image recognition module to obtain an embedding vector of each image patch; an encoding subunit, configured to perform position encoding on the embedding vector of each of the image patches based on a preset position encoding matrix through a position encoding module of the image recognition module, so as to obtain a target embedding vector of each of the image patches; A processing subunit is used to process the target embedding vector of each image patch through the multi-head self-attention mechanism module of the image recognition module to obtain the first feature of the device.

8. The device according to claim 6, characterized in that The first execution unit includes: An acquisition subunit is used to acquire a training data set and an initial safety state recognition model to be trained; the training data set includes a plurality of training samples and a safety state label of each training sample; each training sample includes historical monitoring data of a device, and the safety state label is used to indicate the safety state of the device, and the safety state is one of a normal state, an abnormal state, and a fault state; A selection subunit, used to select a target training sample currently used for training from each training sample in the training data set; A first execution subunit is used to input the target training sample into the initial security state recognition model to obtain a recognition result output by the initial security state recognition model; A calculation subunit, configured to calculate a loss function value of the initial safety state recognition model according to a preset cross entropy loss function, the recognition result and the safety state label; An updating subunit, used for updating the model parameters of the initial safety state identification model according to the loss function value; A second execution subunit is used to return to trigger the selection subunit to execute the step of selecting a target training sample currently used for training from each training sample in the training data set when the initial safety state recognition model after updating the model parameters does not meet the preset training completion condition; The third execution subunit is used to determine the initial security state recognition model as a trained security state recognition model when the initial security state recognition model after updating the model parameters meets a preset training completion condition.

9. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the artificial intelligence-based security monitoring method as described in any one of claims 1 to 5.

10. An electronic device, characterized in that: It comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to implement an artificial intelligence-based security monitoring method as described in any one of claims 1 to 5.

Images